A social robot detection method based on feature fusion and multi-graph convolutional neural networks

Abstract

This invention relates to a method and apparatus for detecting social robots using a multi-graph convolutional neural network with feature fusion, comprising the following steps: acquiring user account data from a social platform; constructing a multi-graph network relationship based on explicit and implicit content relationships in the social network according to the user account data; extracting explicit features from the multi-graph network relationship to obtain first feature data; extracting implicit features from the multi-graph network relationship to obtain second feature data; fusing the first feature data and the second feature data to obtain first feature fusion data; dividing the first graph network relationship and / or the second graph network relationship into overlapping communities, calculating the node features of the overlapping communities to obtain third feature data; fusing the third feature data and the first feature fusion data to obtain second feature fusion data; and inputting the second feature fusion data into a multi-graph convolutional neural network for training to generate a social robot detection classifier.

Description

Technical Field

[0001] This invention relates to the field of computer science and technology, and particularly to the field of social robot detection. Background Technology

[0002] Social bots are accounts active on social media exhibiting clearly robotic behavior. They can be categorized into bots and semi-bots based on the degree of human involvement. The social bot population is growing rapidly, and their pronouncements on social media can have a significant impact, particularly in areas such as spreading misinformation, manipulating reviews, and controlling online buzz, where they have a negative effect. Early social bots were relatively dispersed across social networks, with subtle social relationships, but their robotic behavior was readily apparent, making identification relatively easy. However, with advancements in bot detection technology, bots have begun to use social network relationships to evade detection, forming unique network structures that significantly increase the difficulty of detection. With the application of generative algorithms and increased human involvement, the new generation of social bots exhibits highly realistic simulations, demanding continuous improvement in social bot detection methods. Currently, the evolution of social bots and the advancement of social bot detection technology are locked in a stalemate of competition.

[0003] Most existing social robot detection technologies focus on individual robot detection. The mainstream detection methods combine various user feature information, especially social relationship information, to build graph network models. However, even when various features are fused for social robot detection, several problems still exist. First, existing graph network methods rely heavily on user social relationships and have poor transferability. While social relationships such as followers and following relationships are effective features, whether the advantages of graph network models can still be utilized for detection once these relationships are unavailable requires higher standards for graph network relationship construction methods. Second, a typical application of graph network relationships is community detection, which involves using community detection methods to form different clusters of nodes in the graph and then analyzing them. According to research, there are currently few methods that combine the features mined by community detection methods with deep graph network models, especially those involving overlapping community methods. Introducing this community information can further improve the accuracy of deep graph network detection methods. Third, regarding deep graph network modeling of social bots, deep graph network methods still essentially rely on the construction of various features. As social bots are continuously updated and upgraded, some newly constructed features related to social bots can also continue to improve the detection effect of social bots. These features should be continuously integrated into the latest detection models. Summary of the Invention

[0004] When conducting research on social robot detection, the inventors found that graph network methods rely heavily on social network relationships, have high data requirements, and exhibit poor transferability. Some graph network methods may degrade to the point of being unusable when followers or follower relationships are lacking. Furthermore, there is currently limited research combining graph network methods with graph community detection methods, and even less research combining overlapping community detection methods with graph neural network models, indicating room for improvement. Additionally, features continuously summarized as social robots are updated and upgraded also need to be constantly integrated into the model to improve its detection accuracy.

[0005] To address the aforementioned problems, this invention discloses a feature-fusion-based multi-graph convolutional neural network method for social robot detection, comprising the following steps:

[0006] Obtain user account data from social media platforms;

[0007] Construct a multi-graph network relationship based on explicit and implicit content relationships in social networks using the user account data;

[0008] Explicit feature extraction is performed on the relationships in the multi-graph network to obtain the first feature data;

[0009] Implicit feature extraction is performed on the relationships in the multi-graph network to obtain the second feature data;

[0010] The first feature data and the second feature data are fused to obtain the first feature fused data;

[0011] Overlapping communities are partitioned into explicit content relationships and / or implicit content relationships in the social networks of the multi-graph network relationship, and the node characteristics of the overlapping communities are calculated to obtain the third feature data.

[0012] The third feature data and the first feature fusion data are fused together to obtain the second feature fusion data;

[0013] The second feature fusion data is input into a multi-graph convolutional neural network for training to generate a social robot detection classifier; wherein...

[0014] The step of constructing a multi-graph network relationship based on explicit content relationships in social networks is established using social relationships;

[0015] The step of constructing a multi-graph network relationship based on implicit content relationships in social networks is established using text content relationships;

[0016] The step of performing overlapping community segmentation is carried out using an improved overlapping community detection method, and further includes:

[0017] Step S600: Optimize seed node initialization selection, including:

[0018] Step S6001: Obtain node information in the multigraph network relationship;

[0019] Step S6002: Sort and segment the nodes according to their degree values;

[0020] Step S6003: Select the node with the median degree value of each segment as the representative seed node, which means that the seed node will not be deleted during the community shrinkage process;

[0021] Step S610: Optimize the community expansion and contraction strategy, including:

[0022] Step S6101: Traverse the seed nodes;

[0023] Step S6102: Start building the community from the current seed node;

[0024] Step S6103: Traverse the neighboring nodes representing the seed node in order of degree value;

[0025] Step S6104: Expand the community;

[0026] Step S6105: If the addition of a neighbor node causes the fitness to increase beyond a first fitness threshold, add a neighbor node and proceed to step S6107 to shrink the community; otherwise, proceed to step S6106.

[0027] Step S6106: When the traversal of neighbor nodes is completed and the fitness increment of the community after all neighbor nodes have joined the community is lower than a second fitness threshold, the expansion stops. If the traversal of neighbor nodes is not completed, proceed to step S6105.

[0028] Step S6107: Perform community contraction;

[0029] Step S6108: Traverse the nodes within the community;

[0030] Step S6109: If the increase in fitness due to a node leaving exceeds the first fitness threshold, delete the neighboring node and proceed to step S6103; otherwise, proceed to step S6110.

[0031] Step S6110: When the traversal of nodes within the community is completed, and the fitness increment of the community after all neighboring nodes leave the community is lower than the second fitness threshold, then stop shrinking and proceed to step S6103. If the traversal of nodes within the community is not completed, proceed to step S6108.

[0032] Step S6111: The community of the current seed node has been established. Execute step S6101 to continue traversing the next seed and node.

[0033] Step S6112: After all seed nodes have been traversed, multiple expanded and contracted communities are obtained;

[0034] Step S620: Optimize the aggregation after overlapping community segmentation, including:

[0035] Step S6201: Traverse the communities after shrinking and expanding, calculate the community overlap, and merge two communities with an overlap exceeding a merging threshold.

[0036] Step S6202: Obtain the nodes belonging to the overlapping communities.

[0037] In one embodiment of the present invention, the step of establishing multiple graph network relationships of explicit content relationships in a social network using social relationships further includes:

[0038] Establish networks of followers, fans, comments, reposts, replies, and likes.

[0039] In one embodiment of the present invention, the step of establishing a multi-graph network relationship of implicit content relationships in a social network using text content relationships further includes:

[0040] Establish text copying networks, topic co-occurrence networks, and mention networks.

[0041] In one embodiment of the present invention, the step of establishing a text copying network further includes:

[0042] Step S211: Obtain all accounts and their posting information under multiple datasets or topics;

[0043] Step S212: Vectorize the published information to obtain a text vector;

[0044] Step S213: Calculate the similarity between the text vectors using a similarity threshold; if the similarity exceeds the threshold, they are considered highly similar.

[0045] Step S214: Establish a connection between accounts that have highly similar posts.

[0046] In one embodiment of the present invention, the step of vectorizing the published information further includes:

[0047] The published information is vectorized using word vector calculation and text classification tools.

[0048] In one embodiment of the present invention, the step of establishing a topic co-occurrence network further includes:

[0049] Step S221: Establish link relationships between accounts that post the same topic and / or accounts that comment or repost under the same topic;

[0050] Step S222: Sort the accounts by the number of times they appear from highest to lowest;

[0051] Step S223: Take the median number of times the account appears;

[0052] Step S224: Compare the lower median with a configurable first threshold;

[0053] Step S225: If the lower median is greater than or equal to the first threshold, retain the connection relationship of the account;

[0054] Step S226: If the lower median is less than the first threshold, the connection relationship of the account is not retained.

[0055] In one embodiment of the present invention, the step of explicitly extracting features from the multi-graph network relationships further includes:

[0056] Feature extraction is performed on user metadata features, user behavior features, time series features, content features, and social network features.

[0057] In one embodiment of the present invention, the step of extracting features from user metadata features further includes:

[0058] Step S311: Use an object detection algorithm to extract entity information from the profile picture of the account, identify the entity in the person knowledge base, and obtain the result of whether there is a highly similar profile picture;

[0059] Step S312: Extract the number of IP address changes in the first time period and the number of IP address changes in the same topic for the account.

[0060] Step S313: Extract the number of account IDs on the same terminal device and the same terminal device under the same topic;

[0061] Step S314: Extract the number of columns for personal information to check the completeness of personal information;

[0062] Step S315: Extract the account's registration time.

[0063] In one embodiment of the present invention, the step of extracting features from user behavior features further includes:

[0064] Step S321: Extract the average boosting influence of the posting time sequence;

[0065] Step S322: Extract records of whether the account has repeatedly reposted or commented on its own original content;

[0066] Step S323: Check if the account has a record of unfollowing;

[0067] Step S324: Extract whether the account has a record of long-term dormancy based on a configurable second threshold;

[0068] Step S325: In a configurable second time period, compare the account's activity level in the second time period with the account's daily average activity level to extract whether there are any abnormal account activity levels;

[0069] Step S326: Extract records of whether the account has enabled push notifications.

[0070] In one embodiment of the present invention, the step of extracting features from time-series features further includes:

[0071] Step S331: Calculate the entropy value of the account's posting time series;

[0072] Step S332: Calculate the entropy value of the posting time interval sequence of the account;

[0073] Step S333: Calculate the entropy value of the sequence of the current posting time interval minus the previous posting time interval.

[0074] In one embodiment of the present invention, the step of extracting features from content features further includes:

[0075] Step S341: Calculate the content similarity of extremely short response times under the topic;

[0076] Step S342: Calculate the monthly sentiment polarity variance of posts;

[0077] Step S343: Extract records of whether the account uses multiple languages ​​in its posts.

[0078] In one embodiment of the present invention, the step of extracting features from social network features further includes:

[0079] Step S351: Calculate the proportion of different genders among neighboring nodes based on the node information in the multigraph network relationship;

[0080] Step S352: Calculate the sum of the out-degree and in-degree of the node;

[0081] Step S353: Calculate the central mediability of the network.

[0082] In one embodiment of the present invention, the step of implicitly extracting the relationships in the multi-graph network further includes:

[0083] The content posted by the account is transformed into implicit vectors using the BERT model.

[0084] In one embodiment of the present invention, the step of calculating fitness further includes:

[0085]

[0086] in, Represents fitness. Indicates the degree within the community. Indicates the degree of externality of the community.

[0087] In one embodiment of the present invention, the step of calculating community overlap further includes:

[0088] Calculate the intersection of accounts in two communities and divide it by the union of accounts in the two communities.

[0089] In one embodiment of the present invention, the step of calculating the node characteristics of overlapping communities further includes:

[0090] Calculate the number of communities a node belongs to if it belongs to an overlapping community, the degree value of a node within a community of the top three sizes, the number of neighboring nodes belonging to the same overlapping community, and the degree value characteristics of nodes within non-overlapping communities.

[0091] In one embodiment of the present invention, the multi-graph convolutional neural network further includes:

[0092] Embedding layer, first convolutional layer, second convolutional layer, fully connected layer, and softmax inference layer;

[0093] The loss function is the weighted cross-entropy loss function, which is used to address the problem of uneven distribution of positive and negative samples.

[0094] In one embodiment of the present invention, the social network includes Weibo, Twitter, and Facebook.

[0095] This invention also discloses a feature fusion-based multi-graph convolutional neural network social robot detection device, comprising:

[0096] The user account acquisition module is used to acquire user account data from social media platforms.

[0097] The multi-graph network relationship construction module is used to construct multi-graph network relationships based on explicit and implicit content relationships in social networks according to the user account data.

[0098] An explicit feature extraction module is used to explicitly extract features from the relationships in the multi-graph network to obtain first feature data;

[0099] An implicit feature extraction module is used to extract implicit features from the relationships in the multi-graph network to obtain second feature data;

[0100] The overlapping community segmentation module is used to segment overlapping communities into explicit content relationships and / or implicit content relationships of social networks in the multi-graph network relationship, calculate the node features of overlapping communities, and obtain third feature data.

[0101] The feature data fusion module is used to fuse the first feature data and the second feature data to obtain first feature fusion data, and to fuse the third feature data and the first feature fusion data to obtain second feature fusion data;

[0102] The training module is used to input the second feature fusion data into a multi-graph convolutional neural network for training, thereby generating a social robot detection classifier;

[0103] Overlapping community discovery modules include:

[0104] The seed node selection module is used to optimize the initial selection of seed nodes, including:

[0105] Obtain node information in the multigraph network relationship;

[0106] The nodes are sorted and segmented according to their degree values;

[0107] The node with the median degree value of each segment is selected as the representative seed node, which means that the seed node will not be deleted during the community shrinkage process.

[0108] The community expansion / contraction module is used to optimize community expansion / contraction strategies, including:

[0109] Step S6101: Traverse the seed nodes;

[0110] Step S6102: Start building the community from the current seed node;

[0111] Step S6103: Traverse the neighboring nodes representing the seed node in order of degree value;

[0112] Step S6104: Expand the community;

[0113] Step S6105: If the addition of a neighbor node causes the fitness to increase beyond a first fitness threshold, add a neighbor node and proceed to step S6107 to shrink the community; otherwise, proceed to step S6106.

[0114] Step S6106: When the traversal of neighbor nodes is completed and the fitness increment of the community after all neighbor nodes have joined the community is lower than a second fitness threshold, the expansion stops. If the traversal of neighbor nodes is not completed, proceed to step S6105.

[0115] Step S6107: Perform community contraction;

[0116] Step S6108: Traverse the nodes within the community;

[0117] Step S6109: If the increase in fitness due to a node leaving exceeds the first fitness threshold, delete the neighboring node and proceed to step S6103; otherwise, proceed to step S6110.

[0118] Step S6110: When the traversal of nodes within the community is completed, and the fitness increment of the community after all neighboring nodes leave the community is lower than the second fitness threshold, then stop shrinking and proceed to step S6103. If the traversal of nodes within the community is not completed, proceed to step S6108.

[0119] Step S6111: The community of the current seed node has been established. Execute step S6101 to continue traversing the next seed and node.

[0120] Step S6112: After all seed nodes have been traversed, multiple expanded and contracted communities are obtained;

[0121] The community aggregation module includes:

[0122] Traverse the communities after shrinkage and expansion, calculate the community overlap, and merge two communities with an overlap exceeding a merging threshold.

[0123] Obtain the nodes belonging to the overlapping communities; among them,

[0124] The construction of a multi-graph network relationship based on explicit content relationships in social networks is established using social relationships;

[0125] The construction of a multi-graph network relationship based on implicit content relationships in social networks is established using text content relationships.

[0126] The present invention also discloses an electronic device, comprising: at least one processor, and a memory connected to the processor, wherein the memory stores a computer program executable by the processor, the computer program implementing the steps of any of the methods described above when executed by the processor.

[0127] The present invention also discloses a storage medium for storing a computer control program for performing the steps of the method as described in any one of the preceding claims. Attached Figure Description

[0128] Figure 1 This is a schematic diagram illustrating the steps of a multi-graph convolutional neural network social robot detection method based on feature fusion in one embodiment of the present invention.

[0129] Figure 2This is a block diagram illustrating the construction of a multi-graph network relationship based on explicit and implicit content relationships in a social network, according to one embodiment of the present invention.

[0130] Figure 3 This is a schematic diagram illustrating the steps of establishing a text copying network in one embodiment of the present invention.

[0131] Figure 4 This is a schematic diagram illustrating the steps of establishing a topic co-occurrence network in one embodiment of the present invention.

[0132] Figure 5 This is a schematic diagram illustrating the steps of feature extraction for user metadata features in one embodiment of the present invention.

[0133] Figure 6 This is a schematic diagram illustrating the steps of feature extraction for user behavior features in one embodiment of the present invention.

[0134] Figure 7 This is a schematic diagram illustrating the steps of extracting features from temporal features in one embodiment of the present invention.

[0135] Figure 8 This is a schematic diagram illustrating the steps of feature extraction of content features in one embodiment of the present invention.

[0136] Figure 9 This is a schematic diagram illustrating the steps of feature extraction from social network features in one embodiment of the present invention.

[0137] Figure 10 This is a flowchart illustrating the steps of an improved overlapping community detection method in one embodiment of the present invention.

[0138] Figure 11 This is a block diagram of a multi-graph convolutional neural network social robot detection method based on feature fusion in one embodiment of the present invention.

[0139] Figure 12 This is a block diagram of the architecture of a multi-graph convolutional neural network social robot detection device based on feature fusion in one embodiment of the present invention.

[0140] Figure 13 This is a block diagram of an electronic device according to one embodiment of the present invention.

[0141] In the attached figures, the following labels are used:

[0142] 1: Multigraph Network Relationships

[0143] 2: First feature data

[0144] 3: Second feature data

[0145] 4: Third feature data

[0146] 10: A social robot detection device based on feature fusion and multi-graph convolutional neural network

[0147] 11: User Account Acquisition Module

[0148] 12: Multi-graph network relationship construction module

[0149] 13: Explicit Feature Extraction Module

[0150] 14: Implicit Feature Extraction Module

[0151] 15: Overlapping Community Segmentation Module

[0152] 16: Feature Data Fusion Module

[0153] 17: Training Module

[0154] 20: Multi-Graph Convolutional Neural Networks

[0155] 21: Embedding Layer

[0156] 22: First image: Convolutional layer

[0157] 23: Second convolutional layer

[0158] 24: Fully Connected Layer

[0159] 25: Softmax Inference Layer Detailed Implementation

[0160] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that references to "an embodiment," "embodiment," "example embodiment," etc., in the specification refer to the described embodiment including specific features, structures, or characteristics, but not necessarily including these specific features, structures, or characteristics. Furthermore, such expressions do not refer to the same embodiment. Moreover, when describing specific features, structures, or characteristics in conjunction with embodiments, whether or not explicitly described, it is indicated that incorporating such features, structures, or characteristics into other embodiments is within the knowledge scope of those skilled in the art.

[0161] The specification and subsequent claims use certain terms to refer to specific modules, components, or parts. Those skilled in the art will understand that users or manufacturers may use different names or terms to refer to the same module, component, or part. This specification and subsequent claims do not distinguish modules, components, or parts by differences in name, but rather by differences in function. The terms "comprising" and "including" used throughout the specification and subsequent claims are open-ended and should be interpreted as "including but not limited to." Furthermore, the term "connection" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connections via other means.

[0162] Furthermore, in the following description and claims, numerous terms will be referenced, which should be defined as having the following meanings. The singular forms “a” and “the” include plural referents, unless the context clearly specifies otherwise. “Optional” or “optionally” indicates that an event or situation subsequently described may or may not occur, and the description includes both the scenario where the event occurs and the scenario where the event does not occur.

[0163] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating the steps of a multi-graph convolutional neural network social robot detection method based on feature fusion in one embodiment of the present invention.

[0164] This invention discloses a multi-graph convolutional neural network method for social robot detection based on feature fusion.

[0165] Includes the following steps:

[0166] Step S1: Obtain user account data from social media platforms;

[0167] Step S2: Construct a multi-graph network relationship 1 based on explicit and implicit content relationships in the social network according to the user account data;

[0168] Step S3: Explicitly extract features from the multi-graph network relationship 1 to obtain the first feature data 2;

[0169] Step S4: Implicit feature extraction is performed on the multi-graph network relationship 1 to obtain the second feature data 3;

[0170] Step S5: Fuse the first feature data 2 and the second feature data 3 to obtain the first feature fusion data;

[0171] Step S6: Perform overlapping community segmentation on the explicit content relationship of the social network of the multi-graph network relationship and / or the implicit content relationship of the social network of the multi-graph network relationship, calculate the node characteristics of the overlapping communities, and obtain the third feature data 4.

[0172] Step S7: Fuse the third feature data 4 and the first feature fusion data to obtain the second feature fusion data;

[0173] Step S8: Input the second feature fusion data into the multi-graph convolutional neural network 20 for training to generate a social robot detection classifier.

[0174] It should be noted that the collection and use of relevant user data in this application strictly comply with the "Personal Information Protection Law of the People's Republic of China," the "Cybersecurity Law," and other relevant laws and regulations. All data adheres strictly to information security management requirements, effectively protecting users' personal information rights and network data security, and will not be illegally disclosed or misused. Please refer to... Figure 2 , Figure 2 This is a block diagram illustrating the construction of a multi-graph network relationship based on explicit and implicit content relationships in a social network, according to one embodiment of the present invention.

[0175] The steps described above for constructing a multi-graph network relationship based on explicit content relationships in social networks are established using social relationships.

[0176] In one embodiment of the present invention, the step of establishing a multi-graph network relationship 1 using social relationships to establish explicit content relationships in a social network further includes:

[0177] Establish networks of followers, fans, comments, reposts, replies, and likes.

[0178] The steps described above for constructing a multi-graph network relationship based on implicit content relationships in social networks are established using text content relationships.

[0179] In one embodiment of the present invention, the step of establishing the multi-graph network relationship 1 of the social network implicit content relationship using text content relationship further includes:

[0180] Establish text copying networks, topic co-occurrence networks, and mention networks.

[0181] Text copying networks use the same or highly similar content posted by different accounts to establish connections. The homogenization of posts among social bots provides the foundation for the construction of this network and can better uncover social bot groups with organized behavior.

[0182] Topic co-occurrence network refers to the connection between accounts that post the same topic hashtag in a Weibo topic context.

[0183] Mentioning networks refer to accounts that post content and tag other accounts to establish connections. These networks can reveal core node accounts that are frequently mentioned by other accounts.

[0184] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating the steps of establishing a text copying network in one embodiment of the present invention.

[0185] In one embodiment of the present invention, the step of establishing a text copying network further includes:

[0186] Step S211: Obtain all accounts and their posting information under multiple datasets or topics;

[0187] Step S212: Vectorize the published information to obtain a text vector;

[0188] Step S213: Calculate the similarity between the text vectors using a similarity threshold; if the similarity exceeds the threshold, they are considered highly similar.

[0189] Step S214: Establish a connection between accounts that have highly similar posts.

[0190] The above steps for vectorizing the issued information further include:

[0191] As those skilled in the art will know, text can be vectorized using tools such as fastText, and other word vectorization and text classification tools can also be used for text vectorization.

[0192] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating the steps of establishing a topic co-occurrence network in one embodiment of the present invention.

[0193] In one embodiment of the present invention, the step of establishing a topic co-occurrence network further includes:

[0194] Step S221: Establish link relationships between accounts that post the same topic and / or accounts that comment or repost under the same topic;

[0195] Step S222: Sort the accounts by the number of times they appear from highest to lowest;

[0196] Step S223: Take the median number of times the account appears;

[0197] Step S224: Compare the lower median with a configurable first threshold;

[0198] Step S225: If the lower median is greater than or equal to the first threshold, retain the connection relationship of the account;

[0199] Step S226: If the lower median is less than the first threshold, the connection relationship of the account is not retained.

[0200] The first threshold can be 7, but this invention is not limited to this.

[0201] The steps for explicit feature extraction of the multi-graph network relation 1 described above further include:

[0202] Feature extraction is performed on user metadata features, user behavior features, time series features, content features, and social network features.

[0203] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the steps of feature extraction for user metadata features in one embodiment of the present invention.

[0204] In one embodiment of the present invention, the above-mentioned step of extracting features from user metadata features further includes:

[0205] Step S311: Use the object detection algorithm Faster R-CNN to extract entity information from the account's profile picture, identify entities in the person knowledge base, and obtain the result of whether there are highly similar profile pictures; for example, extract entity information such as anime characters from the image, and check whether there are similarities with other account profile pictures, such as a series of account profile pictures generated from screenshots of the same image of the same cartoon character from different angles.

[0206] Step S312: Extract the number of IP address changes in the first time period and the number of IP address changes in the same topic for the account.

[0207] Step S313: Extract the number of account IDs on the same terminal device and the same terminal device under the same topic;

[0208] Step S314: Extract the number of columns for personal information to check the completeness of personal information;

[0209] Step S315: Extract the account's registration time.

[0210] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the steps of feature extraction for user behavior features in one embodiment of the present invention.

[0211] In one embodiment of the present invention, the above-mentioned step of extracting user behavior features further includes:

[0212] Step S321: Extract the average boosting influence of the posting time sequence. ,in,

[0213] T represents the number of topics the account participated in. Let be the order of the earliest posting times of the accounts under the t-th topic. For the first Total number of posts under this topic;

[0214] Step S322: Extract records of whether the account has repeatedly reposted or commented on its own original content;

[0215] Step S323: Check if the account has a record of unfollowing;

[0216] Step S324: Based on a configurable second threshold, extract whether the account has a record of long-term dormancy;

[0217] Step S325: In a configurable second time period, compare the account's activity level in the second time period with the account's daily average activity level to extract whether there are any abnormal account activity levels;

[0218] Step S326: Extract records of whether the account has enabled push notification functionality.

[0219] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating the steps of extracting features from temporal features in one embodiment of the present invention.

[0220] In one embodiment of the present invention, the above-mentioned step of extracting features from time-series features further includes:

[0221] Step S331: Calculate the entropy value of the account's posting time series.

[0222]

[0223] in, For the first The probability of the distribution of post publication times;

[0224] Step S332: Calculate the entropy value of the posting time interval sequence of the account.

[0225] ,

[0226] in, For the first Post and the The probability distribution of the time difference between the posting times;

[0227] Step S333: Calculate the entropy value of the sequence of the current posting interval minus the previous posting interval.

[0228] ,

[0229] in, For the first The time difference between the first post and the first The probability distribution of the difference in time between posts.

[0230] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating the steps of feature extraction of content features in one embodiment of the present invention.

[0231] In one embodiment of the present invention, the above-mentioned step of extracting features from content features further includes:

[0232] Step S341: Calculate the content similarity of very short response times under the topic.

[0233]

[0234] in The content posted by the representative account This represents the semantic information of the text content as a vector representation. This indicates that the similarity of articles published within a short period of time is calculated.

[0235] Step S342: Calculate the variance of monthly post sentiment polarity.

[0236] ,

[0237] in, Let i be the sentiment value of the i-th post. The average sentiment of all posts within a month;

[0238] Step S343: Extract records of whether the account uses multiple languages ​​in its posts.

[0239] Please see Figure 9 , Figure 9 This is a schematic diagram illustrating the steps of feature extraction from social network features in one embodiment of the present invention.

[0240] In one embodiment of the present invention, the above-mentioned step of extracting features from social network features further includes:

[0241] Step S351: Calculate the proportion of different genders among neighboring nodes based on the node information in the multigraph network relationship 1;

[0242] Step S352: Calculate the sum of the out-degree and in-degree of each node. ;in, Indicates the in-degree of a node. Indicates the out-degree of the node;

[0243] Step S353: Calculate the centrality of the network. Where i and j are two nodes in the network. This is the shortest path bridge node connecting two nodes.

[0244] In one embodiment of the present invention, the step of implicit feature extraction of the above-mentioned multi-graph network relationship 1 further includes:

[0245] The content posted by the account is transformed into implicit vectors using the BERT model.

[0246] Please see Figure 10 , Figure 10 This is a flowchart illustrating the steps of an improved overlapping community detection method in one embodiment of the present invention.

[0247] The steps described above for segmenting overlapping communities were performed using an improved method for discovering overlapping communities.

[0248] In one embodiment of the present invention, the steps of the improved overlapping community detection method further include:

[0249] Step S600: Optimize seed node initialization selection;

[0250] Step S610: Optimize community expansion and contraction strategies;

[0251] Step S620: Optimize the aggregation after overlapping community segmentation.

[0252] In one embodiment of the present invention, the steps of the improved overlapping community detection method further include:

[0253] Step S6001: Obtain node information in the multigraph network relationship 1;

[0254] Step S6002: Sort and segment the nodes according to their degree values;

[0255] Step S6003: Select the node with the median degree value of each segment as the representative seed node, which means that the seed node will not be deleted during the community shrinkage process.

[0256] In one embodiment of the present invention, the steps of the improved overlapping community detection method further include:

[0257] Step S6101: Traverse the seed nodes;

[0258] Step S6102: Start building the community from the current seed node;

[0259] Step S6103: Traverse the neighboring nodes representing the seed node in order of degree value;

[0260] Step S6104: Expand the community;

[0261] Step S6105: If the addition of a neighbor node causes the fitness to increase beyond a first fitness threshold, add a neighbor node and proceed to step S6107 to shrink the community; otherwise, proceed to step S6106.

[0262] Step S6106: When the traversal of neighbor nodes is completed and the fitness increment of the community after all neighbor nodes have joined the community is lower than a second fitness threshold, the expansion stops. If the traversal of neighbor nodes is not completed, proceed to step S6105.

[0263] Step S6107: Perform community contraction;

[0264] Step S6108: Traverse the nodes within the community;

[0265] Step S6109: If the increase in fitness due to a node leaving exceeds the first fitness threshold, delete the neighboring node and proceed to step S6103; otherwise, proceed to step S6110.

[0266] Step S6110: When the traversal of nodes within the community is completed, and the fitness increment of the community after all neighboring nodes leave the community is lower than the second fitness threshold, then stop shrinking and proceed to step S6103. If the traversal of nodes within the community is not completed, proceed to step S6108.

[0267] Step S6111: The community of the current seed node has been established. Execute step S6101 to continue traversing the next seed and node.

[0268] Step S6112: After all seed nodes have been traversed, multiple expanded and contracted communities are obtained.

[0269] In one embodiment of the present invention, the steps of the improved overlapping community detection method further include:

[0270] Step S6201: Traverse the communities after shrinking and expanding, calculate the community overlap, and merge two communities with an overlap exceeding a merging threshold.

[0271] Step S6202: Obtain the nodes belonging to the overlapping communities.

[0272] In one embodiment of the present invention, the step of calculating fitness further includes:

[0273]

[0274] in, Represents fitness. Indicates the degree within the community. Indicates the degree of externality of the community.

[0275] In one embodiment of the present invention, the step of calculating community overlap further includes:

[0276] Calculate the intersection of accounts in two communities and divide it by the union of accounts in the two communities.

[0277] In one embodiment of the present invention, the step of calculating the node characteristics of overlapping communities further includes:

[0278] Calculate the number of communities a node belongs to if it belongs to an overlapping community, the degree value of a node within a community of the top three sizes, the number of neighboring nodes belonging to the same overlapping community, and the degree value characteristics of nodes within non-overlapping communities.

[0279] Please see Figure 11 , Figure 11 This is a block diagram of a multi-graph convolutional neural network social robot detection method based on feature fusion in one embodiment of the present invention.

[0280] In one embodiment of the present invention, the above-mentioned multi-graph convolutional neural network 20 further includes:

[0281] Embedding layer 21, first convolutional layer 22, second convolutional layer 23, fully connected layer 24, and softmax inference layer 25;

[0282] The loss function is the weighted cross-entropy loss function, which is used to address the problem of uneven distribution of positive and negative samples.

[0283] Specifically, the fused first feature data 2, second feature data 3, and third feature data 4 are input into the multi-graph convolutional neural network 20 for end-to-end training to generate a social robot detection classifier. The multi-graph convolutional neural network (R-GCN), with homogeneous nodes but heterogeneous edges, essentially performs graph convolution operations on different homogeneous networks and ultimately merges the outputs of different networks. Here, an embedding layer 21, a first convolutional layer 22, a second convolutional layer 23, a fully connected layer 24, and a softmax inference layer 25 are used to calculate the probability that an account belongs to a social robot. Chebyshev polynomials are used as convolution kernels to aggregate the information of the neighboring nodes through two-step graph convolution operations. Finally, the weighted cross-entropy loss function is used to address the problem of uneven distribution of positive and negative samples.

[0284] In one embodiment of the present invention, the social network includes Weibo, Twitter, and Facebook.

[0285] Please see Figure 12 , Figure 12 This is a block diagram of the architecture of a multi-graph convolutional neural network social robot detection device based on feature fusion in one embodiment of the present invention.

[0286] The present invention also discloses a feature fusion-based multi-graph convolutional neural network social robot detection device 10, comprising:

[0287] User account acquisition module 11 is used to acquire user account data from social platforms;

[0288] The multi-graph network relationship construction module 12 is used to construct a multi-graph network relationship 1 based on explicit and implicit content relationships of social networks according to the user account data;

[0289] The explicit feature extraction module 13 is used to perform explicit feature extraction on the multi-graph network relationship 1 to obtain the first feature data 2;

[0290] The implicit feature extraction module 14 is used to perform implicit feature extraction on the multi-graph network relationship 1 to obtain the second feature data 3;

[0291] The overlapping community segmentation module 15 is used to segment the explicit content relationship of the social network of the multi-graph network relationship and / or the implicit content relationship of the social network of the multi-graph network relationship into overlapping communities, calculate the node features of the overlapping communities, and obtain the third feature data 4.

[0292] The feature data fusion module 16 is used to fuse the first feature data 2 and the second feature data 3 to obtain first feature fusion data, and to fuse the third feature data 4 and the first feature fusion data to obtain second feature fusion data;

[0293] Training module 17 is used to input the second feature fusion data into multi-graph convolutional neural network 20 for training to generate a social robot detection classifier.

[0294] Please see Figure 13 , Figure 13 This is a block diagram of an electronic device according to one embodiment of the present invention.

[0295] The present invention also discloses an electronic device, comprising: at least one processor, and a memory connected to the processor, wherein the memory stores a computer program executable by the processor, the computer program implementing the steps of any of the above methods when executed by the processor.

[0296] The processor 1001 described above can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 1001 can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0297] The present invention also discloses a storage medium for storing a computer control program for performing the steps of the method as described in any one of the preceding claims.

[0298] The aforementioned processor-executable computer program may be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0299] In summary, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can devise various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the protection scope of the patent application of the present invention.

Claims

1. A social robot detection method using multi-graph convolutional neural networks with feature fusion. Its characteristics are Therefore, the following steps are included: Obtain user account data from social media platforms; Based on the user account data, a multi-graph network relationship is constructed based on explicit and implicit content relationships in social networks; Explicit feature extraction is performed on the relationships in the multi-graph network to obtain the first feature data; Implicit feature extraction is performed on the relationships in the multi-graph network to obtain the second feature data; The first feature data and the second feature data are fused to obtain the first feature fused data; Overlapping communities are partitioned into explicit content relationships and / or implicit content relationships in the social networks of the multi-graph network relationship, and the node characteristics of the overlapping communities are calculated to obtain the third feature data. The third feature data and the first feature fusion data are fused together to obtain the second feature fusion data; The second feature fusion data is input into a multi-graph convolutional neural network for training to generate a social robot detection classifier; wherein... The step of constructing a multi-graph network relationship based on explicit content relationships in social networks is established using social relationships; The step of constructing a multi-graph network relationship based on implicit content relationships in social networks is established using text content relationships; The step of performing overlapping community segmentation is carried out using an improved overlapping community detection method, and further includes: Step S600: Optimize seed node initialization selection, including: Step S6001: Obtain node information in the multigraph network relationship; Step S6002: Sort and segment the nodes according to their degree values; Step S6003: Select the node with the median degree value of each segment as the representative seed node, which means that the seed node will not be deleted during the community shrinkage process; Step S610: Optimize the community expansion and contraction strategy, including: Step S6101: Traverse the seed nodes; Step S6102: Start building the community from the current seed node; Step S6103: Traverse the neighboring nodes representing the seed node in order of degree value; Step S6104: Expand the community; Step S6105: If the addition of a neighbor node causes the fitness to increase beyond a first fitness threshold, add a neighbor node and execute step S6107 to shrink the community; otherwise, execute step S6106. Step S6106: When the traversal of neighbor nodes is completed and the fitness increment of the community after all neighbor nodes have joined the community is lower than a second fitness threshold, the expansion stops. If the traversal of neighbor nodes is not completed, proceed to step S6105. Step S6107: Perform community contraction; Step S6108: Traverse the nodes within the community; Step S6109: If the increase in fitness due to a node leaving exceeds the first fitness threshold, delete the neighboring node and proceed to step S6103; otherwise, proceed to step S6110. Step S6110: When the traversal of nodes within the community is completed, and the fitness increment of the community after all neighboring nodes leave the community is lower than the second fitness threshold, then stop shrinking and proceed to step S6103. If the traversal of nodes within the community is not completed, proceed to step S6108. Step S6111: The community of the current seed node has been established. Execute step S6101 to continue traversing the next seed and node. Step S6112: After all seed nodes have been traversed, multiple expanded and contracted communities are obtained; Step S620: Optimize the aggregation after overlapping community segmentation, including: Step S6201: Traverse the communities after shrinking and expanding, calculate the community overlap, and merge two communities with an overlap exceeding a merging threshold. Step S6202: Obtain the nodes belonging to the overlapping communities.

2. The method as described in claim 1, characterized in that, The steps for establishing multiple graph network relationships that utilize social relationships to create explicit content relationships on social networks further include: Establish networks of followers, fans, comments, reposts, replies, and likes.

3. The method as described in claim 1, characterized in that, The steps for establishing implicit content relationships in a multi-graph network using text content relationships further include: Establish text copying networks, topic co-occurrence networks, and mention networks.

4. The method as described in claim 3, characterized in that, The step of establishing the text copying network further includes: Step S211: Obtain all accounts and their posting information under multiple datasets or topics; Step S212: Vectorize the published information to obtain a text vector; Step S213: Calculate the similarity between the text vectors; if the similarity exceeds a similarity threshold, they are considered highly similar. Step S214: Establish a connection between accounts that have highly similar posts.

5. The method as described in claim 4, characterized in that, The step of vectorizing the published information further includes: The published information is vectorized using word vector calculation and text classification tools.

6. The method as described in claim 4, characterized in that, The steps for establishing the topic co-occurrence network further include: Step S221: Establish link relationships between accounts that post the same topic and / or accounts that comment or repost under the same topic; Step S222: Sort the accounts by the number of times they appear from highest to lowest; Step S223: Take the median number of times the account appears; Step S224: Compare the lower median with a configurable first threshold; Step S225: If the lower median is greater than or equal to the first threshold, retain the connection relationship of the account; Step S226: If the lower median is less than the first threshold, the connection relationship of the account is not retained.

7. The method as described in claim 6, characterized in that, The step of explicitly extracting features from the relationships in the multi-graph network further includes: Feature extraction is performed on user metadata features, user behavior features, time series features, content features, and social network features.

8. The method as described in claim 7, characterized in that, The step of extracting features from user metadata features further includes: Step S311: Use an object detection algorithm to extract entity information from the profile picture of the account, identify the entity in the person knowledge base, and obtain the result of whether there is a highly similar profile picture; Step S312: Extract the number of IP address changes in the first time period and the number of IP address changes in the same topic for the account. Step S313: Extract the number of account IDs on the same terminal device and the same terminal device under the same topic; Step S314: Extract the number of columns for personal information to check the completeness of personal information; Step S315: Extract the account's registration time.

9. The method as described in claim 7, characterized in that, The step of extracting features from user behavior characteristics further includes: Step S321: Extract the average boosting influence of the posting time sequence; Step S322: Extract records of whether the account has repeatedly reposted or commented on its own original content; Step S323: Check if the account has a record of unfollowing; Step S324: Extract whether the account has a record of long-term dormancy based on a configurable second threshold; Step S325: In a configurable second time period, compare the account's activity level in the second time period with the account's daily average activity level to extract whether there are any abnormal account activity levels; Step S326: Extract records of whether the account has enabled push notifications.

10. The method as described in claim 7, characterized in that, The step of extracting features from time-series features further includes: Step S331: Calculate the entropy value of the account's posting time series; Step S332: Calculate the entropy value of the posting time interval sequence of the account; Step S333: Calculate the entropy value of the sequence of the current posting time interval minus the previous posting time interval.

11. The method as described in claim 7, characterized in that, The step of extracting features from content features further includes: Step S341: Calculate the content similarity of extremely short response times under the topic; Step S342: Calculate the monthly sentiment polarity variance of posts; Step S343: Extract records of whether the account uses multiple languages ​​in its posts.

12. The method as described in claim 7, characterized in that, The step of feature extraction from social network features further includes: Step S351: Calculate the proportion of different genders among neighboring nodes based on the node information in the multigraph network relationship; Step S352: Calculate the sum of the out-degree and in-degree of the node; Step S353: Calculate the central mediability of the network.

13. The method as described in claim 1, characterized in that, The step of implicitly extracting the relationships in the multigraph network further includes: The content posted by the account is transformed into implicit vectors using the BERT model.

14. The method as described in claim 1, characterized in that, The step of calculating fitness further includes: in, Represents fitness. Indicates the degree within the community. Indicates the degree of externality of the community.

15. The method as described in claim 1, characterized in that, The step of calculating community overlap further includes: Calculate the intersection of accounts in two communities and divide it by the union of accounts in the two communities.

16. The method as described in claim 15, characterized in that, The step of calculating the node characteristics of overlapping communities further includes: Calculate the number of communities a node belongs to if it belongs to an overlapping community, the degree value of a node within a community of the top three sizes, the number of neighboring nodes belonging to the same overlapping community, and the degree value characteristics of nodes within non-overlapping communities.

17. The method as described in claim 1, characterized in that, The multi-graph convolutional neural network further includes: Embedding layer, first convolutional layer, second convolutional layer, fully connected layer, and softmax inference layer; The loss function is the weighted cross-entropy loss function, which is used to address the problem of uneven distribution of positive and negative samples.

18. The method as described in claim 1, characterized in that, The social networks mentioned include Weibo, Twitter, and Facebook.

19. A social robot detection device based on feature fusion and multi-graph convolutional neural network, characterized in that, include: The user account acquisition module is used to acquire user account data from social media platforms. The multi-graph network relationship construction module is used to construct multi-graph network relationships based on explicit and implicit content relationships in social networks according to the user account data. An explicit feature extraction module is used to explicitly extract features from the relationships in the multi-graph network to obtain first feature data; An implicit feature extraction module is used to extract implicit features from the relationships in the multi-graph network to obtain second feature data; The overlapping community segmentation module is used to segment overlapping communities into explicit content relationships and / or implicit content relationships of social networks in the multi-graph network relationship, calculate the node features of overlapping communities, and obtain third feature data. The feature data fusion module is used to fuse the first feature data and the second feature data to obtain first feature fusion data, and to fuse the third feature data and the first feature fusion data to obtain second feature fusion data; The training module is used to input the second feature fusion data into a multi-graph convolutional neural network for training, thereby generating a social robot detection classifier; Overlapping community discovery modules include: The seed node selection module is used to optimize the initial selection of seed nodes, including: Obtain node information in the multigraph network relationship; The nodes are sorted and segmented according to their degree values; The node with the median degree value of each segment is selected as the representative seed node, which means that the seed node will not be deleted during the community shrinkage process. The community expansion / contraction module is used to optimize community expansion / contraction strategies, including: Step S6101: Traverse the seed nodes; Step S6102: Start building the community from the current seed node; Step S6103: Traverse the neighboring nodes representing the seed node in order of degree value; Step S6104: Expand the community; Step S6105: If the addition of a neighbor node causes the fitness to increase beyond a first fitness threshold, add a neighbor node and execute step S6107 to shrink the community; otherwise, execute step S6106. Step S6106: When the traversal of neighbor nodes is completed and the fitness increment of the community after all neighbor nodes have joined the community is lower than a second fitness threshold, the expansion stops. If the traversal of neighbor nodes is not completed, proceed to step S6105. Step S6107: Perform community contraction; Step S6108: Traverse the nodes within the community; Step S6109: If the increase in fitness due to a node leaving exceeds the first fitness threshold, delete the neighboring node and proceed to step S6103; otherwise, proceed to step S6110. Step S6110: When the traversal of nodes within the community is completed, and the fitness increment of the community after all neighboring nodes leave the community is lower than the second fitness threshold, then stop shrinking and proceed to step S6103. If the traversal of nodes within the community is not completed, proceed to step S6108. Step S6111: The community of the current seed node has been established. Execute step S6101 to continue traversing the next seed and node. Step S6112: After all seed nodes have been traversed, multiple expanded and contracted communities are obtained; The community aggregation module includes: Traverse the communities after shrinkage and expansion, calculate the community overlap, and merge two communities with an overlap exceeding a merging threshold. Obtain the nodes belonging to the overlapping communities; among them, The construction of a multi-graph network relationship based on explicit content relationships in social networks is established using social relationships; The construction of a multi-graph network relationship based on implicit content relationships in social networks is established using text content relationships.

20. An electronic device, characterized in that, include: At least one processor, and a memory connected to the processor, wherein the memory stores a computer program executable by the processor, the computer program, when executed by the processor, implementing the steps of any one of claims 1 to 18.

21. A storage medium for storing a computer control program, characterized in that, The computer control program is used to perform the steps of the method as described in any one of claims 1 to 18.

Images