Public opinion robot identification method and system based on gated hybrid expert and prototype comparative learning

Abstract

The invention discloses a public opinion robot identification method and system based on gated hybrid expert and prototype comparative learning, and relates to the technical field of social network user behavior analysis. The method comprises the following steps: acquiring multiple pieces of account information of a social platform and corresponding content text data; performing multi-modal node feature extraction on the multiple pieces of account information and the corresponding content text data; constructing and learning a multi-relation social graph model based on the extracted multi-modal node features; inputting the output features of the multi-relation social graph model into a gating hybrid expert model for training; inputting the multi-relation social graph model and the output features thereof into a comparison learning model based on a prototype for training; when the joint loss reaches a preset condition, stopping training, and storing an optimal configuration parameter; and identifying a social account to be identified through the multi-relation social graph model and the gating hybrid expert model with the optimal configuration parameters. The method can be used for accurate identification of public opinion robot accounts of a social platform.

Description

Technical Field

[0001] This invention relates to the field of social network user behavior analysis technology, specifically to a method and system for identifying opinion robots based on gated hybrid expert and prototype comparative learning. Background Technology

[0002] As social media platforms grow in scale, bot accounts are having a significant impact on public opinion and the platform ecosystem. Bots, also commonly known as social bots, botnets, or paid commenters, are accounts or software programs that automatically or semi-automatically generate, publish, spread, and interact (such as liking, forwarding, and commenting) on ​​online platforms such as social media, forums, and news websites, attempting to influence public opinion by simulating human users. They are not robots in the physical sense, but rather virtual accounts controlled by a string of code.

[0003] For the identification of opinion robots, existing technologies typically employ single discriminators or feature concatenation schemes. However, these traditional rules or single classifiers are difficult to adapt to distribution drift across datasets and over time. Graph models that rely solely on supervised learning are limited in effectiveness when there is insufficient labeling or class imbalance. While existing hybrid expert (MoE) frameworks can improve expressive power, they have limited collaboration with contrastive / clustering self-supervised learning in graph structure scenarios. On the other hand, prototype-based contrastive learning can shape globally separable structures in unlabeled or weakly labeled environments, but its collaboration with supervised branches and sample-level expert selection still needs improvement. Summary of the Invention

[0004] In view of the above problems, this invention proposes a method and system for identifying public opinion robots based on gated hybrid expert and prototype comparative learning, which can achieve higher recognition results in the identification of public opinion robots.

[0005] According to one aspect of the present invention, a method for identifying public opinion robots based on gated hybrid expert and prototype comparative learning is proposed, the method comprising:

[0006] Obtain information on multiple accounts on social media platforms and their corresponding text data.

[0007] Multimodal node feature extraction is performed on multiple account information and corresponding content text data;

[0008] A multi-relationship social graph model is constructed and learned based on the extracted multimodal node features;

[0009] The output features of the multi-relationship social graph model are input into a gated hybrid expert model for training; the multi-relationship social graph model and its output features are input into a prototype-based contrastive learning model for training; the joint loss of the two models is calculated during training; training is stopped when the joint loss reaches a preset condition, and the optimal configuration parameters of each model are saved;

[0010] For the social media account information to be identified and the corresponding content text data, after multimodal node feature extraction, the data is input into a multi-relationship social graph model and a gated hybrid expert model with optimal configuration parameters to obtain the identification result of the social media account to be identified; the identification result is whether the social media account is a public opinion robot or not.

[0011] Furthermore, the account information includes numerical information and modal information. The numerical information includes the number of user followers, the number of user following, the number of user likes, the total number of user posts, the number of active days of the user, and the username length. The modal information includes whether the account is verified, whether geolocation is enabled, whether it is protected, whether a background image is used, whether there is extended information, whether a default avatar is used, and whether a default profile picture is used.

[0012] Furthermore, the multimodal node feature extraction of multiple account information and corresponding content text data includes:

[0013] The pre-trained BERT model is used to extract features from the content text data to obtain the text features and account description features of each account; the numerical information of each account is constructed into a numerical modality vector and normalized to obtain the numerical features; the modality information of each account is constructed into a Boolean modality vector as the modality features; the text features, account description features, numerical features, and modality features of each account constitute the multimodal node features of that account.

[0014] Furthermore, the construction and learning of a multi-relationship social graph model based on the extracted multimodal node features includes:

[0015] The multi-relationship social graph model is constructed as follows: Multiple sets of multimodal node features from multiple accounts are used as a node set V; the relationships between multiple accounts are used as an edge set E; and the relationship types are used as a relationship set R, thereby constructing the multi-relationship social graph model. ;

[0016] The learning process of the multi-relation social graph model includes: linearly projecting the multimodal node features onto a unified dimension and concatenating them to obtain a merged vector; normalizing the concatenated merged vector to obtain the input vector of the multi-relation social graph model; and inputting the input vector, edge set, and relation set into the multi-relation social graph model for relation convolution to obtain the final node representation features output by the multi-relation social graph model.

[0017] Furthermore, the step of inputting the output features of the multi-relationship social graph model into a gated hybrid expert model for training includes:

[0018] The output of the gated network is obtained by calculating the output features of the multi-relationship social graph model. as follows:

[0019] ;

[0020] In the formula, , For gating weights and biases; These are the output features of a multi-relationship social graph model;

[0021] Two groups of experts were set up. And calculate the corresponding marginal discriminant respectively. , :

[0022] ;

[0023] In the formula, This refers to the first expert associated with the opinion robot; This refers to the second expert corresponding to the opinion robot; This refers to the first expert who is not a public opinion robot. The second expert, not a public opinion robot;

[0024] Based on marginal discrimination , and the output of the gating network Perform probability domain fusion to obtain probabilities as follows:

[0025] ;

[0026] In the formula, Represents the Sigmoid function;

[0027] According to probability Calculate symmetric fractions as follows: ;in, .

[0028] Furthermore, the step of inputting the multi-relationship social graph model and its output features into the prototype-based contrastive learning model for training includes:

[0029] Randomly initialize prototype matrix ;

[0030] View augmentation is performed on each node i of the multi-relation social graph model, including: applying Gaussian perturbation to the input vector of the multi-relation social graph model, and then inputting the perturbed input vector, edge set, and relation set into the multi-relation social graph model for relation convolution to obtain the output features. Gaussian perturbation is applied to the edge set of the multi-relation social graph model. The perturbed edge set, input vector, and relation set are then input into the multi-relation social graph model for relation convolution to obtain the output features. ;

[0031] For output features and output features Perform low-dimensional projections separately to obtain the projection vectors. and :

[0032] ;

[0033] ;

[0034] In the formula, P represents the linear projection matrix. This is the projection bias vector;

[0035] Project the vectors respectively and Substituting into the following formula yields the predicted distribution of the first view augmentation. Near-equilibrium soft allocation with enhanced first view And the predicted distribution of the second view enhancement Near-equilibrium soft allocation with enhanced first view :

[0036] ; ;

[0037] ; ;

[0038] In the formula, express Activation function; The prototype matrix; express function; >0 indicates the softmax temperature; >0 is the Sinkhorn balancing parameter, used for near-equilibrium distribution.

[0039] Furthermore, the expression for the joint loss is:

[0040] ;

[0041] In the formula, This represents the supervised loss during the training process of a gated hybrid expert model. , Represents the cross-entropy function. Label; This represents cross-viewing during the training process of a prototype-based contrastive learning model. Figure 1 Sexual damage, , This represents the total number of features for each node; This represents the gated entropy regularization loss during the training process of a gated hybrid expert model. , (·) represents multipath entropy. This represents the output of the gating network corresponding to the i-th node, and N represents the total number of nodes, i.e., the total number of all training samples. It is the loss weighting coefficient.

[0042] Furthermore, the preset condition for stopping training is: the F1 value is at its maximum or the accuracy reaches a preset threshold.

[0043] Furthermore, during the training process of the gated hybrid expert model, in obtaining symmetric scores... Then, for symmetrical fractions Further calibration is required; the calibration formula is as follows:

[0044] ;

[0045] In the formula, Indicates the symmetrical fraction after calibration; express Activation function; , which is the temperature scaling parameter.

[0046] According to another aspect of the present invention, a public opinion robot identification system based on gated hybrid expert and prototype comparative learning is proposed. The system is implemented based on the aforementioned public opinion robot identification method based on gated hybrid expert and prototype comparative learning; the system includes:

[0047] The training data acquisition module is configured to acquire information on multiple accounts on social media platforms and their corresponding text content.

[0048] The feature extraction module is configured to perform multimodal node feature extraction on multiple account information and corresponding content text data;

[0049] The graph model learning module is configured to construct and learn a multi-relationship social graph model based on the extracted multimodal node features;

[0050] The recognition model training module is configured to input the output features of the multi-relationship social graph model into a gated hybrid expert model for training; input the multi-relationship social graph model and its output features into a prototype-based contrastive learning model for training; calculate the joint loss of the two models during training; stop training when the joint loss reaches a preset condition and save the optimal configuration parameters of each model.

[0051] The robot recognition module is configured to extract multimodal node features from the social account information to be identified and the corresponding content text data, and then input the data into a multi-relationship social graph model and a gated hybrid expert model with optimal configuration parameters to obtain the recognition result of the social account to be identified; the recognition result is that the social account is a public opinion robot or a non-public opinion robot.

[0052] The beneficial technical effects of this invention are:

[0053] This invention proposes a method and system for identifying public opinion robots based on gated hybrid expert and prototype comparative learning. By integrating four-way modality alignment, RGCN representation learning, gate circuit probability fusion, and threshold / calibration decision into an integrated process, it achieves efficient reasoning and stable returns on massive social data. Compared with traditional single discriminator or feature splicing schemes, this invention can achieve higher recognition results at the same cost and provide auditable decision evidence, making it suitable for engineering implementation in content deduplication, hotspot aggregation, intelligent summarization, and risk control inspection. Attached Figure Description

[0054] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example and not limitation, wherein:

[0055] Figure 1 This is a flowchart of the public opinion robot identification method based on gated hybrid expert and prototype comparative learning as described in the embodiments of the present invention;

[0056] Figure 2 This is a schematic diagram of the structure of the public opinion robot recognition system based on gated hybrid expert and prototype comparative learning as described in an embodiment of the present invention. Detailed Implementation

[0057] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0058] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. It should be understood herein that any number of elements in the accompanying drawings is for illustrative purposes only and not as a limitation, and any naming is for distinction only and has no limiting meaning.

[0059] This invention proposes a method for identifying public opinion robots based on gated hybrid expert and prototype comparative learning, such as... Figure 1 As shown, the method includes:

[0060] S1. Obtain information on multiple accounts on social media platforms and their corresponding text data.

[0061] S2. Perform multimodal node feature extraction on multiple account information and corresponding content text data;

[0062] S3. Construct and learn a multi-relationship social graph model based on the extracted multimodal node features;

[0063] S4. Input the output features of the multi-relationship social graph model into the gated hybrid expert model for training; input the multi-relationship social graph model and its output features into the prototype-based contrastive learning model for training; calculate the joint loss of the two models during training; stop training when the joint loss reaches the preset condition and save the optimal configuration parameters of each model.

[0064] S5. For the social account information to be identified and the corresponding content text data, after multimodal node feature extraction, the data is input into a multi-relationship social graph model and a gated hybrid expert model with optimal configuration parameters to obtain the identification result of the social account to be identified; the identification result is that the social account is a public opinion robot or a non-public opinion robot.

[0065] The method begins with S1. In S1, information on multiple accounts on social media platforms and their corresponding text data is obtained.

[0066] According to an embodiment of the present invention, taking the aggregation of content from a social media platform account as an example, 5000 pieces of original text (i.e., content text data) are retrieved for each account and the account is labeled to obtain the tag y for each account. y=1 indicates that the account is a bot; y=0 indicates that the account is not a bot. Each account's information includes numerical and modal information. Numerical information includes the number of followers, the number of accounts followed, the number of likes, the total number of posts, the number of active days, and the username length. Modal information includes whether the account is verified, whether geolocation is enabled, whether it is protected, whether it uses a background image, whether it has extended information, whether it uses a default profile picture, and whether it uses a default profile picture style. The obtained account information and corresponding text data are used as the training set. Other publicly available datasets can also be used as training and testing sets.

[0067] Then, in S2, multimodal node feature extraction is performed on multiple account information and corresponding content text data.

[0068] According to an embodiment of the present invention, firstly, weak text filtering is performed on each of the 5000 original texts for each account, including removing URLs, topic tags, and @ mentions, and compressing whitespace. For example, texts with a net length of less than 8 characters are considered invalid and removed; timestamps are retained. After the weak text filtering step, 3500 valid texts are retained. The total number of likes, comments, and reposts for each text is calculated, sorted in descending order, and the top 20 texts are selected as representative texts for that account.

[0069] Then, based on each account's information and its retained content text data, initial features are generated using three paths: text, numerical, and category / Boolean. Specifically, this includes: 1) using a pre-trained BERT model to obtain the text features for each account. and account description features The dimension of the initial CLS vector in the input sequence of the BERT model. , =768, which is used to aggregate global semantic information of the input sequence. Text features for each account node. 1) Extract the CLS vector of each text and perform mean pooling. 2) Construct numerical modality vectors from the numerical information of each account and normalize them to obtain numerical features. , 3) Construct a Boolean mode vector from the modal information of each account. As modal features, the text features, account description features, numerical features, and modal features of each account constitute the multimodal node features of that account.

[0070] Then, in S3, a multi-relationship social graph model is constructed and learned based on multimodal node features.

[0071] According to an embodiment of the present invention, the construction of a multi-relationship social graph model is as follows: Multiple sets of multimodal node features from multiple accounts are used as a node set V, the relationships between multiple accounts are used as an edge set E, and the relationship types are used as a relationship set R, thereby constructing a multi-relationship social graph model. Wherein, the set of relationships R = {followers, followers, mentions, reposts, quotes, replies}. In this embodiment, the graph neural network model adopts a relational graph convolutional network (RGCN).

[0072] The learning process for a multi-relationship social graph model is as follows: First, the features of multimodal nodes are linearly projected onto a unified 128-dimensional matrix and then concatenated to obtain a merged vector. :

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] in, This represents the account description features after linear projection; Represents text features after linear projection; Represents the numerical characteristics after linear projection; Represents the modal characteristics after linear projection; , , , Both represent linear weights. , , , ; , , , Both indicate bias.

[0078] Then, the concatenated merged vector Normalization is performed to obtain the input vector x of the graphical model:

[0079] ;

[0080] in, Indicates normalization; , Linear weights and biases are applied to each feature to a unified dimension.

[0081] Then, the input vector x, edge set E, and relation set R are input into the multi-relation social graph model for relation convolution; where the first... +1 hidden layer is represented as :

[0082] ;

[0083] in, Indicates the target node currently being updated. Indicate to Neighboring nodes that pass messages; For relation r ( The field under ); These are the normalization coefficients; For learnable weights, Represents a non-linear activation function; Indicates the first The corresponding neighbor node is the th Layered hidden representation; Indicates the first The first target node corresponding to the Hidden representations; multi-relationship social graph models output final node representation features. .

[0084] Then, in S4, the output features of the multi-relationship social graph model are input into a gated hybrid expert model for training; the multi-relationship social graph model and its output features are input into a prototype-based contrastive learning model for training; the joint loss of the two models is calculated during training; training is stopped when the joint loss reaches a preset condition, and the optimal configuration parameters of each model are saved.

[0085] According to an embodiment of the present invention, firstly, the final node representation feature is output by the multi-relationship social graph model. As input, a gating network generates sample-level weights and performs convex combination in the probability domain to obtain stable fusion probabilities and symmetric scores, which are used for subsequent supervised training and inference. The process is represented as follows:

[0086] 1) Gated network: ; , For gating weights and biases; For gating network output;

[0087] 2) Probability domain fusion: ; Represents the Sigmoid function; , Two groups of experts were set up , The resulting marginal discriminant represents the difference between the expert's positive class score (logits) and negative class score (logits). ; This refers to the first expert associated with the opinion robot; This refers to the second expert corresponding to the opinion robot; This refers to the first expert who is not a public opinion robot. The second expert, not a public opinion robot;

[0088] 3) Symmetric fractions (logits): .

[0089] Furthermore, in obtaining symmetrical fractions Then, symmetrical fractions can be considered. Further calibration is performed to make the output probability more consistent with the actual confidence level; the calibration formula is:

[0090] ;

[0091] In the formula, Indicates the symmetrical fraction after calibration; express Activation function; , which is the temperature scaling parameter.

[0092] The above process enables adaptive selection of different discrimination preferences and provides interpretable gating and marginal signals.

[0093] Then, a prototype-based contrastive learning model is trained.

[0094] Specifically, this includes: First, randomly initializing the prototype matrix. , d is the projection dimension. For example, the number of prototypes. =100, d=64;

[0095] Then, view augmentation is performed on each node i of the multi-relation social graph model, including: applying Gaussian perturbation to the input vector of the multi-relation social graph model, and then inputting the perturbed input vector, edge set, and relation set into the multi-relation social graph model for relation convolution to obtain the output features. Gaussian perturbation is applied to the edge set of the multi-relation social graph model. The perturbed edge set, input vector, and relation set are then input into the multi-relation social graph model for relation convolution to obtain the output features. As an example, in Gaussian interference, the neighbor sampling retention rate is set to 0.9, which means that 90% of the input vector or edge set is retained.

[0096] Then, for the output features and output features Perform low-dimensional projections separately to obtain the projection vectors. and :

[0097] ;

[0098] ;

[0099] In the formula, P represents the linear projection matrix. This is the projection bias vector;

[0100] Then, the projection vectors are respectively and Substituting into the following formula yields the predicted distribution of the first view augmentation. Near-equilibrium soft allocation with enhanced first view And the predicted distribution of the second view enhancement Near-equilibrium soft allocation with enhanced first view :

[0101] ; ;

[0102] ; ;

[0103] In the formula, express Activation function; The prototype matrix; express function; >0 indicates the softmax temperature; >0 represents the Sinkhorn balancing parameter, used for near-equilibrium distribution. As an example, the soft-distribution temperature... Sinkhorn balance parameters .

[0104] Calculate the joint loss of the two models during training. Specifically, the supervised branch and the self-supervised branch are jointly optimized, with a joint loss. Represented as:

[0105] ;

[0106] In the formula, This represents the supervised loss during the training process of a gated hybrid expert model. , Represents the cross-entropy function. It's a tag; This represents cross-viewing during the training process of a prototype-based contrastive learning model. Figure 1 Sexual damage, , This represents the total number of features for each node. Indicates the inner product; This represents the gated entropy regularization loss during the training process of a gated hybrid expert model. , Represents multipath entropy. This represents the output of the gating network corresponding to the i-th node, and N represents the total number of nodes, i.e., the total number of all training samples. These are loss weighting coefficients; as an example, , .

[0107] Among them, cross-vision Figure 1 Inductive loss is only combined with the supervised branch during the training period to improve robustness, which is achieved through cross-view... Figure 1 Self-supervised optimization with consistency as the goal enables the representation space to form a global prior of "semantic cluster-prototype".

[0108] When joint loss Training stops when preset conditions are met, and the optimal configuration parameters for each model are saved. The preset conditions are either the maximum F1 score or the accuracy reaching a preset threshold. Optimal configuration parameters include the decision threshold. :

[0109] ;

[0110] in Indicates precision, For the preset threshold, As an example, =0.58.

[0111] Then, in S5, for the social account information to be identified and the corresponding content text data, after multimodal node feature extraction, it is input into a multi-relational social graph model and a gated hybrid expert model with optimal configuration parameters to obtain the identification result of the social account to be identified; the identification result is that the social account is a public opinion robot or a non-public opinion robot.

[0112] According to an embodiment of the present invention, during the inference phase, the trained graphical model RGCN and a gated hybrid expert model with optimal configuration parameters are loaded; the node to be identified is extracted through multimodal node features and then input into the trained graphical model RGCN to obtain the representation feature z, which is then processed by the gated hybrid expert model with optimal configuration parameters to obtain the fused symmetric score. The fused symmetric score logit is temperature-scaled to obtain the calibration score. and with calibration score ≥ Decision Threshold Determine the category (i.e., whether the social media account is a bot or not). It can also return interpretable information, including the gating weight g and the marginal judgments of the two experts. , wait.

[0113] The system performs batch reasoning and ensures low latency, forming a reasoning and evidence feedback process that can be directly deployed online.

[0114] The method in this embodiment of the invention jointly trains the node representation of the RGCN backbone, the gated MoE binary classification supervision branch, and the prototype comparison self-supervised branch. The trained model can still maintain robustness and generalization ability in class imbalance, cross-domain, and noisy environments.

[0115] This invention also proposes a public opinion robot identification system based on gated hybrid expert and prototype comparative learning, which is implemented based on the public opinion robot identification method based on gated hybrid expert and prototype comparative learning described in the above embodiments; as follows Figure 2 As shown, the system includes:

[0116] The training data acquisition module 210 is configured to acquire information on multiple accounts on social media platforms and their corresponding content text data.

[0117] The feature extraction module 220 is configured to perform multimodal node feature extraction on multiple account information and corresponding content text data;

[0118] Graph model learning module 230 is configured to construct and learn a multi-relationship social graph model based on extracted multimodal node features;

[0119] The recognition model training module 240 is configured to input the output features of the multi-relationship social graph model into a gated hybrid expert model for training; input the multi-relationship social graph model and its output features into a prototype-based contrastive learning model for training; calculate the joint loss of the two models during training; stop training when the joint loss reaches a preset condition and save the optimal configuration parameters of each model.

[0120] The robot recognition module 250 is configured to extract multimodal node features from the social account information to be identified and the corresponding content text data, and then input the data into a multi-relationship social graph model and a gated hybrid expert model with optimal configuration parameters to obtain the recognition result of the social account to be identified; the recognition result is that the social account is a public opinion robot or a non-public opinion robot.

[0121] The functionality of the public opinion robot identification system based on gated hybrid expert and prototype comparative learning described in this embodiment of the invention can be explained by the aforementioned public opinion robot identification method based on gated hybrid expert and prototype comparative learning. Therefore, for the parts not detailed in the system embodiment, please refer to the above method embodiment, and they will not be repeated here.

[0122] It should be noted that although several units, modules, or sub-modules are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0123] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0124] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for identifying public opinion robots based on gated hybrid expert and prototype comparative learning, characterized in that, include: Obtain information on multiple accounts on social media platforms and their corresponding text data. Multimodal node feature extraction is performed on multiple account information and corresponding content text data; A multi-relationship social graph model is constructed and learned based on the extracted multimodal node features; The output features of the multi-relationship social graph model are input into a gated hybrid expert model for training; the multi-relationship social graph model and its output features are input into a prototype-based contrastive learning model for training; the joint loss of the two models is calculated during training; training is stopped when the joint loss reaches a preset condition, and the optimal configuration parameters of each model are saved; For the social media account information to be identified and the corresponding content text data, after multimodal node feature extraction, the data is input into a multi-relationship social graph model and a gated hybrid expert model with optimal configuration parameters to obtain the identification result of the social media account to be identified; the identification result is whether the social media account is a public opinion robot or not.

2. The method for identifying public opinion robots based on gated hybrid expert and prototype comparative learning according to claim 1, characterized in that, The account information includes numerical information and modal information. The numerical information includes the number of user followers, the number of user following, the number of user likes, the total number of user posts, the number of active days of the user, and the length of the username. The modal information includes whether the account is verified, whether geolocation is enabled, whether it is protected, whether a background image is used, whether there is extended information, whether a default avatar is used, and whether a default profile picture is used.

3. The method for identifying public opinion robots based on gated hybrid expert and prototype comparative learning according to claim 1, characterized in that, The multimodal node feature extraction of multiple account information and corresponding content text data includes: The pre-trained BERT model is used to extract features from the content text data to obtain the text features and account description features of each account; the numerical information of each account is constructed into a numerical modality vector and normalized to obtain the numerical features; the modality information of each account is constructed into a Boolean modality vector as the modality features; the text features, account description features, numerical features, and modality features of each account constitute the multimodal node features of that account.

4. The public opinion robot identification method based on gated hybrid expert and prototype comparative learning according to claim 3, characterized in that, The construction and learning of a multi-relationship social graph model based on extracted multimodal node features includes: The multi-relationship social graph model is constructed as follows: Multiple sets of multimodal node features from multiple accounts are used as a node set V; the relationships between multiple accounts are used as an edge set E; and the relationship types are used as a relationship set R, thereby constructing the multi-relationship social graph model. ; The learning process of the multi-relation social graph model includes: linearly projecting the multimodal node features onto a unified dimension and concatenating them to obtain a merged vector; normalizing the concatenated merged vector to obtain the input vector of the multi-relation social graph model; and inputting the input vector, edge set, and relation set into the multi-relation social graph model for relation convolution to obtain the final node representation features output by the multi-relation social graph model.

5. The public opinion robot identification method based on gated hybrid expert and prototype comparative learning according to claim 4, characterized in that, The step of inputting the output features of the multi-relationship social graph model into a gated hybrid expert model for training includes: The output of the gated network is obtained by calculating the output features of the multi-relationship social graph model. as follows: ； In the formula, , For gating weights and biases; These are the output features of a multi-relationship social graph model; Two groups of experts were set up. And calculate the corresponding marginal discriminant respectively. , : ； In the formula, This refers to the first expert associated with the opinion robot; This refers to the second expert corresponding to the opinion robot; This refers to the first expert who is not a public opinion robot. The second expert, not a public opinion robot; Based on marginal discrimination , and the output of the gating network Perform probability domain fusion to obtain probabilities as follows: ； In the formula, Represents the Sigmoid function; According to probability Calculate symmetric fractions as follows: ;in, .

6. The public opinion robot identification method based on gated hybrid expert and prototype comparative learning according to claim 5, characterized in that, The step of inputting the multi-relationship social graph model and its output features into the prototype-based contrastive learning model for training includes: Randomly initialize prototype matrix ; View augmentation is performed on each node i of the multi-relation social graph model, including: applying Gaussian perturbation to the input vector of the multi-relation social graph model, and then inputting the perturbed input vector, edge set, and relation set into the multi-relation social graph model for relation convolution to obtain the output features. Gaussian perturbation is applied to the edge set of the multi-relation social graph model. The perturbed edge set, input vector, and relation set are then input into the multi-relation social graph model for relation convolution to obtain the output features. ; For output features and output features Perform low-dimensional projections separately to obtain the projection vectors. and : ； ； In the formula, P represents the linear projection matrix. This is the projection bias vector; Project the vectors respectively and Substituting into the following formula yields the predicted distribution of the first view augmentation. Near-equilibrium soft allocation with enhanced first view And the predicted distribution of the second view enhancement Near-equilibrium soft allocation with enhanced first view : ； ； ； ； In the formula, express Activation function; The prototype matrix; express function; >0 indicates the softmax temperature; >0 is the Sinkhorn balancing parameter, used for near-equilibrium distribution.

7. The public opinion robot identification method based on gated hybrid expert and prototype comparative learning according to claim 6, characterized in that, The expression for the joint loss is: ； In the formula, This represents the supervised loss during the training process of a gated hybrid expert model. , Represents the cross-entropy function. Label; This represents the cross-view consistency loss during the training process of a prototype-based contrastive learning model. , This represents the total number of features for each node; This represents the gated entropy regularization loss during the training process of a gated hybrid expert model. , (·) represents multipath entropy. This represents the output of the gating network corresponding to the i-th node, and N represents the total number of nodes, i.e., the total number of all training samples. It is the loss weighting coefficient.

8. The method for identifying public opinion robots based on gated hybrid expert and prototype comparative learning according to claim 7, characterized in that, The preset conditions for stopping training are: the F1 value is at its maximum or the accuracy reaches a preset threshold.

9. The method for identifying public opinion robots based on gated hybrid expert and prototype comparative learning according to claim 5, characterized in that, During the training of the gated hybrid expert model, symmetric scores are obtained. Then, for symmetrical fractions Further calibration is required; the calibration formula is as follows: ； In the formula, Indicates the symmetrical fraction after calibration; express Activation function; , which is the temperature scaling parameter.

10. A public opinion robot recognition system based on gated hybrid expert and prototype comparative learning, characterized in that, The system is implemented based on the public opinion robot identification method based on gated hybrid expert and prototype comparative learning as described in any one of claims 1-9; the system includes: The training data acquisition module is configured to acquire information on multiple accounts on social media platforms and their corresponding text content. The feature extraction module is configured to perform multimodal node feature extraction on multiple account information and corresponding content text data; The graph model learning module is configured to construct and learn a multi-relationship social graph model based on the extracted multimodal node features; The recognition model training module is configured to input the output features of the multi-relationship social graph model into a gated hybrid expert model for training; input the multi-relationship social graph model and its output features into a prototype-based contrastive learning model for training; calculate the joint loss of the two models during training; stop training when the joint loss reaches a preset condition and save the optimal configuration parameters of each model. The robot recognition module is configured to extract multimodal node features from the social account information to be identified and the corresponding content text data, and then input the data into a multi-relationship social graph model and a gated hybrid expert model with optimal configuration parameters to obtain the recognition result of the social account to be identified; the recognition result is that the social account is a public opinion robot or a non-public opinion robot.

Images