Intelligent network false information identification method based on multi-modal behavior map

Abstract

The invention relates to the technical field of network information processing, in particular to an intelligent network false information identification method based on a multi-modal behavior map. Comprising the following steps: acquiring to-be-identified target network information and multi-modal content thereof, and collecting propagation behavior data of the information in a propagation process; based on the propagation behavior data, constructing an initial propagation graph which takes propagation users as nodes and propagation relationships as edges; based on the initial propagation graph and the multi-modal content, constructing a double-layer coupling behavior graph comprising a propagation structure layer and a cognitive emotion layer; performing cross-modal credibility verification based on the double-layer coupling behavior map and the multi-modal content, and generating a credibility verification result; and judging the authenticity of the target network information according to the verification result, thereby realizing automatic and multi-dimensional comprehensive research and judgment on the authenticity of the network information.

Description

Technical Field

[0001] This application relates to the field of network information processing technology, and in particular to an intelligent network fake information identification method based on multimodal behavior graphs. Background Technology

[0002] Online misinformation poses a serious threat to the building of trust and the stable operation of digital society, and its effective identification and governance are critical issues that urgently need to be addressed. Traditional misinformation identification methods mostly rely on the analysis of the information content itself, such as text-based fact-checking, image authentication, or single-modal semantic analysis. These methods belong to a post-event content verification model.

[0003] However, with the evolution of methods for creating and disseminating misinformation, especially the widespread use of multimodal content and the emergence of complex techniques such as manipulating dissemination paths and public sentiment through social bots and online troll accounts, methods relying solely on content analysis face significant challenges. For information that spreads rapidly and widely on social networks, the group behavior patterns, emotional polarization, and network topology anomalies that emerge during its dissemination are themselves key signals for determining its credibility. Ignoring in-depth analysis of the dissemination dynamics will make it difficult to effectively identify misinformation that is inherently concealed but spreads through abnormal behavioral patterns, leading to delayed identification and insufficient accuracy. Summary of the Invention

[0004] The purpose of this application is to provide an intelligent network misinformation identification method based on multimodal behavior graphs to solve the above-mentioned technical problems. The method aims to improve the accuracy and robustness of identifying complex misinformation by integrating multi-dimensional in-depth analysis of information content and its dissemination behavior.

[0005] In some embodiments of this application, a method for identifying fake information in intelligent networks based on multimodal behavior graphs is provided, including:

[0006] Acquire the target network information to be identified and its multimodal content, and collect the propagation behavior data of the target network information during the propagation process; Based on the aforementioned propagation behavior data, an initial propagation graph is constructed with propagating users as nodes and propagation relationships as edges; Based on the initial propagation graph and the multimodal content, a two-layer coupled behavioral graph containing a propagation structure layer and a cognitive-emotional layer is constructed. Based on the dual-layer coupled behavior map and the multimodal content, cross-modal credibility verification is performed to generate credibility verification results. Based on the credibility verification results, the authenticity of the target network information is determined.

[0007] In some embodiments of this application, when acquiring target network information to be identified and its multimodal content, and collecting propagation behavior data of the target network information during the propagation process, the following steps are included: Receive target network information containing core textual claims through the data interface; Extract the multimodal original content of the target network information, and perform standardized encoding and feature alignment on the multimodal original content to form multimodal content data; Using the target network information as the root node, the propagation behavior data is tracked and collected on social platforms. During the collection process, based on the multi-dimensional behavioral characteristics of the propagation nodes, a pre-trained account filtering model is used to perform initial credibility screening and filtering of the collected propagation nodes. The propagation behavior data includes propagation node data, propagation topology data, and node interaction data.

[0008] In some embodiments of this application, when constructing an initial propagation graph with propagating users as nodes and propagation relationships as edges based on the propagation behavior data, the following steps are included: The target network information entity is taken as the root node, and each collected propagation user account is defined as a propagation node; Based on the propagation topology data, directed edges are established between the root node and each propagation node, as well as between each propagation node, to generate propagation paths; Each propagation node is assigned a corresponding propagation node data attribute, and each directed edge is assigned a corresponding additional attribute, including a timestamp and an interaction type label. Based on all nodes, directed edges, and their additional attributes, generate structured initial propagation graph data.

[0009] In some embodiments of this application, when constructing a two-layer coupled behavioral graph containing a propagation structure layer and a cognitive-emotion layer based on the initial propagation graph and the multimodal content, the following steps are included: Based on the initial propagation graph data, the macroscopic topology and node attributes are extracted to construct a propagation structure layer that explicitly represents the information diffusion path. Based on the multimodal content and node interaction data, implicit cognitive state vectors and sentiment tendency vectors are generated for each propagation node through semantic and sentiment analysis models to construct a cognitive sentiment layer. By using a cross-layer attention mechanism, the propagation structure layer and the cognitive emotion layer are dynamically associated and information is fused to generate the dual-layer coupled behavior map.

[0010] In some embodiments of this application, when extracting the macroscopic topology and node attributes based on the initial propagation graph data to construct a propagation structure layer that explicitly represents the information diffusion path, the following steps are included: Graph structure features are extracted from the initial propagation graph data, and macroscopic topological indices, including node centrality, propagation path length, and subgraph clustering coefficient, are calculated. The user attribute features in the propagation node data are fused to form an enhanced node attribute vector; The macroscopic topology indicators and enhanced node attribute vectors are structured and encoded to construct the propagation structure layer.

[0011] In some embodiments of this application, based on the multimodal content and node interaction data, implicit cognitive state vectors and sentiment tendency vectors are generated for each propagation node using a semantic and sentiment analysis model. When constructing the cognitive-sentiment layer, the following steps are included: Semantic understanding and intent recognition are performed on the comment text in the node interaction data to classify and quantify the user's cognitive state of certainty, doubt, and neutrality towards the target network information, and generate a cognitive state vector. Fine-grained sentiment analysis is performed on comment text and associated emojis to identify and quantify the positive, negative, or complex emotions expressed by users during the dissemination process, and to generate sentiment tendency vectors. Based on the cognitive state vector and the emotional tendency vector, the historical behavior credibility features of the node are fused to form a comprehensive state representation of the node in the cognitive-emotional layer.

[0012] In some embodiments of this application, when generating the dual-layer coupled behavioral graph by dynamically associating and fusing information between the propagation structure layer and the cognitive-emotion layer through a cross-layer attention mechanism, the following steps are included: The comprehensive state representation of each node in the cognitive emotion layer is used as the query vector; The topological environment information of the corresponding nodes in the propagation structure layer is mapped into key vectors and value vectors; Calculate the similarity weights between the query vector and all relevant key vectors, and then perform a weighted summation of the value vectors based on the similarity weights; A weighted summation is performed on all nodes, and an updated set of node representations that incorporates information from both layers is output, forming the dual-layer coupled behavior graph.

[0013] In some embodiments of this application, when performing cross-modal credibility verification based on the dual-layer coupled behavioral graph and the multimodal content, and generating credibility verification results, the process includes: The topological features of the propagation structure layer and the node state features of the cognitive emotion layer are extracted and spliced ​​together to form a fused node graph joint representation. Calculate the semantic distance between multimodal content features and the joint representation of the node graph; Based on the results of the cross-modal alignment and consistency calculation, a verification result score representing the credibility of the target network information is output through a classification model.

[0014] In some embodiments of this application, based on the results of the cross-modal alignment and consistency calculation, a verification result score characterizing the credibility of the target network information is output through a classification model, including: The consistency metric vector generated by the cross-modal alignment and consistency calculation is used as the core input feature of the classification model; The key propagation path features and abnormal interaction pattern features in the joint representation of the node graph are used as auxiliary discriminant features and input into the classification model. The classification model performs joint inference based on all input features and outputs the verification result score.

[0015] In some embodiments of this application, determining the authenticity of the target network information based on the credibility verification result includes: The verification result score is compared with a preset authenticity judgment threshold to generate a comparison result; Based on the comparison results, the target network information is classified into a predefined authenticity category; The output includes the target network information and the corresponding authenticity category.

[0016] Compared with existing technologies, the intelligent network fake information identification method based on multimodal behavior graphs provided in this application has the following advantages: This method constructs a dual-layer coupled behavioral graph integrating a propagation structure layer and a cognitive-emotional layer, enabling a multi-dimensional comprehensive analysis of information content and its propagation behavior patterns. This method not only analyzes the multimodal semantics of the information itself but also deeply mines its topological structure of diffusion within social networks and the collective psychological state of the propagating subjects. Furthermore, it innovatively employs a cross-modal consistency verification mechanism to compare the semantic distance between content features and behavioral patterns, thereby effectively identifying hidden false information where content and propagation behavior contradict each other, significantly improving the accuracy and depth of the discrimination.

[0017] Meanwhile, this method introduces a real-time filtering and purification mechanism based on a pre-trained model during the data acquisition and processing stages, enhancing the overall robustness of the system. By performing initial credibility screening and filtering on propagation nodes during the construction of the propagation graph, noisy data such as bot accounts and paid commenters can be effectively eliminated, ensuring that subsequent analysis is based on more authentic user behavior. This design, which improves quality from the data source, combined with the dynamic coupling analysis of the two-layer graph, jointly guarantees the reliability and stability of the identification results when facing complex and organized disinformation propagation. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a preferred embodiment of an intelligent network misinformation identification method based on multimodal behavior graphs. Detailed Implementation

[0019] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0020] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0022] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0023] like Figure 1 As shown in the preferred embodiments of this application, some embodiments of this application provide a method for identifying false information in intelligent networks based on multimodal behavior graphs, including: Acquire the target network information to be identified and its multimodal content, and collect the propagation behavior data of the target network information during the propagation process; Based on the aforementioned propagation behavior data, an initial propagation graph is constructed with propagating users as nodes and propagation relationships as edges; Based on the initial propagation graph and the multimodal content, a two-layer coupled behavioral graph containing a propagation structure layer and a cognitive-emotional layer is constructed. Based on the dual-layer coupled behavior map and the multimodal content, cross-modal credibility verification is performed to generate credibility verification results. Based on the credibility verification results, the authenticity of the target network information is determined.

[0024] In some embodiments of this application, when acquiring target network information to be identified and its multimodal content, and collecting propagation behavior data of the target network information during the propagation process, the following steps are included: Receive target network information containing core textual claims through the data interface; Extract the multimodal original content of the target network information, and perform standardized encoding and feature alignment on the multimodal original content to form multimodal content data; Using the target network information as the root node, the propagation behavior data is tracked and collected on social platforms. During the collection process, based on the multi-dimensional behavioral characteristics of the propagation nodes, a pre-trained account filtering model is used to perform initial credibility screening and filtering of the collected propagation nodes. The propagation behavior data includes propagation node data, propagation topology data, and node interaction data.

[0025] In this embodiment, the core text claim refers to the core viewpoint, assertion, or factual statement that the target online information to be verified attempts to convey. It is used for subsequent multimodal content analysis and dissemination behavior tracking.

[0026] In this embodiment, target network information refers to a specific information unit that is disseminated in public or semi-public cyberspaces such as social networks and news platforms, and whose authenticity needs to be determined. It includes a core textual claim, as well as related multimodal content such as images, videos, audio, and links, and all metadata and user interaction records generated during its dissemination.

[0027] In this embodiment, multimodal raw content refers to unprocessed raw media data directly associated with the target network information. This includes, but is not limited to: the main text of the information, accompanying images, embedded video or audio clips, attached emoticons or voting components, etc.

[0028] In this embodiment, normalized encoding and feature alignment refer to the key technical steps in preprocessing and representing multimodal raw content. Normalized encoding refers to converting raw data from different modalities, such as text, image pixels, and audio waveforms, into numerical vectors of a unified dimension through corresponding pre-trained models. Feature alignment refers to using cross-modal learning techniques in the vector space to make feature vectors from different modalities but expressing the same semantics more similar in spatial distribution.

[0029] In this embodiment, multimodal content data is a structured data set that can be used for model computation, formed by standardizing and aligning the original multimodal content.

[0030] In this embodiment, propagation behavior data refers to the sum of all behavioral records generated by users during the spread of target network information on a social network. It is a structured dataset specifically including: Propagation node data: user accounts participating in the propagation and their basic attributes, such as nickname, number of followers, registration time, etc. Propagation topology data: records the propagation relationships between nodes, such as forwarding chains, referencing relationships, comment reply relationships, and the timestamp of each relationship, used to describe the path and sequence of information propagation. Node interaction data: additional interactive content generated by users at propagation nodes, such as comment text, emojis used, likes, dislikes, etc., reflecting users' subjective reactions to the information.

[0031] In this embodiment, the multi-dimensional behavioral characteristics of the propagation node refer to a series of behavioral indicators used to assess the credibility or anomaly of the propagation user account. For example: the account's historical posting frequency, content topic distribution, social relationship network characteristics, and behavioral patterns in this propagation, such as whether it forwarded multiple times in a very short period of time.

[0032] In this embodiment, the pre-trained account filtering model refers to a multi-class machine learning model trained using a large amount of labeled data. Its function is to receive multi-dimensional behavioral features of an account as input and output a probability score indicating whether the account belongs to a "suspicious account" or a "normal account," which is used for preliminary screening during the data collection phase.

[0033] In this embodiment, the initial credibility screening and filtering refers to the process of evaluating each captured propagation node in real time using a pre-trained account filtering model while collecting propagation behavior data. Accounts and their related behavioral data that the model determines to be highly suspicious are removed or downweighted from the subsequent analysis pipeline, aiming to improve the data quality of the constructed propagation map, reduce noise interference, and enhance the robustness of subsequent authenticity determination.

[0034] In this embodiment, the system receives target network information to be verified through a data interface, with its core textual claims serving as the anchor point for authenticity verification. All multimodal raw content directly related to this information is extracted, including text, images, and videos. Key technical processing is applied to the extracted raw content. Heterogeneous modal data are transformed into unified numerical vectors through standardized encoding, and these vectors are aligned in the semantic space using feature alignment technology, ultimately forming multimodal content data representing the complete semantics of the information. Simultaneously with content processing, the system uses the target network information as a virtual propagation source, i.e., the root node, and tracks its spread trajectory on social platforms in real-time or retrospectively. This process systematically collects propagation behavior data, which includes: propagation node data, propagation topology data, and node interaction data. To improve the accuracy of subsequent analysis, credibility assessment is performed on each propagation node during the collection of behavior data. The system extracts multi-dimensional behavioral features of the nodes and inputs them into a pre-trained account filtering model for real-time analysis. This model allows for initial credibility screening and filtering of nodes, effectively eliminating or downgrading suspicious noise data such as bot accounts and paid troll accounts, thereby ensuring that the collected dissemination behavior data better reflects the dissemination dynamics of real users.

[0035] This step utilizes two parallel data pipelines to transform raw network information into analytical raw materials suitable for deep computing. One pipeline performs multimodal semantic understanding and alignment of the information ontology, while the other pipeline performs structured modeling and purification of the information dissemination ecosystem.

[0036] In some embodiments of this application, when constructing an initial propagation graph with propagating users as nodes and propagation relationships as edges based on the propagation behavior data, the following steps are included: The target network information entity is taken as the root node, and each collected propagation user account is defined as a propagation node; Based on the propagation topology data, directed edges are established between the root node and each propagation node, as well as between each propagation node, to generate propagation paths; Each propagation node is assigned a corresponding propagation node data attribute, and each directed edge is assigned a corresponding additional attribute, including a timestamp and an interaction type label. Based on all nodes, directed edges, and their additional attributes, generate structured initial propagation graph data.

[0037] In this embodiment, using the target network information entity as the root node means that during the graph data modeling process, the target network information to be verified is instantiated as a special virtual node representing the information source. This node is set as the sole source and logical starting point of the entire propagation network, i.e., the root node, and all direct propagation behaviors will be linked to this node. This modeling method clearly anchors the origin of information propagation, ensuring the integrity and traceability of the propagation topology.

[0038] In this embodiment, each collected user account participating in information dissemination is defined as a dissemination node, meaning that each collected user account participating in information dissemination is mapped to an independent node in a graph data structure. Each dissemination node is an entity in the graph, representing a participant in information dissemination.

[0039] In this embodiment, a propagation path refers to a chain representing the direction of information flow, formed by establishing directed edges between nodes with direct propagation relationships based on propagation topology data. A propagation path consists of a series of interconnected nodes and directed edges, visually depicting the specific trajectory and direction of information spreading step by step from the root node through one or more user nodes.

[0040] In this embodiment, attaching corresponding propagation node data attributes refers to assigning associated static or dynamic characteristic information to each propagation node. These attributes are directly derived from the propagation node data, such as the number of a user's followers, number of followers, account creation time, historical credibility rating, etc.

[0041] In this embodiment, timestamp and interaction type label refer to two core attributes attached to each directed edge connecting nodes. Timestamp: Precisely records the moment the propagation behavior occurred. It gives the propagation graph a temporal dimension, which can be used to analyze temporal dynamic characteristics such as propagation speed and burst patterns. Interaction type label: Identifies the specific type of propagation behavior represented by the directed edge, such as forwarding, comment reply, or quoting. This label is used to distinguish propagation relationships of different natures.

[0042] In this embodiment, the process of integrating and instantiating the results of the above definitions and additional operations based on all nodes, directed edges, and their additional attributes refers to the process of integrating and instantiating the results of the above definitions and additional operations. Specifically, this involves organizing and storing the root node, all propagating nodes, the directed edges connecting them, and all data attributes, timestamps, and interaction type tags attached to the nodes and edges in a unified manner according to the specifications of the graph data model, forming a complete and self-consistent data structure.

[0043] In this embodiment, the structured initial propagation graph data refers to the data object finally generated through the above steps, which can be directly processed by subsequent graph computation and machine learning models. It typically exists in a standardized graph data format. This data not only contains the topological structure of information propagation, but also deeply integrates the spatiotemporal and type information of node attributes and edges, providing a high-quality, semantically rich input foundation for the next stage of constructing a more complex two-layer coupled behavioral graph.

[0044] In this embodiment, the target network information entity to be verified is defined as the root node of the entire propagation network, serving as the single logical starting point for information diffusion. All user accounts participating in the propagation are mapped one by one to propagation nodes, representing participating entities in the network. Based on the user relationships recorded in the propagation topology data, directional connections are established between the root node and propagation nodes, and among the propagation nodes themselves, thereby generating propagation paths representing the direction of information flow. Each propagation node is appended with its corresponding propagation node data attributes, integrating user profile information into the graph model. Key additional attributes are appended to each directed edge, where a timestamp records the precise moment the propagation behavior occurs, enabling the network to perform time-series analysis; and an interaction type label clarifies the specific nature of the propagation relationship. Finally, based on all nodes, directed edges, and their additional attributes, integration and instantiation are performed to generate a complete, self-consistent, structured initial propagation graph data object. This data object exists in a standard graph data structure form, fully encapsulating the topological structure, time series, and attribute characteristics of the participating entities in the information propagation.

[0045] In some embodiments of this application, when constructing a two-layer coupled behavioral graph containing a propagation structure layer and a cognitive-emotion layer based on the initial propagation graph and the multimodal content, the following steps are included: Based on the initial propagation graph data, the macroscopic topology and node attributes are extracted to construct a propagation structure layer that explicitly represents the information diffusion path. Based on the multimodal content and node interaction data, implicit cognitive state vectors and sentiment tendency vectors are generated for each propagation node through semantic and sentiment analysis models to construct a cognitive sentiment layer. By using a cross-layer attention mechanism, the propagation structure layer and the cognitive emotion layer are dynamically associated and information is fused to generate the dual-layer coupled behavior map.

[0046] In some embodiments of this application, when extracting the macroscopic topology and node attributes based on the initial propagation graph data to construct a propagation structure layer that explicitly represents the information diffusion path, the following steps are included: Graph structure features are extracted from the initial propagation graph data, and macroscopic topological indices, including node centrality, propagation path length, and subgraph clustering coefficient, are calculated. The user attribute features in the propagation node data are fused to form an enhanced node attribute vector; The macroscopic topology indicators and enhanced node attribute vectors are structured and encoded to construct the propagation structure layer.

[0047] In this embodiment, graph structure feature extraction refers to the process of calculating and analyzing the topological connections of the initial propagation graph data using graph theory algorithms and complex network analysis methods. Its purpose is to quantitatively extract key indicators from the original graph structure that can characterize the overall network morphology, node importance, and community structure.

[0048] In this embodiment, node centrality refers to a type of index calculated through graph structure feature extraction, used to quantify the influence of a single propagation node in the entire propagation network. It includes degree centrality (number of direct connections), betweenness centrality (frequency of acting as a bridge on the shortest path), and proximity centrality (ease of reaching other nodes).

[0049] In this embodiment, the subgraph clustering coefficient is a metric used to measure the degree of clustering of nodes in a network. It calculates the ratio of the actual number of connections between a node's neighbors to the maximum possible number of connections. A high clustering coefficient indicates the existence of tightly connected sub-communities within the network.

[0050] In this embodiment, user attribute features refer to static or dynamic attributes related to the user account itself, derived from the data of the propagation nodes. Examples include the number of followers, the number of accounts followed, the account's age, the distribution of historical post topics, and the authentication status. These features are independent of the graph structure but reflect the basic profile of the propagation entity.

[0051] In this embodiment, structured coding refers to the process of normalizing and vectorizing heterogeneous macroscopic topological indicators and user attribute features. Specifically, it involves integrating features of different scales and dimensions into a unified, fixed-dimensional numerical vector through operations such as normalization, concatenation, or projection.

[0052] In this embodiment, the representation of the propagation structure layer refers to the output vector ultimately generated through structured encoding, which represents the propagation structure layer. It is a macroscopic, explicit, and structured mathematical description of the information diffusion process. This representation not only encodes how information flows in the network but also who is driving the information flow, providing a solid external behavioral structural foundation for subsequent coupling with the cognitive-emotional layer.

[0053] In this embodiment, the system extracts graph structure features from the initial propagation graph data. By applying graph theory and complex network analysis algorithms, a series of macro-topological indicators are calculated, including but not limited to: node centrality: quantifying the influence and hub status of each propagation node in the network; propagation path length: measuring the distance information travels from the source to the end node, reflecting propagation depth and efficiency; and subgraph clustering coefficient: assessing the density of local communities in the network. Simultaneously, the system integrates user attribute features from the propagation node data to generate a profile vector for each node reflecting its social capital and background. Finally, through structured coding techniques, the extracted macro-topological indicators and the enhanced node attribute vectors are normalized, vectorized, and integrated.

[0054] In some embodiments of this application, based on the multimodal content and node interaction data, implicit cognitive state vectors and sentiment tendency vectors are generated for each propagation node using a semantic and sentiment analysis model. When constructing the cognitive-sentiment layer, the following steps are included: Semantic understanding and intent recognition are performed on the comment text in the node interaction data to classify and quantify the user's cognitive state of certainty, doubt, and neutrality towards the target network information, and generate a cognitive state vector. Fine-grained sentiment analysis is performed on comment text and associated emojis to identify and quantify the positive, negative, or complex emotions expressed by users during the dissemination process, and to generate sentiment tendency vectors. Based on the cognitive state vector and the emotional tendency vector, the historical behavior credibility features of the node are fused to form a comprehensive state representation of the node in the cognitive-emotional layer.

[0055] In this embodiment, semantic understanding and intent recognition refer to the process of performing deep natural language processing on comment text in node interaction data. Its core objective is to go beyond surface emotional coloring, analyze the deep semantics of the text, and determine the fundamental intent behind the user's comment, particularly their stance on the authenticity of the target online information. Specifically, the system uses a pre-trained language model to perform contextual encoding on the comment text. The intent classification module identifies whether the user expresses a cognitive attitude related to fact-checking, such as "convinced," "skeptical," or "neutral," and quantifies this attitude into a numerical vector, namely, a cognitive state vector. This process aims to capture the user's rational judgment dimension regarding the information content.

[0056] In this embodiment, fine-grained sentiment analysis refers to a deep and detailed sentiment analysis process of comment text and associated emoticons in node interaction data. Unlike simple sentiment binary classification, it aims to identify and quantify the specific emotional types, polarities, and intensities exhibited by users. For text, a sentiment dictionary or deep learning model is typically used to analyze vocabulary and syntax; for emoticons, they are mapped to predefined sentiment categories. The analysis results are quantified into a sentiment tendency vector to capture the emotional responses and fluctuations of users during communication, reflecting their emotional response to the information content.

[0057] In this embodiment, the comprehensive state representation of a node at the cognitive-emotional layer refers to the final vector generated for each dissemination node through information fusion technology, uniformly representing its internal psychological state. This representation is constructed by fusing the following three parts of information: Cognitive state vector: representing the user's rational judgment of the authenticity of information (belief / doubt / neutrality); Emotional tendency vector: representing the user's emotional reaction (e.g., positive, negative) during dissemination; Node's historical behavior credibility features: a credibility score calculated based on the user's past behavior (e.g., the authenticity of past statements, the types of information disseminated). Through feature concatenation, weighted summation, or a neural network-based fusion processor, the above multi-source information is integrated into a fixed-dimensional, high-order comprehensive state representation.

[0058] In some embodiments of this application, when generating the dual-layer coupled behavioral graph by dynamically associating and fusing information between the propagation structure layer and the cognitive-emotion layer through a cross-layer attention mechanism, the following steps are included: The comprehensive state representation of each node in the cognitive emotion layer is used as the query vector; The topological environment information of the corresponding nodes in the propagation structure layer is mapped into key vectors and value vectors; Calculate the similarity weights between the query vector and all relevant key vectors, and then perform a weighted summation of the value vectors based on the similarity weights; A weighted summation is performed on all nodes, and an updated set of node representations that incorporates information from both layers is output, forming the dual-layer coupled behavior graph.

[0059] In this embodiment, the comprehensive state representation refers to the high-dimensional vector generated for each propagation node during the construction of the cognitive-emotional layer, which integrates its cognitive state, emotional tendency, and historical credibility. This representation is a digital implicit expression of the user's internal psychological state.

[0060] In this embodiment, the query vector refers to the vector generated after passing the comprehensive state representation of each node in the cognitive-emotion layer through a linear transformation layer. It carries the user's query intent regarding what contextual information the user's current internal state needs to obtain from the propagation structure.

[0061] In this embodiment, topological context information refers to the context information defined by a node's own attributes, its neighboring nodes, and the network structure within the propagation structure layer. Key vector: Mapped from this topological context information, it is used to calculate similarity with the query vector, determining the matchability of the information. Value vector: Also mapped from this topological context information, it contains the specific content information to be retrieved and aggregated.

[0062] In this embodiment, the similarity weights between the query vector and all relevant key vectors are calculated. For the query vector of the current node, the system calculates the similarity between it and the key vectors of all relevant nodes in the network. Subsequently, these similarity scores are normalized into a probability distribution, i.e., the similarity weights. This weight quantifies the degree of attention the current node pays to the information carried by each other node in the network.

[0063] In this embodiment, the node representation set refers to the set of fused vectors of all nodes obtained after performing the aforementioned weighted summation operation on all nodes in the graph. Each vector in this set is a unified representation of a node after its update, deeply fusing its internal state of cognitive emotion and external environmental information of propagation structure. This node representation set constitutes the final, two-layer coupled behavioral graph.

[0064] In this embodiment, the comprehensive state representation of each node in the cognitive-emotion layer is converted into a query vector, representing what the node needs to focus on from the network structure based on its own state. The topological environment information of each node in the propagation structure layer is mapped into key vectors and value vectors, forming an information base. The key vectors are used for matching queries, and the value vectors store the specific content to be aggregated. For each node, the similarity between its query vector and the key vectors of all related nodes is calculated, and a similarity weight is obtained through normalization. This weight dynamically quantifies the importance of the structural environment information of other nodes in the propagation network to the current node in its specific psychological state. Based on the calculated weights, the value vectors of all related nodes are weighted and summed. The result of this operation is that each node obtains a new, fused vector. This vector not only contains its original internal state but also adaptively and on-demand injects the most relevant information from its network structure context. The above aggregation operation is performed in parallel on all nodes in the graph, ultimately outputting an updated set of node representations. Each vector in this set is a unified representation of a node that deeply integrates its internal psychological state and external structural environment. This set as a whole constitutes a two-layer coupled behavior map.

[0065] In some embodiments of this application, when performing cross-modal credibility verification based on the dual-layer coupled behavioral graph and the multimodal content, and generating credibility verification results, the process includes: The topological features of the propagation structure layer and the node state features of the cognitive emotion layer are extracted and spliced ​​together to form a fused node graph joint representation. Calculate the semantic distance between multimodal content features and the joint representation of the node graph; Based on the results of the cross-modal alignment and consistency calculation, a verification result score representing the credibility of the target network information is output through a classification model.

[0066] In this embodiment, topological feature extraction involves extracting graph theory features from the constructed propagation structure layer that explicitly characterize the macroscopic pattern of information diffusion, such as network diameter, average clustering coefficient, and statistical features of the critical path. These features describe the overall form of how information flows within the network. Node state feature extraction involves extracting statistical features reflecting the psychological state of the group from the node representations of the constructed cognitive-emotional layer or the two-layer coupled behavioral graph, such as the distribution of node cognitive states and the distribution and intensity of emotional polarity. These features describe the overall tendency of the propagator group to perceive the information. Feature concatenation involves combining the feature vectors from different levels and describing different dimensions into a longer, unified vector through a concatenation operation. This process achieves statistical information fusion from macroscopic structure to microscopic psychology, laying the foundation for subsequent holistic comparison with content features.

[0067] In this embodiment, the node graph joint representation refers to a fixed-dimensional comprehensive vector generated after the aforementioned feature concatenation operation. This vector is a highly generalized and statistically summarized result of the rich information contained in the two-layer coupled behavioral graph. It represents the behavioral patterns induced and presented by the target network information during the propagation process.

[0068] In this embodiment, calculating the semantic distance between multimodal content features and the joint representation of the node graph aims to quantify the consistency between information content and its propagation behavior. The system encodes and aligns the generated multimodal content data, projecting it into a semantic space compatible with the joint representation of the node graph. The semantic distance between the two is then calculated within this space. A closer distance indicates greater consistency between the semantic content of the information and the resulting propagation behavior pattern; a greater distance indicates a potential discrepancy between content and behavior. For example, a seemingly neutral news item might trigger highly emotional and clustered dissemination, often a sign of misinformation.

[0069] In some embodiments of this application, based on the results of the cross-modal alignment and consistency calculation, a verification result score characterizing the credibility of the target network information is output through a classification model, including: The consistency metric vector generated by the cross-modal alignment and consistency calculation is used as the core input feature of the classification model; The key propagation path features and abnormal interaction pattern features in the joint representation of the node graph are used as auxiliary discriminant features and input into the classification model. The classification model performs joint inference based on all input features and outputs the verification result score.

[0070] In this embodiment, the consistency metric vector is a numerical vector whose elements are generated by calculating the semantic distance between the multimodal content features and the joint representation of the node graph. Specifically, this vector includes not only the calculated overall semantic distance value, but also the sub-distances between different modal pairs, consistency confidence, etc.

[0071] In this embodiment, key propagation path features refer to specific diffusion patterns identified and quantified from the propagation structure layer. Examples include single-point bursts (information instantly spreading from one node to a large number of nodes) and deep penetration (information slowly spreading along a long chain). False information often exhibits specific propagation path patterns. Abnormal interaction pattern features refer to abnormal group behavior patterns identified from the cognitive-emotional layer and interaction data. Examples include emotional polarization (both supporters and opponents have extreme and opposing emotions) and cognitive-emotional divergence (comments expressing skepticism but accompanied by a large number of likes). As auxiliary discriminative features, these features, while not directly measuring the consistency between content and behavior, are extracted from the internal propagation behavior, highly correlated with false information, and can provide powerful auxiliary discriminative signals for the classification model, improving its ability to identify complex and hidden false information patterns.

[0072] In this embodiment, joint inference is performed based on all input features: the classification model receives and processes the core input features and auxiliary discriminant features mentioned above simultaneously. The model's internal decision rules interact, weight, and jointly analyze all features to capture the complex correlation patterns between them. The verification result score is output: after joint inference, the classification model finally outputs a scalar score. This score is a comprehensive quantitative assessment of the credibility of the target network information; a higher score indicates a greater likelihood that the information is judged to be true.

[0073] In some embodiments of this application, determining the authenticity of the target network information based on the credibility verification result includes: The verification result score is compared with a preset authenticity judgment threshold to generate a comparison result; Based on the comparison results, the target network information is classified into a predefined authenticity category; The output includes the target network information and the corresponding authenticity category.

[0074] In this embodiment, the verification result score output by the classification model is compared with a preset authenticity judgment threshold. These thresholds are pre-set based on historical data, business rules, and experimental optimization, and are used to divide different credibility level intervals. Based on the comparison result generated by the threshold comparison, the target network information is classified into a predefined authenticity category, such as "real," "suspicious," or "false." The system generates and outputs a structured final judgment result. This result clearly includes the identifier of the target network information and its determined authenticity category, constituting a complete and traceable judgment record.

[0075] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of this application, and these improvements and substitutions should also be considered within the scope of protection of this application.

Claims

1. A method for identifying fake information in intelligent networks based on multimodal behavior graphs, characterized in that, include: Acquire the target network information to be identified and its multimodal content, and collect the propagation behavior data of the target network information during the propagation process; Based on the aforementioned propagation behavior data, an initial propagation graph is constructed with propagating users as nodes and propagation relationships as edges; Based on the initial propagation graph and the multimodal content, a two-layer coupled behavioral graph containing a propagation structure layer and a cognitive-emotional layer is constructed. Based on the dual-layer coupled behavior map and the multimodal content, cross-modal credibility verification is performed to generate credibility verification results. Based on the credibility verification results, the authenticity of the target network information is determined.

2. The intelligent network fake information identification method based on multimodal behavior graph as described in claim 1, characterized in that, When acquiring target network information to be identified and its multimodal content, and collecting propagation behavior data of the target network information during the propagation process, the following is included: Receive target network information containing core textual claims through the data interface; Extract the multimodal original content of the target network information, and perform standardized encoding and feature alignment on the multimodal original content to form multimodal content data; Using the target network information as the root node, the propagation behavior data is tracked and collected on social platforms. During the collection process, based on the multi-dimensional behavioral characteristics of the propagation nodes, a pre-trained account filtering model is used to perform initial credibility screening and filtering of the collected propagation nodes. The propagation behavior data includes propagation node data, propagation topology data, and node interaction data.

3. The method for identifying false information in intelligent networks based on multimodal behavior graphs as described in claim 2, characterized in that, When constructing an initial propagation graph based on the propagation behavior data, with propagating users as nodes and propagation relationships as edges, the following steps are included: The target network information entity is taken as the root node, and each collected propagation user account is defined as a propagation node; Based on the propagation topology data, directed edges are established between the root node and each propagation node, as well as between each propagation node, to generate propagation paths; Each propagation node is assigned a corresponding propagation node data attribute, and each directed edge is assigned a corresponding additional attribute, including a timestamp and an interaction type label. Based on all nodes, directed edges, and their additional attributes, generate structured initial propagation graph data.

4. The method for identifying false information in intelligent networks based on multimodal behavior graphs as described in claim 3, characterized in that, When constructing a two-layer coupled behavioral graph containing a propagation structure layer and a cognitive-emotional layer based on the initial propagation graph and the multimodal content, the following steps are included: Based on the initial propagation graph data, the macroscopic topology and node attributes are extracted to construct a propagation structure layer that explicitly represents the information diffusion path. Based on the multimodal content and node interaction data, implicit cognitive state vectors and sentiment tendency vectors are generated for each propagation node through semantic and sentiment analysis models to construct a cognitive sentiment layer. By using a cross-layer attention mechanism, the propagation structure layer and the cognitive emotion layer are dynamically associated and information is fused to generate the dual-layer coupled behavior map.

5. The intelligent network fake information identification method based on multimodal behavior graph as described in claim 4, characterized in that, When extracting the macroscopic topology and node attributes based on the initial propagation graph data, and constructing a propagation structure layer that explicitly represents the information diffusion path, the process includes: Graph structure features are extracted from the initial propagation graph data, and macroscopic topological indices, including node centrality, propagation path length, and subgraph clustering coefficient, are calculated. The user attribute features in the propagation node data are fused to form an enhanced node attribute vector; The macroscopic topology indicators and enhanced node attribute vectors are structured and encoded to construct the propagation structure layer.

6. The method for identifying false information in intelligent networks based on multimodal behavior graphs as described in claim 5, characterized in that, Based on the aforementioned multimodal content and node interaction data, implicit cognitive state vectors and sentiment tendency vectors are generated for each propagation node using a semantic and sentiment analysis model. When constructing the cognitive-sentiment layer, the following steps are included: Semantic understanding and intent recognition are performed on the comment text in the node interaction data to classify and quantify the user's cognitive state of certainty, doubt, and neutrality towards the target network information, and generate a cognitive state vector. Fine-grained sentiment analysis is performed on comment text and associated emojis to identify and quantify the positive, negative, or complex emotions expressed by users during the dissemination process, and to generate sentiment tendency vectors. Based on the cognitive state vector and the emotional tendency vector, the historical behavior credibility features of the node are fused to form a comprehensive state representation of the node in the cognitive-emotional layer.

7. The intelligent network fake information identification method based on multimodal behavior graph as described in claim 6, characterized in that, When generating the dual-layer coupled behavioral graph by dynamically associating and fusing information between the propagation structure layer and the cognitive-emotion layer through a cross-layer attention mechanism, the following steps are included: The comprehensive state representation of each node in the cognitive emotion layer is used as the query vector; The topological environment information of the corresponding nodes in the propagation structure layer is mapped into key vectors and value vectors; Calculate the similarity weights between the query vector and all relevant key vectors, and then perform a weighted summation of the value vectors based on the similarity weights; A weighted summation is performed on all nodes, and an updated set of node representations that incorporates information from both layers is output, forming the dual-layer coupled behavior graph.

8. The method for identifying false information in intelligent networks based on multimodal behavior graphs as described in claim 4, characterized in that, Based on the dual-layer coupled behavioral map and the multimodal content, cross-modal credibility verification is performed, and the credibility verification result is generated, including: The topological features of the propagation structure layer and the node state features of the cognitive emotion layer are extracted and spliced ​​together to form a fused node graph joint representation. Calculate the semantic distance between multimodal content features and the joint representation of the node graph; Based on the results of the cross-modal alignment and consistency calculation, a verification result score representing the credibility of the target network information is output through a classification model.

9. The method for identifying false information in intelligent networks based on multimodal behavior graphs as described in claim 8, characterized in that, Based on the results of the cross-modal alignment and consistency calculation, a verification result score characterizing the credibility of the target network information is output through a classification model, including: The consistency metric vector generated by the cross-modal alignment and consistency calculation is used as the core input feature of the classification model; The key propagation path features and abnormal interaction pattern features in the joint representation of the node graph are used as auxiliary discriminant features and input into the classification model. The classification model performs joint inference based on all input features and outputs the verification result score.

10. The method for identifying false information in intelligent networks based on multimodal behavior graphs as described in claim 9, characterized in that, When determining the authenticity of the target network information based on the credibility verification result, the following are included: The verification result score is compared with a preset authenticity judgment threshold to generate a comparison result; Based on the comparison results, the target network information is classified into a predefined authenticity category; The output includes the target network information and the corresponding authenticity category.

Images