A multi-modal social network public opinion hidden danger checking method and system
By collecting multimodal data and segmenting communities, combined with cross-modal validation and propagation path simulation, the limitations of single-modal data processing in social network public opinion monitoring have been overcome, enabling efficient and accurate identification and prediction of potential public opinion risks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DATA SPACE RES INST
- Filing Date
- 2025-06-25
- Publication Date
- 2026-05-15
AI Technical Summary
Existing social network public opinion monitoring technologies mainly rely on single-modal data processing, which cannot effectively capture the correlation of multimodal data, lack quantitative analysis of the dynamic changes and impact range of public opinion dissemination paths, and have insufficient automated analysis capabilities, resulting in the risk of missed detection in potential risks.
Multimodal data is collected in real time by distributed edge nodes. Social network communities are divided based on user interaction relationships. Community propagation dynamics, group topology and content features are extracted. Cross-modal conflict detection is carried out using a dual-stream cross-validation network and a LightGBM classifier. The propagation path is simulated by combining Time-LLM and Graph Transformer to generate intervention strategies.
It significantly improves the comprehensiveness and accuracy of identifying potential public opinion risks, overcomes the limitations of single-modal data processing, and enables scientific, accurate prediction and efficient intervention in the spread of public opinion.
Smart Images
Figure CN120765004B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of social network public opinion monitoring technology, and in particular to a method and system for investigating potential public opinion risks on multimodal social networks. Background Technology
[0002] Current social network public opinion monitoring technologies primarily employ a single-modal data processing model, analyzing independent data types such as text, images, audio, or video, and performing analysis by extracting textual information. This lack of a collaborative processing mechanism for multimodal data results in public opinion analysis being limited to a single data format, failing to effectively capture the correlation between textual semantics, visual features, and auditory information. Regarding dynamic propagation characteristics analysis, existing methods fail to fully model the topological evolution of social networks, lacking quantitative analysis tools for key characteristics such as the dynamic changes in public opinion propagation paths and the diffusion mechanisms of influence, leading to insufficient accuracy in prediction models. Operationally, the main reliance on a serial processing model with manual intervention creates gaps in the automated analysis chain, limiting cross-platform data synchronization and real-time computing capabilities, making it difficult to adapt to the rapid evolution of public opinion situations. Particularly in the multimedia content recognition stage, existing systems lack robustness in detecting unstructured data such as image-sensitive features and voiceprint violations, and have not established a correlation analysis model between user interaction relationships and content features, resulting in a significant risk of missed detections in multimodal public opinion scenarios. Summary of the Invention
[0003] To address the technical problems existing in the background art, this invention proposes a method and system for investigating potential public opinion risks on multimodal social networks.
[0004] This invention proposes a method for investigating potential public opinion risks on multimodal social networks, comprising the following steps:
[0005] S1. Real-time collection of multimodal data from social platforms via distributed edge nodes, including text, image, and audio data;
[0006] S2. Divide multiple social network communities based on user interaction relationships, including but not limited to topic circles, friend relationships, comment interactions, forwarding and liking;
[0007] S3. Based on multimodal data, extract community propagation dynamics features, group topology features, and content features from multiple social network communities one by one;
[0008] S4. Based on the characteristics of community propagation dynamics, group topology, content characteristics, and preset screening strategies, multiple suspected potential communities are selected from multiple social network communities.
[0009] S5. After detecting and classifying multiple suspected hazardous communities based on multimodal data, output multiple hazardous categories corresponding to the multiple suspected hazardous communities. The hazardous categories include high risk, early warning, and attention.
[0010] Preferably, the community propagation dynamics features specifically include the time derivative of the forwarding rate and the network modularity; the group topology features specifically include the user clustering Gini coefficient and the sentiment polarization index; and the content features include the density of sensitive words and the probability of visual violations.
[0011] Preferably, the preset filtering strategy specifically includes:
[0012] The community communication dynamics, group topology, and content characteristics are compared one by one with the corresponding preset screening threshold ranges;
[0013] If any of the community propagation dynamics, group topology, or content characteristics does not meet the preset screening threshold, the corresponding social network community will be considered a suspected potential risk community.
[0014] Preferably, the hazard detection in step S5 specifically includes:
[0015] Cross-modal verification of user content in multiple communities with suspected potential risks is performed based on multimodal data. The semantic consistency between text and image or audio data in the community is analyzed by a pre-trained dual-stream cross-validation network. Cross-modal contradictory content is marked as high-risk nodes to obtain cross-modal verification results.
[0016] Preferably, the marker of cross-modal contradictory content as a high-risk node must meet any of the following conditions:
[0017] (a) The probability of the image / audio being prohibited is greater than the threshold P, and the semantics of the text description contradict the prohibited content, i.e., the semantic similarity is less than the threshold Q;
[0018] (b) The independent probabilities of text, images, and audio are all greater than the threshold R, and the cross-modal attention weight is greater than the threshold S;
[0019] (c) The difference in probabilities of single-modal violations is greater than the threshold T, and the cross-modal semantic similarity is less than the threshold U.
[0020] Preferably, the hazard classification in step S5 specifically includes:
[0021] The community transmission dynamics characteristics, group topology characteristics, and cross-modal validation results are input into a pre-trained LightGBM hierarchical classifier to output a hazard classification, which includes high risk, early warning, and concern.
[0022] Preferably, the method further includes: after step S5, the method further includes:
[0023] S6. Conduct propagation simulations for communities specifically classified as high-risk for potential hazards, and encode the propagation path using a Time-LLM time series model; simulate the propagation topology and impact range based on graph transformation networks and neural differential equations;
[0024] S7. Generate intervention strategies based on the simulation results. The intervention strategies include content deletion, user guidance, or official clarification.
[0025] Preferably, the propagation simulation in step S6 supports minute-level trajectory prediction, specifically including: generating a heat map of the propagation of the primary event for uninterrupted communities; and generating a waveform diagram of the duration of the prevention and control event for intervened communities.
[0026] This invention proposes a multimodal social network public opinion risk assessment system, comprising:
[0027] The data acquisition module is used to collect multimodal data from social platforms in real time through distributed edge nodes. The multimodal data includes text, image, audio, and video data.
[0028] The community segmentation module is used to segment multiple social network communities based on user interaction relationships, including but not limited to topic circles, friend relationships, comment interactions, forwarding and liking;
[0029] The feature extraction module is used to extract community propagation dynamics features, group topology features, and content features from multiple social network communities based on multimodal data.
[0030] The community screening module is used to filter out multiple potentially problematic communities from multiple social network communities based on community propagation dynamics, group topology, content characteristics, and preset screening strategies.
[0031] The hazard detection and classification module is used to detect and classify multiple suspected hazard communities based on multimodal data, and then output multiple hazard classifications corresponding to the multiple suspected hazard communities. The hazard classifications include high risk, early warning, and attention.
[0032] Preferably, it further includes:
[0033] The propagation simulation module is used to perform propagation simulations for communities specifically classified as high-risk hazards. It encodes the propagation path using a Time-LLM time series model and simulates the propagation topology and impact range based on graph transformation networks and neural differential equations.
[0034] The intervention module is used to generate intervention strategies based on the simulation results. These intervention strategies include content deletion, user guidance, or official clarification.
[0035] This invention proposes a multimodal social network public opinion risk assessment method and system. It collects multimodal data through distributed edge nodes; divides communities based on user interaction relationships, extracts propagation dynamics, group topology, and content features, and filters potentially risky communities; employs a dual-stream cross-validation network to achieve cross-modal conflict detection, and combines a LightGBM classifier to output three-level labels: high-risk, warning, and attention; further, it simulates propagation paths using Time-LLM and Graph Transformer to generate intervention strategies; finally, it optimizes system parameters through a closed-loop intelligent agent based on effect evaluation. This significantly improves the comprehensiveness and accuracy of public opinion risk identification, overcoming the limitations of single-modal data processing in existing technologies. It also compensates for the shortcomings of existing technologies in propagation dynamic modeling, making the prediction of public opinion risks more scientific and accurate. Attached Figure Description
[0036] Figure 1 This is a schematic diagram illustrating the workflow of a multimodal social network public opinion risk assessment method proposed in this invention.
[0037] Figure 2 This is a schematic diagram of one embodiment of the multimodal social network public opinion risk investigation method proposed in this invention;
[0038] Figure 3 This is a schematic diagram of the system architecture of a multimodal social network public opinion risk investigation system proposed in this invention. Detailed Implementation
[0039] Reference Figure 1-3 The present invention proposes a method for investigating potential public opinion risks on multimodal social networks, comprising the following steps:
[0040] S1. Collect multimodal data from social platforms in real time through distributed edge nodes. The multimodal data includes text, images, and audio data.
[0041] S2. Divide into multiple social network communities based on user interaction relationships, including but not limited to topic circles, friend relationships, comment interactions, forwarding and liking.
[0042] In this embodiment, the partitioning method for user interaction relationships across multiple social network communities includes, but is not limited to, partitioning algorithms such as modularity optimization algorithm, edge betweenness algorithm, spectral clustering algorithm, and graph convolutional network.
[0043] S3. Based on multimodal data, extract community propagation dynamics features, group topology features, and content features from multiple social network communities one by one.
[0044] Specifically, the extraction methods for extracting community propagation dynamics features from multiple social network communities include, but are not limited to, using domain-adapted vector libraries and feature-level fusion to extract text, image, and audio data.
[0045] In this embodiment, the community propagation dynamics features specifically include the time derivative of the forwarding rate and the network modularity; the group topology features specifically include the user clustering Gini coefficient and the sentiment polarization index; and the content features include the density of sensitive words and the probability of visual violations.
[0046] S4. Based on the characteristics of community propagation dynamics, group topology, content features, and preset screening strategies, multiple suspected potential communities are selected from multiple social network communities.
[0047] In this embodiment, the preset filtering strategy specifically includes:
[0048] The community communication dynamics, group topology, and content characteristics are compared one by one with the corresponding preset screening threshold ranges;
[0049] If any of the community propagation dynamics, group topology, or content characteristics does not meet the preset screening threshold, the corresponding social network community will be considered a suspected potential risk community.
[0050] S5. Based on multimodal data, after detecting and classifying multiple suspected hazardous communities, output multiple hazardous categories corresponding to the multiple suspected hazardous communities. The hazardous categories include high risk, early warning, and attention.
[0051] In this embodiment, the hazard detection in step S5 specifically includes:
[0052] Cross-modal verification of user content in multiple communities with suspected potential risks is performed based on multimodal data. The semantic consistency between text and image or audio data in the community is analyzed by a pre-trained dual-stream cross-validation network. Cross-modal contradictory content is marked as high-risk nodes to obtain cross-modal verification results.
[0053] Specifically, marking cross-modal contradictory content as a high-risk node requires meeting any of the following conditions:
[0054] (a) The probability of the image / audio being prohibited is greater than the threshold P, and the semantics of the text description contradict the prohibited content, i.e., the semantic similarity is less than the threshold Q;
[0055] (b) The independent probabilities of text, images, and audio are all greater than the threshold R, and the cross-modal attention weight is greater than the threshold S;
[0056] (c) The difference in probabilities of single-modal violations is greater than the threshold T, and the cross-modal semantic similarity is less than the threshold U.
[0057] It should be noted that multiple communities suspected of having potential risks were prioritized for investigation by the detection agent. High-dimensional risk detection was conducted on these communities, examining users' content risk characteristics such as sensitive word density, visual violation probability, and voiceprint violation, to deeply detect potentially risky individuals or objects. Simultaneously, a dual-stream cross-validation network was used to analyze the semantic consistency between text descriptions and image or audio content, achieving cross-modal risk verification and marking users with potentially risky content as risky nodes within the community.
[0058] Specifically, the training process of the dual-stream cross-validation network involves extracting features in parallel through dual-modal streams and exchanging them to generate adversarial supervision signals in order to iteratively optimize the cross-modal consistency loss.
[0059] In this embodiment, the hazard classification in step S5 specifically includes:
[0060] The community transmission dynamics characteristics, group topology characteristics, and cross-modal validation results are input into a pre-trained LightGBM hierarchical classifier to output a hazard classification, which includes high risk, early warning, and concern.
[0061] In this embodiment, the method further includes: after step S5, the method further includes:
[0062] S6. Conduct propagation simulations for communities specifically classified as high-risk for potential hazards, and encode the propagation path using a Time-LLM time series model; simulate the propagation topology and impact range based on graph transformation networks and neural differential equations;
[0063] S7. Generate intervention strategies based on the simulation results. Intervention strategies include content deletion, user guidance, or official clarification.
[0064] Specifically, the propagation simulation in step S6 supports minute-level trajectory prediction, including: generating a heat map of the primary event propagation for uninterrupted communities; and generating a waveform diagram of the duration of the prevention and control event for intervened communities.
[0065] Specifically, a high-dimensional feature vector dataset for potential risks is constructed by extracting surface features, contextual environment, and community features from multimodal data. Surface features include, but are not limited to, keywords and visual elements; contextual environment includes, but is not limited to, spatiotemporal labels and device fingerprints; and community features include, but are not limited to, user profiles and group behavior patterns. A large-scale Time-LLM model is introduced to encode propagation path data into a learnable token sequence, and the spatiotemporal dependencies of cross-platform propagation are analyzed through a spatiotemporal attention mechanism.
[0066] Specifically, the core idea of Graph Transformer is to apply the Transformer architecture to graph-structured data. By fusing Graph Neural Networks (GNNs) with self-attention mechanisms, it enables the processing of relationships between nodes and the capture of long-range dependencies. A dynamic evolution engine is built using Graph Transformer, combined with a neural differential equation solver, to achieve continuous time-series extrapolation of hazard states, supporting minute-level propagation trajectory prediction and intervention strategy simulation. Support for minute-level propagation trajectory prediction specifically includes: for uninterrupted hazard communities, simulating the propagation path topology, impact range heatmap, and duration waveform after tagged users within the community send hazard information, generating the original hazard propagation event. For high-risk communities after implementing intervention strategies, the propagation path, impact range, and duration of hazard information are simulated again, generating hazard prevention and control propagation events.
[0067] Specifically, the intervention strategies include:
[0068] a. Content Management: Delete or restrict the dissemination of illegal content.
[0069] b. User guidance: Warn or educate users who publish sensitive information.
[0070] c. Official clarification: Release authoritative information to guide the public to view the event correctly.
[0071] d. Social network regulation: Adjust the dissemination mechanism of social networks to reduce the speed of the spread of potentially harmful information.
[0072] In this embodiment, it also includes generating a hazard investigation report by combining the original hazard propagation events without intervention and the hazard propagation events after intervention.
[0073] This embodiment also includes a closed-loop feedback core process for intelligent agent swarms. This process is responsible for real-time monitoring, data analysis, and effect evaluation of the entire process of hazard investigation, simulation, and decision-making. The generated strategies can be evaluated, a simulated review team can review them and provide feedback, and unsatisfactory strategies will be returned to the process for re-simulation. Based on public opinion feedback after real hazard occurrences, each link of the system is quantitatively evaluated, providing data support for system optimization and improvement, ensuring continuous iterative optimization, forming a complete closed-loop feedback mechanism, and guaranteeing the efficient and accurate operation of the system.
[0074] Reference Figure 1-3 This invention proposes a multimodal social network public opinion risk assessment system, comprising:
[0075] The data acquisition module is used to collect multimodal data from social platforms in real time through distributed edge nodes. The multimodal data includes text, images, audio, and video data.
[0076] The community segmentation module is used to segment multiple social network communities based on user interaction relationships, including but not limited to topic circles, friend relationships, comment interactions, forwarding and liking;
[0077] The feature extraction module is used to extract community propagation dynamics features, group topology features, and content features from multiple social network communities based on multimodal data.
[0078] The community screening module is used to filter out multiple potentially problematic communities from multiple social network communities based on community propagation dynamics, group topology, content characteristics, and preset screening strategies.
[0079] The hazard detection and classification module is used to detect and classify multiple suspected hazard communities based on multimodal data, and output multiple hazard classifications corresponding to the multiple suspected hazard communities. Hazard classifications include high risk, early warning, and attention.
[0080] In this embodiment, it also includes:
[0081] The propagation simulation module is used to perform propagation simulations for communities specifically classified as high-risk hazards. It encodes the propagation path using a Time-LLM time series model and simulates the propagation topology and impact range based on graph transformation networks and neural differential equations.
[0082] The intervention module is used to generate intervention strategies based on the simulation results. These strategies include content deletion, user guidance, or official clarification.
[0083] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for investigating potential public opinion risks on multimodal social networks, characterized in that, Includes the following steps: S1. Real-time collection of multimodal data from social platforms via distributed edge nodes, including text, image, and audio data; S2. Divide multiple social network communities based on user interaction relationships, including but not limited to topic circles, friend relationships, comment interactions, forwarding and liking; S3. Based on multimodal data, extract community propagation dynamics features, group topology features, and content features from multiple social network communities one by one; S4. Based on the characteristics of community propagation dynamics, group topology, content characteristics, and preset screening strategies, multiple suspected potential communities are selected from multiple social network communities. S5. After detecting and classifying multiple suspected hazardous communities based on multimodal data, output multiple hazardous categories corresponding to the multiple suspected hazardous communities. The hazardous categories include high risk, early warning, and attention. Specifically, the hazard detection in step S5 includes: Cross-modal verification of user content in multiple communities with suspected potential risks is performed based on multimodal data. The semantic consistency between text and image or audio data in the community is analyzed by a pre-trained dual-stream cross-validation network. Cross-modal contradictory content is marked as high-risk nodes to obtain cross-modal verification results.
2. The method for investigating potential public opinion risks on multimodal social networks according to claim 1, characterized in that, The community propagation dynamics features specifically include the time derivative of the forwarding rate and network modularity; the group topology features specifically include the user clustering Gini coefficient and sentiment polarization index; and the content features include sensitive word density and visual violation probability.
3. The method for investigating potential public opinion risks on multimodal social networks according to claim 2, characterized in that, The preset filtering strategy specifically includes: The community communication dynamics, group topology, and content characteristics are compared one by one with the corresponding preset screening threshold ranges; If any of the community propagation dynamics, group topology, or content characteristics does not meet the preset screening threshold, the corresponding social network community will be considered a suspected potential risk community.
4. The method for investigating potential public opinion risks on multimodal social networks according to claim 1, characterized in that, The marker for cross-modal contradictory content to be a high-risk node must meet any of the following conditions: (a) The probability of the image / audio being prohibited is greater than the threshold P, and the semantics of the text description contradict the prohibited content, i.e., the semantic similarity is less than the threshold Q; (b) The independent probabilities of text, images, and audio are all greater than the threshold R, and the cross-modal attention weight is greater than the threshold S; (c) The difference in probabilities of single-modal violations is greater than the threshold T, and the cross-modal semantic similarity is less than the threshold U.
5. The method for investigating potential public opinion risks on multimodal social networks according to claim 1, characterized in that, The hazard classification in step S5 specifically includes: The community transmission dynamics characteristics, group topology characteristics, and cross-modal validation results are input into a pre-trained LightGBM hierarchical classifier to output a hazard classification, which includes high risk, early warning, and concern.
6. The method for investigating potential public opinion risks on multimodal social networks according to claim 1, characterized in that, Also includes: The process after step S5 also includes: S6. Conduct propagation simulations for communities specifically classified as high-risk for potential hazards, and encode the propagation path using a Time-LLM time series model; simulate the propagation topology and impact range based on graph transformation networks and neural differential equations; S7. Generate intervention strategies based on the simulation results. The intervention strategies include content deletion, user guidance, or official clarification.
7. The method for investigating potential public opinion risks on multimodal social networks according to claim 6, characterized in that, The propagation simulation in step S6 supports minute-level trajectory prediction, specifically including: generating a heat map of the primary event propagation for uninterrupted communities; and generating a waveform diagram of the duration of the prevention and control event for intervened communities.
8. A multimodal social network public opinion risk assessment system, characterized in that, include: The data acquisition module is used to collect multimodal data from social platforms in real time through distributed edge nodes. The multimodal data includes text, image, audio, and video data. The community segmentation module is used to segment multiple social network communities based on user interaction relationships, including but not limited to topic circles, friend relationships, comment interactions, forwarding and liking; The feature extraction module is used to extract community propagation dynamics features, group topology features, and content features from multiple social network communities based on multimodal data. The community screening module is used to filter out multiple potentially problematic communities from multiple social network communities based on community propagation dynamics, group topology, content characteristics, and preset screening strategies. The hazard detection and classification module is used to detect and classify multiple suspected hazard communities based on multimodal data, and then output multiple hazard classifications corresponding to the multiple suspected hazard communities. The hazard classifications include high risk, early warning, and attention. Specifically, hazard detection includes: Cross-modal verification of user content in multiple communities with suspected potential risks is performed based on multimodal data. The semantic consistency between text and image or audio data in the community is analyzed by a pre-trained dual-stream cross-validation network. Cross-modal contradictory content is marked as high-risk nodes to obtain cross-modal verification results.
9. The multimodal social network public opinion risk assessment system according to claim 8, characterized in that, Also includes: The propagation simulation module is used to perform propagation simulations for communities specifically classified as high-risk hazards, and encodes the propagation path using a Time-LLM time series model. Simulation of propagation topology and influence range based on graph transform networks and neural differential equations; The intervention module is used to generate intervention strategies based on the simulation results. These intervention strategies include content deletion, user guidance, or official clarification.