News propagation influence quantitative evaluation method and system based on data mining

Abstract

The invention relates to the technical field of data analysis, in particular to a news propagation influence quantitative evaluation method and system based on data mining, and the system comprises a data collection and preprocessing module, a dynamic index management module, an intelligent modeling and analysis module, a causal inference and explanation module and a storage and retrieval module. In order to solve the problem that a traditional evaluation model is insufficient in scene adaptability due to the fact that a fixed weight and a linear attenuation hypothesis are adopted, a dynamic attenuation model and a reinforcement learning mechanism are introduced in the scheme, and chain reactions in the information spreading process are described through modeling of a self-excitation effect in a spreading event; on the basis, an evaluation index weight distribution mechanism is dynamically optimized in combination with an adaptive algorithm, so that the model can automatically adapt to rapid outbreak of sudden news and long-acting propagation characteristics of policy news, and a nonlinear rule presented in the propagation process is accurately captured; according to the method, the scene adaptability and timeliness of the evaluation result are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to a method and system for quantitatively evaluating the influence of news dissemination based on data mining. Background Technology

[0002] In today's digital environment, news dissemination has transformed from a one-way transmission from media to audiences to a cross-platform, multi-format, high-concurrency real-time interactive network. The rapid evolution of social networks, news applications, short videos, and other channels enables news to reach hundreds of millions of users instantly, triggering multi-level dissemination and chain reactions.

[0003] However, corresponding methods for assessing the impact of communication still largely rely on traditional statistical models and single data indicators, making it difficult to match the dynamic and diverse nature of the current communication ecosystem. Existing methods typically rely solely on surface-level indicators such as reposts and comments, employing fixed weights or linear regression for analysis. This fails to effectively integrate heterogeneous data from different platforms, neglects the semantic relationships between text, images, and videos, and lacks dynamic modeling of the spatiotemporal patterns of communication. For example, the spread of breaking news often exhibits explosive growth, while in-depth reports or policy interpretations may have a sustained and profound impact; traditional static models struggle to accurately depict these differences.

[0004] Furthermore, multi-source data collection is often subject to noise interference such as fake traffic and cross-platform duplication, affecting the reliability of the assessment; and the "black box" model makes it difficult for decision-makers to trace the key factors influencing dissemination and to clarify the causal relationship between news releases and the evolution of public opinion. Although some studies have introduced machine learning methods, they are still generally limited by practical bottlenecks such as reliance on manual feature extraction, response lag, and insufficient model interpretability.

[0005] Therefore, the industry urgently needs to build an evaluation system that can integrate multimodal real-time data processing, dynamic propagation modeling, causal inference and interpretable analysis, so as to scientifically, accurately and reliably reveal the intrinsic influence mechanism of news dissemination. Summary of the Invention

[0006] To overcome the problems mentioned in the background art, this invention proposes a method and system for quantitatively evaluating the influence of news dissemination based on data mining.

[0007] The technical solution of this invention is: a method for quantitatively evaluating the influence of news dissemination based on data mining, comprising the following steps: S11: Data acquisition and preprocessing, acquiring multi-source data and cleaning and fusing cross-modal features in real time through a stream processing framework; S12: Construction of indicator system and optimization of weights, based on semantic association of knowledge graph and Hawkes process modeling propagation decay, and dynamic adjustment of indicator weights using reinforcement learning; S13: Feature engineering and modeling, using neural differential equations to fit propagation curves, constructing heterogeneous graph neural networks to predict diffusion paths, and combining AutoML to automatically generate high-order features; S14: Causal reasoning, verifying influence through causal discovery and propensity score matching, and generating a natural language interpretation report based on SHAP value and GPT-3.

[0008] As a preferred method, when collecting multi-source data and cleaning and fusing cross-modal features in real time using a stream processing framework, the specific steps include: S21: Multi-source heterogeneous data crawling. It crawls text, image and video data from multiple platforms through APIs and web crawlers, and uses OpenCV to extract video keyframes and CLIP model to analyze cross-modal semantics, and deploys Apache Kafka to achieve real-time stream processing. S22: Robot detection upgrade, based on the XGBoost model to detect robot accounts, with features including posting time interval variance, follower ratio and content duplication rate, and intercepting high-probability robot data; S23: Multimodal data fusion, which performs cross-modal attention-weighted fusion of text data, image data and video data to generate a unified semantic embedding vector; S24: Real-time cleaning and storage, filtering outliers and deleting advertising links based on the 3σ principle, normalizing the data and storing it in Elasticsearch, and storing the original multimedia files in MinIO object storage.

[0009] As a preferred approach, when detecting bot accounts based on the XGBoost model, features include posting time interval variance, followers, follower ratio, and content duplication rate. Specifically, when blocking high-probability bot data, the following features are included: S31: Data Preparation and Feature Extraction: Extract account behavior data from social media APIs and logs, and calculate key features of the account behavior data. These key features include posting interval variance, follower ratio, and content duplication rate. The formula for calculating the posting interval variance is as follows: ; in, This is the timestamp of the i-th post. Let N be the average interval between posts, and N be the total number of posts. S32: Model training and validation. A training set is constructed using manual annotation and rule-based pre-screening, with a positive to negative sample ratio of 1:3. The constructed training set is then used to train and validate the XGBoost model. S33: Real-time monitoring and interception. Deploy the XGBoost model to receive account behavior data in real time, output the probability of bots, set a basic threshold, and adjust the threshold in real time according to the false negative rate data. Judge the account behavior data, and intercept bot accounts if they are judged to be bot accounts.

[0010] As a preferred embodiment, when performing cross-modal attention-weighted fusion of text data, image data, and video data to generate a unified semantic embedding vector, the specific steps include: S41: Feature extraction. For text data, a pre-trained BERT model is used to extract 768-dimensional sentence vectors. For image data, a ResNet-50 pre-trained model is used to extract 2048-dimensional features after global average pooling. For video data, a 3DCNN model is used to extract 512-dimensional temporal features for each video segment. S42: Modal alignment, calculates cross-modal similarity between text data, image data, and video data using the CLIP model, and filters semantically matching image-text pairs; S43: Attention weight calculation. A multi-head attention mechanism is used to calculate the dynamic attention weights for text, image, and video data. The calculation principle formula is as follows: ; Where Q is the query vector, and , For text features, Used to project text features into d-dimensional space For key vectors, , For image features, For video features, The dimension of the key vector is d. This indicates that the similarity between the query vector and the query vector is calculated. These are the dynamic attention weights for text data, image data, and video data, respectively. S44: Feature fusion, which dynamically fuses text features, image features and video features by calculating dynamic attention weights to obtain a unified embedding vector; S45: Post-processing, normalizes the unified embedding vector obtained by fusion, and stores it in the vector database.

[0011] As a preferred approach, when modeling propagation decay based on semantic associations in knowledge graphs and the Hawkes process, and dynamically adjusting indicator weights using reinforcement learning, the specific methods include: S51: Indicator expansion and semantic association, constructing a knowledge graph of news and policy entities, calculating semantic path similarity between entities through the TransR model, and expanding the deep association between indicators; S52: Dynamic decay model construction, using Hawkes process to model the self-excitation effect of propagation, and using learnable parameters to dynamically fit the time decay characteristics of propagation events; S53: Adaptive weight allocation. Design a reinforcement learning model based on DQN, using historical dissemination effects as environmental feedback to dynamically optimize the weight allocation of dissemination breadth, depth, and social impact.

[0012] Preferably, when modeling the self-excitation effect of propagation using the Hawkes process and dynamically fitting the time decay characteristics of the propagation event using learnable parameters, the specific methods include: S61: Data preparation and event sequence definition, collect timestamp data of news dissemination events, construct time series by news ID, and record the occurrence time of each dissemination behavior; S62: Model parameter initialization, initialize the base propagation rate, trigger strength, and decay rate as trainable parameters, and set the optimizer to prepare for learning; S63: Model training and parameter optimization, which involves iteratively updating parameters by maximizing the log-likelihood function and using gradient descent. S64: Calculation and application of decay weights, generating normalized decay weights based on the trained model, and dynamically adjusting the influence score at each time step. S65: Model validation and tuning, calculating RMSE to evaluate fitting accuracy, analyzing parameter sensitivity, and segmenting and modeling for long-cycle news.

[0013] As a preferred approach, when designing a reinforcement learning model based on DQN, using historical dissemination effects as environmental feedback to dynamically optimize the weight allocation of dissemination breadth, depth, and social impact, the specific steps include: S71: Reinforcement learning environment design, defining state actions and reward functions, and constructing a dynamic decision-making environment; S72: DQN model training, designing a dual hidden layer network, learning the Q-value mapping relationship between state and action through experience replay and temporal difference error minimization; S73: Online weight adjustment and verification, real-time reasoning to select the optimal weight adjustment action, and verification of the propagation effect through A / B testing.

[0014] Preferably, when using neural differential equations to fit the propagation curve, constructing a heterogeneous graph neural network to predict the diffusion path, and combining this with AutoML to automatically generate high-order features, the specific steps include: S81: The propagation curve is fitted using neural differential equations. The propagation curve is dynamically fitted using neural differential equations, and the nonlinear changes in the propagation rate are automatically captured by the neural network. S82: Heterogeneous graph neural network modeling, constructing a heterogeneous graph of users-news-comments, using a relational graph convolutional network to aggregate multiple types of node relationships, learn complex interaction structures and predict key propagation paths; S83: AutoML-driven feature generation automatically searches for feature combinations through TPOT to generate high-order interactive features.

[0015] As a preferred method, when verifying influence through causal discovery and propensity score matching, and generating a natural language interpretation report based on SHAP values ​​and GPT-3, the specific steps include: S91: Robust causal analysis, using PC algorithm to identify confounding variables, and applying propensity score matching to estimate the net effect of news on public opinion; S92: Natural language interpretation generation, extracting feature contribution based on SHAP value, and generating readable conclusions by combining GPT-3; S93: Contextualized time decay, dynamically selects decay function, and corrects influence score to adapt to the timeliness patterns of different news types.

[0016] A data mining-based system for quantifying the impact of news dissemination includes: The data acquisition and preprocessing module is used to acquire multi-source data in real time, clean up anomalies, and fuse cross-modal features. The dynamic indicator management module is used to build a dynamic indicator system and optimize weight allocation through reinforcement learning; The intelligent modeling and analysis module is used to generate high-order features using neural differential equations, heterogeneous graph networks, and AutoML to achieve propagation path prediction and nonlinear modeling. The causal inference and explanation module is used to verify the causal effect of news on public opinion, generate an explainable report, and dynamically adjust the timeliness decay strategy. The storage and retrieval module is used to efficiently store structured data and multimodal vectors, and supports fast retrieval and similarity analysis.

[0017] The beneficial effects of this invention are: 1. Existing evaluation methods are usually limited to single-format data and the cleaning process is lagging, often resulting in incomplete information sources and excessive noise. To address this, this solution constructs a real-time stream processing system that simultaneously collects text, image, and video data from different platforms in various formats. Through cross-modal semantic alignment technology and robot behavior recognition models, the system can perform dynamic cleaning and integration upon access. Furthermore, by leveraging an attention mechanism to perform weighted fusion of multimodal features, it not only enhances the richness and semantic coherence of data representation but also effectively eliminates interference from abnormal accounts, thereby providing a reliable and high-quality data foundation for subsequent analysis. 2. To address the insufficient scenario adaptability of traditional evaluation models due to their fixed weights and linear decay assumptions, this solution introduces a dynamic decay model and reinforcement learning mechanism. By modeling the self-excitation effect in a dissemination event, it characterizes the chain reaction in the information dissemination process. Based on this, an adaptive algorithm is used to dynamically optimize the weight allocation mechanism of evaluation indicators, enabling the model to automatically adapt to the rapid outbreak of breaking news and the long-term dissemination characteristics of policy news, accurately capturing the nonlinear patterns presented in the dissemination process. This method effectively improves the scenario adaptability and timeliness of the evaluation results. 3. To address the shortcomings of current methods in causal inference and result interpretation, which lead to difficulties in removing confounding factors and low reliability of conclusions, this solution employs causal discovery techniques to identify potential confounding variables, uses causal effect estimation methods to measure the net impact of news on public opinion, and automatically generates a natural language analysis report using an interpretable model. This design not only scientifically quantifies the actual effect of news dissemination but also enhances the reliability of results through transparency in the decision-making process, thereby providing decision support for public opinion management that combines scientific rigor with practical guidance. Attached Figure Description

[0018] Figure 1 The diagram shown is a flowchart of the data mining-based method for quantitatively evaluating the influence of news dissemination according to the present invention. Figure 2 The diagram shown is a schematic representation of the structure of the data mining-based news dissemination influence quantitative evaluation system of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Please see Figures 1-2 This invention provides an embodiment: a method for quantitatively evaluating the influence of news dissemination based on data mining, comprising the following steps: S11: Data acquisition and preprocessing, acquiring multi-source data and cleaning and fusing cross-modal features in real time through a stream processing framework; S12: Construction of indicator system and optimization of weights, based on semantic association of knowledge graph and Hawkes process modeling propagation decay, and dynamic adjustment of indicator weights using reinforcement learning; S13: Feature engineering and modeling, using neural differential equations to fit propagation curves, constructing heterogeneous graph neural networks to predict diffusion paths, and combining AutoML to automatically generate high-order features; S14: Causal reasoning, verifying influence through causal discovery and propensity score matching, and generating a natural language interpretation report based on SHAP value and GPT-3.

[0021] As described above, this invention achieves real-time acquisition and cross-modal feature fusion of multi-source heterogeneous data (text, images, videos) through a stream processing framework, constructs an indicator system based on knowledge graph semantic association and Hawkes process dynamic propagation decay, and adaptively adjusts weight allocation using a reinforcement learning mechanism. Simultaneously, it employs neural differential equations to accurately capture the nonlinear variation of the propagation curve, combines heterogeneous graph neural networks to deeply analyze the complex interaction structure between users, news, and comments, and automatically generates high-order feature combinations through AutoML technology. Furthermore, it introduces a causal discovery algorithm to identify confounding variables and applies propensity score matching to verify the net effect of news on public opinion. Finally, it quantifies feature contribution based on SHAP values ​​and generates a natural language interpretation report using GPT-3, significantly improving the comprehensiveness, timeliness, accuracy, interpretability, and scientific decision support capabilities of news dissemination influence assessment.

[0022] As a preferred method, when collecting multi-source data and cleaning and fusing cross-modal features in real time using a stream processing framework, the specific steps include: S21: Multi-source heterogeneous data crawling. It crawls text, image and video data from multiple platforms through APIs and web crawlers, and uses OpenCV to extract video keyframes and CLIP model to analyze cross-modal semantics, and deploys Apache Kafka to achieve real-time stream processing. S22: Robot detection upgrade, based on the XGBoost model to detect robot accounts, with features including posting time interval variance, follower ratio and content duplication rate, and intercepting high-probability robot data; S23: Multimodal data fusion, which performs cross-modal attention-weighted fusion of text data, image data and video data to generate a unified semantic embedding vector; S24: Real-time cleaning and storage, filtering outliers and deleting advertising links based on the 3σ principle, normalizing the data and storing it in Elasticsearch, and storing the original multimedia files in MinIO object storage.

[0023] As described above, this invention achieves real-time acquisition and cross-modal feature fusion of multi-source heterogeneous data (text, images, and videos) through a stream processing framework. It utilizes OpenCV to extract keyframes from videos and combines them with the CLIP model to analyze cross-modal semantic relationships. Apache Kafka is deployed to ensure low latency and high throughput in data processing. The XGBoost model is used to detect robot accounts and intercept abnormal data, effectively reducing noise interference. A cross-modal attention mechanism is employed to dynamically weight and fuse text, image, and video features, generating a unified semantic embedding vector to enhance data representation capabilities. Simultaneously, by filtering outliers using the 3σ principle, normalization processing, and distributed storage (Elasticsearch and MinIO), it ensures data quality while supporting efficient retrieval and backtracking, significantly improving the real-time performance, accuracy, and scalability of news dissemination data acquisition and preprocessing.

[0024] As a preferred approach, when detecting bot accounts based on the XGBoost model, features include posting time interval variance, followers, follower ratio, and content duplication rate. Specifically, when blocking high-probability bot data, the following features are included: S31: Data Preparation and Feature Extraction: Extract account behavior data from social media APIs and logs, and calculate key features of the account behavior data. These key features include posting interval variance, follower ratio, and content duplication rate. The formula for calculating the posting interval variance is as follows: ; in, This is the timestamp of the i-th post. Let N be the average interval between posts, and N be the total number of posts. S32: Model training and validation. A training set is constructed using manual annotation and rule-based pre-screening, with a positive to negative sample ratio of 1:3. The constructed training set is then used to train and validate the XGBoost model. S33: Real-time monitoring and interception. Deploy the XGBoost model to receive account behavior data in real time, output the probability of bots, set a basic threshold, and adjust the threshold in real time according to the false negative rate data. Judge the account behavior data, and intercept bot accounts if they are judged to be bot accounts.

[0025] As described above, this invention extracts account behavior data from social media APIs and logs, calculates key features such as posting time interval variance, follower ratio, and content duplication rate, and uses manual annotation and rule pre-screening to construct a training set with a positive-to-negative sample ratio of 1:3 to train and validate the XGBoost model, achieving high-precision identification of bot accounts. By monitoring account behavior data in real time and outputting bot probabilities, combined with a dynamic threshold adjustment mechanism (optimized in real time based on the false negative rate), precise interception is achieved, effectively filtering out abnormal account interference. While ensuring a high recall rate, the false positive rate is significantly reduced, ensuring the purity and reliability of the data collection stage and providing a solid data quality guarantee for subsequent analysis.

[0026] As a preferred embodiment, when performing cross-modal attention-weighted fusion of text data, image data, and video data to generate a unified semantic embedding vector, the specific steps include: S41: Feature extraction. For text data, a pre-trained BERT model is used to extract 768-dimensional sentence vectors. For image data, a ResNet-50 pre-trained model is used to extract 2048-dimensional features after global average pooling. For video data, a 3DCNN model is used to extract 512-dimensional temporal features for each video segment. S42: Modal alignment, calculates cross-modal similarity between text data, image data, and video data using the CLIP model, and filters semantically matching image-text pairs; S43: Attention weight calculation. A multi-head attention mechanism is used to calculate the dynamic attention weights for text, image, and video data. The calculation principle formula is as follows: ; Where Q is the query vector, and , For text features, Used to project text features into d-dimensional space For key vectors, , For image features, For video features, The dimension of the key vector is d. This indicates that the similarity between the query vector and the query vector is calculated. These are the dynamic attention weights for text data, image data, and video data, respectively. S44: Feature fusion, which dynamically fuses text features, image features and video features by calculating dynamic attention weights to obtain a unified embedding vector; S45: Post-processing, normalizes the unified embedding vector obtained by fusion, and stores it in the vector database.

[0027] As described above, this invention extracts textual semantic features using a pre-trained BERT model, extracts global image features using ResNet-50, and captures temporal video features using 3DCNN. It combines this with the CLIP model to achieve cross-modal similarity calculation and semantic alignment. A multi-head attention mechanism is employed to dynamically allocate weights for text, image, and video features, accurately capturing intermodal relationships and generating unified semantic embedding vectors. Through normalization processing and vector database storage, it achieves efficient fusion and semantically consistent representation of multimodal data, significantly improving cross-modal information complementarity, solving the semantic gap problem in heterogeneous data fusion, and providing more comprehensive and accurate semantic representation support for subsequent propagation analysis.

[0028] As a preferred approach, when modeling propagation decay based on semantic associations in knowledge graphs and the Hawkes process, and dynamically adjusting indicator weights using reinforcement learning, the specific methods include: S51: Indicator expansion and semantic association, constructing a knowledge graph of news and policy entities, calculating semantic path similarity between entities through the TransR model, and expanding the deep association between indicators; S52: Dynamic decay model construction, using Hawkes process to model the self-excitation effect of propagation, and using learnable parameters to dynamically fit the time decay characteristics of propagation events; S53: Adaptive weight allocation. Design a reinforcement learning model based on DQN, using historical dissemination effects as environmental feedback to dynamically optimize the weight allocation of dissemination breadth, depth, and social impact.

[0029] As described above, this invention constructs a knowledge graph of news and policy entities and uses the TransR model to calculate the semantic path similarity between entities, thereby deeply mining the implicit associations between indicators. It employs the Hawkes process to accurately model the self-excitation effect and time decay characteristics of dissemination events, achieving dynamic capture of dissemination patterns. It designs a reinforcement learning model based on DQN, using historical dissemination effects as feedback signals to adaptively optimize the weight allocation of dissemination breadth, depth, and social influence, overcoming the limitations of traditional static weights and significantly improving the dynamic adaptability and interpretability of the indicator system. This provides a scientific, accurate, and adaptive quantitative analysis framework for influence assessment in complex dissemination scenarios.

[0030] Preferably, when modeling the self-excitation effect of propagation using the Hawkes process and dynamically fitting the time decay characteristics of the propagation event using learnable parameters, the specific methods include: S61: Data preparation and event sequence definition, collect timestamp data of news dissemination events, construct time series by news ID, and record the occurrence time of each dissemination behavior; S62: Model parameter initialization, initialize the base propagation rate, trigger strength, and decay rate as trainable parameters, and set the optimizer to prepare for learning; S63: Model training and parameter optimization, which involves iteratively updating parameters by maximizing the log-likelihood function and using gradient descent. S64: Calculation and application of decay weights, generating normalized decay weights based on the trained model, and dynamically adjusting the influence score at each time step. S65: Model validation and tuning, calculating RMSE to evaluate fitting accuracy, analyzing parameter sensitivity, and segmenting and modeling for long-cycle news.

[0031] As described above, this invention collects timestamps of news dissemination events and constructs time series by news ID. It initializes the basic propagation rate, trigger strength, and decay rate of the Hawkes process as trainable parameters, and uses gradient descent to maximize the log-likelihood function to iteratively optimize the parameters, accurately capturing the self-excitation effect and time decay law of the dissemination event. Based on the trained model, it generates normalized decay weights to dynamically adjust the influence score at each moment, and evaluates the fitting accuracy and analyzes the parameter sensitivity through RMSE. For long-cycle news, it implements segmented modeling optimization, significantly improving the accuracy and timeliness of the propagation curve fitting. It effectively solves the problem that traditional fixed decay models cannot adapt to heterogeneous propagation modes, providing an adaptive and high-precision quantitative analysis tool for propagation dynamics modeling.

[0032] As a preferred approach, when designing a reinforcement learning model based on DQN, using historical dissemination effects as environmental feedback to dynamically optimize the weight allocation of dissemination breadth, depth, and social impact, the specific steps include: S71: Reinforcement learning environment design, defining state actions and reward functions, and constructing a dynamic decision-making environment; S72: DQN model training, designing a dual hidden layer network, learning the Q-value mapping relationship between state and action through experience replay and temporal difference error minimization; S73: Online weight adjustment and verification, real-time reasoning to select the optimal weight adjustment action, and verification of the propagation effect through A / B testing.

[0033] As described above, this invention designs a reinforcement learning environment that uses the breadth, depth, and social impact of dissemination as action dimensions, constructs a reward function based on historical dissemination effect indicators, and forms a dynamic decision-making framework. It employs a dual-hidden-layer neural network combined with an experience replay mechanism and temporal differential error optimization to efficiently learn the Q-value mapping relationship between state and action, accurately capturing the complex interactive effects between dissemination indicators. Through online inference, it adjusts the weight allocation strategy in real time and uses A / B testing to verify the optimization effect, significantly improving the dynamic adaptability and scientific nature of weight allocation. This overcomes the limitations of traditional static weights and provides adaptive, data-driven intelligent decision support for indicator optimization in complex dissemination scenarios.

[0034] Preferably, when using neural differential equations to fit the propagation curve, constructing a heterogeneous graph neural network to predict the diffusion path, and combining this with AutoML to automatically generate high-order features, the specific steps include: S81: The propagation curve is fitted using neural differential equations. The propagation curve is dynamically fitted using neural differential equations, and the nonlinear changes in the propagation rate are automatically captured by the neural network. S82: Heterogeneous graph neural network modeling, constructing a heterogeneous graph of users-news-comments, using a relational graph convolutional network to aggregate multiple types of node relationships, learn complex interaction structures and predict key propagation paths; S83: AutoML-driven feature generation automatically searches for feature combinations through TPOT to generate high-order interactive features.

[0035] As described above, this invention dynamically fits the propagation curve using neural differential equations, automatically captures the nonlinear variation characteristics of the propagation rate using neural networks, and overcomes the limitations of traditional fixed models in representing complex propagation patterns. It constructs a heterogeneous user-news-comment graph and uses a relational graph convolutional network to aggregate multi-type node interaction relationships, accurately predicting key propagation paths and deeply analyzing complex propagation network structures. Combined with AutoML technology, it automatically searches for high-order feature combinations, significantly improving feature engineering efficiency and model prediction capabilities. This forms a complete solution from dynamic propagation modeling to complex network analysis and automated feature generation, effectively enhancing the accuracy and interpretability of propagation influence assessment.

[0036] As a preferred method, when verifying influence through causal discovery and propensity score matching, and generating a natural language interpretation report based on SHAP values ​​and GPT-3, the specific steps include: S91: Robust causal analysis, using PC algorithm to identify confounding variables, and applying propensity score matching to estimate the net effect of news on public opinion; S92: Natural language interpretation generation, extracting feature contribution based on SHAP value, and generating readable conclusions by combining GPT-3; S93: Contextualized time decay, dynamically selects decay function, and corrects influence score to adapt to the timeliness patterns of different news types.

[0037] As described above, this invention effectively isolates the causal effects between news dissemination and public opinion changes by identifying confounding variables using PC algorithms and applying propensity score matching, eliminating interference from external factors such as holidays, and accurately quantifying the net impact of news on public opinion. Based on SHAP values, it quantifies the contribution of each feature and generates a natural language interpretation report using GPT-3, presenting the model's decision-making basis with highly readable and logically clear conclusions, significantly improving the interpretability and credibility of the analysis results. It dynamically selects scenario-based time decay functions (such as exponential decay for breaking news and polynomial decay for policy news) to adapt to the timeliness patterns of different news types, further enhancing the accuracy and timeliness adaptability of influence assessment, forming a scientific evaluation system integrating causal verification, transparent explanation, and dynamic adaptation.

[0038] A data mining-based system for quantifying the impact of news dissemination includes: The data acquisition and preprocessing module is used to acquire multi-source data in real time, clean up anomalies, and fuse cross-modal features. The dynamic indicator management module is used to build a dynamic indicator system and optimize weight allocation through reinforcement learning; The intelligent modeling and analysis module is used to generate high-order features using neural differential equations, heterogeneous graph networks, and AutoML to achieve propagation path prediction and nonlinear modeling. The causal inference and explanation module is used to verify the causal effect of news on public opinion, generate an explainable report, and dynamically adjust the timeliness decay strategy. The storage and retrieval module is used to efficiently store structured data and multimodal vectors, and supports fast retrieval and similarity analysis.

[0039] As described above, this invention uses a data acquisition and preprocessing module to capture multi-source heterogeneous data (text, images, and videos) in real time, and utilizes cross-modal feature fusion technology to build a high-quality data foundation. A dynamic indicator management module combines knowledge graphs and Hawkes processes to model propagation patterns, and dynamically optimizes indicator weights through reinforcement learning, significantly improving the adaptive capability of the evaluation system. An intelligent modeling and analysis module uses neural differential equations to capture nonlinear propagation characteristics, leverages heterogeneous graph neural networks to analyze complex propagation paths, and automatically generates high-order interaction features through AutoML, achieving accurate propagation prediction and in-depth analysis. A causal inference and explanation module verifies the net effect of news on public opinion through causal discovery technology, combines SHAP values ​​and GPT-3 to generate transparent and interpretable reports, and dynamically adjusts timeliness decay strategies to adapt to different scenarios. A storage and retrieval module efficiently manages structured data and multimodal vectors, supporting rapid similarity retrieval and retrospective analysis. The system as a whole forms a comprehensive quantitative assessment solution for the influence of news dissemination, integrating data fusion, dynamic modeling, causal verification, intelligent analysis and efficient storage. It significantly improves the comprehensiveness, accuracy, interpretability and real-time performance of the assessment, and provides scientific support for public opinion management, policy making and business decision-making.

[0040] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for quantitatively evaluating the influence of news dissemination based on data mining; characterized in that: Includes the following steps: S11: Data Acquisition and Preprocessing. This involves acquiring multi-source data and cleaning and fusing cross-modal features in real time using a stream processing framework. Specifically, this includes: S21: Multi-source heterogeneous data crawling. It crawls text, image and video data from multiple platforms through APIs and web crawlers, and uses OpenCV to extract video keyframes and CLIP model to analyze cross-modal semantics, and deploys Apache Kafka to achieve real-time stream processing. S22: Robot detection upgrade, based on the XGBoost model to detect robot accounts, with features including posting time interval variance, follower ratio and content duplication rate, and intercepting high-probability robot data; S23: Multimodal data fusion, which performs cross-modal attention-weighted fusion of text data, image data and video data to generate a unified semantic embedding vector; S24: Real-time cleaning and storage, filtering outliers and deleting advertising links based on the 3σ principle, normalizing the data and storing it in Elasticsearch, and storing the original multimedia files in MinIO object storage. S12: Construction of indicator system and optimization of weights, based on semantic association of knowledge graph and Hawkes process modeling propagation decay, and dynamic adjustment of indicator weights using reinforcement learning; S13: Feature engineering and modeling, using neural differential equations to fit propagation curves, constructing heterogeneous graph neural networks to predict diffusion paths, and combining AutoML to automatically generate high-order features; S14: Causal reasoning, verifying influence through causal discovery and propensity score matching, and generating a natural language interpretation report based on SHAP value and GPT-3.

2. The method for quantitatively evaluating the influence of news dissemination based on data mining as described in claim 1, characterized in that: When detecting bot accounts based on the XGBoost model, features include posting time interval variance, followers, follower ratio, and content duplication rate. Specifically, when blocking high-probability bot data, the following features are included: S31: Data Preparation and Feature Extraction: Extract account behavior data from social media APIs and logs, and calculate key features of the account behavior data, including posting time interval variance, follower ratio, and content duplication rate. S32: Model training and validation. A training set is constructed using manual annotation and rule-based pre-screening, with a positive to negative sample ratio of 1:

3. The constructed training set is then used to train and validate the XGBoost model. S33: Real-time monitoring and interception. Deploy the XGBoost model to receive account behavior data in real time, output the probability of bots, set a basic threshold, and adjust the threshold in real time according to the false negative rate data. Judge the account behavior data, and intercept bot accounts if they are judged to be bot accounts.

3. The method for quantitatively evaluating the influence of news dissemination based on data mining according to claim 2, characterized in that: In the data preparation and feature extraction steps, the formula for calculating the variance of the posting time interval is: ; in, This is the timestamp of the i-th post. denoted as the average posting interval, and N as the total number of posts.

4. The method for quantitatively evaluating the influence of news dissemination based on data mining according to claim 3, characterized in that: When performing cross-modal attention-weighted fusion of text, image, and video data to generate a unified semantic embedding vector, the specific steps include: S41: Feature extraction. For text data, a pre-trained BERT model is used to extract 768-dimensional sentence vectors. For image data, a ResNet-50 pre-trained model is used to extract 2048-dimensional features after global average pooling. For video data, a 3DCNN model is used to extract 512-dimensional temporal features for each video segment. S42: Modal alignment, calculates cross-modal similarity between text data, image data, and video data using the CLIP model, and filters semantically matching image-text pairs; S43: Attention weight calculation. A multi-head attention mechanism is used to calculate the dynamic attention weights for text, image, and video data. The calculation principle formula is as follows: ; Where Q is the query vector, and , For text features, Used to project text features into d-dimensional space For key vectors, , For image features, For video features, The dimension of the key vector is d. This indicates that the similarity between the query vector and the query vector is calculated. These are the dynamic attention weights for text data, image data, and video data, respectively. S44: Feature fusion, which dynamically fuses text features, image features and video features by calculating dynamic attention weights to obtain a unified embedding vector; S45: Post-processing, normalizes the unified embedding vector obtained by fusion, and stores it in the vector database.

5. The method for quantitatively evaluating the influence of news dissemination based on data mining according to claim 4, characterized in that: When modeling propagation decay based on semantic associations in knowledge graphs and the Hawkes process, and dynamically adjusting indicator weights using reinforcement learning, the specific steps include: S51: Indicator expansion and semantic association, constructing a knowledge graph of news and policy entities, calculating semantic path similarity between entities through the TransR model, and expanding the deep association between indicators; S52: Dynamic decay model construction, using Hawkes process to model the self-excitation effect of propagation, and using learnable parameters to dynamically fit the time decay characteristics of propagation events; S53: Adaptive weight allocation. Design a reinforcement learning model based on DQN, using historical dissemination effects as environmental feedback to dynamically optimize the weight allocation of dissemination breadth, depth, and social impact.

6. The method for quantitatively evaluating the influence of news dissemination based on data mining according to claim 5, characterized in that: When modeling the self-excitation effect of propagation using Hawkes processes and dynamically fitting the time decay characteristics of propagation events using learnable parameters, the specific steps include: S61: Data preparation and event sequence definition, collect timestamp data of news dissemination events, construct time series by news ID, and record the occurrence time of each dissemination behavior; S62: Model parameter initialization, initialize the base propagation rate, trigger strength, and decay rate as trainable parameters, and set the optimizer to prepare for learning; S63: Model training and parameter optimization, which involves iteratively updating parameters by maximizing the log-likelihood function and using gradient descent. S64: Calculation and application of decay weights, generating normalized decay weights based on the trained model, and dynamically adjusting the influence score at each time step. S65: Model validation and tuning, calculating RMSE to evaluate fitting accuracy, analyzing parameter sensitivity, and segmenting and modeling for long-cycle news.

7. The method for quantitatively evaluating the influence of news dissemination based on data mining as described in claim 6, characterized in that: When designing a reinforcement learning model based on DQN, using historical dissemination effects as environmental feedback to dynamically optimize the weight allocation of dissemination breadth, depth, and social impact, the specific steps include: S71: Reinforcement learning environment design, defining state actions and reward functions, and constructing a dynamic decision-making environment; S72: DQN model training, designing a dual hidden layer network, learning the Q-value mapping relationship between state and action through experience replay and temporal difference error minimization; S73: Online weight adjustment and verification, real-time reasoning to select the optimal weight adjustment action, and verification of the propagation effect through A / B testing.

8. The method for quantitatively evaluating the influence of news dissemination based on data mining according to claim 7, characterized in that: When using neural differential equations to fit propagation curves, constructing heterogeneous graph neural networks to predict diffusion paths, and combining this with AutoML to automatically generate high-order features, the specific steps include: S81: The propagation curve is fitted using neural differential equations. The propagation curve is dynamically fitted using neural differential equations, and the nonlinear changes in the propagation rate are automatically captured by the neural network. S82: Heterogeneous graph neural network modeling, constructing a heterogeneous graph of users-news-comments, using a relational graph convolutional network to aggregate multiple types of node relationships, learn complex interaction structures and predict key propagation paths; S83: AutoML-driven feature generation, which automatically searches for feature combinations through TPOT to generate high-order interactive features.

9. The method for quantitatively evaluating the influence of news dissemination based on data mining according to claim 8, characterized in that: When validating influence through causal discovery and propensity score matching, and generating natural language interpretation reports based on SHAP values ​​and GPT-3, the specific steps include: S91: Robust causal analysis, using PC algorithm to identify confounding variables, and applying propensity score matching to estimate the net effect of news on public opinion; S92: Natural language interpretation generation, extracting feature contribution based on SHAP value, and generating readable conclusions by combining GPT-3; S93: Contextualized time decay, dynamically selects decay function, and corrects influence score to adapt to the timeliness patterns of different news types.

10. A data mining-based system for quantitatively evaluating the influence of news dissemination, characterized in that: include: The data acquisition and preprocessing module is used to acquire multi-source data in real time, clean up anomalies, and fuse cross-modal features. The dynamic indicator management module is used to build a dynamic indicator system and optimize weight allocation through reinforcement learning; The intelligent modeling and analysis module is used to generate high-order features using neural differential equations, heterogeneous graph networks, and AutoML to achieve propagation path prediction and nonlinear modeling. The causal inference and explanation module is used to verify the causal effect of news on public opinion, generate an explainable report, and dynamically adjust the timeliness decay strategy. The storage and retrieval module is used to efficiently store structured data and multimodal vectors, and supports fast retrieval and similarity analysis.

Images