Public opinion event multi-mode semantic fusion modeling and abstract generation method and system

Abstract

The invention discloses a public opinion event multi-mode semantic fusion modeling and abstract generation method and system, and relates to the field of natural language processing and social network analysis. Through the multi-mode semantic fusion technology, the short text understanding ability is improved, and the problems of semantic fuzziness and network language diversification are solved. Meanwhile, through a cross-window event cluster matching technology, an event evolution path with time continuity is constructed, and comprehensive capture of event dynamic characteristics is realized. Besides, the structured event abstract is automatically generated by utilizing the generative model, so that the consistency and the information density of the abstract are improved, and the actual application requirements are met. Through the innovations, the defects in the aspects of semantic comprehension, dynamic modeling and abstract generation in the prior art can be effectively overcome, a more efficient and accurate solution is provided for monitoring and analysis of public opinion events, and the method has wide application prospects in the fields of public opinion monitoring, emergency early warning, social media data analysis and the like.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and social network analysis, specifically to a method and system for multimodal semantic fusion modeling and summarization of public opinion events. Background Technology

[0002] In recent years, with the rapid development of social media, the monitoring and analysis of public opinion events has become an important requirement in fields such as information management, public safety, and business decision-making. However, the dynamic and complex nature of public opinion events poses new challenges to existing technologies. Traditional methods mainly rely on keyword detection and static analysis, which have significant shortcomings in capturing the deep semantic relationships and dynamic evolution paths of events. For example, keyword detection methods identify events by statistically analyzing the frequency of keyword occurrences, but this method ignores the contextual relationships of the text, making it impossible to accurately track the evolution of events. Furthermore, static analysis methods and independent time window processing are difficult to adapt to the dynamic characteristics of events and cannot effectively identify early warning signals or predict subsequent development trajectories.

[0003] Existing technologies attempt to address problems in public opinion event detection and analysis using methods such as deep learning and graph clustering. For example, a social media event detection method combining deep learning classification and graph clustering (patent number: CN202410373064.X) constructs heterogeneous message graphs and multi-relationship message graphs, uses a deep learning classification model to determine the similarity of message pairs, and performs event detection using a graph clustering algorithm. However, this method relies on message associations within a fixed time window and does not design a dynamic matching mechanism across time windows, leading to broken event evolution paths. For example, when the focus of discussion on the same event shifts in different time windows (e.g., from "epidemic lockdown" to "economic impact"), static graph clustering cannot establish cross-window semantic associations. Furthermore, semantic modeling relies solely on text similarity calculations, without integrating lexical co-occurrence statistics and deep semantic features, resulting in insufficient robustness to semantic ambiguity in short texts (e.g., the contextual difference between "health code" and "travel code"). Furthermore, a deep learning-based method and system for identifying public opinion information (patent number: CN202410240298.7) learns and extracts features from public opinion data using a Transformer model, and combines time-series features for trend prediction. While this method improves semantic understanding and temporal correlation, it still falls short in adapting to the fragmented nature of short social media texts and in modeling dynamic evolution paths, making it difficult to fully capture the dynamic characteristics of events. Regarding summary generation, a long text event extraction method and system based on summary generation (patent number: CN202411856361.6) improves the accuracy of long text event extraction by generating summary content from long texts and extracting events based on the summary content. However, this method mainly targets event extraction from long texts, has limited processing capabilities for short texts and dynamic public opinion events, and lacks modeling of event evolution paths.

[0004] Further analysis of existing technologies reveals the following shortcomings: First, insufficient semantic fusion. Existing methods struggle to effectively integrate lexical features and deep semantic information when processing short texts, resulting in insufficient capacity to handle semantic ambiguity and the diversity of online language. Second, limited dynamic modeling capabilities. Existing methods primarily focus on static features or analysis within a single time window, lacking the ability to model the dynamic evolution path of events and failing to track the complete development process of events. Third, limitations in summary generation. Existing summary generation methods often rely on manually generated rules or fail to optimize the generation model for public opinion scenarios, leading to information omissions or semantic distortions, making it difficult to meet the application needs of public opinion event analysis. Finally, insufficient cross-window event correlation. When processing event fragments across time windows, existing methods easily misclassify different stages of the same event as independent events, resulting in the omission of key information. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method and system for multimodal semantic fusion modeling and summary generation of public opinion events, enabling dynamic modeling, cross-window evolution tracking, and high-quality summary generation of public opinion events in short text scenarios.

[0006] The technical solution of this invention is as follows:

[0007] On the one hand, this invention provides a method for multimodal semantic fusion modeling and summary generation of public opinion events, including the following steps:

[0008] Acquire blog post data from social media platforms, preprocess the blog post data, and then construct an initial public opinion time series dataset;

[0009] Multimodal semantic fusion is performed on blog post data within each time window of the initial public opinion time series dataset to generate a unified semantic representation, and then a fusion matrix is ​​obtained.

[0010] Based on the fusion matrix, dynamic multi-level clustering is performed on blog post data within each time window of the initial public opinion time series dataset to generate event clusters;

[0011] Based on cluster sets in different time windows, a global event evolution graph is constructed using cross-window event cluster matching technology, which includes the evolution path of events;

[0012] Based on the event evolution path, a summary of the public opinion event is generated;

[0013] Store the event evolution path and the generated summary in a structured form.

[0014] Furthermore, the acquisition of blog post data from social media platforms, and the preprocessing of this data to construct an initial public opinion time-series dataset, specifically includes:

[0015] A1: Obtain several blog post data from social media platforms using web scraping tools. Each blog post data includes text content, author ID, publication time, and interaction data.

[0016] A2: Clean the original blog post data to obtain preprocessed blog post data;

[0017] Specifically, a three-layer filtering mechanism is used to clean the original blog post data, including rule-based basic filtering, supervised learning-based semantic filtering, and filtering based on social relationship networks and user behavior analysis.

[0018] The first layer is rule-based basic filtering: a predefined list of stop words and regular expression rules are used to perform preliminary filtering of blog post data based on the text content of each blog post.

[0019] The second layer is semantic filtering based on supervised learning: a supervised learning model is introduced to evaluate the event relevance of the text content of blog posts, and further filter out blog posts that are relevant to the event.

[0020] The third layer of filtering is based on social relationship networks and user behavior analysis: combining social relationship network analysis and user behavior pattern recognition technology, it identifies and removes automatically generated blog post data and blog post data published by malicious accounts;

[0021] A3: The preprocessed blog post data is divided into continuous time windows with a fixed time granularity using the sliding window technique. The blog post data in each time window is merged as a blog post dataset and sorted by time to form an initial public opinion time series dataset including several blog post data sets.

[0022] Furthermore, the step of performing multimodal semantic fusion on blog post data within each time window of the initial public opinion time series dataset to generate a unified semantic representation, and then obtaining a fusion matrix, specifically includes:

[0023] B1: Extract named entities with event indication significance from each blog post data, and then construct a set of named entities;

[0024] For a set of blog post data within a time window , This indicates the ID of the blog post dataset. This represents blog post data. This indicates the blog post number. This indicates the number of blog posts in the blog post dataset; named entity recognition is performed using a pre-trained BERT model, and its annotation process is defined as follows:

[0025] (1);

[0026] in, Represents a collection of named entities. This represents the vocabulary units in the blog post data. Indicates the entity type corresponding to the lexical unit. For a predefined collection of entity types, Indicates a person's name. Indicates a place name. Indicates the organization name. Indicates time;

[0027] B2: Extract topic tags from blog post data using regular expressions, form a topic tag set, and merge it with the named entity set to form a candidate entity set;

[0028] The candidate entity set is represented as follows:

[0029] (2);

[0030] in, Represents a collection of blog posts The corresponding set of candidate entities, Represents blog post data The corresponding set of topic tags, and , This refers to topic tags in blog post data. Indicates the hashtag number;

[0031] B3: Construct an entity distribution weight matrix based on the candidate entity set. ,in Name the total number of entities in the candidate entity set. For blog post data collection Number of blog posts;

[0032] The elements in the entity distribution weight matrix represent the weighted frequency of named entities in the blog post data:

[0033] (3);

[0034] in, Represents named entities, Indicates the number of the named entity. The elements in the entity distribution weight matrix represent named entities. In blog data Weighted frequency of occurrence in; Named entities In blog data word frequency in Inverse document frequency, For containing named entities The number of blog posts;

[0035] B4: For each named entity in the candidate entity set, construct its associated blog post set, and sample from it. Use the sampled blog post data to construct the context sample set for that named entity. ;

[0036] Specifically: First, for each named entity in the candidate entity set... A collection of related blog posts can be constructed using the blog post data that appears in the named entity. Sampled from related blog posts Building a contextual sample set from blog post data When sampling, both influence and diversity were considered. On the one hand, blog posts with high dissemination volume were selected to capture mainstream semantics, that is, the top posts were selected in descending order of repost volume. / 2 blog post data, on the other hand, randomly select from the remaining blog post data. / 2 items;

[0037] B5: For context sample sets Each sampled blog post data The BERT model and FastText model are used to generate sentence vectors and word vectors respectively, and then weighted and fused to obtain the comprehensive contextual semantic vector of each named entity.

[0038] The BERT model is used to capture the overall semantics of blog post data, resulting in the following sentence vectors:

[0039] (4);

[0040] in, Represents blog post data The sentence vector; Represents the BERT model;

[0041] Fine-grained semantics of named entity-related words are extracted using the FastText model, resulting in the following word vectors:

[0042] (5);

[0043] in, Representing word vectors, Represents blog post data The collection of all words contained therein Representing words, This represents the word vector representation of the FastText model's output of vocabulary;

[0044] Based on sentence vectors and word vectors, a comprehensive contextual semantic vector for each named entity is generated through metadata-weighted multimodal fusion:

[0045] (6);

[0046] in, Represents named entities The comprehensive contextual semantic vector, Represents the context sample set The number of blog posts in China and These are the weighting coefficients. and These represent blog post data. The number of reposts and likes, Indicates a splicing operation;

[0047] B6: Construct a context semantic matrix based on the comprehensive context semantic vector of each named entity. The row vectors are the comprehensive contextual semantic vectors for each named entity;

[0048] B7: The design is based on a cosine distance fusion mechanism, which constructs a co-occurrence distribution distance matrix and a semantic space distance matrix according to the entity distribution weight matrix and the context semantic matrix, respectively;

[0049] Co-occurrence distribution distance matrix:

[0050] (7);

[0051] in, , Indicates the number of the named entity; , This represents the weighted co-occurrence distribution vector of the named entity in the blog post data, including the weighted number of times the named entity appears in multiple blog post data. The elements in the co-occurrence distribution distance matrix represent the distance between two named entities in the lexical co-occurrence space;

[0052] Semantic space distance matrix:

[0053] (8);

[0054] in, , , Represents a comprehensive contextual semantic vector. Represents the vector normalization function. The elements in the semantic space distance matrix represent the semantic space distance between named entities;

[0055] B8: The co-occurrence distribution distance matrix and the semantic space distance matrix are fused into a unified distance matrix by linear combination, resulting in the fusion matrix;

[0056] (9);

[0057] in, Represents the fusion matrix. Represents the co-occurrence distribution distance matrix. Represents the semantic space distance matrix; For weight fusion.

[0058] Furthermore, based on the fusion matrix, dynamic multi-level clustering is performed on the blog post data within each time window of the initial public opinion time series dataset to generate event clusters. Specifically, this involves: firstly, based on the fusion matrix... Construct a weighted average Nearest neighbor graph The weighted average represents the number of nearest neighbors. The nearest neighbor graph is an undirected weighted graph consisting of nodes and edges, with each node corresponding to a named entity. An edge represents the proximity relationship between two named entities. When two named entities are close neighbors, an undirected edge is established between the nodes corresponding to these two named entities. Then, the core distance and reachability distance of each named entity are calculated. Based on the core distance and reachability distance of the named entities, a hierarchical density tree is constructed. Clusters are automatically identified through a dynamic density threshold criterion. For each cluster, its intra-cluster cohesion is calculated to quantify the popularity level of the event. Finally, the cluster set corresponding to each blog post dataset is output. , A cluster is a collection of semantically related named entities and their associated blog post data within a time window. The number of clusters.

[0059] Furthermore, the construction of a global event evolution graph based on cluster sets from different time windows, using cross-window event cluster matching technology, specifically includes:

[0060] C1: Based on the cluster sets within adjacent time windows, perform association modeling to construct a bipartite graph network structure with weighted features;

[0061] Let the cluster sets in the previous time window and the current time window be represented as follows:

[0062] (10);

[0063] in, This represents the set of clusters within the previous time window. This represents the set of clusters within the current time window. Indicates time, Indicates a cluster, This indicates the number of clusters in the cluster set within the previous time window. This indicates the number of clusters in the cluster set within the current time window;

[0064] Construct a bipartite graph network structure based on the above cluster set:

[0065] (11);

[0066] in, Represents the bipartite graph network structure. This represents the cross-window cluster association weight matrix, and the elements in the cross-window cluster association weight matrix are... Indicates the cluster in the previous time window Clusters in the current time window The strength of the correlation between them;

[0067] Elements in the cross-window inter-cluster association weight matrix The calculation uses a linear weighted fusion mechanism, and the specific calculation formula is as follows:

[0068] (12);

[0069] in, As a regulating factor; This represents the improved weighted Jaccard coefficient; This represents a measure of semantic consistency.

[0070] The improved formula for calculating the weighted Jaccard coefficient is as follows:

[0071] (13);

[0072] in, , These represent the sets of named entities in the two clusters, Represents named entities, The constructed entity weight evaluation model is as follows:

[0073] (14);

[0074] in, Represents named entities Word frequencies in the blog post dataset within the current time window. Represents named entities Inverse document frequency, Indicates the presence of named entities The total popularity of blog posts; Represents the normalization function. Indicates the heat adjustment coefficient; The decay function represents the time-varying decay, expressed in exponential decay form. , The time interval between the two clusters within their respective time windows. The attenuation coefficient;

[0075] The calculation process for the semantic consistency metric is as follows:

[0076] First, construct the multimodal cluster center vector: for any cluster within a time window... Calculate its multimodal cluster center vector This vector is the weighted average of the multimodal feature vectors of all blog posts within the cluster, used to represent the core semantic content of the cluster. The calculation formula is as follows:

[0077] (15);

[0078] in, Cluster The multimodal cluster center vector, Cluster The number of blog post data included. This represents a single blog post within a cluster; blog post The sentence vector, blog post Word vectors, This indicates a feature fusion operation. and These are the modal weighting coefficients;

[0079] Obtaining the center vector of the multimodal cluster Then, calculate the clusters within the previous time window. Clusters within the current time window The semantic consistency measure between them is calculated using the following formula:

[0080] (16);

[0081] in, and Clusters and cluster The multimodal cluster center vector;

[0082] C2: Based on the bipartite graph network structure with weighted features, an improved bipartite graph optimization matching algorithm is used to perform cluster pair matching to obtain matching pairs, which include two clusters;

[0083] C3: After cluster pair matching, the global event evolution graph is updated based on the cluster pair matching results, and the event evolution path is obtained by combining the dynamic maintenance mechanism. The event evolution path is a path composed of nodes and associated edges.

[0084] Furthermore, C2 specifically includes:

[0085] C2.1: Traversing the cross-window inter-cluster association weight matrix all elements ,like Less than the preset effective association threshold If the element is set to 0, the bipartite graph network is transformed into a sparse bipartite graph network with weighted features.

[0086] C2.2: A solver based on maximum weight matching is used to solve the sparse bipartite graph network with weight features to obtain matching pairs. Each cluster in the matching pair is used as a node, and each node has an event ID. At the same time, an association edge is established between the two nodes to represent the evolution process, thereby obtaining the global event evolution graph.

[0087] During the solution process, dual constraints are set. Specifically, a one-to-one uniqueness constraint is used to ensure that a cluster in the previous time window can establish a relationship with at most one cluster in the current time window. A maximum weight sum constraint is used to find the weights that maximize all matching pairs while satisfying the uniqueness constraint. The combination that maximizes the sum;

[0088] C2.3: For clusters within the current time window that still fail to match after solving C2.2, calculate their weights relative to clusters within all historical time windows. If the set branch threshold is reached, it is determined as a branch node of an existing event, and an association edge is established between the cluster in the current time window and the clusters in the historical time windows; if the weight between the cluster in the current time window and all clusters in the historical time windows is... If all events are below the set branch threshold, they are determined to be independent events, added to the global event evolution graph as initial sections, and assigned new event IDs as the starting point of a new evolution path, thus obtaining the updated global event evolution graph.

[0089] Furthermore, the generation of a summary of public opinion events based on the event evolution path specifically includes:

[0090] D1: For the set of blog post data associated in the event evolution path, a semantically aware key phrase extraction method is adopted, and key phrase units are captured through a sliding window mechanism;

[0091] Specifically, the method involves using a sliding window technique to perform local semantic scanning centered on named entities in blog post data, and extracting key phrase units containing the named entity and its modifiers.

[0092] D2: Construct a phrase-blog post mapping dataset by establishing a mapping relationship between the original blog post data and its extracted key phrase units;

[0093] D3: DeepSeek-V3 is used as the basic generative model and fine-tuned based on the phrase-blog post mapping dataset to obtain the fine-tuned basic generative model;

[0094] D4: First, the key phrase units extracted in D1 are semantically encoded using the BERT model to obtain phrase vectors. Then, the HDBSCAN density clustering algorithm is used to perform cluster analysis on the phrase vectors to generate semantic clusters.

[0095] D5: In each semantic cluster, the key phrase unit with the highest cosine similarity is selected as the representative semantic unit. Then, the representative semantic units arranged in time sequence are input into the fine-tuned basic generative model to form a coherent text and obtain a summary.

[0096] D6: Establish a multi-dimensional evaluation system, evaluate the generated summaries, and adjust the basic generation model in real time based on the evaluation results.

[0097] On the other hand, the present invention also provides a system for multimodal semantic fusion modeling and summarizing of public opinion events, used to implement a method for multimodal semantic fusion modeling and summarizing of public opinion events, including:

[0098] The dataset construction module is used to acquire blog post data from social media platforms, preprocess the blog post data, and then construct the initial public opinion time series dataset.

[0099] The multimodal semantic fusion module is used to perform multimodal semantic fusion on blog post data within each time window in the initial public opinion time series dataset, generate a unified semantic representation, and then obtain a fusion matrix;

[0100] The clustering module is used to perform dynamic multi-level clustering of blog post data within each time window in the initial public opinion time series dataset based on the fusion matrix, generating event clusters.

[0101] The evolution module is used to construct a global event evolution graph, including the evolution path of events, based on cluster sets of different time windows and cross-window event cluster matching technology.

[0102] The summary generation module generates summaries of public opinion events based on the event evolution path;

[0103] The storage module is used to store the event evolution path and the generated summary in a structured form.

[0104] Thirdly, this application proposes an electronic device, including: one or more processors, and a memory, the memory being used to store instructions, which, when executed by the one or more processors, cause the one or more processors to execute the aforementioned public opinion event multimodal semantic fusion modeling and summarization generation method.

[0105] Fourthly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the aforementioned method for multimodal semantic fusion modeling and summarization of public opinion events.

[0106] Fifthly, this application proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned method for multimodal semantic fusion modeling and summarization of public opinion events.

[0107] The beneficial effects of adopting the above technical solution are as follows:

[0108] This invention enhances the understanding of short texts through multimodal semantic fusion technology, addressing the issues of semantic ambiguity and the diversity of online language. Simultaneously, it constructs a temporally continuous event evolution path through cross-window event cluster matching technology, achieving comprehensive capture of the dynamic characteristics of events. Furthermore, this invention utilizes a generative model to automatically generate structured event summaries, improving the coherence and information density of the summaries to meet practical application needs. Through these innovations, this invention effectively addresses the shortcomings of existing technologies in semantic understanding, dynamic modeling, and summarization generation, providing a more efficient and accurate solution for monitoring and analyzing public opinion events, and has broad application prospects in fields such as public opinion monitoring, emergency warning, and social media data analysis. Attached Figure Description

[0109] Figure 1 This is a general framework diagram of the multimodal semantic fusion modeling and summary generation method for public opinion events in this embodiment of the invention. Detailed Implementation

[0110] To facilitate understanding of the present invention, specific embodiments are described in further detail below with reference to the accompanying drawings and examples. These embodiments are illustrative of the invention but are not intended to limit its scope. Rather, they are provided to provide a more thorough and complete understanding of the disclosure of the present invention.

[0111] In this embodiment, due to the highly dynamic and complex nature of public opinion event data on social media platforms, traditional keyword detection and static analysis methods are insufficient to effectively capture the deep semantic relationships and dynamic evolution paths of events. Therefore, this invention combines lexical features and contextual semantic information to track the dynamic development process of events in real time and generate high-quality event summaries.

[0112] The principle of this invention includes: extracting and aligning semantic representations from multimodal data, constructing a unified distance metric based on the fusion of lexical co-occurrence and deep contextual semantics, obtaining event evolution chains through dynamic clustering and cross-window matching on time series data, and finally generating structured summaries.

[0113] This embodiment proposes a method for multimodal semantic fusion modeling and summarization generation of public opinion events, such as... Figure 1 As shown, it includes the following steps:

[0114] Step 1: Obtain blog post data from social media platforms, preprocess the blog post data, and then construct an initial public opinion time series dataset;

[0115] In a specific embodiment of the present invention, a systematic method for collecting and preprocessing blog post data on social media platforms is proposed, providing high-quality data support for the subsequent analysis of public opinion events.

[0116] The specific implementation steps are as follows:

[0117] Step 1.1: Use web scraping tools to obtain data from several blog posts on social media platforms. Each blog post includes information such as text content, author ID, publication time, and interaction data (e.g., number of likes, comments, and reposts).

[0118] Step 1.2: Clean the original blog post data to obtain preprocessed blog post data;

[0119] Specifically, a three-layer filtering mechanism is used to clean the original blog post data, including rule-based basic filtering, supervised learning-based semantic filtering, and filtering based on social relationship networks and user behavior analysis, in order to remove noisy data and retain text content related to the event.

[0120] The first layer is rule-based basic filtering: a noise feature library is constructed by analyzing a large-scale corpus. Based on the predefined stop word list and regular expression rules in the noise feature library, the blog post data is initially filtered based on the text content of each blog post. The efficient matching operation is achieved through a hash table to quickly remove obviously useless noise data.

[0121] The second layer is semantic filtering based on supervised learning: a supervised learning model is introduced, which combines semantic analysis and lexical features to evaluate the event relevance of the text content of blog posts, and further filters out blog posts that are relevant to the event.

[0122] The third layer of filtering is based on social relationship networks and user behavior analysis: combining social relationship network analysis and user behavior pattern recognition technology, it identifies and removes automatically generated blog post data and blog post data published by malicious accounts to ensure the authenticity and validity of blog post data;

[0123] Step 1.3: The preprocessed blog post data is divided into continuous time windows with a fixed time granularity using the sliding window technique. The blog post data in each time window is merged as a blog post dataset and sorted by time to form an initial public opinion time series dataset including several blog post data sets.

[0124] In this embodiment, a certain degree of overlap is allowed between adjacent time windows to ensure the continuity and integrity of events and avoid the omission of key information due to the division of time window boundaries. The blog post data in each time window is stored in the form of a structure, including time interval, text content and metadata.

[0125] Step 2: Perform multimodal semantic fusion on the blog post data within each time window of the initial public opinion time series dataset to generate a unified semantic representation, and then obtain the fusion matrix;

[0126] Step 2.1: As Figure 1 The framework diagram shown addresses the sparsity and unstructured nature of short texts on social media by first extracting named entities with event indications from each blog post, and then constructing a set of named entities.

[0127] For a set of blog post data within a time window , This indicates the ID of the blog post dataset. This represents blog post data. This indicates the blog post number. This represents the number of blog posts in the blog post dataset; Named Entity Recognition (NER) is performed using a pre-trained BERT model, and its annotation process is defined as follows:

[0128] (1);

[0129] in, Represents a collection of named entities. This represents the vocabulary units in the blog post data. Indicates the entity type corresponding to the lexical unit. For a predefined collection of entity types, Indicates a person's name. Indicates a place name. Indicates the organization name. Indicates time;

[0130] Step 2.2: Extract topic tags from blog post data using regular expressions, form a topic tag set, and merge it with the named entity set to form a candidate entity set;

[0131] The candidate entity set is represented as follows:

[0132] (2);

[0133] in, Represents a collection of blog posts The corresponding set of candidate entities, Represents blog post data The corresponding set of topic tags, and , This refers to topic tags in blog post data. Indicates the hashtag number;

[0134] Step 2.3: Next, to quantify the distribution pattern of named entities in the blog post data, an entity distribution weight matrix is ​​constructed based on the candidate entity set. ,in Name the total number of entities in the candidate entity set. For blog post data collection Number of blog posts;

[0135] The elements in the entity distribution weight matrix represent the weighted occurrence count of named entities in the blog post data. This weighted occurrence count is weighted by TF-IDF to suppress the interference of high-frequency general entities, while enhancing the distinguishability of low-frequency specific entities. The weight of high-frequency general words is significantly reduced, while the weight of professional terms is increased. This weighting method effectively solves the problem of uneven entity distribution in short texts and improves the accuracy of entity recognition.

[0136] (3);

[0137] in, Represents named entities, Indicates the number of the named entity. The elements in the entity distribution weight matrix represent named entities. In blog data Weighted frequency of occurrence in; Named entities In blog data word frequency in Inverse document frequency, For containing named entities The number of blog posts;

[0138] Step 2.4: Construct the associated blog post set for each named entity in the candidate entity set, and sample from it. Use the sampled blog post data to construct the context sample set for that named entity. ;

[0139] Specifically, to overcome the semantic incompleteness and polysemy problems of short texts, this invention proposes a deep semantic modeling method based on dynamic context sampling. Firstly, for each named entity in the candidate entity set... A collection of related blog posts can be constructed using the blog post data that appears in the named entity. Sampled from related blog posts Building a contextual sample set from blog post data When sampling, both influence and diversity were considered. On the one hand, blog posts with high dissemination volume were selected to capture mainstream semantics, that is, the top posts were selected in descending order of repost volume. / 2 blog post data, on the other hand, randomly select from the remaining blog post data. / 2 entries, covering semantic variations and long-tail expressions;

[0140] Step 2.5: For the context sample set Each sampled blog post data The BERT model and FastText model are used to generate sentence vectors and word vectors respectively, and then weighted and fused to obtain the comprehensive contextual semantic vector of each named entity.

[0141] The BERT model is used to capture the overall semantics of blog post data, resulting in the following sentence vectors:

[0142] (4);

[0143] in, Represents blog post data The sentence vector; This represents the base version of the BERT model.

[0144] Fine-grained semantics of named entity-related words are extracted using the FastText model, resulting in the following word vectors:

[0145] (5);

[0146] in, Representing word vectors, Represents blog post data The collection of all words contained therein Representing words, This represents the word vector representation of the FastText model's output of vocabulary;

[0147] Based on sentence vectors and word vectors, a comprehensive contextual semantic vector for each named entity is generated through metadata-weighted multimodal fusion:

[0148] (6);

[0149] in, Represents named entities The comprehensive contextual semantic vector, Represents the context sample set The number of blog posts in China and These are weighting coefficients used to enhance the semantic contribution of highly viral tweets. and These represent blog post data. The number of reposts and likes, This indicates a concatenation operation, which fuses semantic and lexical features to generate a hybrid feature vector using the methods described above.

[0150] Step 2.6: Construct a context semantic matrix based on the comprehensive context semantic vector of each named entity. The row vectors are the comprehensive contextual semantic vectors for each named entity;

[0151] Step 2.7: To unify the lexical co-occurrence pattern and contextual semantic association, a cosine distance-based fusion mechanism is designed. Based on the entity distribution weight matrix and the contextual semantic matrix, a co-occurrence distribution distance matrix and a semantic space distance matrix are constructed respectively.

[0152] The co-occurrence distribution distance matrix is ​​obtained by calculating the cosine distance of the row vectors of the TF-IDF-weighted entity distribution weight matrix, and is used to measure the intensity of explicit co-occurrence between named entities.

[0153] (7);

[0154] in, , Indicates the number of the named entity; , This represents the weighted co-occurrence distribution vector of the named entity in the blog post data, including the weighted number of times the named entity appears in multiple blog post data. The elements in the co-occurrence distribution distance matrix represent the distance between two named entities in the lexical co-occurrence space;

[0155] The semantic space distance matrix calculates normalized cosine distance based on the comprehensive context semantic vectors, capturing implicit semantic relationships between named entities:

[0156] (8);

[0157] in, , , Represents a comprehensive contextual semantic vector. Represents the vector normalization function. The elements in the semantic space distance matrix represent the semantic space distance between named entities;

[0158] Step 2.8: Merge the co-occurrence distribution distance matrix and the semantic space distance matrix into a unified distance matrix through linear combination to obtain the fusion matrix;

[0159] (9);

[0160] in, Represents the fusion matrix. Represents the co-occurrence distribution distance matrix. Represents the semantic space distance matrix; To integrate weights, considering that lexical co-occurrence and deep semantics each have value in event discovery, and that assigning equal importance to both is a common benchmark setting in the absence of prior knowledge or detailed tuning, this invention empirically... The value is set to 0.5. This parameter is chosen to balance the influence of explicit co-occurrence patterns and implicit semantic associations between entities. Further research can adjust the value based on specific tasks or data characteristics. The values ​​are optimized.

[0161] For example, when When the value approaches 0, the model relies more on deep semantic associations and is suitable for handling lexical variations; when... As the value approaches 1, the model focuses more on explicit co-occurrence patterns, making it suitable for detecting sudden combinations of entities. The final generated fusion matrix... As input for subsequent clustering, the smaller the value, the stronger the relationship between entities.

[0162] Step 3: Based on the fusion matrix, perform dynamic multi-level clustering on the blog post data within each time window of the initial public opinion time series dataset to generate event clusters;

[0163] In this implementation, this step employs the Hierarchical Density Clustering (HDBSCAN) algorithm to adaptively partition the multimodal semantically fused data, generating event clusters. HDBSCAN is a density-based clustering algorithm that can automatically identify dense regions in the data and divide these regions into clusters, while effectively handling noise points and marginal topics. By constructing a weighted nearest neighbor graph, the core distance and reachability distance of entities are calculated, thereby constructing a hierarchical density tree. Based on a dynamic density threshold criterion, cluster structures with significant density differences are automatically identified, distinguishing main event clusters from noise points or marginal topics. The final output is a set of clusters, each containing a set of semantically related named entities and their associated blog post data, and the popularity level of events is quantified by the cohesion within the clusters.

[0164] Specifically, the HDBSCAN algorithm is first based on the fusion matrix Construct a weighted average Nearest neighbor graph The weighted average represents the number of nearest neighbors. The nearest neighbor graph is an undirected weighted graph consisting of nodes and edges, with each node corresponding to a named entity. An edge represents the nearest neighbor relationship between two named entities. When two named entities are nearest neighbors, an undirected edge is established between the nodes corresponding to these two named entities. Then, the core distance and reachability distance of each named entity are calculated. The core distance is defined as the distance from the current named entity to its nearest neighbor. The nearest neighbor distance is used, while the reachability distance is the minimum core distance between named entities. Based on the core distance and reachability distance of named entities, a hierarchical density tree is constructed. Through a dynamic density threshold criterion, clusters with significant density differences are automatically identified. For each cluster, its intra-cluster cohesion is calculated to quantify the popularity level of events. Finally, the cluster set corresponding to each blog post dataset is output. , A cluster, comprising a collection of semantically related named entities and their associated blog post data within a time window, provides the foundation for subsequent event evolution path modeling. The number of clusters;

[0165] Step 4: Based on the cluster sets of different time windows, construct a global event evolution graph using cross-window event cluster matching technology, which includes the evolution path of events;

[0166] ;

[0167] Step 4.1: Perform association modeling based on the cluster sets within adjacent time windows to construct a bipartite graph network structure with weighted features;

[0168] First, we model the associations of cluster sets within adjacent time windows, constructing a bipartite graph network structure with weighted features. We define the cluster sets within the previous and current time windows as follows:

[0169] (10);

[0170] in, This represents the set of clusters within the previous time window. This represents the set of clusters within the current time window. Indicates time, Indicates a cluster, This indicates the number of clusters in the cluster set within the previous time window. This indicates the number of clusters in the cluster set within the current time window;

[0171] Construct a bipartite graph network structure based on the above cluster set:

[0172] (11);

[0173] in, Represents the bipartite graph network structure. This represents the cross-window cluster association weight matrix, and the elements in the cross-window cluster association weight matrix are... Indicates the cluster in the previous time window Clusters in the current time window The strength of the correlation between them;

[0174] Elements in the cross-window inter-cluster association weight matrix A linear weighted fusion mechanism is used for calculation, combining named entity continuity (based on an improved weighted Jaccard coefficient) and semantic consistency (based on temporal similarity of multimodal cluster center vectors). The specific calculation formula is as follows:

[0175] (12);

[0176] in, This is a moderating factor used to balance the contributions of explicit associations and implicit semantic associations of named entities; This represents the improved weighted Jaccard coefficient; This represents a measure of semantic consistency.

[0177] The improved formula for calculating the weighted Jaccard coefficient is as follows:

[0178] (13);

[0179] in, , These represent the sets of named entities in the two clusters, Represents named entities, The constructed entity weight evaluation model is as follows:

[0180] (14);

[0181] in, Represents named entities Word frequencies in the blog post dataset within the current time window. Represents named entities Inverse document frequency, Indicates the presence of named entities The total popularity score of the blog post data is calculated based on the weighted sum of the number of reposts, comments, and likes of the blog post data; Represents the normalization function. This represents the heat adjustment coefficient, used to control the proportion of the dissemination influence in the total weight; The decay function represents the time-varying decay, expressed in exponential decay form. , The time interval between the two clusters within their respective time windows. The attenuation coefficient;

[0182] Semantic consistency aims to measure the continuity of clusters in terms of topic content across time windows. Unlike explicit matching based on named entities, semantic consistency, through the comparison of deep semantic features, can accurately capture the evolution of events even when named entities change but the event topic remains the same. First, multimodal cluster center vectors are constructed: for any cluster within a time window... First, calculate its multimodal cluster center vector. This vector is the weighted average of the multimodal feature vectors of all blog posts within the cluster, used to represent the core semantic content of the cluster. The calculation formula is as follows:

[0183] (15);

[0184] in, Cluster The multimodal cluster center vector, Cluster The number of blog post data included. This represents a single blog post within a cluster; blog post The sentence vector, blog post Word vectors, This indicates a feature fusion operation. and , which is the modality weight coefficient, used to adjust the contribution ratio of deep semantic features and lexical features;

[0185] Obtaining the center vector of the multimodal cluster Then, calculate the clusters within the previous time window. Clusters within the current time window To measure semantic consistency between public opinion events and reflect their timeliness, this embodiment introduces a time decay mechanism based on standard cosine similarity. The calculation formula is as follows:

[0186] (16);

[0187] in, and Clusters and cluster The multimodal cluster center vector;

[0188] Step 4.2: Based on the bipartite graph network structure with weighted features, an improved bipartite graph optimization matching algorithm is used to perform cluster pair matching to obtain matching pairs, which include two clusters;

[0189] Specifically, the following steps are included:

[0190] Step 4.2.1: To improve computational efficiency and filter out noisy associations, traverse the cross-window inter-cluster association weight matrix. all elements ,like Less than the preset effective association threshold If the element is not found, it is set to 0 (the corresponding connection edge is disconnected). This process removes weakly related connections and transforms the fully connected bipartite graph network with weighted features into a sparse bipartite graph network with weighted features, which serves as the input for subsequent matching algorithms.

[0191] Step 4.2.2: Use a solver based on maximum weight matching to solve the sparse bipartite graph network with weight features to obtain matching pairs. Treat each cluster in the matching pair as a node, and each node has an event ID. At the same time, establish an association edge between the two nodes to represent the evolution process, and then obtain the global event evolution graph.

[0192] During the solution process, dual constraints are set to ensure the accuracy of matching. Specifically, a one-to-one uniqueness constraint is used to ensure that a cluster in the previous time window can establish a "lead evolution" relationship with at most one cluster in the current time window, avoiding confusion in the main logic of the evolution path. A maximum weight sum constraint is used to find the weights that make all matching pairs satisfactory, while satisfying the uniqueness constraint. The combination that maximizes the sum;

[0193] Step 4.2.3: For clusters within the current time window that still fail to match after step 4.2.2 (i.e., newly formed clusters).

[0194] Calculate its weights with all clusters within all historical time windows. If the set branch threshold is reached, it is determined as a branch node of an existing event, and an association edge is established between the cluster in the current time window and the clusters in the historical time windows; if the weight between the cluster in the current time window and all clusters in the historical time windows is... If all events are below the set branch threshold, they are determined to be independent events, added to the global event evolution graph as initial nodes (new event outbreaks), and assigned a new event ID as the starting point of a new evolution path, thus obtaining an updated global event evolution graph.

[0195] To ensure the real-time performance and accuracy of the evolutionary chain, this invention features a dynamic maintenance mechanism. If a node does not detect a new associated edge within a preset time window, it is marked as terminated and archived, avoiding interference from stale nodes in real-time analysis while preserving the multi-path possibilities of event evolution. Here, a node is defined as a cluster within a time window. An edge refers to an edge connecting only adjacent time windows.

[0196] Step 4.3: After cluster pair matching, based on the cluster pair matching results, update the global event evolution graph and, in conjunction with the dynamic maintenance mechanism designed in this invention, obtain the event evolution path;

[0197] The event evolution path is a path composed of nodes and associated edges;

[0198] Step 5: Generate a summary of the public opinion event based on the event evolution path;

[0199] In its implementation, the core objective is to leverage intelligent text generation technology to extract structured semantic expressions from complex social event information. Based on the event evolution path obtained during the event detection phase, the OpinionDigest framework is used, combining semantic understanding and natural language generation techniques to extract knowledge from social media events. Compared to traditional methods that only output discrete labels, the text summaries generated by this method can intuitively present the core elements of the event, reveal the logic of event development, and provide highly understandable semantic output for public opinion analysis.

[0200] Specifically, the following steps are included:

[0201] Step 5.1: For the set of blog post data associated in the event evolution path, a semantically aware key phrase extraction method is adopted to capture key phrase units through a sliding window mechanism;

[0202] Specifically, a sliding window technique is used to perform local semantic scanning centered on named entities in blog post data, extracting key phrase units containing the named entity and its modifiers. This mechanism preserves local contextual features while ensuring information density, providing basic semantic units for subsequent generation.

[0203] Step 5.2: To address the lack of labeled summaries in social media data, a "phrase-post" mapping dataset is constructed by establishing a mapping relationship between the original blog post data and its extracted key phrase units;

[0204] Step 5.3: Use DeepSeek-V3 as the base generative model and fine-tune it based on the "phrase-blog post" mapping dataset to obtain the fine-tuned base generative model;

[0205] By predicting missing content by masking some words, we learn the generation pattern of reconstructing complete blog post data from discrete phrases and master the logical relationship between event elements and the rules of language organization.

[0206] Step 5.4: First, use the BERT model to semantically encode the key phrase units extracted in Step 5.1 to obtain phrase vectors. Then, combine the HDBSCAN density clustering algorithm to perform cluster analysis on the above phrase vectors, and aggregate different key phrase units with similar semantics or expressing the same meaning into the same group to generate semantic clusters.

[0207] Step 5.5: In each semantic cluster, select the key phrase unit with the highest cosine similarity as the representative semantic unit, which preserves information diversity and avoids redundancy. Then, input the representative semantic units arranged in time sequence into the fine-tuned basic generative model to form a coherent text and obtain the summary.

[0208] Step 5.6: Establish a multi-dimensional evaluation system, evaluate the generated abstracts, and adjust the basic generation model in real time based on the evaluation results;

[0209] To ensure output quality, this framework constructs a multi-dimensional evaluation system. By designing quantitative indicators such as semantic coverage, temporal coherence, and information fidelity, combined with manual semantic verification, a closed-loop optimization mechanism for generation quality is established. Semantic coverage measures whether the generated text contains representative semantic units, requiring a coverage rate of over 80%; temporal coherence assesses text fluency using a language model perplexity index (target value <30); and information fidelity calculates the topic similarity between the generated summary and the original blog post data (based on BERT sentence vector cosine similarity ≥0.8).

[0210] Finally, a feedback optimization mechanism is established to use the evaluation results for model optimization, forming a closed loop for quality improvement. When relevant information is detected to be missing, the model will automatically increase the generation weight of relevant phrases.

[0211] Step 6: Store the event evolution path and the generated summary in a structured format to complete the event archiving.

[0212] In practice, detailed information is collected for each detected event. This information includes the event ID, the time period in which the event occurred, the event summary, and related entity and blog post data. The event ID is a unique identifier automatically generated by the system to distinguish different events. The event time period records the time span from the start to the end of the event. The event summary is a brief description of the event, directly referencing the text generated in the previous stage. In addition, the system extracts entity information related to the event, such as named entities and hashtags, which are aggregated from event-related tweets. To provide richer context, the system also records key tweets related to the event, which typically have high numbers of likes, replies, or retweets, reflecting the event's social impact.

[0213] After the data is processed, this information is organized into a JSON object. JSON is a lightweight data exchange format that is easy to read and write, and also easy for machines to parse and generate. In this invention, a JSON file contains the following fields: "eventId" (event ID), "timePeriod" (event time period), "summary" (event summary), "entities" (list of related entities), and "topTweets" (key tweets and their metadata). This standardized format has good compatibility and can be easily read and stored by various database systems. Finally, these files are stored in a database. Different types of data storage schemes can be selected according to actual needs, including traditional relational databases or new non-relational databases, to meet different query and analysis requirements.

[0214] Experimental comparison:

[0215] In this embodiment, the method of the present invention is compared with six benchmark models, LDA, Word2vec, BERT, RevDet, STREAMCUBE, and STED, on one commonly used public dataset, Twitter dataset, and one real microblog dataset. The quality of event clustering and its evolution path is evaluated using cohesion and discrimination.

[0216] Cohesion primarily measures a model's ability to aggregate related elements (such as tweets, tags, or named entities) into tight clusters of events spanning consecutive time windows. Specifically, for an event e spanning a time window set T, cohesion is defined as follows:

[0217] (17);

[0218] Here, the total number of entity pairs refers to the total number of entity pairs output by the system at time t, while the number of relevant entity pairs refers to the number of entity pairs that are correctly included in the output, i.e., both entities are manually labeled as entity pairs related to event e. A higher cohesion score means that the event clusters are more tightly packed. In the study, entity pairs are considered relevant if they appear in tweets about the same event.

[0219] Distinctiveness is used to evaluate a model's ability to separate unrelated entities to avoid confusion between different events. It is defined as:

[0220] (18);

[0221] Here, the number of irrelevant entity pairs refers to the number of entity pairs that appear in the output, where at least one entity is labeled as irrelevant to event e. Unlike cohesion, discriminability has a lower ideal value because it indicates that fewer irrelevant items are incorrectly grouped together.

[0222] The experimental results are shown in Tables 1 and 2. The experimental data demonstrate that, addressing the semantic ambiguity issue in short texts on social media, this invention, by fusing lexical co-occurrence and deep contextual semantics to construct a unified distance metric, improves the integration degree in the experiment by 7.39% and 5.21% respectively compared to the suboptimal model. Regarding the challenge of easily broken evolution paths in public opinion events, this invention achieves accurate matching of cross-window event clusters through an improved weighted Jaccard coefficient and semantic consistency metric, reducing the discrimination by 4.7% and 4.74% respectively. In summary, this invention not only significantly enhances the cohesion within event clusters but also effectively suppresses the confusion of irrelevant entities, exhibiting higher clustering quality and stronger semantic recognition robustness in handling dynamic public opinion analysis tasks across time windows.

[0223] Table 1. Experimental results on the PerSED dataset;

[0224]

[0225] Table 2 shows the experimental results on the Weibo dataset;

[0226]

[0227] Example 2:

[0228] A system for multimodal semantic fusion modeling and summarization of public opinion events, used to implement a method for multimodal semantic fusion modeling and summarization of public opinion events, includes:

[0229] The dataset construction module is used to acquire blog post data from social media platforms, preprocess the blog post data, and then construct the initial public opinion time series dataset.

[0230] The multimodal semantic fusion module is used to perform multimodal semantic fusion on blog post data within each time window in the initial public opinion time series dataset, generate a unified semantic representation, and then obtain a fusion matrix;

[0231] The clustering module is used to perform dynamic multi-level clustering of blog post data within each time window in the initial public opinion time series dataset based on the fusion matrix, generating event clusters.

[0232] The evolution module is used to construct a global event evolution graph, including the evolution path of events, based on cluster sets of different time windows and cross-window event cluster matching technology.

[0233] The summary generation module generates summaries of public opinion events based on the event evolution path;

[0234] The storage module is used to store the event evolution path and the generated summary in a structured form.

[0235] This invention proposes a multimodal semantic fusion modeling and summarization method and system for public opinion events, primarily addressing the difficulties in understanding short texts and the discontinuous event tracking in current public opinion analysis. For semantic modeling, this method combines co-occurrence relationships in the text with semantic information in the context to construct a unified calculation method, alleviating the problems of incomplete content and unclear word meanings in short texts on social media, and improving the semantic association between different content. For event tracking, this method designs a cross-time period event matching method, utilizing an improved overlap calculation method and semantic similarity index to connect related content of the same event in different time periods, solving the problems of event segmentation and discontinuous development paths in the original methods.

[0236] Example 3:

[0237] This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors perform the aforementioned public opinion event multimodal semantic fusion modeling and summary generation.

[0238] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the multimodal semantic fusion modeling and summary generation of public opinion events as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0239] The processor is used to execute all or part of the steps in the multimodal semantic fusion modeling and summarization generation of public opinion events as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in an electronic device, as well as application-related data.

[0240] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to perform the multimodal semantic fusion modeling and summary generation of public opinion events described in the above embodiments.

[0241] Example 4:

[0242] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0243] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of multimodal semantic fusion modeling and summarization of public opinion events as described in the various embodiments of this application.

[0244] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) app stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can perform the various steps of multimodal semantic fusion modeling and summary generation of public opinion events as described above.

[0245] Example 5:

[0246] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, realizes the multimodal semantic fusion modeling and summary generation of public opinion events.

[0247] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0248] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0249] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A method for multimodal semantic fusion modeling and summarization generation of public opinion events, characterized in that, Includes the following steps: Acquire blog post data from social media platforms, preprocess the blog post data, and then construct an initial public opinion time series dataset; Multimodal semantic fusion is performed on blog post data within each time window of the initial public opinion time series dataset to generate a unified semantic representation, and then a fusion matrix is ​​obtained. Based on the fusion matrix, dynamic multi-level clustering is performed on blog post data within each time window of the initial public opinion time series dataset to generate event clusters; Based on cluster sets in different time windows, a global event evolution graph is constructed using cross-window event cluster matching technology, which includes the evolution path of events; Based on the event evolution path, a summary of the public opinion event is generated; Store the event evolution path and the generated summary in a structured form.

2. The method for multimodal semantic fusion modeling and summarization of public opinion events according to claim 1, characterized in that, The process of acquiring blog post data from social media platforms, preprocessing the blog post data, and then constructing an initial public opinion time series dataset specifically includes: A1: Obtain several blog post data from social media platforms using web scraping tools. Each blog post data includes text content, author ID, publication time, and interaction data. A2: Clean the original blog post data to obtain preprocessed blog post data; Specifically, a three-layer filtering mechanism is used to clean the original blog post data, including rule-based basic filtering, supervised learning-based semantic filtering, and filtering based on social relationship networks and user behavior analysis. The first layer is rule-based basic filtering: a predefined list of stop words and regular expression rules are used to perform preliminary filtering of blog post data based on the text content of each blog post. The second layer is semantic filtering based on supervised learning: a supervised learning model is introduced to evaluate the event relevance of the text content of blog posts, and further filter out blog posts that are relevant to the event. The third layer of filtering is based on social relationship networks and user behavior analysis: combining social relationship network analysis and user behavior pattern recognition technology, it identifies and removes automatically generated blog post data and blog post data published by malicious accounts; A3: The preprocessed blog post data is divided into continuous time windows with a fixed time granularity using the sliding window technique. The blog post data in each time window is merged as a blog post dataset and sorted by time to form an initial public opinion time series dataset including several blog post data sets.

3. The method for multimodal semantic fusion modeling and summarization of public opinion events according to claim 1, characterized in that, The process of performing multimodal semantic fusion on blog post data within each time window of the initial public opinion time series dataset to generate a unified semantic representation and thus obtain a fusion matrix specifically includes: B1: Extract named entities with event indication significance from each blog post data, and then construct a set of named entities; For a set of blog post data within a time window , This indicates the ID of the blog post dataset. This represents blog post data. This indicates the blog post number. This indicates the number of blog posts in the blog post dataset; named entity recognition is performed using a pre-trained BERT model, and its annotation process is defined as follows: （1）； in, Represents a collection of named entities. This represents the vocabulary units in the blog post data. Indicates the entity type corresponding to the lexical unit. For a predefined collection of entity types, Indicates a person's name. Indicates a place name. Indicates the organization name. Indicates time; B2: Extract topic tags from blog post data using regular expressions, form a topic tag set, and merge it with the named entity set to form a candidate entity set; The candidate entity set is represented as follows: （2）； in, Represents a collection of blog posts The corresponding set of candidate entities, Represents blog post data The corresponding set of topic tags, and , This refers to topic tags in blog post data. Indicates the hashtag number; B3: Construct an entity distribution weight matrix based on the candidate entity set. ,in Name the total number of entities in the candidate entity set. For blog post data collection Number of blog posts; The elements in the entity distribution weight matrix represent the weighted frequency of named entities in the blog post data: （3）； in, Represents named entities, Indicates the number of the named entity. The elements in the entity distribution weight matrix represent named entities. In blog data Weighted frequency of occurrence in; Named entities In blog data word frequency in Inverse document frequency, For containing named entities The number of blog posts; B4: For each named entity in the candidate entity set, construct its associated blog post set, and sample from it. Use the sampled blog post data to construct the context sample set for that named entity. ; Specifically: First, for each named entity in the candidate entity set... A collection of related blog posts can be constructed using the blog post data that appears in the named entity. Sampled from related blog posts Building a contextual sample set from blog post data When sampling, both influence and diversity were considered. On the one hand, blog posts with high dissemination volume were selected to capture mainstream semantics, that is, the top posts were selected in descending order of repost volume. / 2 blog post data, on the other hand, randomly select from the remaining blog post data. / 2 items; B5: For context sample sets Each sampled blog post data The BERT model and FastText model are used to generate sentence vectors and word vectors respectively, and then weighted and fused to obtain the comprehensive contextual semantic vector of each named entity. The BERT model is used to capture the overall semantics of blog post data, resulting in the following sentence vectors: （4）； in, Represents blog post data The sentence vector; Represents the BERT model; Fine-grained semantics of named entity-related words are extracted using the FastText model, resulting in the following word vectors: （5）； in, Representing word vectors, Represents blog post data The collection of all words contained therein Representing words, This represents the word vector representation of the FastText model's output of vocabulary; Based on sentence vectors and word vectors, a comprehensive contextual semantic vector for each named entity is generated through metadata-weighted multimodal fusion: （6）； in, Represents named entities The comprehensive contextual semantic vector, Represents the context sample set The number of blog posts in China and These are the weighting coefficients. and These represent blog post data. The number of reposts and likes, Indicates a splicing operation; B6: Construct a context semantic matrix based on the comprehensive context semantic vector of each named entity. The row vectors are the comprehensive contextual semantic vectors for each named entity; B7: The design is based on a cosine distance fusion mechanism, which constructs a co-occurrence distribution distance matrix and a semantic space distance matrix according to the entity distribution weight matrix and the context semantic matrix, respectively; Co-occurrence distribution distance matrix: （7）； in, , Indicates the number of the named entity; , This represents the weighted co-occurrence distribution vector of the named entity in the blog post data, including the weighted number of times the named entity appears in multiple blog post data. The elements in the co-occurrence distribution distance matrix represent the distance between two named entities in the lexical co-occurrence space; Semantic space distance matrix: （8）； in, , , Represents a comprehensive contextual semantic vector. Represents the vector normalization function. The elements in the semantic space distance matrix represent the semantic space distance between named entities; B8: The co-occurrence distribution distance matrix and the semantic space distance matrix are fused into a unified distance matrix by linear combination, resulting in the fusion matrix; （9）； in, Represents the fusion matrix. Represents the co-occurrence distribution distance matrix. Represents the semantic space distance matrix; For weight fusion.

4. The method for multimodal semantic fusion modeling and summarization of public opinion events according to claim 1, characterized in that, The process involves dynamically clustering blog post data within each time window of the initial public opinion time-series dataset using a fusion matrix to generate event clusters. Specifically, this is achieved by first performing dynamic multi-level clustering on the fusion matrix... Construct a weighted average Nearest neighbor graph The weighted average represents the number of nearest neighbors. The nearest neighbor graph is an undirected weighted graph consisting of nodes and edges, with each node corresponding to a named entity. An edge represents the proximity relationship between two named entities. When two named entities are close neighbors, an undirected edge is established between the nodes corresponding to these two named entities. Then, the core distance and reachability distance of each named entity are calculated. Based on the core distance and reachability distance of the named entities, a hierarchical density tree is constructed. Clusters are automatically identified through a dynamic density threshold criterion. For each cluster, its intra-cluster cohesion is calculated to quantify the popularity level of the event. Finally, the cluster set corresponding to each blog post dataset is output. , A cluster is a collection of semantically related named entities and their associated blog post data within a time window. The number of clusters.

5. The method for multimodal semantic fusion modeling and summarization of public opinion events according to claim 1, characterized in that, The construction of a global event evolution graph based on cluster sets within different time windows, using cross-window event cluster matching technology, specifically includes: C1: Based on the cluster sets within adjacent time windows, perform association modeling to construct a bipartite graph network structure with weighted features; Let the cluster sets in the previous time window and the current time window be represented as follows: （10）； in, This represents the set of clusters within the previous time window. This represents the set of clusters within the current time window. Indicates time, Indicates a cluster, This indicates the number of clusters in the cluster set within the previous time window. This indicates the number of clusters in the cluster set within the current time window; Construct a bipartite graph network structure based on the above cluster set: （11）； in, Represents the bipartite graph network structure. This represents the cross-window cluster association weight matrix, and the elements in the cross-window cluster association weight matrix are... Indicates the cluster in the previous time window Clusters in the current time window The strength of the correlation between them; Elements in the cross-window inter-cluster association weight matrix The calculation uses a linear weighted fusion mechanism, and the specific calculation formula is as follows: （12）； in, As a regulating factor; This represents the improved weighted Jaccard coefficient; This represents a measure of semantic consistency. The improved formula for calculating the weighted Jaccard coefficient is as follows: （13）； in, , These represent the sets of named entities in the two clusters, Represents named entities, The constructed entity weight evaluation model is as follows: （14）； in, Represents named entities Word frequencies in the blog post dataset within the current time window. Represents named entities Inverse document frequency, Indicates the presence of named entities The total popularity of blog posts; Represents the normalization function. Indicates the heat adjustment coefficient; The decay function represents the time-varying decay, expressed in exponential decay form. , The time interval between the two clusters within their respective time windows. The attenuation coefficient; The calculation process for the semantic consistency metric is as follows: First, construct the multimodal cluster center vector: for any cluster within a time window... Calculate its multimodal cluster center vector This vector is the weighted average of the multimodal feature vectors of all blog posts within the cluster, used to represent the core semantic content of the cluster. The calculation formula is as follows: （15）； in, Cluster The multimodal cluster center vector, Cluster The number of blog post data included. This represents a single blog post within a cluster; blog post The sentence vector, blog post Word vectors, This indicates a feature fusion operation. and These are the modal weighting coefficients; Obtaining the center vector of the multimodal cluster Then, calculate the clusters within the previous time window. Clusters within the current time window The semantic consistency measure between them is calculated using the following formula: （16）； in, and Clusters and cluster The multimodal cluster center vector; C2: Based on the bipartite graph network structure with weighted features, an improved bipartite graph optimization matching algorithm is used to perform cluster pair matching to obtain matching pairs, which include two clusters; C3: After cluster pair matching, the global event evolution graph is updated based on the cluster pair matching results, and the event evolution path is obtained by combining the dynamic maintenance mechanism. The event evolution path is a path composed of nodes and associated edges.

6. The method for multimodal semantic fusion modeling and summarization of public opinion events according to claim 5, characterized in that, C2 specifically includes: C2.1: Traversing the cross-window inter-cluster association weight matrix all elements ,like Less than the preset effective association threshold If the element is set to 0, the bipartite graph network is transformed into a sparse bipartite graph network with weighted features. C2.2: A solver based on maximum weight matching is used to solve the sparse bipartite graph network with weight features to obtain matching pairs. Each cluster in the matching pair is used as a node, and each node has an event ID. At the same time, an association edge is established between the two nodes to represent the evolution process, thereby obtaining the global event evolution graph. During the solution process, dual constraints are set. Specifically, a one-to-one uniqueness constraint is used to ensure that a cluster in the previous time window can establish a relationship with at most one cluster in the current time window. A maximum weight sum constraint is used to find the weights that maximize all matching pairs while satisfying the uniqueness constraint. The combination that maximizes the sum; C2.3: For clusters within the current time window that still fail to match after solving C2.2, calculate their weights relative to clusters within all historical time windows. If the set branch threshold is reached, it is determined as a branch node of an existing event, and an association edge is established between the cluster in the current time window and the clusters in the historical time windows; if the weight between the cluster in the current time window and all clusters in the historical time windows is... If all events are below the set branch threshold, they are determined to be independent events, added to the global event evolution graph as initial sections, and assigned new event IDs as the starting point of a new evolution path, thus obtaining the updated global event evolution graph.

7. The method for multimodal semantic fusion modeling and summarization of public opinion events according to claim 1, characterized in that, The process of generating a summary of public opinion events based on the event evolution path specifically includes: D1: For the set of blog post data associated in the event evolution path, a semantically aware key phrase extraction method is adopted, and key phrase units are captured through a sliding window mechanism; Specifically, the method involves using a sliding window technique to perform local semantic scanning centered on named entities in blog post data, and extracting key phrase units containing the named entity and its modifiers. D2: Construct a phrase-blog post mapping dataset by establishing a mapping relationship between the original blog post data and its extracted key phrase units; D3: DeepSeek-V3 is used as the basic generative model and fine-tuned based on the phrase-blog post mapping dataset to obtain the fine-tuned basic generative model; D4: First, the key phrase units extracted in D1 are semantically encoded using the BERT model to obtain phrase vectors. Then, the HDBSCAN density clustering algorithm is used to perform cluster analysis on the phrase vectors to generate semantic clusters. D5: In each semantic cluster, the key phrase unit with the highest cosine similarity is selected as the representative semantic unit. Then, the representative semantic units arranged in time sequence are input into the fine-tuned basic generative model to form a coherent text and obtain a summary. D6: Establish a multi-dimensional evaluation system, evaluate the generated summaries, and adjust the basic generation model in real time based on the evaluation results.

8. A system for multimodal semantic fusion modeling and summarizing public opinion events, used to implement the method for multimodal semantic fusion modeling and summarizing public opinion events as described in any one of claims 1-7, characterized in that, include: The dataset construction module is used to acquire blog post data from social media platforms, preprocess the blog post data, and then construct the initial public opinion time series dataset. The multimodal semantic fusion module is used to perform multimodal semantic fusion on blog post data within each time window in the initial public opinion time series dataset, generate a unified semantic representation, and then obtain a fusion matrix; The clustering module is used to perform dynamic multi-level clustering of blog post data within each time window in the initial public opinion time series dataset based on the fusion matrix, generating event clusters. The evolution module is used to construct a global event evolution graph, including the evolution path of events, based on cluster sets of different time windows and cross-window event cluster matching technology. The summary generation module generates summaries of public opinion events based on the event evolution path; The storage module is used to store the event evolution path and the generated summary in a structured form.

9. An electronic device, characterized in that, include: One or more processors, and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the public opinion event multimodal semantic fusion modeling and summarization method according to any one of claims 1-7.

10. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the public opinion event multimodal semantic fusion modeling and summarization method according to any one of claims 1-7.

Images