Internet-based converged media data relation analysis system

Abstract

The invention relates to the field of data analysis, in particular to an internet-based convergence media data relation analysis system. The fusion media data relation analysis system based on the Internet comprises a fusion media data acquisition module, a fusion media data processing module and a hot topic analysis module. According to the method, the keywords of the convergence media data are extracted, the corresponding clustering clusters are constructed, then the correlation degree of the two pieces of convergence media data is calculated on the basis of the clustering clusters, and the convergence media data higher than the correlation threshold value are combined to be used for analyzing the hot topic condition of the current time period; and the correlation degree is calculated based on the similarity of the clustering clusters and the TF-IDF value, and the distribution condition of keywords corresponding to the clustering clusters is also considered in the calculation process, so that the accuracy of fusion media data matching can be improved.

Description

Technical Field

[0001] This invention relates to the field of data analysis, specifically to an internet-based converged media data relationship analysis system. Background Art

[0002] With the rapid development of internet technology and the widespread adoption of various converged media formats such as social media, news platforms, and short video platforms, the speed and reach of information dissemination have reached unprecedented breadth and depth. In this context, users can access information and participate in discussions through multiple channels, forming an unprecedentedly complex information flow and interaction pattern. However, the speed and diversity of information dissemination have also brought new challenges, especially in the dissemination of public events, breaking news, or socially sensitive topics, where trending topics often experience explosive growth.

[0003] Against this backdrop, there is an urgent need for an internet-based converged media data relationship analysis system that can promptly and accurately grasp the dynamics of trending topics and provide decision-makers with comprehensive and objective information support on these topics. Summary of the Invention

[0004] This invention extracts keywords from converged media data and constructs corresponding clusters. Then, it calculates the correlation between two sets of converged media data based on the clusters and combines converged media data with correlation thresholds to analyze the hot topics in the current time period. Furthermore, the correlation is calculated based on the similarity and TF-IDF value of the clusters, and the distribution of keywords corresponding to the clusters is also considered in the calculation process, which can improve the accuracy of converged media data matching.

[0005] The internet-based converged media data relationship analysis system includes: The converged media data acquisition module is used to continuously acquire converged media data within a preset period until the data analysis time point is reached. It combines all converged media data within the preset period into a converged media dataset, and the time period between two adjacent data analysis time points is the preset period. The converged media data processing module is used to traverse the converged media data in the converged media dataset. For each selected converged media data, the following operations are performed: the selected converged media data is designated as the target converged media data, and the target converged media data is deleted from the converged media dataset. A converged media data association library is constructed to store the target converged media data. Then, the target converged media data is matched one by one with the remaining converged media data in the converged media dataset. During each matching, the converged media data selected from the converged media dataset is designated as the matched converged media data. Based on the similarity of keyword distribution between the target converged media data and the matched converged media data, and the similarity between the target converged media data and the matched converged media data, the processing module performs the following operations: The propagation correlation between the media data is calculated by separately calculating the keyword distribution similarity and propagation correlation. Then, the keyword distribution similarity and propagation correlation are weighted and summed to obtain the corresponding media convergence data correlation value. It is then determined whether the media convergence data correlation value is higher than the correlation threshold. If the media convergence data correlation value is higher than the correlation threshold, the matching media convergence data is deleted from the media convergence dataset and stored in the media convergence data correlation library. If the media convergence data correlation value is not higher than the correlation threshold, no operation is performed, and the media convergence data selected from the media convergence dataset is matched with the target media convergence data. This process continues until the media convergence dataset is empty, at which point the entire media convergence data correlation library is output. The hot topic analysis module is used to traverse all converged media data association libraries and select those converged media data association libraries with a quantity of converged media data greater than a certain threshold as target converged media data association libraries. The target converged media data association library includes the most concentrated content on the converged media platform in the current time period. This content can reflect the current hot topic situation. The quantity threshold is determined according to the size of the converged media platform. The larger the size of the converged media platform, the larger the quantity threshold. Based on the target converged media data association library, hot topics can be monitored.

[0006] Preferably, in the converged media data processing module, the keyword distribution similarity is calculated based on the similarity of keyword distribution between the target converged media data and the matched converged media data, specifically including the following steps: Preprocessing is performed on the target converged media data to construct a target keyword set, which includes the target keywords corresponding to the target converged media data. Then, word embedding is performed on all target keywords in the target keyword set to construct corresponding target keyword vectors. Based on the target keyword vectors, cluster analysis is performed on the target keyword set to output several target clusters. Each target cluster includes several target keywords, and the target feature word vector corresponding to the center of each target cluster is output. Preprocessing is also performed on the matching converged media data to construct a matching keyword set, which includes the matching keywords corresponding to the matching converged media data. Then, word embedding is performed on all matching keywords in the matching keyword set to construct corresponding matching keyword vectors. Based on the matching keyword vectors, cluster analysis is performed on the matching keyword set to output several matching clusters. Each matching cluster includes several matching keywords, and the matching feature word vector corresponding to the center of each matching cluster is output. Construct a target keyword vector matrix and a matching keyword vector matrix. Each row of the target keyword vector matrix stores a target feature word vector corresponding to the target converged media data, and each row of the matching keyword vector matrix stores a matching feature word vector corresponding to the matching converged media data. Add the TF-IDF value of the target cluster corresponding to the target feature word vector to the end of each row of the target keyword vector matrix to construct the target converged media feature matrix. Add the TF-IDF value of the matching cluster corresponding to the matching feature word vector to the end of each row of the matching keyword vector matrix to construct the matching converged media feature matrix. The target converged media feature matrix and the matched converged media feature matrix after performing the transpose operation are multiplied to construct a similarity matrix, and the elements in the similarity matrix are denoted as E. i，j , E i，j The similarity between the i-th row of the target integrated media feature matrix and the j-th row of the matching integrated media feature matrix is ​​represented by the cosine similarity algorithm. i = 1, 2, 3, ..., I, where I is the total number of rows in the target integrated media feature matrix (i.e., the total number of keywords corresponding to the target integrated media feature matrix), and j = 1, 2, 3, ..., J, where J is the total number of rows in the matching integrated media feature matrix. Based on the similarity matrix, several optimal cluster pairs F are constructed using a swarm optimization algorithm. k k = 1, 2, 3, ..., K, where K is the total number of cluster pairs. The cluster pairs F... k The target cluster and the matching cluster are denoted as A and B respectively. k and B k Cluster the target cluster A k The corresponding TF-IDF value is denoted as a. k This will match cluster B. kThe corresponding TF-IDF value is denoted as b. k Calculate the keyword distribution similarity between the target integrated media data and the matching integrated media data. .

[0007] Preferably, in the converged media data processing module, the propagation correlation degree is calculated based on the propagation correlation between the target converged media data and the matching converged media data, specifically including the following steps: Map all target feature word vectors corresponding to the target integrated media data to the integrated media data propagation association graph to segment the target propagation feature graph. Map all matching feature word vectors corresponding to the matching integrated media data to the integrated media data propagation association graph to segment the matching propagation feature graph. The integrated media data propagation association graph includes several feature nodes and connecting edges. The feature nodes store integrated media keyword vectors. Determine all cluster pairs F k The number N of cluster pairs with similarity higher than the similarity threshold is calculated. Then, the ratio η of N to the larger value of I and J is calculated. The product of η and the upper limit of the neighborhood range is calculated and rounded up to obtain the neighborhood range H. The target propagation feature map is extended into the H-order neighborhood in the media convergence data propagation association map to obtain the target propagation extension map. The matching propagation feature map is extended into the H-order neighborhood in the media convergence data propagation association map to obtain the matching propagation extension map. The overlap between the target propagation extension map and the matching extension map is calculated as the propagation association degree. The construction method of the converged media data propagation association graph is as follows: Acquire converged media data from historical records; preprocess the converged media data to construct a converged media keyword set, which includes the converged media keywords corresponding to the converged media data; perform word embedding operations on all converged media keywords in the keyword set to construct corresponding converged media keyword feature vectors; based on the feature vectors of the converged media keywords, perform cluster analysis on the converged media keyword set to output several converged media keyword clusters, each containing several converged media keywords; output the converged media keyword vector corresponding to the center of each cluster, using the converged media keyword vector as feature nodes; and traverse the converged media data. If two feature nodes in the converged media data have a contextual relationship, construct a connection edge between these two feature nodes; finally, assemble all feature nodes and all corresponding connection edges into a converged media data propagation association graph.

[0008] Preferably, in the converged media data processing module, word embedding operations are performed on all target keywords in the target keyword set to construct corresponding target keyword vectors, and word embedding operations are performed on all matching keywords in the matching keyword set to construct corresponding matching keyword vectors. Specifically, this includes the following steps: The target keywords in the target keyword set are matched with words in the word vector library. The word vectors corresponding to the successfully matched words in the word vector library are output as the target keyword vectors corresponding to the target keywords. The word vector library includes one-to-one corresponding words and word vectors, and the word vector library is generated by improving the CBOW model based on context. The matching keywords in the matching keyword set are matched with words in the word vector library, and the word vectors corresponding to the successfully matched words in the word vector library are output as the matching keyword vectors corresponding to the matching keywords.

[0009] Preferably, in the converged media data processing module, the improved CBOW model includes an input layer, a word embedding layer, a knowledge reinforcement layer, a context weighting layer, and an output layer; The input layer is used to receive the one-hot encodings of the context words located in the sliding window of the center word. The center word is the target predicted word during the training process, and the size of the sliding window is C. The word embedding layer is used to multiply the one-hot encoding received by the input layer by the weight matrix W to obtain the word embedding vector corresponding to each one-hot encoding. In the knowledge enhancement layer, the word embedding vector is enhanced based on the semantic knowledge graph, and the enhanced word embedding vector is output. The semantic knowledge graph includes several word triples. The word triples are in the form of entity word-semantic relation-entity dimension. The semantic relations include synonyms and antonyms. In the context-weighted layer, the attention weight of each reinforced word embedding vector with respect to the center word is calculated, and then all reinforced word embedding vectors are weighted and summed according to their corresponding attention weights to obtain the hidden vector; The output layer multiplies the hidden vector by the weight matrix between the context-weighted layer and the output layer to obtain the output vector. The output vector is then normalized by softmax to obtain the probability vector. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

[0010] Preferably, for the converged media data processing module, the word vector library is generated by improving the CBOW model, specifically including the following steps: Obtain several word vector training samples, including converged media data. Combine all word vector training samples into a word vector training set, and then feed the word vector training set into the parameter-initialized improved CBOW model for training. During training, randomly select a training text through a sliding window, use the one-hot encodings of the words in the training text excluding the center word as input, and use the center word of the training text as the target prediction word. Determine whether the training conditions are met. If the training conditions are met, output the trained improved CBOW model; otherwise, continue iterative training. Obtain the weight matrix W from the trained improved CBOW model, and multiply the one-hot encoding corresponding to any word with the weight matrix W to obtain the word vector corresponding to the word. Then, combine all the words and their corresponding word vectors into a word vector library.

[0011] Preferably, for the converged media data processing module, the improved CBOW model's processing of training samples specifically includes the following: The training samples are segmented into words, and the resulting training words arranged in word order are combined into a training word set. The training word set is selected by sliding window, and the training word located in the center of the selected set is recorded as the center word, and the remaining training words are recorded as context words. In the input layer, the one-hot encodings corresponding to the context words are received; In the word embedding layer, the one-hot encoding corresponding to each context word is multiplied by the weight matrix W to obtain the word embedding vector corresponding to each one-hot encoding; In the knowledge reinforcement layer, the following operation is performed for each word embedding vector: Based on the context words corresponding to the word embedding vector, matching is performed on the semantic knowledge graph, outputting all entity words that have semantic relationships with the context words corresponding to the word embedding vector, denoted as neighbor words. Then, the one-hot encoding corresponding to the neighbor words is multiplied with the weight matrix W in the previously trained improved CBOW model to construct the corresponding neighbor word vector. All neighbor word vectors form the neighbor set, and the word embedding vector is reinforced using the following formula: G v (l) is the output vector of the l-th iteration, B is the neighbor set, W(l) is the reinforcement weight matrix corresponding to the l-th iteration, and G is the neighbor set. u Let γ be the word vectors of the neighbors in the neighbor set, l=1, 2, 3, ..., L, γ vu The entity words corresponding to the word embedding vectors and their neighbor word vectors G u The semantic weights corresponding to the semantic relationships between the entity words are defined by σ, which is the activation function. The output vector of the last iteration is used as the reinforcement word embedding vector. In the context-weighted layer, the one-hot encoding corresponding to the center word is multiplied with the weight matrix W in the previously trained improved CBOW model to construct the corresponding center word vector. The dot product between each reinforcement word embedding vector and the center word vector is calculated and recorded as the attention score. Then, all attention scores are normalized to obtain the attention weights corresponding to the reinforcement word embedding vectors. Finally, all reinforcement word embedding vectors are weighted and summed according to their corresponding attention weights to obtain the hidden vector. In the output layer, the hidden vector is multiplied by the weight matrix between the context-weighted layer and the output layer to obtain the output vector. The output vector is then normalized by softmax to obtain the probability vector. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

[0012] Preferably, in the converged media data processing module, the TF-IDF values ​​of the target cluster and the matching cluster are obtained through the following steps: Get the number of times the target keyword in the target cluster appears in the target converged media data H1, get the number of all target keywords in the target converged media data H2, get the total number of converged media data in the converged media data H3, get the total number of converged media data including the target keyword in the target cluster H4, and the TF-IDF value of the target cluster is [(H1 / H2)·log(H3 / (H4+1)). Get the number of times the matching keywords in the matching cluster appear in the matching converged media data (H5), get the total number of all matching keywords in the matching converged media data (H6), get the total number of converged media data in the converged media dataset (H3), get the total number of converged media data including the matching keywords in the matching cluster (H7), and get the TF-IDF value of the matching cluster is [(H5 / H6)·log(H3 / (H7+1))].

[0013] Preferably, in the converged media data processing module, several optimal cluster pairs F are constructed using a group optimization algorithm. k Specifically, it includes the following steps: Construct several simulated cluster sets, each containing several cluster pairs, and combine all simulated cluster sets into a population set, setting a maximum number of iterations; The fitness γ corresponding to the simulated cluster set is calculated using the following formula: ; Where U is the set of simulated clusters; Based on the fitness γ corresponding to the simulated cluster set, the population set is iteratively updated using a population optimization algorithm; Until the maximum number of iterations is reached, the set of simulated clusters with the highest fitness is selected as several optimal cluster pairs F. k Output the results.

[0014] Preferably, the population optimization algorithm uses a genetic algorithm.

[0015] The present invention has the following advantages: 1. This invention extracts keywords from converged media data and constructs corresponding clusters. Then, it calculates the correlation between two sets of converged media data based on the clusters and combines converged media data with correlation thresholds to analyze the hot topics in the current time period. The correlation is calculated based on the similarity and TF-IDF value of the clusters. The distribution of keywords corresponding to the clusters is also considered in the calculation process, which can improve the accuracy of converged media data matching.

[0016] 2. By introducing knowledge graphs, this invention enables the constructed word vectors to better conform to the semantic relationships of synonyms and antonyms, thereby achieving higher accuracy in determining whether they belong to the same topic. It can also identify new words in a timely manner. Furthermore, an attention weighting mechanism is introduced, which, under the guidance of the central word, can further analyze the contribution of context words to the target word, thereby improving the accuracy of target word prediction and making the resulting word vector library more accurate. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of the Internet-based converged media data relationship analysis system used in an embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this invention.

[0019] Internet-based converged media data relationship analysis system, such as Figure 1 As shown, it includes: The converged media data acquisition module is used to continuously acquire converged media data within a preset period. Converged media data includes news, articles, posts, comments, and video texts from various converged media platforms. This content constitutes an important part of converged media. Until the data analysis time point is reached, all converged media data within the preset period is combined into a converged media dataset. The time period between two adjacent data analysis time points is the preset period. The length of the preset period is determined by the information update data corresponding to the converged media platform. Generally, the faster the information update speed, the shorter the preset period, so as to better monitor hot topics. The converged media data processing module is used to traverse the converged media data in the converged media dataset. For each selected converged media data, the following operations are performed: the selected converged media data is designated as the target converged media data, and then deleted from the converged media dataset. A converged media data association library is constructed to store the target converged media data. Next, the target converged media data is matched one by one with the remaining converged media data in the converged media dataset. During each match, the converged media data selected from the dataset is designated as the matched converged media data. Based on the similarity of keyword distribution between the target and matched converged media data, and the dissemination correlation between them, keyword distribution similarity and dissemination correlation are calculated respectively. Then, the keyword distribution similarity and dissemination correlation are weighted and summed to obtain the corresponding converged media data association value. The weights in this weighted summation operation are set by the operator based on the accuracy determined by the hot topics. The module then determines whether the converged media data association value is higher than the association threshold, which is determined by the developers based on the converged media platform. The efficiency of the system dictates that, generally, the content of a converged media platform is relatively uniform, so the association threshold will be increased to achieve a higher matching accuracy. If the association value of the converged media data is higher than the association threshold, it means that the target converged media data and the matching converged media data are highly similar in content. In this case, the matching converged media data is deleted from the converged media dataset and stored in the converged media data association library. If the association value of the converged media data is not higher than the association threshold, it means that the target converged media data and the matching converged media data are not similar in content. No operation is performed, and the system continues to select converged media data from the converged media dataset to match the target converged media data. This continues until the converged media dataset is empty, at which point all converged media data association libraries are output. Each converged media data association library represents a type of data with similar content and reflects the hot topics in the current time period, providing a reference for subsequent hot topic processing. It should be noted that during the process of traversing the converged media data in the converged media dataset, a converged media data association library will be output for each selected converged media data. In the converged media data processing module, the converged media data correlation value corresponding to the target converged media data and the matching converged media data is calculated, which specifically includes the following steps: The target multimedia data undergoes preprocessing, including word segmentation and stop word removal, to construct a target keyword set. This set includes target keywords corresponding to the multimedia data, such as various content words in the article. Word embedding is then performed on all target keywords in the set to construct corresponding target keyword vectors. Based on these vectors, cluster analysis is performed on the target keyword set using the K-means algorithm. To merge words with similar meanings in the multimedia data, several target clusters are output, each containing several target keywords. The target feature word vectors corresponding to the centers of each target cluster are analyzed. Preprocessing is performed on the matching converged media data to construct a set of matching keywords, which includes matching keywords corresponding to the matching converged media data, such as various content words in the article. Then, word embedding is performed on all matching keywords in the set of matching keywords to construct corresponding matching keyword vectors. Based on the matching keyword vectors corresponding to the matching keywords, cluster analysis is performed on the set of matching keywords. In order to merge words with similar meanings in the matching converged media data, several matching clusters are output, which include several matching keywords. The matching feature word vectors corresponding to the centers of each matching cluster are output. Construct a target keyword vector matrix and a matching keyword vector matrix. Each row of the target keyword vector matrix stores a target feature word vector corresponding to the target converged media data, and each row of the matching keyword vector matrix stores a matching feature word vector corresponding to the matching converged media data. Add the TF-IDF value of the target cluster corresponding to the target feature word vector to the end of each row of the target keyword vector matrix to construct the target converged media feature matrix. Similarly, add the TF-IDF value of the matching cluster corresponding to the matching feature word vector to the end of each row of the matching keyword vector matrix to construct the matching converged media feature matrix. The target converged media feature matrix and the matching converged media feature matrix respectively represent the features of the content corresponding to the target converged media data and the matching converged media data, and are used for subsequent content similarity judgment. The TF-IDF values ​​of the target cluster and the matching cluster are obtained through the following steps: Get the number of times the target keyword in the target cluster appears in the target converged media data H1, get the number of all target keywords in the target converged media data H2, get the total number of converged media data in the converged media data H3, get the total number of converged media data including the target keyword in the target cluster H4, and the TF-IDF value of the target cluster is [(H1 / H2)·log(H3 / (H4+1)). Get the number of times the matching keywords in the matching cluster appear in the matching converged media data (H5), get the total number of all matching keywords in the matching converged media data (H6), get the total number of converged media data in the converged media dataset (H3), get the total number of converged media data including the matching keywords in the matching cluster (H7), and get the TF-IDF value of the matching cluster is [(H5 / H6)·log(H3 / (H7+1)). The target converged media feature matrix and the matched converged media feature matrix after performing the transpose operation are multiplied to construct a similarity matrix, and the elements in the similarity matrix are denoted as E. i，j , E i，j The similarity between the i-th row of the target integrated media feature matrix and the j-th row of the matching integrated media feature matrix is ​​represented by the cosine similarity algorithm. i = 1, 2, 3, ..., I, where I is the total number of rows in the target integrated media feature matrix (i.e., the total number of keywords corresponding to the target integrated media feature matrix), and j = 1, 2, 3, ..., J, where J is the total number of rows in the matching integrated media feature matrix (i.e., the total number of keywords corresponding to the matching integrated media feature matrix). Based on the similarity matrix, several optimal cluster pairs F are constructed using a group optimization algorithm. k k = 1, 2, 3, ..., K, where K is the total number of cluster pairs. The cluster pairs F... k The target cluster and the matching cluster are denoted as A and B respectively. k and B k Cluster the target cluster A k The corresponding TF-IDF value is denoted as a. k This will match cluster B. k The corresponding TF-IDF value is denoted as b. k Calculate the keyword distribution similarity between the target integrated media data and the matching integrated media data. The correlation value of converged media data can reflect the similarity of keyword distribution between target converged media data and matching converged media data, and thus characterize the content similarity between the target converged media data and matching converged media data. The dissemination correlation degree is calculated based on the dissemination correlation between the target converged media data and the matching converged media data, specifically including the following steps: The target feature word vectors corresponding to the target integrated media data are mapped to the integrated media data propagation association graph to segment the target propagation feature graph. Similarly, the matching feature word vectors corresponding to the matching integrated media data are mapped to the integrated media data propagation association graph to segment the matching propagation feature graph. The integrated media data propagation association graph includes several feature nodes and connecting edges. The feature nodes store integrated media keyword vectors. Specifically, the mapping operation involves querying the feature node corresponding to the integrated media keyword vector with the highest similarity to the target feature word vector or the matching feature word vector in the integrated media data propagation association graph, and then segmenting the target propagation feature graph or the matching propagation feature graph, including connecting edges, from the integrated media data propagation association graph based on the queried feature node. Determine all cluster pairs F k The number N of cluster pairs with similarity higher than the similarity threshold is calculated. This similarity threshold is set by the operator, typically 0.75. Then, the ratio η between N and the larger of I and J is calculated. The product of η and the upper limit of the neighborhood range is calculated and rounded up to obtain the neighborhood range H. The upper limit of the neighborhood range is typically set to 5. The target propagation feature map is then expanded into an H-order neighborhood in the converged media data propagation association map to obtain the target propagation extension map. Specifically, the expansion operation involves selecting all feature nodes located at the edges in the target propagation feature map. Taking the feature stage as the starting point, all feature nodes in the converged media data propagation association map that are within the H-order neighborhood of the starting point are selected and their corresponding connecting edges are added to the target propagation feature map. This expands the matching propagation feature map into an H-order neighborhood in the converged media data association map, resulting in... The overlap between the target propagation expansion graph and the matching propagation expansion graph is calculated as the propagation correlation. Through the expansion operation, the keyword distribution of the converged media data after propagation can be determined based on the propagation correlation of the topic, thus obtaining the propagation correlation. Based on the propagation correlation, the propagation correlation can be used to analyze whether the target converged media data and the matching converged media data belong to the same propagation logic chain, thereby more accurately determining whether the target converged media data and the matching converged media data belong to the same theme. It should be noted that the overlap between the target propagation expansion graph and the matching propagation expansion graph is specifically the ratio between twice the number of overlapping connection pairs and the sum of the total number of connection pairs in the target propagation expansion graph and the total number of connection pairs in the matching propagation expansion graph. A connection pair is a combination of feature nodes with connecting edges. The construction method of the converged media data dissemination association graph is as follows: Acquire converged media data from historical records; preprocess the converged media data to construct a converged media keyword set, which includes the converged media keywords corresponding to the converged media data; then perform word embedding operations on all converged media keywords in the keyword set to construct corresponding converged media keyword feature vectors; based on the feature vectors of the converged media keywords, perform cluster analysis on the converged media keyword set, outputting several converged media keyword clusters, each containing several converged media keywords; output the converged media keyword vector corresponding to the center of each cluster, using the converged media keyword vector as feature nodes; and then traverse the converged media data. If two features are present in the converged media data... If a feature node has a contextual relationship, a connection edge is constructed between the two feature nodes. Then, all feature nodes and their corresponding connection edges are combined to form a converged media data propagation association graph. Here, the contextual relationship refers to the small distance between corresponding keywords in the converged media data. It should also be noted that the converged media data propagation association graph can be expanded. Specifically, the latest converged media data is selected, and the corresponding converged media keywords are determined. If the similarity between the converged media keywords and the existing feature nodes in the converged media data propagation association graph is higher than the similarity threshold, no new feature node is added; otherwise, a feature node is added. It is then determined whether the connection edges corresponding to the contextual relationships of all converged media keywords in the converged media data exist. If they exist, no new connection edges are added; otherwise, connection edges are added between the corresponding feature nodes.

[0020] The hot topic analysis module is used to traverse all converged media data association libraries and select converged media data association libraries with a number of converged media data exceeding a certain threshold as target converged media data association libraries. The target converged media data association library includes the most concentrated content on the converged media platform in the current time period. This content can reflect the current hot topic situation. The quantity threshold is determined according to the size of the converged media platform. Here, size generally refers to the number of users of the converged media platform. The larger the size of the converged media platform, the larger the quantity threshold. Based on the target converged media data association library, hot topics are monitored. This application extracts keywords from converged media data and constructs corresponding clusters. Then, it calculates the correlation between two sets of converged media data based on the clusters and combines converged media data with correlation thresholds to analyze the hot topics in the current time period. Furthermore, the correlation is calculated based on the similarity and TF-IDF value of the clusters, and the distribution of keywords corresponding to the clusters is also considered in the calculation process, which can improve the accuracy of converged media data matching.

[0021] It should be noted that existing technologies for determining whether converged media data belong to the same topic typically involve constructing TF-IDF vectors from the converged media data and then determining whether the converged media data belongs to the same topic based on the similarity between TF-IDF vectors. This method is only sensitive to surface-level words. In tests, using 100% manual matching as a standard, the accuracy rate of the TF-IDF vector method was 71%. Alternatively, all keyword vectors in the converged media data are weighted and summed to construct text vectors, and then the similarity between text vectors is used to determine whether the converged media data belongs to the same topic. In tests, the accuracy rate of the text vector method was 83%. The method in this application, however, has an accuracy rate of 92%, which significantly improves the accuracy of determining whether converged media data belongs to the same topic.

[0022] The process involves performing word embedding operations on all target keywords in the target keyword set to construct corresponding target keyword vectors, and performing word embedding operations on all matching keywords in the matching keyword set to construct corresponding matching keyword vectors. Specifically, this includes the following steps: The target keywords in the target keyword set are matched with words in the word vector library. The word vectors corresponding to the successfully matched words in the word vector library are output as the target keyword vectors corresponding to the target keywords. The word vector library includes one-to-one corresponding words and word vectors, and the word vector library is generated by improving the CBOW model based on context. The matching keywords in the matching keyword set are matched with words in the word vector library, and the word vectors corresponding to the successfully matched words in the word vector library are output as the matching keyword vectors corresponding to the matching keywords.

[0023] The improved CBOW model includes an input layer, a word embedding layer, a knowledge reinforcement layer, a context weighting layer, and an output layer. The input layer is used to receive the one-hot encodings of the context words located in the sliding window of the center word. The center word is the target predicted word during the training process, and the size of the sliding window is C. The word embedding layer is used to multiply the one-hot encoding received by the input layer by the weight matrix W to obtain the word embedding vector corresponding to each one-hot encoding. In the knowledge reinforcement layer, word embedding vectors are reinforced based on semantic knowledge graphs, and reinforced word embedding vectors are output. The semantic knowledge graph includes several word triples, which take the form of entity word-semantic relation-entity dimension. Semantic relations include synonyms and antonyms. Since many new words often appear in the analysis of converged media, and the training samples for these new words are few, the new words can be associated through artificially constructed semantic knowledge graphs, and the word embedding effect of the CBOW model can be enhanced through the relationships of synonyms and antonyms. In the context-weighted layer, the attention weight of each enhanced word embedding vector to the center word is calculated. The attention weight here represents the contribution of the enhanced word embedding vector to the prediction of the center word. Then, all enhanced word embedding vectors are weighted and summed according to their corresponding attention weights to obtain the hidden vector. The output layer multiplies the hidden vector by the weight matrix between the context-weighted layer and the output layer to obtain the output vector. The output vector is then normalized by softmax to obtain the probability vector. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

[0024] The process of generating a word vector library by improving the CBOW model includes the following steps: Several word vector training samples are obtained, including converged media data, which is acquired by developers from actual converged media platforms. All word vector training samples are combined into a word vector training set, which is then fed into the parameter-initialized improved CBOW model for training. During training, a training text is randomly selected through a sliding window. The one-hot codes of the words in the training text, excluding the center word, are used as input, and the center word of the training text is used as the target prediction word. It is determined whether the training conditions are met. The training conditions are generally reaching a certain number of training iterations or the accuracy exceeding a certain threshold. If the training conditions are met, the trained improved CBOW model is output; otherwise, the training continues iteratively. Obtain the weight matrix W from the trained improved CBOW model, and multiply the one-hot encoding corresponding to any word with the weight matrix W to obtain the word vector corresponding to the word. Then, combine all the words and their corresponding word vectors into a word vector library.

[0025] The specific processing of training samples in the CBOW model includes the following: The training samples are segmented into words, and the resulting training words arranged in word order are combined into a training word set. The training word set is selected by sliding window, and the training word located in the center of the selected set is recorded as the center word, and the remaining training words are recorded as context words. In the input layer, the one-hot encodings corresponding to the context words are received; In the word embedding layer, the one-hot encoding corresponding to each context word is multiplied by the weight matrix W to obtain the word embedding vector corresponding to each one-hot encoding; In the knowledge reinforcement layer, the following operations are performed for each word embedding vector: Matching is performed on the semantic knowledge graph based on the context words corresponding to the word embedding vector. This matching refers to matching the context words with the entity words in the word triples. It should be noted that all words here refer to strings. All entity words that have a semantic relationship with the context words corresponding to the word embedding vector are output and denoted as neighbor words. Then, the one-hot encoding corresponding to the neighbor words is multiplied by the weight matrix W in the previously trained improved CBOW model to construct the corresponding neighbor word vectors. All neighbor word vectors form the neighbor set, and the word embedding vector is reinforced using the following formula: G v (l) is the output vector of the l-th iteration, B is the neighbor set, W(l) is the reinforcement weight matrix corresponding to the l-th iteration, and G is the neighbor set. u Let γ be the word vector of the neighbor set, l=1, 2, 3, ..., L, where L is typically 3. vu The entity words corresponding to the word embedding vectors and their neighbor word vectors G u The semantic weights corresponding to the semantic relationships between the entity words are as follows: the semantic weight of a synonym is generally 1, and the semantic weight of an antonym is -1. σ is the activation function, which is generally ReLU. The output vector of the last iteration is used as the reinforcement word embedding vector. In the context-weighted layer, the one-hot encoding corresponding to the center word is multiplied with the weight matrix W in the previously trained improved CBOW model to construct the corresponding center word vector. The dot product between each reinforcement word embedding vector and the center word vector is calculated and recorded as the attention score. Then, all attention scores are normalized to obtain the attention weights corresponding to the reinforcement word embedding vectors. Finally, all reinforcement word embedding vectors are weighted and summed according to their corresponding attention weights to obtain the hidden vector. In the output layer, the hidden vector is multiplied by the weight matrix between the context-weighted layer and the output layer to obtain the output vector. The output vector is then normalized by softmax to obtain the probability vector. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

[0026] This application introduces knowledge graphs, which enable the constructed word vectors to better reflect the semantic relationships between synonyms and antonyms, thereby achieving higher accuracy in determining whether words belong to the same topic. It also allows for timely identification of new words. Furthermore, an attention weighting mechanism is introduced, which, under the guidance of the central word, further analyzes the contribution of context words to the target word, thereby improving the accuracy of target word prediction and making the resulting word vector library more accurate.

[0027] Several optimal cluster pairs F are constructed using a population optimization algorithm. kSpecifically, it includes the following steps: Construct several simulated cluster sets, each containing several cluster pairs. Combine all simulated cluster sets into a population set and set a maximum number of iterations. Note that when constructing the simulated cluster sets, first determine the size of I and J. If I is greater than J, use the matching cluster number as the first cluster pair and randomly assign the target cluster number to I cluster pairs as the second. If I is not greater than J, use the target cluster number as the first cluster pair and randomly assign the matching cluster number to I cluster pairs as the second. For example, {(1,2), (2,7), (3,11)...}. The fitness γ corresponding to the simulated cluster set is calculated using the following formula: ; Where U is the set of simulated clusters, that is, the sum of the similarities of all cluster pairs in the set of simulated clusters is used as the fitness γ of the set of simulated clusters; Based on the fitness γ corresponding to the simulated cluster set, the population set is iteratively updated using a genetic algorithm. It should be noted that the recombination and mutation operations in the genetic algorithm must be performed under the one-to-one constraint of all clusters in the simulated cluster set. For example, for the recombination operation of {(1,2), (2,7), (3,11)...} and {(1,5), (2,11), (3,3)...}, swapping (2,7) and (2,11) will form {(1,2), (2,11), (3,11)...} and {(1,5), (2,7), (3,3)...}. In {(1,2), (2,11), (3,11)...}, (2,11) and (3,11) do not conform to the one-to-one constraint, so (3,11) which has not been swapped needs to be randomized to become (3,17); the mutation operation is similar. Until the maximum number of iterations is reached, the set of simulated clusters with the highest fitness is selected as several optimal cluster pairs F. k Output the results.

[0028] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Parts not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. A converged media data relationship analysis system based on the Internet, characterized in that: include: The converged media data acquisition module is used to continuously acquire converged media data within a preset period until the data analysis time point is reached. It combines all converged media data within the preset period into a converged media dataset, and the time period between two adjacent data analysis time points is the preset period. The converged media data processing module is used to determine several converged media data association libraries based on the similarity of keyword distribution among converged media data and the dissemination correlation among converged media data. The converged media data association library stores several converged media data belonging to the same theme. The hot topic analysis module is used to traverse all integrated media data association libraries and select those with a number of integrated media data exceeding a certain threshold as the target integrated media data association library. The target integrated media data association library includes the most concentrated content on the integrated media platform during the current time period. This content reflects the current hot topics. The quantity threshold is determined based on the size of the integrated media platform. The larger the size of the integrated media platform, the larger the quantity threshold. Based on the target integrated media data association library, hot topics can be monitored.

2. The Internet-based converged media data relationship analysis system according to claim 1, characterized in that, In the converged media data processing module, several converged media data association databases are determined based on the similarity of keyword distribution and the dissemination correlation between converged media data. Specifically, the steps include: Iterate through the converged media data in the converged media dataset. For each selected converged media data, perform the following operations: Record the selected converged media data as the target converged media data, delete the target converged media data from the converged media dataset, construct a converged media data association library to store the target converged media data, and then match the target converged media data with the remaining converged media data in the converged media dataset one by one. During each match, record the converged media data selected from the converged media dataset as the matched converged media data. The matching is based on the similarity of keyword distribution between the target converged media data and the matched converged media data, as well as the similarity between the target converged media data and the matched converged media data. The dissemination relevance is calculated by separately calculating the keyword distribution similarity and dissemination relevance. Then, the keyword distribution similarity and dissemination relevance are weighted and summed to obtain the corresponding media convergence data relevance value. It is then determined whether the media convergence data relevance value is higher than the relevance threshold. If the media convergence data relevance value is higher than the relevance threshold, the matching media convergence data is deleted from the media convergence dataset and stored in the media convergence data relevance library. If the media convergence data relevance value is not higher than the relevance threshold, no operation is performed, and the process continues to select media convergence data from the media convergence dataset to match the target media convergence data. This continues until the media convergence dataset is empty, at which point the entire media convergence data relevance library is output.

3. The Internet-based converged media data relationship analysis system according to claim 2, characterized in that, In the converged media data processing module, keyword distribution similarity is calculated based on the similarity of keyword distribution between the target converged media data and the matched converged media data. The specific steps include the following: Preprocessing is performed on the target converged media data to construct a target keyword set, which includes the target keywords corresponding to the target converged media data. Then, word embedding is performed on all target keywords in the target keyword set to construct corresponding target keyword vectors. Based on the target keyword vectors, cluster analysis is performed on the target keyword set to output several target clusters. Each target cluster includes several target keywords, and the target feature word vector corresponding to the center of each target cluster is output. Preprocessing is also performed on the matching converged media data to construct a matching keyword set, which includes the matching keywords corresponding to the matching converged media data. Then, word embedding is performed on all matching keywords in the matching keyword set to construct corresponding matching keyword vectors. Based on the matching keyword vectors, cluster analysis is performed on the matching keyword set to output several matching clusters. Each matching cluster includes several matching keywords, and the matching feature word vector corresponding to the center of each matching cluster is output. Construct a target keyword vector matrix and a matching keyword vector matrix. Each row of the target keyword vector matrix stores a target feature word vector corresponding to the target converged media data, and each row of the matching keyword vector matrix stores a matching feature word vector corresponding to the matching converged media data. Add the TF-IDF value of the target cluster corresponding to the target feature word vector to the end of each row of the target keyword vector matrix to construct the target converged media feature matrix. Add the TF-IDF value of the matching cluster corresponding to the matching feature word vector to the end of each row of the matching keyword vector matrix to construct the matching converged media feature matrix. The target converged media feature matrix and the matched converged media feature matrix after performing the transpose operation are multiplied to construct a similarity matrix, and the elements in the similarity matrix are denoted as E. i，j , E i，j The similarity between the i-th row of the target integrated media feature matrix and the j-th row of the matching integrated media feature matrix is ​​represented by the cosine similarity algorithm. i = 1, 2, 3, ..., I, where I is the total number of rows in the target integrated media feature matrix (i.e., the total number of keywords corresponding to the target integrated media feature matrix), and j = 1, 2, 3, ..., J, where J is the total number of rows in the matching integrated media feature matrix. Based on the similarity matrix, several optimal cluster pairs F are constructed using a swarm optimization algorithm. k k = 1, 2, 3, ..., K, where K is the total number of cluster pairs. The cluster pairs F... k The target cluster and the matching cluster are denoted as A and B respectively. k and B k Cluster the target group A k The corresponding TF-IDF value is denoted as a. k This will match cluster B. k The corresponding TF-IDF value is denoted as b. k Calculate the keyword distribution similarity between the target integrated media data and the matching integrated media data. .

4. The Internet-based converged media data relationship analysis system according to claim 3, characterized in that, In the converged media data processing module, the propagation correlation degree is calculated based on the propagation correlation between the target converged media data and the matching converged media data. The specific steps include the following: Map all target feature word vectors corresponding to the target integrated media data to the integrated media data propagation association graph to segment the target propagation feature graph. Map all matching feature word vectors corresponding to the matching integrated media data to the integrated media data propagation association graph to segment the matching propagation feature graph. The integrated media data propagation association graph includes several feature nodes and connecting edges. The feature nodes store integrated media keyword vectors. Determine all cluster pairs F k The number N of cluster pairs with similarity higher than the similarity threshold is calculated. Then, the ratio η of N to the larger value of I and J is calculated. The product of η and the upper limit of the neighborhood range is calculated and rounded up to obtain the neighborhood range H. The target propagation feature map is extended into the H-order neighborhood in the media convergence data propagation association map to obtain the target propagation extension map. The matching propagation feature map is extended into the H-order neighborhood in the media convergence data propagation association map to obtain the matching propagation extension map. The overlap between the target propagation extension map and the matching extension map is calculated as the propagation association degree. The construction method of the converged media data propagation association graph is as follows: Acquire converged media data from historical records; preprocess the converged media data to construct a converged media keyword set, which includes the converged media keywords corresponding to the converged media data; perform word embedding operations on all converged media keywords in the keyword set to construct corresponding converged media keyword feature vectors; based on the feature vectors of the converged media keywords, perform cluster analysis on the converged media keyword set to output several converged media keyword clusters, each containing several converged media keywords; output the converged media keyword vector corresponding to the center of each cluster, using the converged media keyword vector as feature nodes; and traverse the converged media data. If two feature nodes in the converged media data have a contextual relationship, construct a connection edge between these two feature nodes; finally, assemble all feature nodes and all corresponding connection edges into a converged media data propagation association graph.

5. The Internet-based converged media data relationship analysis system according to claim 4, characterized in that, In the converged media data processing module, word embedding operations are performed on all target keywords in the target keyword set to construct corresponding target keyword vectors, and word embedding operations are performed on all matching keywords in the matching keyword set to construct corresponding matching keyword vectors. Specifically, the steps include: The target keywords in the target keyword set are matched with words in the word vector library. The word vectors corresponding to the successfully matched words in the word vector library are output as the target keyword vectors corresponding to the target keywords. The word vector library includes one-to-one corresponding words and word vectors, and the word vector library is generated by improving the CBOW model based on context. The matching keywords in the matching keyword set are matched with words in the word vector library, and the word vectors corresponding to the successfully matched words in the word vector library are output as the matching keyword vectors corresponding to the matching keywords.

6. The Internet-based converged media data relationship analysis system according to claim 5, characterized in that, In the converged media data processing module, the improved CBOW model includes an input layer, a word embedding layer, a knowledge reinforcement layer, a context weighting layer, and an output layer; The input layer is used to receive the one-hot encodings of the context words located in the sliding window of the center word. The center word is the target predicted word during the training process, and the size of the sliding window is C. The word embedding layer is used to multiply the one-hot encoding received by the input layer by the weight matrix W to obtain the word embedding vector corresponding to each one-hot encoding. In the knowledge enhancement layer, the word embedding vector is enhanced based on the semantic knowledge graph, and the enhanced word embedding vector is output. The semantic knowledge graph includes several word triples. The word triples are in the form of entity word-semantic relation-entity dimension. The semantic relations include synonyms and antonyms. In the context-weighted layer, the attention weight of each reinforced word embedding vector with respect to the center word is calculated, and then all reinforced word embedding vectors are weighted and summed according to their corresponding attention weights to obtain the hidden vector; The output layer multiplies the hidden vector by the weight matrix between the context-weighted layer and the output layer to obtain the output vector. The output vector is then normalized by softmax to obtain the probability vector. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

7. The Internet-based converged media data relationship analysis system according to claim 6, characterized in that, For the converged media data processing module, a word vector library is generated by improving the CBOW model, specifically including the following steps: Obtain several word vector training samples, including converged media data. Combine all word vector training samples into a word vector training set, and then feed the word vector training set into the parameter-initialized improved CBOW model for training. During training, randomly select a training text through a sliding window, use the one-hot encodings of the words in the training text excluding the center word as input, and use the center word of the training text as the target prediction word. Determine whether the training conditions are met. If the training conditions are met, output the trained improved CBOW model; otherwise, continue iterative training. Obtain the weight matrix W from the trained improved CBOW model, and multiply the one-hot encoding corresponding to any word with the weight matrix W to obtain the word vector corresponding to the word. Then, combine all the words and their corresponding word vectors into a word vector library.

8. The Internet-based converged media data relationship analysis system according to claim 7, characterized in that, For the converged media data processing module, the improved CBOW model's processing of training samples specifically includes the following: The training samples are segmented into words, and the resulting training words arranged in word order are combined into a training word set. The training word set is selected by sliding window, and the training word located in the center of the selected set is recorded as the center word, and the remaining training words are recorded as context words. In the input layer, the one-hot encodings corresponding to the context words are received; In the word embedding layer, the one-hot encoding corresponding to each context word is multiplied by the weight matrix W to obtain the word embedding vector corresponding to each one-hot encoding; In the knowledge reinforcement layer, the following operation is performed for each word embedding vector: Based on the context words corresponding to the word embedding vector, matching is performed on the semantic knowledge graph, outputting all entity words that have semantic relationships with the context words corresponding to the word embedding vector, denoted as neighbor words. Then, the one-hot encoding corresponding to the neighbor words is multiplied with the weight matrix W in the previously trained improved CBOW model to construct the corresponding neighbor word vector. All neighbor word vectors form the neighbor set, and the word embedding vector is reinforced using the following formula: G v (l) is the output vector of the l-th iteration, B is the neighbor set, W(l) is the reinforcement weight matrix corresponding to the l-th iteration, and G is the neighbor set. u Let γ be the word vectors of the neighbors in the neighbor set, l=1, 2, 3, ..., L, γ vu The entity words corresponding to the word embedding vectors and their neighbor word vectors G u The semantic weights corresponding to the semantic relationships between the entity words are defined by σ, which is the activation function. The output vector of the last iteration is used as the reinforcement word embedding vector. In the context-weighted layer, the one-hot encoding corresponding to the center word is multiplied with the weight matrix W in the previously trained improved CBOW model to construct the corresponding center word vector. The dot product between each reinforcement word embedding vector and the center word vector is calculated and recorded as the attention score. Then, all attention scores are normalized to obtain the attention weights corresponding to the reinforcement word embedding vectors. Finally, all reinforcement word embedding vectors are weighted and summed according to their corresponding attention weights to obtain the hidden vector. In the output layer, the hidden vector is multiplied by the weight matrix between the context-weighted layer and the output layer to obtain the output vector. The output vector is then normalized by softmax to obtain the probability vector. The word corresponding to the highest probability value in the probability vector is taken as the prediction result.

9. The Internet-based converged media data relationship analysis system according to claim 8, characterized in that, In the converged media data processing module, the TF-IDF values ​​of the target cluster and the matching cluster are obtained through the following steps: Get the number of times the target keyword in the target cluster appears in the target converged media data H1, get the number of all target keywords in the target converged media data H2, get the total number of converged media data in the converged media data H3, get the total number of converged media data including the target keyword in the target cluster H4, and the TF-IDF value of the target cluster is [(H1 / H2)·log(H3 / (H4+1)). Get the number of times the matching keywords in the matching cluster appear in the matching converged media data (H5), get the total number of all matching keywords in the matching converged media data (H6), get the total number of converged media data in the converged media dataset (H3), get the total number of converged media data including the matching keywords in the matching cluster (H7), and get the TF-IDF value of the matching cluster is [(H5 / H6)·log(H3 / (H7+1))].

10. The Internet-based converged media data relationship analysis system according to claim 9, characterized in that, In the converged media data processing module, several optimal cluster pairs F are constructed using a group optimization algorithm. k Specifically, it includes the following steps: Construct several simulated cluster sets, each containing several cluster pairs, and combine all simulated cluster sets into a population set, setting a maximum number of iterations; The fitness γ corresponding to the simulated cluster set is calculated using the following formula: ； Where U is the set of simulated clusters; Based on the fitness γ corresponding to the simulated cluster set, the population set is iteratively updated using a population optimization algorithm; Until the maximum number of iterations is reached, the set of simulated clusters with the highest fitness is selected as several optimal cluster pairs F. k Output the results.

Images