Intelligent statistical analysis method for news transmission cross-platform data

Abstract

This invention relates to the field of data processing technology, specifically an intelligent statistical analysis method for cross-platform news dissemination data. The method includes: decomposing and analyzing keywords from the original news text and extracting news data; extracting extended vocabulary from the extracted news text, using keywords as anchors to cluster the extended vocabulary and keywords, constructing an extended vocabulary library, and determining the sentiment range of the extended vocabulary; outputting sentiment analysis results including sentiment tags and positive ratings based on the current sentiment range of the extended vocabulary; constructing a news tree with the original news text as the root node, the mapping relationship between extended vocabulary and keywords as branch nodes, and positive ratings and sentiment tags as leaf nodes; and statistically analyzing the fuzzy sentiment features of each branch node for each combination of sentiment tags and positive ratings under the same branch node. This achieves both efficiency and accuracy in cross-platform news data statistics.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to an intelligent statistical analysis method for cross-platform data in news dissemination. Background Technology

[0002] With the development of the internet, news information has broken the limitations of traditional single-channel dissemination, achieving rapid spread through multiple candidate platforms such as Weibo, Douyin, industry websites, and social forums. Statistical analysis of cross-platform news dissemination data has become an important tool for understanding public opinion dynamics, judging public attitudes, and assisting in decision-making. However, traditional news classification methods mainly rely on simple keyword matching; they lack comprehensive consideration of semantic relationships of words, news topic hierarchy, and industry attributes, which easily leads to data redundancy across platforms and makes it difficult to grasp the emotional trends of cross-platform news dissemination.

[0003] For example, Chinese Patent Publication No. CN117786119A discloses a method, apparatus, device, and storage medium for automatic data classification, relating to the field of natural language processing technology. The method includes: obtaining keywords from a target news article and, based on the keywords, preliminarily determining the industry to which the target news article belongs; obtaining entity words from the text content of the target news article and, based on the entity words, determining the theme corresponding to the target news article; counting high-frequency words in the text content of the target news article that appear more than a preset number of times, and generating related words for the target news article based on the high-frequency words; and combining the industry to which the target news article belongs, the theme corresponding to the target news article, and the related words to determine the final industry of the target news article.

[0004] For example, Chinese Patent Publication No. CN114564675A discloses an information recommendation method, device, and storage medium. Through the setting of a behavior acquisition module, a classification and sorting module, a behavior judgment module, and a balanced push module, it realizes the recording, acquisition, and analysis of users' news browsing behavior. It judges the subjective tendency of users' comments and the subjective tendency of news content to understand users' understanding of the news, thereby controlling the news push scheme for users. It can effectively push news content that supplements the user's understanding of the news, that is, help solve the problem of incomplete news information acquisition, help users better understand the news content, improve users' thinking ability, and reduce the probability of online disputes caused by information imbalance.

[0005] Existing technologies classify news by industry to determine its industry category and filter news pushes based on behavioral tendencies to complete news analysis. However, these methods are easily affected by the data types of various platforms, making it difficult for the vocabulary of news texts to match the data requirements of each platform. This results in unclear data levels, single features, and low accuracy in subsequent sentiment analysis and data statistics, and a lack of effective statistics on sentiment. Summary of the Invention

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: an intelligent statistical analysis method for cross-platform news dissemination data, including: S1, for users' data capture needs on any candidate platform, taking the original news text as input, extracting keywords from the original news text, decomposing and analyzing the keywords, and then capturing the corresponding news data.

[0007] S2 extracts extended vocabulary from the crawled news text, uses keywords as anchors, performs vocabulary clustering on extended vocabulary and keywords, constructs an extended vocabulary library, and determines the sentiment range of extended vocabulary in the positive or negative dimension by combining the mapping relationship between each extended vocabulary and keywords after clustering.

[0008] S3, based on the sentiment range of the current expanded vocabulary, performs semantic splicing using the contextual sentence structure of the paragraph where the sentiment range is located, and uses a multi-head attention mechanism to output sentiment analysis results containing sentiment labels and positive ratings.

[0009] S4 constructs a news tree with the original news text as the root node, the mapping relationship between extended vocabulary and keywords as branch nodes, and the data corresponding to positive ratings and sentiment labels as leaf nodes.

[0010] S5, for each sentiment label and positive score combination under the same branch node, clarifies its comprehensive sentiment tendency, and then obtains the fuzzy sentiment characteristics of each branch node.

[0011] The beneficial effects of this invention are as follows: First, this invention constructs candidate keyword combinations according to a time-subject-action structure, determines the main keyword through semantic similarity comparison of word vectors, then filters secondary theme keywords based on semantic similarity with the main keyword, and obtains industry-specific keywords by calling an industry dictionary to form a hierarchical keyword set; and it also counts the extended vocabulary corresponding to the keywords, so that each candidate platform only captures data types that match the corresponding keywords, improving the accuracy of cross-platform data capture, enhancing the semantic correlation between data from different platforms during news data dissemination, and providing a processing foundation for subsequent data processing.

[0012] Second, this invention determines the clustering dimension based on user needs and clusters extended vocabulary and keywords according to semantic similarity to form an extended vocabulary library. Within each cluster, it establishes a precise mapping between extended vocabulary and keywords by using rules such as direct mapping with a single keyword, mapping with the maximum similarity among multiple keywords, and mapping with the maximum similarity among keywords in the absence of keywords. It collects paragraphs corresponding to the extended vocabulary, records the sentiment polarity through part-of-speech tagging, and outputs the corresponding positions of positive and negative sentiments in the paragraphs as sentiment intervals. This ensures the structure and effectiveness of the extended vocabulary library, avoids the loss of sentiment expression range, and allows sentiment intervals to correspond to specific paragraph segments, providing a position index for subsequent context fusion analysis.

[0013] Third, this invention maps the sentiment intensity value of the sentiment interval to an initial sentiment vector, extracts the context sentence structure and converts it into an embedded sentiment vector through word embedding, and concatenates the two to construct a sentiment feature vector; inputs the feature vector into a multi-head attention sentiment classifier, takes the maximum probability value of the classification branch as the sentiment label, and obtains the output of the regression branch as the positive score; records the ratio of positive and negative sentiment words in the corresponding paragraphs of the extended vocabulary to the total vocabulary, and updates it to the sentiment interval; making the quantitative recording of sentiment intervals and sentiment information more comprehensive, and making the value range of sentiment labels and positive scores corresponding to each extended vocabulary clearer when the tree structure is summarized, so as to reflect the sentiment tendency of each platform under the news extended capture in a timely manner.

[0014] Fourth, this invention constructs branch nodes with the original news text as the root node, including keyword branches, extended word branches, and mapping relationship association layers. The mapping relationship association layer records related information. Leaf nodes such as positive ratings and sentiment tags are bound to extended word branches to complete the hierarchical construction. It realizes the explicit storage of keyword and extended word association information and sorts the same level according to semantic similarity, giving priority to branch nodes with high core relevance, thus improving the efficiency of data retrieval and information acquisition. It makes the range of fuzzy sentiment features in the final output clear, providing clear guidance for back-end staff to conduct in-depth public opinion mining and other operations. It improves the efficiency and accuracy of cross-platform statistics of news data. Attached Figure Description

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0016] Figure 1 This is a flowchart illustrating an intelligent statistical analysis method for cross-platform data in news dissemination.

[0017] Figure 2 This is a flowchart illustrating step S1 of an intelligent statistical analysis method for cross-platform data in news dissemination.

[0018] Figure 3 This is a flowchart illustrating step S2 of an intelligent statistical analysis method for cross-platform data in news dissemination.

[0019] Figure 4 This is a flowchart illustrating step S4 of an intelligent statistical analysis method for cross-platform data in news dissemination.

[0020] Figure 5 This is a flowchart illustrating step S5 of an intelligent statistical analysis method for cross-platform data in news dissemination. Detailed Implementation

[0021] The embodiments of the present invention are described in detail below. The embodiments described below are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Where specific techniques or conditions are not specified in the embodiments, they shall be performed in accordance with the techniques or conditions described in the literature in the art or in accordance with the product manual.

[0022] See Figure 1 A method for intelligent statistical analysis of cross-platform news dissemination data includes: S1, for users' data capture needs on any candidate platform, taking the original news text as input, extracting keywords from the original news text, decomposing and analyzing the keywords, and then capturing the corresponding news data.

[0023] S2 extracts extended vocabulary from the crawled news text, uses keywords as anchors, performs vocabulary clustering on extended vocabulary and keywords, constructs an extended vocabulary library, and determines the sentiment range of extended vocabulary in the positive or negative dimension by combining the mapping relationship between each extended vocabulary and keywords after clustering.

[0024] S3, based on the sentiment range of the current expanded vocabulary, performs semantic splicing using the contextual sentence structure of the paragraph where the sentiment range is located, and uses a multi-head attention mechanism to output sentiment analysis results containing sentiment labels and positive ratings.

[0025] S4 constructs a news tree with the original news text as the root node, the mapping relationship between extended vocabulary and keywords as branch nodes, and the data corresponding to positive ratings and sentiment labels as leaf nodes.

[0026] S5, for each sentiment label and positive score combination under the same branch node, clarifies its comprehensive sentiment tendency, and then obtains the fuzzy sentiment characteristics of each branch node.

[0027] Data scraping needs can be represented as users viewing original news text as the subject, collecting interactive data, content extension data, and sentiment data from multiple platforms such as social platforms, news media platforms, and vertical forums.

[0028] Interactive data only captures clicks, reposts, comments, etc., corresponding to content matching keywords, ignoring redundant non-interactive data, and is used to determine the interaction status after the current news spread; content extension data is based on keywords to capture extended terms, such as new energy vehicle technology breakthrough → technology implementation models, impact on car purchase costs, etc., and simultaneously captures the interaction data and sentiment data of sub-topics and similar topics corresponding to these extended terms on various platforms; sentiment data captures at least 100 text data (comments, posts, etc.) matching keywords on each candidate platform, recording the text content and labeling it with positive, negative, and neutral sentiment tags, for subsequent sentiment percentage statistics and sentiment meaning interpretation.

[0029] like Figure 2 As shown, the implementation of step S1 includes: S11, receiving the original news text input by the user, performing word segmentation on the original news text, and breaking it down into main keywords, secondary theme keywords and industry keywords to form a keyword set.

[0030] At this point, the main keywords are such as "2025 New Energy Vehicle Technology Breakthrough," and the secondary keywords are such as "New Energy Vehicle Range Improvement" and "Charging Technology Optimization." The industry keywords are such as "New Energy Vehicle" and "Automobile Manufacturing." The news themes, industry terms, and event terms that appear in the original news text are divided into multiple keywords according to their main description, secondary description, and industry-specific forms. The keywords are then compared with the corresponding data types to form a platform-data mapping table.

[0031] S12, count the number of candidate platforms corresponding to the original news text, assign a number to each candidate platform, and record the data type captured by each candidate platform. The data type will be represented as interactive data, content extension data, and sentiment data.

[0032] S13: Map each data type to the keyword set to obtain the crawling results corresponding to each keyword.

[0033] Extracting three categories of keywords from original news text requires following a logical flow: text preprocessing → initial vocabulary screening → classification determination. First, the text is cleaned and segmented, a general stop word list is loaded, and meaningless words and symbols are removed. The extracted words are then converted into word vectors, completing text preprocessing. Next, a combination of TF-IDF and TextRank can be used. The TF-IDF algorithm emphasizes the frequency of words in the current news, while the TextRank algorithm emphasizes the association weights of words in the text's semantic network. The weighted sum of the two algorithms is used as the basis for word selection, choosing the top 10% of words as candidate keywords. The TF-IDF and TextRank algorithms are publicly available technologies and will not be explained in detail here. The weighted sum used here represents the calculations of the two algorithms for corresponding words in the original news text. The result of the TF-IDF algorithm is considered the index value corresponding to the frequency of occurrence, and the result of the TextRank algorithm is considered the index value of the association weights.

[0034] Then, based on the semantics of the words, high-frequency words that are unrelated to the news topic are extracted. For example, if the word "mobile phone" appears frequently in a certain technology news article, but the actual topic is "new energy vehicles", then "mobile phone" will be removed; only words that are strongly related to the core semantics of the text will be retained.

[0035] Then, using the semantic similarity between the word vectors of the candidate keyword combination and the word vectors of the original news text as a quantitative indicator, the main keyword is selected from the candidate keywords; using the semantic similarity with the main keyword as a quantitative indicator, secondary theme keywords are selected from the remaining candidate keywords excluding the main keyword; and using industry semantic similarity as a quantitative indicator, industry-specific keywords are selected.

[0036] That is, the implementation of step S11 also includes: preprocessing the original news text and filtering candidate keywords based on the frequency of occurrence and association weight of words in the original news text.

[0037] Candidate keywords are grouped into candidate keyword combinations according to the structure of time, subject, and action. The semantic similarity between the word vector of the candidate keyword combination and the word vector of the original news text is calculated, and the combination with the highest semantic similarity is selected as the main keyword.

[0038] Secondary topic keywords are selected based on the semantic similarity between the remaining candidate keywords after excluding the main keyword and the main keyword.

[0039] The process involves comparing and filtering candidate keywords from a dictionary of industry-specific keywords to obtain the relevant industry-specific keywords. The industry dictionary records keywords relevant to a particular industry and is used to identify industry-related terms among high-frequency words.

[0040] When filtering these three types of keywords, the cosine similarity is used to calculate the semantic similarity between words. When selecting the main keyword, multiple words are combined (e.g., time, 2025), subject (e.g., new energy vehicles), and core action (e.g., technological breakthrough) to filter the main keyword and select the combination with the highest semantic similarity. If the input news text does not contain time, its subject and action are recorded, and the main keyword is extracted according to the form of subject and action.

[0041] When selecting secondary keywords, they should be used as supplementary descriptions to the primary keywords, supplementing them with dimensions such as effect, impact, and description. Multiple words with a semantic similarity greater than 70% can be selected for supplementation. As for the remaining words, they need to be compared with standard vocabulary such as industry dictionaries. Here, multiple words with a semantic similarity greater than 70% can also be selected as industry keywords.

[0042] In one embodiment of the present invention, when setting up an extended vocabulary database, the clustering is selected based on the type of keywords, and matching is performed step by step according to the order of primary keywords, secondary theme keywords, and industry keywords. For example, the primary keywords correspond to derivative words, synonyms, or near-synonyms, such as extended vocabulary in the manner of "2025 New Energy Vehicle Technology Breakthrough → 2025 New Energy Vehicle Enterprise Technology Innovation, New Energy Vehicle Core Technology Upgrade 2025," etc.; secondary theme keywords are further subdivided into scenario words and topic branch words, such as extended vocabulary describing specific scenarios such as "range improvement → winter range attenuation optimization, high-speed range stability, and urban commuting range performance"; industry keywords describe industry terms and upstream and downstream related words, such as "new energy vehicles → power batteries, charging pile construction, new energy vehicle purchase tax policy, and used car residual value rate," indicating the expression of the term in the upstream and downstream of the industry, and recording the corresponding sentiments and interactions of these expressions.

[0043] For each acquired extended vocabulary, the topic attribute of the vocabulary needs to be labeled, such as the form of subtopic or similar topic, and the current extended vocabulary should be labeled accordingly. The extraction method of extended vocabulary is the same as that of keyword extraction. Extended words will also be set according to the three types of data extracted from keywords. However, when setting words corresponding to the main keyword, at least one word should be set in the text extracted from each candidate platform. That is, each candidate platform will obtain multiple sets of news, and each set of news will set one extended vocabulary corresponding to the main keyword.

[0044] like Figure 3As shown, the implementation method of step S2 includes: S21, determining the clustering dimension of the current extended vocabulary and keywords based on the user's data crawling needs; the clustering dimension represents the needs of the currently viewed vocabulary to be derived words, synonyms or near-synonyms, or scene words, topic branch words, industry terms and upstream and downstream related words. Based on the current user's data crawling needs, the clustering dimension of the extended vocabulary and keywords involved at this time is determined.

[0045] S22, based on the clustering dimension corresponding to the current extended vocabulary, clusters are formed by semantic similarity between the extended vocabulary and keywords, and the clusters formed after clustering and the extended vocabulary within the clusters are regarded as the output extended vocabulary library.

[0046] The K-means clustering algorithm can be used. First, determine the keywords and extended words to be clustered according to the current clustering dimension. Calculate the semantic similarity between each extended word and the keyword. Take the maximum cosine similarity between the extended word and all keyword vectors. If this maximum value is greater than 0.7, the corresponding extended word is retained to avoid weakly related words interfering with the anchor point direction.

[0047] Then, initial centroids are selected from keywords and extended vocabulary with high semantic similarity, and the k value is set according to the elbow rule. The clusters after iterative clustering are used as the output data at this time.

[0048] S23, based on the keywords and extended vocabulary contained in each cluster, determine the mapping relationship between keywords and extended vocabulary; after the determination is completed, proceed directly to the next step of processing.

[0049] During the judgment, it is used to determine the mapping relationship between the extended words corresponding to each keyword in scenarios where only extended words exist in the cluster and multiple keywords exist in the cluster, so as to facilitate the subsequent classification of data.

[0050] The implementation of step S23 includes: when there is only one keyword in the cluster, directly establishing a mapping relationship between the keyword in the cluster and the extended vocabulary.

[0051] When a cluster contains more than one keyword, the semantic similarity between each keyword in the cluster and the extended vocabulary is calculated sequentially, and a mapping relationship is established between the extended vocabulary and the keyword with the highest semantic similarity value.

[0052] When a cluster does not contain a keyword, the semantic similarity of the extended vocabulary in the current cluster with the keywords in other clusters is calculated, and the keyword with the highest semantic similarity to the extended vocabulary in the current cluster is selected to establish a mapping relationship.

[0053] Suppose there are clusters A, B, and C, each containing five words. Cluster A contains only one keyword A1, cluster B contains two keywords B2 and B3, and cluster C contains no keywords. Cluster A can directly establish mapping relationships between A1 and other keywords to illustrate the multiple words extended by A1. In cluster B, for each extended word, semantic similarity is calculated with B2 and B3 in turn. The keyword with the highest similarity to these two keywords is found, and a mapping relationship is established. In cluster C, keywords from all other clusters are extracted, and the keyword with the highest similarity value calculated with the extended words in cluster C is selected as the current mapping relationship.

[0054] S24. After the mapping relationship is processed, collect the paragraphs corresponding to the extended vocabulary, record the sentiment polarity of the paragraphs corresponding to the extended vocabulary by using part-of-speech tagging, and use the positions of the positive and negative sentiments of the paragraphs as the output sentiment range.

[0055] The sentiment interval refers to the semantic unit in the text that directly carries the sentiment tendency (such as positive or negative). When setting the sentiment interval, the BERT-WWM model will be used to perform part-of-speech tagging on the input text, identify basic sentiment words (such as like) and their sentiment polarity (positive / negative), and label the sentiment intensity value (1-5 levels) of each word.

[0056] It should be explained that after labeling the emotional intensity corresponding to positive and negative emotions in the current sentiment interval, the remaining neutral sentiment words will be recorded. When outputting sentiment tags in the future, the existence form of neutral emotions will be determined by aggregating the context text, and used in the subsequent summary and statistics of fuzzy sentiment features.

[0057] In one embodiment of the present invention, in step S3, the context of the paragraph containing the sentiment interval will be checked, and the sentiment intervals of strong positive / weak positive / neutral / weak negative / strong negative will be converted into annotable sentiment categories. At the same time, cross-platform text semantic refinement classification will be combined to ensure that the category matches the news dissemination scenario.

[0058] The implementation of step S3 also includes: S31, based on the sentiment range of the current expanded vocabulary, mapping the sentiment intensity value of the sentiment range to the initial sentiment vector.

[0059] S32, extract the contextual sentence structure of the paragraph containing the sentiment interval, and transform the contextual sentence structure into an embedded sentiment vector through word embedding; concatenate the embedded sentiment vector with the initial sentiment vector to construct the sentiment feature vector.

[0060] S33. Based on the output of the multi-head attention sentiment classifier, obtain the sentiment label probability value of the classification branch output, and take the part with the largest probability value as the output sentiment label.

[0061] S34. Simultaneously, based on the output of the multi-head attention sentiment classifier, the positive score of the regression branch output is obtained as the positive score of the output.

[0062] In step S3, the position index of the sentiment words, intensity values ​​and sentiment intervals marked in step S24 will be directly referenced, and the text fragments where the sentiment regions are located will be embedded in the sentence structure according to the context of the paragraph they belong to, so as to achieve multi-attention processing of the context.

[0063] First, determine the part-of-speech tagging and sentiment intensity value of the paragraph containing the extended vocabulary. Normalize the sentiment intensity value of the corresponding sentiment word tagging into an initial sentiment vector. Then, based on the position index of the sentiment word, extract the context sentences before and after the corresponding paragraph. Use a sliding window (e.g., window size = 5 words, or use continuous paragraphs) to extract sentence fragments. Obtain the embedded sentiment vector after word embedding through the BERT model. Then, concatenate the two to obtain a multi-dimensional sentiment feature vector. The dimension can be 770 or other sizes.

[0064] Then, a multi-head attention mechanism is applied to the sentiment feature vector to output a multi-head attention output vector, which can be set to a 770-dimensional sentiment feature vector. For the sentiment probability distribution of the classification branch and the original positive score of the regression branch, the implementation process will be based on the multi-head attention output vector, and the hierarchical weights and bias weights will be initialized respectively.

[0065] Such as multi-head attention output vector Layered weighting of sentiment tags and bias weights ;in, The matrix represents a 770-row x 5-column matrix over the real number field, mapping the 770 dimensions of multi-head attention to five sentiment categories (strong positive, weak positive, neutral, weak negative, and strong negative). These five sentiment categories represent the sentiment category of specific words. The bias weights describe... This represents the bias generated when mapped to the five categories. At this point, more emotion categories or dimensions can be set as needed.

[0066] Stratified weighting of positive ratings and bias weights The positivity score will be mapped to one dimension, which is the score that the positivity score can be obtained after passing through the multi-head attention output vector.

[0067] Raw scores of classification branches ;in, The raw scores represent the sentiment classification, corresponding to five sentiment categories. Each value is the raw score for the corresponding sentiment. The higher the value, the more the model tends to be in that category. Then, the raw scores are normalized using softmax to obtain the sentiment probability distribution, and thus the output sentiment label.

[0068] The calculation process for the original score of the regression branch is the same as that for the classification label. However, the original score output by the Sigmoid needs to be processed to map its value to the range of 0-1. Then, it is multiplied by 10 to obtain a score of 0-10. This score is the positive score output by the regression branch.

[0069] Preferably, step S3 is further implemented by: based on the output sentiment tags, recording the ratio of the number of words containing positive and negative sentiments in the paragraph corresponding to each extended vocabulary to the total number of words, setting the positive sentiment ratio and negative sentiment ratio of each extended vocabulary in turn, and updating it to the sentiment interval.

[0070] At this point, multiple paragraphs corresponding to each extended term after data capture will be recorded. These paragraphs will be statistically analyzed according to positive and negative sentiment. The ratio of the number of terms with positive sentiment to the total number of terms corresponding to that extended term will be considered the positive sentiment ratio, and the ratio of the number of terms with negative sentiment to the total number of terms corresponding to that extended term will be considered the negative sentiment ratio. For example, if there are 62 / 90 positive comments on Weibo and 40 / 60 positive comments on Douyin regarding technological innovations in new energy vehicle companies, then the positive sentiment ratio is 68%. The negative comments are 18 / 90 and 10 / 60 respectively, so the negative sentiment ratio is 18.7%. This allows us to know the specific sentiment ratio in the text paragraphs corresponding to the extended terms. These data will describe the intensity and distribution range of the positive / negative sentiment, providing a data foundation for subsequent detailed sentiment classification and explaining the macro-sentiment distribution of the current extended terms in the positive dimension. Finally, the sentiment intensity of each extended term will be clarified, and the overall sentiment ratio of the original news text can be obtained after summarizing, which will help to locate relevant terms in public opinion dissemination.

[0071] In one embodiment of the present invention, when constructing a news tree, the middle nodes represent the mapping relationship between keywords and extended vocabulary in the news text, and the leaf nodes describe the candidate platforms for the corresponding vocabulary query, the number of texts queried, and the sentiment tags and positive ratings obtained from these texts. According to the mapping relationship, the sentiment tendency and positive rating of each extended vocabulary can be statistically analyzed, and these sentiment tendencies and positive ratings can be mapped to the corresponding keywords to describe the macro trend obtained by each candidate platform based on vocabulary decomposition when querying the extended platform.

[0072] The root node of the news tree is the original input news text. Its branch nodes are intermediate nodes with child nodes. It is necessary to determine the keywords, extended vocabulary, and their mapping relationships within these nodes. Figure 4 As shown, the implementation of step S4 also includes: S41, constructing branch nodes of the news tree. The branch nodes include keyword branches, extended word branches, and mapping relationship association layers. The mapping relationship association layers are used to establish mapping relationships between keyword branches and extended word branches, and record the association information corresponding to the mapping relationships.

[0073] S42 associates and binds the leaf nodes with the corresponding extended word branches to complete the hierarchical construction of the news tree.

[0074] S43, based on the mapping relationship corresponding to the branch nodes, arrange the branch nodes at the same level in descending order of semantic similarity.

[0075] The resulting tree structure will have a relationship of root node → keyword branch → extended word branch → leaf node. Based on the number of keywords and extended words, a tree structure for cross-platform data aggregation will be formed.

[0076] In one embodiment of the present invention, the same branch node represents the branch node at the same level. The average positive score and sentiment tendency after the statistics are calculated based on the keyword branch and the extended word branch respectively, and set as the fuzzy sentiment feature of the corresponding branch.

[0077] The implementation of step S5 includes: S51, based on the sentiment label of each word under the branch node, the sentiment ratio under each branch node is standardized according to the proportion of positive sentiment and the proportion of negative sentiment. That is, the neutral sentiment label corresponding to the sentiment label in each branch stage is removed and not included in the proportion calculation of positive and negative sentiment, and is only used for subsequent fuzzy judgment.

[0078] S52, if the current branch node is a keyword, retrieve the percentage of text corresponding to the extended terms under each branch node, set the weight of each extended term under the same branch node, and use the standard deviation of the sentiment percentage weighted by the positive score and the weighted average positive score as the output fuzzy sentiment feature. Here, the percentage of text refers to the ratio of the number of texts counted under the extended terms associated with the current keyword to the total number of texts under the current keyword.

[0079] It should be noted that the number of texts counted in these statistics refers to texts containing sentiment labels, in order to clearly identify specific texts containing sentiment labels of strong positive / weak positive / neutral / weak negative / strong negative.

[0080] For multiple branch nodes under a keyword, data statistics are performed on the extended vocabulary under that keyword. The emphasis will be on the number of texts corresponding to different extended vocabulary, as well as the standard deviation of the positive sentiment ratio and the negative sentiment ratio after summarizing the extended vocabulary, to explain the dispersion of sentiment and whether the multiple branch nodes corresponding to the keyword belong to the part with clear sentiment.

[0081] The standard deviation of the weighted emotional proportions represents the weighting of the positive and negative emotional proportions respectively.

[0082] Such as the standard deviation of the weighted proportion of positive sentiment , ;in, This represents the weight of the i-th extended vocabulary, where i represents the index of its extended vocabulary, and the number is determined based on the number of words mapped to the current keyword; This represents the percentage of positive sentiment in the text statistics of the i-th extended word at the corresponding node. This represents the average percentage of positive sentiment; the standard deviation calculated here represents the sentiment trend of the expanded vocabulary. The percentage of negative sentiment is calculated in the same way and is not shown. The sentiment percentage can also be specified as a percentage of strong positive sentiment, a percentage of weak positive sentiment, etc.

[0083] Positive score weighting can emphasize the number of sentiment concentration trends after branch node statistics. These data will be output as fuzzy sentiment features in the form of text + numbers, according to the way they are assigned values.

[0084] S53, if the current branch node is an extended vocabulary, it means that only its associated leaf nodes are counted, the number of candidate platforms corresponding to the current branch node is determined, and weights are set according to the proportion of texts corresponding to each candidate platform. The weighted average sentiment proportion and the value range of the positive score are used as the output fuzzy sentiment features. Here, the proportion of texts represents the ratio of the number of texts counted under the candidate platforms associated with the current extended vocabulary to the total number of texts under the current extended vocabulary.

[0085] When processing keyword branches and extended term branches, the keyword branch focuses on sentiment trends and differences, reflecting the overall sentiment trend of the keyword and the description of the consistency of sentiment tendency among various extended terms; the extended term branch needs to determine the overall sentiment tendency and positive sentiment fluctuations of the candidate platforms to reflect the consistency and fluctuation range of sentiment across different platforms, so as to realize news dissemination statistics in cross-platform scenarios.

[0086] Neutral sentiment labels are output based on the position index and dominant tendency when the extended vocabulary is statistically analyzed. For example, if the extended vocabulary W3 contains words with neutral sentiment labels, the proportion of positive sentiment is greater than the proportion of negative sentiment when W3 is statistically analyzed alone, positive sentiment is the dominant tendency, and the proportion of positive sentiment is close to that of negative sentiment, etc., the fuzzy sentiment features after the statistics are obtained.

[0087] like Figure 5 As shown, the implementation of step S5 also includes: taking the root node of the current news tree as the starting point, mapping the fuzzy sentiment features statistically analyzed on the same branch node to the root node, and determining the scalable range of each branch node based on the number of nodes connected to each branch node, and updating the scalable range to the fuzzy sentiment features.

[0088] The scalability range represents the number of nodes connected to the branch nodes, namely the number of extended word branches associated with the keyword branch and the number of leaf nodes associated with the extended word branches. The larger this value, the more detailed the keyword is when it is expanded, and the clearer the scope of news dissemination it describes will be. When these associated numbers are synchronized to the output fuzzy sentiment features, the dimensions of news expansion retrieved by different keywords can be clarified, providing guidance for subsequent public opinion trend analysis and making the use of tree structures or other decision-making methods more valuable for decision-making.

[0089] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, which are still covered within the protection scope of the present invention.

Claims

1. A method for intelligent statistical analysis of cross-platform data in news dissemination, characterized in that, include: S1, targeting users' data scraping needs on any candidate platform, takes the original news text as input, extracts the keywords from the original news text, decomposes and analyzes the keywords, and then scrapes the corresponding news data. S2: Extract extended vocabulary from the crawled news text, use keywords as anchors, cluster the extended vocabulary and keywords to build an extended vocabulary library, and combine the mapping relationship between each extended vocabulary and keywords after clustering to determine the sentiment range of the extended vocabulary in the positive or negative dimension. S3, based on the sentiment range of the current expanded vocabulary, performs semantic splicing using the contextual sentence structure of the paragraph where the sentiment range is located, and uses a multi-head attention mechanism to output sentiment analysis results containing sentiment labels and positive ratings; S4 constructs a news tree with the original news text as the root node, the mapping relationship between extended words and keywords as branch nodes, and the data corresponding to positive ratings and sentiment labels as leaf nodes. S5. For each sentiment label and positive score combination under the same branch node, clarify its comprehensive sentiment tendency, and then obtain the fuzzy sentiment characteristics of each branch node. The implementation of step S2 includes: S21, determining the clustering dimension of the current extended vocabulary and keywords based on the user's data crawling needs; S22, clustering based on the semantic similarity between the extended vocabulary and keywords according to the clustering dimension corresponding to the current extended vocabulary, and considering the clusters formed after clustering and the extended vocabulary within the clusters as the output extended vocabulary library; S23, determining the mapping relationship between keywords and extended vocabulary based on the keywords and extended vocabulary contained in each cluster; S24, after the mapping relationship is processed, collecting the paragraphs corresponding to the extended vocabulary, recording the sentiment polarity of the paragraphs corresponding to the extended vocabulary in the form of part-of-speech tagging, and using the positions corresponding to the positive and negative sentiments of the paragraphs as the output sentiment intervals; The implementation of step S5 includes: S51, based on the sentiment label of each word under the branch node, standardizing the sentiment ratio of each branch node according to the proportion of positive sentiment and the proportion of negative sentiment; S52, if the current branch node is a keyword, retrieving the proportion of text corresponding to the extended words under each branch node, setting the weight of each extended word under the same branch node, and using the standard deviation of the weighted sentiment ratio and the weighted average positive score as the output fuzzy sentiment feature; S53, if the current branch node is an extended word, determining the number of candidate platforms corresponding to the current branch node, setting the weight according to the proportion of text corresponding to each candidate platform, and using the range of the weighted average sentiment ratio and the positive score as the output fuzzy sentiment feature.

2. The intelligent statistical analysis method for cross-platform data in news dissemination according to claim 1, characterized in that, The implementation methods for step S1 include: S11 receives the original news text input by the user, performs word segmentation on the original news text, and breaks it down into main keywords, secondary theme keywords and industry keywords to form a keyword set; S12, count the number of candidate platforms corresponding to the original news text, assign a number to each candidate platform, and record the data type captured by each candidate platform; S13: Map each data type to the keyword set to obtain the crawling results corresponding to each keyword.

3. The intelligent statistical analysis method for cross-platform data in news dissemination according to claim 2, characterized in that, The implementation of step S11 also includes: The original news text is preprocessed to filter candidate keywords based on the frequency of occurrence and association weight of words in the original news text; Candidate keywords are grouped into candidate keyword combinations according to the structure of time, subject, and action. The semantic similarity between the word vector of the candidate keyword combination and the word vector of the original news text is calculated, and the combination with the highest semantic similarity is used as the main keyword. Secondary topic keywords are selected based on the semantic similarity between the remaining candidate keywords after excluding the main keyword and the main keyword. After comparing and filtering secondary topic keywords using an industry dictionary, candidate keywords are obtained to determine the relevant industry keywords.

4. The intelligent statistical analysis method for cross-platform data of news dissemination according to claim 3, characterized in that, The implementation methods of step S23 include: When a cluster contains only one keyword, a mapping relationship is directly established between the keyword in the cluster and the extended vocabulary; When a cluster contains more than one keyword, the semantic similarity between each keyword in the cluster and the extended vocabulary is calculated sequentially, and a mapping relationship is established between the extended vocabulary and the keyword with the highest semantic similarity value. When a cluster does not contain a keyword, the semantic similarity of the extended vocabulary in the current cluster with the keywords in other clusters is calculated, and the keyword with the highest semantic similarity to the extended vocabulary in the current cluster is selected to establish a mapping relationship.

5. The intelligent statistical analysis method for cross-platform data in news dissemination according to claim 1, characterized in that, The implementation of step S3 also includes: S31, based on the sentiment range of the current expanded vocabulary, map the sentiment intensity value of the sentiment range to the initial sentiment vector; S32, extract the contextual sentence structure of the paragraph containing the sentiment interval, and transform the contextual sentence structure into an embedded sentiment vector through word embedding; concatenate the embedded sentiment vector with the initial sentiment vector to construct the sentiment feature vector; S33. Based on the output of the multi-head attention sentiment classifier, obtain the sentiment label probability value of the classification branch output, and take the part with the largest probability value as the output sentiment label. S34. Simultaneously, based on the output of the multi-head attention sentiment classifier, the positive score of the regression branch output is obtained as the positive score of the output.

6. The intelligent statistical analysis method for cross-platform news dissemination data according to claim 5, characterized in that, The implementation of step S3 also includes: Based on the output sentiment tags, the ratio of the number of words with positive and negative sentiments in the corresponding paragraph of each extended vocabulary to the total number of words is recorded. The positive sentiment ratio and negative sentiment ratio of each extended vocabulary are set sequentially and updated to the sentiment range.

7. The intelligent statistical analysis method for cross-platform data in news dissemination according to claim 1, characterized in that, The implementation of step S4 also includes: S41, Construct the branch nodes of the news tree. The branch nodes include keyword branches, extended word branches, and mapping relationship association layers. The mapping relationship association layers are used to establish the mapping relationship between keyword branches and extended word branches, and at the same time record the association information corresponding to the mapping relationship. S42, associate and bind the leaf nodes with the corresponding extended word branches to complete the hierarchical construction of the news tree; S43, based on the mapping relationship corresponding to the branch nodes, arrange the branch nodes at the same level in descending order of semantic similarity.

8. The intelligent statistical analysis method for cross-platform data in news dissemination according to claim 1, characterized in that, The implementation of step S5 also includes: Starting from the root node of the current news tree, the fuzzy sentiment features statistically analyzed on the same branch node are mapped to the root node. Based on the number of nodes connected to each branch node, the scalable range of each branch node is determined, and the scalable range is updated to the fuzzy sentiment features.

Images