A method for filtering public opinion knowledge graph based on the credibility of news sources

Abstract

The present invention discloses a method for filtering a public opinion knowledge graph based on the credibility of news sources, including S1, generating an initial news data set; S2, establishing a credibility evaluation model; S3, retaining high-credibility news data; S4, using natural language processing technology to perform text preprocessing on the retained news data; S5, generating preliminary entity and relationship data; S6, constructing an initial public opinion knowledge graph based on the preliminary entity and relationship data; S7, dynamically updating the public opinion knowledge graph; S8, identifying public opinion attitudes and trends towards specific events or topics; S9, optimizing the data quality of the knowledge graph; S10, performing public opinion analysis based on the optimized public opinion knowledge graph and generating a public opinion report. The present invention effectively solves the defects and deficiencies in the prior art, realizes high-quality and time-effective public opinion analysis, and provides users with reliable decision support and management reference.

Description

Technical Field

[0001] The present invention relates to the technical field of public opinion knowledge graphs, and in particular to a public opinion knowledge graph filtering method based on the credibility of news sources. Background Art

[0002] With the rapid development of the Internet and information technology, news media and online platforms have become important channels for obtaining information. However, the massive amount of information also brings about problems with information quality and reliability. Especially in the field of public opinion analysis, how to filter out high-quality news data from numerous information sources has become a technical problem that needs to be solved urgently. Existing public opinion analysis systems face the following major defects when processing large amounts of news data:

[0003] First, it is difficult for existing public opinion analysis systems to ensure the reliability of news sources. In the Internet environment, news sources are numerous and complex. Traditional public opinion analysis methods usually treat all news data equally and lack an effective evaluation and screening mechanism for the credibility of news sources. This results in low-credibility news information being mixed in the data set, affecting the accuracy and reliability of public opinion analysis. For example, although the amount of user-generated content on some social media platforms is huge, its credibility is often difficult to guarantee, which directly affects the reliability of the analysis results. The traditional public opinion knowledge graph construction method lacks a dynamic update mechanism. Public opinion analysis needs to reflect the latest news events and public opinion changes in a timely manner, while existing knowledge graphs are usually static and difficult to dynamically update according to newly emerging news data. This lack of timeliness in knowledge graphs often cannot provide accurate and timely public opinion information in practical applications and cannot meet users' needs for real-time data. In addition, the accuracy of the knowledge graph will gradually decrease over time, affecting the effect of public opinion analysis. In the process of public opinion analysis, the existing technology is relatively rough in the sentiment analysis and emotional tendency evaluation of news content. Sentiment analysis is an important part of public opinion analysis. By identifying the emotional tendencies in news texts, we can judge the public's attitudes and trends towards specific events or topics. However, existing sentiment analysis technologies usually only perform simple positive and negative classifications and cannot deeply understand the complex emotional information of news texts. For example, when faced with texts containing complex emotions such as sarcasm and humor, traditional sentiment analysis technologies often have difficulty accurately identifying them, resulting in biased analysis results.

[0004] In addition, the existing public opinion analysis system lacks a deep semantic understanding of news data during data processing and analysis. The development of natural language processing technology has provided new methods and tools for text analysis, but its application in the field of public opinion analysis is not yet extensive and in-depth enough. Traditional methods usually only perform basic word segmentation and part-of-speech tagging, lack a deep semantic understanding and analysis of news texts, and are difficult to fully and accurately extract key information and relationships in news.

[0005] Finally, in the process of public opinion analysis, the existing technology lacks an effective filtering mechanism to remove low-quality and irrelevant information. A large amount of irrelevant information not only increases the burden of data processing, but also interferes with the results of public opinion analysis. Existing filtering methods are usually based on keyword matching or simple statistical models, which cannot effectively screen according to the actual quality and relevance of news content, resulting in a large amount of noise data in the analysis results.

[0006] In summary, the existing public opinion analysis technology has many defects in news source credibility evaluation, knowledge graph dynamic update, sentiment analysis depth, semantic understanding and information filtering, and it is difficult to meet the needs of practical applications. In response to these technical problems, a public opinion knowledge graph filtering method based on news source credibility is proposed, aiming to improve the reliability and accuracy of public opinion analysis data, and realize the dynamic update and optimization of public opinion knowledge graph, so as to provide higher quality public opinion analysis services. Summary of the invention

[0007] One purpose of the present invention is to propose a method for filtering public opinion knowledge graphs based on the credibility of news sources. The present invention effectively solves the defects and deficiencies in the prior art, realizes high-quality and timely public opinion analysis, and provides users with reliable decision-making support and management reference.

[0008] According to an embodiment of the present invention, a method for filtering a public opinion knowledge graph based on the credibility of news sources includes the following steps:

[0009] S1, automatically collect news reports from various news media and online platforms to generate an initial news dataset;

[0010] S2. By establishing a credibility evaluation model, the credibility of each news source in the initial news data set is analyzed and evaluated;

[0011] S3. Based on the credibility analysis and evaluation results, filter out low-credibility news sources and only retain high-credibility news data;

[0012] S4. Use natural language processing technology to perform text preprocessing on the retained news data, including word segmentation, stop word removal, and syntactic analysis;

[0013] S5, performing named entity recognition and relationship extraction on the preprocessed news text to generate preliminary entity and relationship data;

[0014] S6. Based on the preliminary entity and relationship data, construct the initial public opinion knowledge graph and represent the entities and their relationships in the news as structured data;

[0015] S7. Dynamically update the public opinion knowledge graph as new news data emerges;

[0016] S8. Use sentiment analysis technology to evaluate the sentiment tendency of news content in the public opinion knowledge graph and identify public opinion attitudes and trends towards specific events or topics;

[0017] S9. Based on the credibility evaluation and sentiment analysis results, further filter the public opinion knowledge graph to optimize the data quality of the knowledge graph;

[0018] S10. Conduct public opinion analysis based on the optimized public opinion knowledge graph and generate a public opinion report.

[0019] Optionally, the S1 specifically includes:

[0020] S11. Collect news coverage data from various news media and online platforms, where news media include but are not limited to news websites, social media platforms, and blogs;

[0021] S12: The collected news report data is formatted and processed to generate the initial news data set D 0 ;

[0022] S13. Define the initial news dataset:

[0023]

[0024] in, represents the i-th news report data;

[0025] S14. Report data for each news item Add source tag

[0026]

[0027] in, Indicates the source of the i-th news;

[0028] S15. News dataset D 1 Timestamp marking, the marking format is

[0029]

[0030] in, Indicates the release time of the i-th news.

[0031] Optionally, the S2 specifically includes:

[0032] S21. For each news source Define a credibility scoring model M;

[0033] S22. Use the credibility scoring model M to evaluate the initial news dataset D.2 Analyze historical data of each news source in the system and calculate its credibility score

[0034] S23. Credibility score The calculation formula is:

[0035]

[0036] in, Indicates the source of news The historical reliability of Indicates the quality of news reports, represents the expert evaluation of the news source, α, β, and γ are weight coefficients;

[0037] S24. For each news source Conduct multi-dimensional data collection, including news release frequency Historical Accuracy User feedback

[0038]

[0039] in, Indicates the source of news The accuracy score in the jth news, μ P is the average value of accuracy score, N is the total number of news;

[0040]

[0041] in, represents the quality score of news k, For news sources The average quality score of , M is the total number of quality scores;

[0042]

[0043] in, Indicates expert opinion on news source Rating, w l is the weight of expert l, and L is the total number of experts participating in the scoring;

[0044] S25. Input the multi-dimensional data into the credibility scoring model M to calculate the comprehensive credibility score And perform normalization;

[0045] S26. Based on the credibility score set C, the initial news dataset D 2 Filter to keep only credibility scores Higher than the preset threshold T cnews data, generate a credible news dataset D 3 :

[0046]

[0047] Optionally, the S4 specifically includes:

[0048] S41. For the credible news dataset D 3 Each news data in Perform word segmentation and generate word sequences

[0049] S42, word sequence Perform stop word processing to remove common but meaningless words and generate purified word sequences

[0050] S43. Purified word sequence Perform syntactic analysis and generate syntactic structures Including part-of-speech tagging and dependency analysis:

[0051]

[0052] in, Expressive words Part of speech, Expressive words Dependencies with other words;

[0053] S44, the news data after word segmentation, stop word removal and syntactic analysis As input, generate the preprocessed news dataset D 4 .

[0054] Optionally, the S5 specifically includes:

[0055] S51, the preprocessed news dataset D 4 Each news data in Use the enhanced pre-trained language model BERT for deep semantic understanding and generate embedding vectors containing contextual information

[0056]

[0057] in, Represents news data The context embedding vector of the jth word in , where n is the total number of words. The specific formula for generating the embedding vector is as follows:

[0058]

[0059] Among them, BERT represents the BERT model, represents the jth word and its context words;

[0060] S52. Using the transformer-based named entity recognition model SpanBERT to embed the vector Perform entity recognition and generate a set containing entity boundaries and categories

[0061]

[0062] Among them, b j and e j Respectively represent the starting and ending positions of the jth entity, c j represents the entity category, p is the total number of entities identified, and the specific identification process is as follows:

[0063]

[0064] in, Represents entity e j The probability of SpanBERT represents the SpanBERT model. and Respectively represent category c j The weights and biases of

[0065] S53. Using the relationship extraction model based on graph neural network, the entity set and its context embedding vector to perform relationship inference and generate a relationship set

[0066]

[0067] in, Represents entity pair The relationship types include cooperation, conflict, and affiliation. The specific formula for relationship inference is as follows:

[0068]

[0069] in, Represents the entity in the l+1th layer The embedding vector of represents the neighbor set of an entity pair, W (l) represents the weight matrix of the lth layer, represents the attention coefficient, a represents the parameter vector of the attention mechanism, and ∥ represents the vector connection operation;

[0070] S54. Use self-supervised learning methods to verify and correct entity and relationship data to ensure their accuracy and consistency, and generate corrected entity and relationship data D6 :

[0071]

[0072] The specific process of verification and correction is as follows:

[0073]

[0074] in, represents the loss function of self-supervised learning, Represents a given entity pair Type of relationship The probability of , λ is the regularization parameter, and Θ represents the model parameter.

[0075] Optionally, the S6 specifically includes:

[0076] S61, the modified entity and relationship data D 6 Each news data in Perform structural processing to generate a preliminary set of entities and relationships;

[0077]

[0078] Among them, e j Represents an entity, T j Indicates the timestamp, T ab and W ab Represent the timestamp and weight of the relationship respectively;

[0079] S62. Define the node set N and edge set E of the public opinion knowledge graph G, and improve it in combination with the specific attributes of the news data;

[0080]

[0081] S63. Represent each entity e in the entity set N as a graph node, where the node attributes include entity type, occurrence frequency, related news source, and timestamp:

[0082] e=(type,freq,sources,timestamp);

[0083] type = EntityType(e);

[0084]

[0085] timestamp=max{T j |e=e j};

[0086] Among them, type indicates the entity type, freq indicates the frequency of the entity, sources indicates the source of the relevant news, and timestamp indicates the time when the news was released. is the indicator function, when 1 when it is, otherwise 0;

[0087] S64. Represent each relation r in the relation set E as a graph edge, where the edge attributes include relation type, relation weight and timestamp:

[0088] r=(type,weight,timestamp);

[0089] type = RelationType(r);

[0090]

[0091] timestamp=max{T ab |r=(e a ,e b )};

[0092] Among them, type indicates the relationship type, weight indicates the relationship weight, and timestamp indicates the release time of the news corresponding to the relationship;

[0093] S65. Use time series analysis methods to dynamically update and maintain entities and relationships based on timestamps to ensure the timeliness of the graph;

[0094]

[0095] Where T represents the current time, Update(G,T) represents the updated graph, N' and E' represent the updated node and edge sets respectively;

[0096] S66. Use the graph database system to store entity and relationship data as a graph structure and generate an initial public opinion knowledge graph G:

[0097] G = (N, E);

[0098] S67. According to the frequency of occurrence of entities, the weight of relationships and the time of news release, the public opinion knowledge graph G is optimized and streamlined, and noise data and low-relevance nodes and edges are removed to ensure the accuracy and timeliness of the graph;

[0099] N={e∈N|freq>θ f and Recency(e,T)<θ t};

[0100] E={r∈E|weight>θ wand Recency(r,T)<θ t};

[0101] Among them, θ f ,θ w ,θ t are the thresholds of frequency, weight and time, and Recency(x,T) represents the recency of a node or edge relative to the current time T.

[0102] Optionally, the S8 specifically includes:

[0103] S81. Perform text sentiment analysis on each piece of news data in the public opinion knowledge graph G to generate a sentiment score set A:

[0104] A={a 1 ,a 2 ,…,a m};

[0105] Among them, a j represents the sentiment score of the jth news, and m is the total number of news;

[0106] S82. Use a deep learning-based sentiment analysis model to classify the sentiment of news texts:

[0107]

[0108] Among them, P(c j |d j ) indicates news j Belongs to the emotion category c j The probability of Represents the emotion category c j The parameter, f(d j ) indicates news text d j The eigenvector of

[0109] S83, normalizing the sentiment score set A to generate a normalized sentiment score set A';

[0110] S84. Associate the normalized sentiment score A' with the entities and relationships in the public opinion knowledge graph G to generate a public opinion knowledge graph G' with enhanced sentiment tendency:

[0111] G'=(N,E,A');

[0112] Among them, N is the node set, E is the edge set, and A' is the normalized sentiment score set;

[0113] S85. Analyze the public opinion knowledge graph G' with enhanced sentiment tendency, identify the public opinion attitude and trend towards specific events or topics, and generate a public opinion report R.

[0114] Optionally, the S85 specifically includes:

[0115] S851. Perform topic modeling on the public opinion knowledge graph G' and extract the main topics using the latent Dirichlet distribution model:

[0116] θ d ~Dirichlet(α);

[0117]

[0118] z d,n ~Multinomial(θd);

[0119]

[0120] Among them, θ d represents the topic distribution of document d, represents the word distribution of topic k, z d,n represents the topic of the nth word in the dth document, w d,n Represents the nth word in the dth document;

[0121] S852. Based on the results of topic modeling, identify the main sentiment of each topic k :

[0122]

[0123] Among them, D k represents the document set containing topic k, a′ d represents the sentiment score of document d;

[0124] S853. Calculate the public opinion trend of each topic through time series analysis method k :

[0125]

[0126] Among them, t i Indicates a point in time, Indicates time point t i The corresponding sentiment score, Slope represents the slope of the calculated sentiment score over time;

[0127] S854, comprehensive emotional attitude sentiment k and public opinion trends k Generate a public opinion report R, in which each topic contains the corresponding emotional attitude and public opinion trend information.

[0128] The beneficial effects of the present invention are:

[0129] (1) The present invention establishes a credibility assessment model to analyze and evaluate the credibility of news sources, ensuring that only highly credible news data is retained. Through a multi-dimensional data collection and evaluation model based on historical data, news quality, and expert ratings, low-credible news sources are effectively filtered out, significantly improving the quality of input data, and thereby improving the reliability and accuracy of public opinion analysis results.

[0130] (2) The present invention constructs an initial public opinion knowledge graph by performing structured processing on the corrected entity and relationship data, and uses a time series analysis method to dynamically update and maintain entities and relationships according to timestamps, thereby ensuring that the public opinion knowledge graph can reflect the latest news information in real time and maintain the timeliness and accuracy of the data.

[0131] (3) The present invention uses a deep learning-based sentiment analysis model to classify and evaluate the sentiment of news texts. Through the sentiment score calculation and normalization processing described in detail, combined with the entities and relationships in the public opinion knowledge graph, a public opinion knowledge graph with enhanced sentiment tendency is generated. This not only improves the depth of sentiment analysis, but also can identify the complex sentiment tendencies towards specific events or topics, and generate a more comprehensive and accurate public opinion report.

[0132] (4) The present invention introduces a multi-level filtering mechanism in multiple steps, filters low-credibility news data through credibility evaluation results, further filters low-credibility information in the knowledge graph based on sentiment analysis and credibility evaluation results, and optimizes and streamlines the public opinion knowledge graph according to the frequency of entity occurrence, the weight of the relationship and the time of news release. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0134] Figure 1 A flowchart of a method for filtering public opinion knowledge graph based on the credibility of news sources proposed by the present invention;

[0135] Figure 2 A flowchart of the specific steps for constructing an initial public opinion knowledge graph for a public opinion knowledge graph filtering method based on the credibility of news sources proposed in the present invention. DETAILED DESCRIPTION

[0136] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0137] refer to Figure 1-2, a method for filtering public opinion knowledge graph based on the credibility of news sources, comprising the following steps:

[0138] S1, automatically collect news reports from various news media and online platforms to generate an initial news dataset;

[0139] S2. By establishing a credibility evaluation model, the credibility of each news source in the initial news data set is analyzed and evaluated;

[0140] S3. Based on the credibility analysis and evaluation results, filter out low-credibility news sources and only retain high-credibility news data;

[0141] S4. Use natural language processing technology to perform text preprocessing on the retained news data, including word segmentation, stop word removal, and syntactic analysis;

[0142] S5, performing named entity recognition and relationship extraction on the preprocessed news text to generate preliminary entity and relationship data;

[0143] S6. Based on the preliminary entity and relationship data, construct the initial public opinion knowledge graph and represent the entities and their relationships in the news as structured data;

[0144] S7. Dynamically update the public opinion knowledge graph as new news data emerges;

[0145] S8. Use sentiment analysis technology to evaluate the sentiment tendency of news content in the public opinion knowledge graph and identify public opinion attitudes and trends towards specific events or topics;

[0146] S9. Based on the credibility evaluation and sentiment analysis results, further filter the public opinion knowledge graph to optimize the data quality of the knowledge graph;

[0147] S10. Conduct public opinion analysis based on the optimized public opinion knowledge graph and generate a public opinion report.

[0148] In this implementation, S1 specifically includes:

[0149] S11. Collect news coverage data from various news media and online platforms, where news media include but are not limited to news websites, social media platforms, and blogs;

[0150] S12: The collected news report data is formatted and processed to generate the initial news data set D 0 ;

[0151] S13. Define the initial news dataset:

[0152]

[0153] in, represents the i-th news report data;

[0154] S14. Report data for each news item Add source tag

[0155]

[0156] in, Indicates the source of the i-th news;

[0157] S15. News dataset D 1 Timestamp marking, the marking format is

[0158]

[0159] in, Indicates the release time of the i-th news.

[0160] In this implementation, S2 specifically includes:

[0161] S21. For each news source Define a credibility scoring model M;

[0162] S22. Use the credibility scoring model M to evaluate the initial news dataset D. 2 Analyze historical data of each news source in the system and calculate its credibility score

[0163] S23. Credibility score The calculation formula is:

[0164]

[0165] in, Indicates the source of news The historical reliability of Indicates the quality of news reports, represents the expert evaluation of the news source, α, β, and γ are weight coefficients;

[0166] S24. For each news source Conduct multi-dimensional data collection, including news release frequency Historical Accuracy User feedback

[0167]

[0168] in, Indicates the source of news The accuracy score in the jth news, μ P is the average value of accuracy score, N is the total number of news;

[0169]

[0170] in, represents the quality score of news k, For news sources The average quality score of , M is the total number of quality scores;

[0171]

[0172] in, Indicates expert opinion on news source Rating, w l is the weight of expert l, and L is the total number of experts participating in the scoring;

[0173] S25. Input the multi-dimensional data into the credibility scoring model M to calculate the comprehensive credibility score And perform normalization;

[0174] S26. Based on the credibility score set C, the initial news dataset D 2 Filter to keep only credibility scores Higher than the preset threshold T c news data, generate a credible news dataset D 3 :

[0175]

[0176] In this implementation, S4 specifically includes:

[0177] S41. For the credible news dataset D 3 Each news data in Perform word segmentation and generate word sequences

[0178] S42, word sequence Perform stop word processing to remove common but meaningless words and generate purified word sequences

[0179] S43. Purified word sequence Perform syntactic analysis and generate syntactic structures Including part-of-speech tagging and dependency analysis:

[0180]

[0181] in, Expressive words Part of speech, Expressive words Dependencies with other words;

[0182] S44, the news data after word segmentation, stop word removal and syntactic analysis As input, generate the preprocessed news dataset D 4 .

[0183] In this implementation, S5 specifically includes:

[0184] S51, the preprocessed news dataset D 4 Each news data in Use the enhanced pre-trained language model BERT for deep semantic understanding and generate embedding vectors containing contextual information

[0185]

[0186] in, Represents news data The context embedding vector of the jth word in , where n is the total number of words. The specific formula for generating the embedding vector is as follows:

[0187]

[0188] Among them, BERT represents the BERT model, represents the jth word and its context words;

[0189] S52. Using the transformer-based named entity recognition model SpanBERT to embed the vector Perform entity recognition and generate a set containing entity boundaries and categories

[0190]

[0191] Among them, b j and e j Respectively represent the starting and ending positions of the jth entity, c j represents the entity category, p is the total number of entities identified, and the specific identification process is as follows:

[0192]

[0193] in, Represents entity e j The probability of SpanBERT represents the SpanBERT model. and Respectively represent category c j The weights and biases of

[0194] S53. Using the relationship extraction model based on graph neural network, the entity set and its context embedding vector to perform relationship inference and generate a relationship set

[0195]

[0196] in, Represents entity pair The relationship types include cooperation, conflict, and affiliation. The specific formula for relationship inference is as follows:

[0197]

[0198] in, Represents the entity in the l+1th layer The embedding vector of represents the neighbor set of an entity pair, W (l) represents the weight matrix of the lth layer, represents the attention coefficient, a represents the parameter vector of the attention mechanism, and ∥ represents the vector connection operation;

[0199] S54. Use self-supervised learning methods to verify and correct entity and relationship data to ensure their accuracy and consistency, and generate corrected entity and relationship data D 6 :

[0200]

[0201] The specific process of verification and correction is as follows:

[0202]

[0203] in, represents the loss function of self-supervised learning, Represents a given entity pair Type of relationship The probability of , λ is the regularization parameter, and Θ represents the model parameter.

[0204] In this implementation, S6 specifically includes:

[0205] S61, the modified entity and relationship data D 6 Each news data in Perform structural processing to generate a preliminary set of entities and relationships;

[0206]

[0207] Among them, e j Represents an entity, Tj Indicates the timestamp, T ab and W ab Represent the timestamp and weight of the relationship respectively;

[0208] S62. Define the node set N and edge set E of the public opinion knowledge graph G, and improve it in combination with the specific attributes of the news data;

[0209]

[0210] S63. Represent each entity e in the entity set N as a graph node, where the node attributes include entity type, occurrence frequency, related news source, and timestamp:

[0211] e=(type,freq,sources,timestamp);

[0212] type = EntityType(e);

[0213]

[0214] timestamp=max{T j |e=e j};

[0215] Among them, type indicates the entity type, freq indicates the frequency of the entity, sources indicates the source of the relevant news, and timestamp indicates the time when the news was released. is the indicator function, when 1 when it is, otherwise 0;

[0216] S64. Represent each relation r in the relation set E as a graph edge, where the edge attributes include relation type, relation weight and timestamp:

[0217] r=(type,weight,timestamp);

[0218] type = RelationType(r);

[0219]

[0220] timestamp=max{T ab |r=(e a ,e b )};

[0221] Among them, type indicates the relationship type, weight indicates the relationship weight, and timestamp indicates the release time of the news corresponding to the relationship;

[0222] S65. Use time series analysis methods to dynamically update and maintain entities and relationships based on timestamps to ensure the timeliness of the graph;

[0223]

[0224] Where T represents the current time, Update(G,T) represents the updated graph, N' and E' represent the updated node and edge sets respectively;

[0225] S66. Use the graph database system to store entity and relationship data as a graph structure and generate an initial public opinion knowledge graph G:

[0226] G = (N, E);

[0227] S67. According to the frequency of occurrence of entities, the weight of relationships and the time of news release, the public opinion knowledge graph G is optimized and streamlined, and noise data and low-relevance nodes and edges are removed to ensure the accuracy and timeliness of the graph;

[0228] N′={e∈N|freq>θ f and Recency(r,T)<θ t};

[0229] E′={r∈E|weight>θ w and Recency(r,T)<θ t};

[0230] Among them, θ f ,θ w ,θ t are the thresholds of frequency, weight and time, and Recency(x,T) represents the recency of a node or edge relative to the current time T.

[0231] In this implementation, S8 specifically includes:

[0232] S81. Perform text sentiment analysis on each piece of news data in the public opinion knowledge graph G to generate a sentiment score set A:

[0233] A={a 1 ,a 2 ,…,a m};

[0234] Among them, a j represents the sentiment score of the jth news, and m is the total number of news;

[0235] S82. Use a deep learning-based sentiment analysis model to classify the sentiment of news texts:

[0236]

[0237] Among them, P(c j |d j ) indicates news j Belongs to the emotion category c j The probability of Represents the emotion category c j The parameter, f(d j ) indicates news text d j The eigenvector of

[0238] S83, normalizing the sentiment score set A to generate a normalized sentiment score set A';

[0239] S84. Associate the normalized sentiment score A' with the entities and relationships in the public opinion knowledge graph G to generate a public opinion knowledge graph G' with enhanced sentiment tendency:

[0240] G'=(N,E,A');

[0241] Among them, N is the node set, E is the edge set, and A' is the normalized sentiment score set;

[0242] S85. Analyze the public opinion knowledge graph G' with enhanced sentiment tendency, identify the public opinion attitude and trend towards specific events or topics, and generate a public opinion report R.

[0243] In this implementation, S85 specifically includes:

[0244] S851. Perform topic modeling on the public opinion knowledge graph G' and extract the main topics using the latent Dirichlet distribution model:

[0245] θ d ~Dirichlet(α);

[0246]

[0247] z d,n ~Multinomial(θd);

[0248]

[0249] Among them, θ d represents the topic distribution of document d, represents the word distribution of topic k, z d,n represents the topic of the nth word in the dth document, w d,n Represents the nth word in the dth document;

[0250] S852. Based on the results of topic modeling, identify the main sentiment of each topic k :

[0251]

[0252] Among them, D k represents the document set containing topic k, a′ d represents the sentiment score of document d;

[0253] S853. Calculate the public opinion trend of each topic through time series analysis method k :

[0254]

[0255] Among them, t i Indicates a point in time, Indicates time point t i The corresponding sentiment score, Slope represents the slope of the calculated sentiment score over time;

[0256] S854, comprehensive emotional attitude sentiment k and public opinion trends k Generate a public opinion report R, in which each topic contains the corresponding emotional attitude and public opinion trend information.

[0257] Example 1: In this example, an important social event is selected as the analysis object to simulate the collection, analysis and filtering process of news reports. The specific event is an emergency that occurred during an international sports event. The event was widely disseminated on multiple news media and social platforms, generating a large number of news reports and user comments. The goal of public opinion analysis is to identify the public's attitude and trend towards this event, and reflect it in the public opinion knowledge graph in a timely manner for relevant departments to make decisions and management.

[0258] In this scenario, news report data about the sports event incident is first automatically collected from various news media and online platforms to generate an initial news data set. This includes data obtained from multiple channels such as news websites, social media platforms, blogs, etc. In order to ensure the quality and credibility of the data, the method of the present invention is used to evaluate the credibility of news sources, filter out low-credibility news sources, and retain only high-credibility news data.

[0259] Next, the retained news data is preprocessed, including word segmentation, stop word removal, and syntactic analysis. Then, named entity recognition and relationship extraction techniques are used to extract key entities and relationship data from the preprocessed news text to generate preliminary entity and relationship data.

[0260] Based on the preliminary entity and relationship data, the initial public opinion knowledge graph is constructed to represent the entities and their relationships in the news as structured data. As new news data emerges, the public opinion knowledge graph is updated in real time through a dynamic update mechanism to ensure its timeliness and accuracy.

[0261] Subsequently, sentiment analysis technology is used to evaluate the sentiment tendency of the news content in the public opinion knowledge graph, identify the public's attitudes and trends towards specific events or topics, and generate a public opinion knowledge graph with enhanced sentiment tendency.

[0262] Finally, public opinion analysis is conducted based on the optimized public opinion knowledge graph, a public opinion report is generated, and the analysis results are displayed through a visual interface to provide high-quality decision-making support for relevant departments.

[0263] In this example, we selected news reports of an international sports event for simulation. A total of 5,000 news reports were collected from 20 news media and online platforms, including news websites (such as Toutiao), social media platforms (such as Sina Weibo), etc.

[0264] Through the credibility evaluation model, the credibility of these news sources was analyzed, and low-credibility news sources were filtered out. Finally, 15 news sources were retained, totaling 3,000 high-credibility news data. After text preprocessing, these high-credibility news data generated a data set containing keywords, entities, and relationships.

[0265] Using named entity recognition technology, key entities related to emergencies are extracted from news texts, such as "event name", "emergency description", "names of relevant personnel", etc. Relationship extraction technology further identifies the relationships between these entities, such as "participation", "influence", "report", etc., and ultimately generates preliminary entity and relationship data.

[0266] Based on these entity and relationship data, an initial public opinion knowledge graph is constructed. As the event progresses and new news reports emerge, the public opinion knowledge graph is dynamically updated to ensure that it contains the latest news data.

[0267] Sentiment analysis technology is used to evaluate the sentiment tendency of news content. The results show that 60% of news reports have a negative attitude towards the event, 30% of reports have a neutral attitude, and 10% of reports have a positive attitude. These sentiment tendency data are integrated into the public opinion knowledge graph to generate a public opinion knowledge graph with enhanced sentiment tendency.

[0268] In order to verify the effectiveness of the method of the present invention, the method of the present invention is compared with the traditional public opinion analysis method. The following is the comparison data:

[0269] Table 1 Comparison between the method of the present invention and the traditional public opinion analysis method

[0270] index Method of the present invention Traditional methods Data processing time (hours) 2 5 High confidence data ratio 60％ 40％ Sentiment classification accuracy 85％ 70％ Knowledge graph update frequency Real-time updates Once a day Noise data ratio 5％ 20％ Decision support satisfaction 90％ 65％

[0271] The data in Table 1 above show that the method of the present invention is significantly superior to the traditional method in multiple key indicators. Specifically, the method of the present invention effectively increases the proportion of high-credibility data through the credibility assessment model, significantly reduces data processing time, and improves the accuracy of sentiment classification and the update frequency of the knowledge graph. At the same time, the proportion of noise data is significantly reduced, and the decision support satisfaction is significantly improved.

[0272] Through the above embodiments, the practical application effect of the method of the present invention in public opinion analysis is fully demonstrated, which effectively solves the problems of data quality, timeliness and accuracy in the prior art and provides strong technical support for public opinion analysis.

[0273] The present invention establishes a credibility assessment model to conduct credibility analysis and evaluation on news sources, ensuring that only highly credible news data is retained. Through a multi-dimensional data collection and evaluation model based on historical data, news quality and expert ratings, low-credible news sources are effectively filtered out, significantly improving the quality of input data, and thereby improving the reliability and accuracy of public opinion analysis results.

[0274] The present invention constructs an initial public opinion knowledge graph by performing structured processing on the corrected entity and relationship data, and uses a time series analysis method to dynamically update and maintain entities and relationships according to timestamps, thereby ensuring that the public opinion knowledge graph can reflect the latest news information in real time and maintain the timeliness and accuracy of the data.

[0275] The present invention uses a deep learning-based sentiment analysis model to perform sentiment classification and evaluation on news texts. Through the sentiment score calculation and normalization processing described in detail, combined with the entities and relationships in the public opinion knowledge graph, a public opinion knowledge graph with enhanced sentiment tendency is generated. This not only improves the depth of sentiment analysis, but also can identify complex sentiment tendencies towards specific events or topics, and generate more comprehensive and accurate public opinion reports.

[0276] The present invention introduces a multi-level filtering mechanism in multiple steps, filters low-credibility news data through credibility evaluation results, further filters low-credibility information in the knowledge graph based on sentiment analysis and credibility evaluation results, and optimizes and streamlines the public opinion knowledge graph according to the frequency of entity occurrence, the weight of the relationship and the time of news release.

[0277] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for filtering public opinion knowledge graph based on the credibility of news sources, characterized in that: The steps include: S1, automatically collect news reports from various news media and online platforms to generate an initial news dataset; S2. By establishing a credibility evaluation model, the credibility of each news source in the initial news data set is analyzed and evaluated; S3. Based on the credibility analysis and evaluation results, filter out low-credibility news sources and only retain high-credibility news data; S4. Use natural language processing technology to perform text preprocessing on the retained news data, including word segmentation, stop word removal, and syntactic analysis; S5, performing named entity recognition and relationship extraction on the preprocessed news text to generate preliminary entity and relationship data; S6. Based on the preliminary entity and relationship data, construct the initial public opinion knowledge graph and represent the entities and their relationships in the news as structured data; S7. Dynamically update the public opinion knowledge graph as new news data emerges; S8. Use sentiment analysis technology to evaluate the sentiment tendency of news content in the public opinion knowledge graph and identify public opinion attitudes and trends towards specific events or topics; S9. Based on the credibility evaluation and sentiment analysis results, further filter the public opinion knowledge graph to optimize the data quality of the knowledge graph; S10. Perform public opinion analysis based on the optimized public opinion knowledge graph and generate a public opinion report; The S1 specifically includes: S11. Collect news coverage data from various news media and online platforms, where news media include but are not limited to news websites, social media platforms, and blogs; S12, processing the collected news report data into a unified format to generate an initial news data set D0; S13. Define the initial news dataset: in, represents the i-th news report data; S14. Report data for each news item Add source tag in, Indicates the source of the i-th news; S15. Timestamp the news dataset D1. The timestamp format is: in, Indicates the release time of the i-th news; The S2 specifically includes: S21. For each news source Define a credibility scoring model M; S22: Use the credibility scoring model M to analyze the historical data of each news source in the initial news data set D2 and calculate its credibility score. S23. Credibility score The calculation formula is: in, Indicates the source of news The historical reliability of Indicates the quality of news reports, represents the expert evaluation of the news source, α, β, and γ are weight coefficients; S24. For each news source Conduct multi-dimensional data collection, including news release frequency Historical Accuracy User feedback in, Indicates the source of news The accuracy score in the jth news, μ P is the average value of accuracy score, N is the total number of news; in, represents the quality score of news k, For news sources The average quality score of , M is the total number of quality scores; in, Indicates expert opinion on news source Rating, w l is the weight of expert l, and L is the total number of experts participating in the scoring; S25. Input the multi-dimensional data into the credibility scoring model M to calculate the comprehensive credibility score And perform normalization processing; S26. Filter the initial news dataset D2 according to the credibility score set C, and only keep the credibility scores Higher than the preset threshold T c News data from the source code, generate the credible news dataset D3:

2. According to claim 1, a method for filtering public opinion knowledge graph based on the credibility of news sources is characterized in that: The S4 specifically includes: S41. For each piece of news data in the credible news dataset D3 Perform word segmentation and generate word sequences S42, word sequence Perform stop word processing to remove common but meaningless words and generate purified word sequences S43. Purified word sequence Perform syntactic analysis and generate syntactic structures Including part-of-speech tagging and dependency analysis: in, Expressive words Part of speech, Expressive words Dependencies with other words; S44, the news data after word segmentation, stop word removal and syntactic analysis As input, the preprocessed news dataset D4 is generated.

3. According to claim 2, a method for filtering public opinion knowledge graph based on the credibility of news sources is characterized in that: The S5 specifically includes: S51, for each piece of news data in the preprocessed news data set D4 Use the enhanced pre-trained language model BERT for deep semantic understanding and generate embedding vectors containing contextual information in, Represents news data The context embedding vector of the jth word in , where n is the total number of words. The specific formula for generating the embedding vector is as follows: Among them, BERT represents the BERT model, Represents the jth word and its context words; S52. Using the transformer-based named entity recognition model SpanBERT to embed the vector Perform entity recognition and generate a set containing entity boundaries and categories Among them, b j and e j Respectively represent the starting and ending positions of the jth entity, c j represents the entity category, p is the total number of entities identified, and the specific identification process is as follows: in, Represents entity e j The probability of SpanBERT represents the SpanBERT model. and Respectively represent category c j The weights and biases of S53. Using the relationship extraction model based on graph neural network, the entity set and its context embedding vector to perform relationship inference and generate a relationship set in, Represents entity pair The relationship types include cooperation, conflict, and affiliation. The specific formula for relationship inference is as follows: in, Represents the entity in the l+1th layer The embedding vector of represents the neighbor set of entity pairs, W (l) represents the weight matrix of the lth layer, represents the attention coefficient, a represents the parameter vector of the attention mechanism, and || represents the vector connection operation; S54. Use self-supervised learning methods to verify and correct entity and relationship data to ensure their accuracy and consistency, and generate corrected entity and relationship data D6: The specific process of verification and correction is as follows: in, represents the loss function of self-supervised learning, Represents a given entity pair Type of relationship The probability of , λ is the regularization parameter, and Θ represents the model parameter.

4. According to claim 3, a method for filtering public opinion knowledge graph based on the credibility of news sources is characterized in that: The S6 specifically includes: S61, for each piece of news data in the revised entity and relationship data D6 Perform structural processing to generate a preliminary set of entities and relationships; Among them, e j Represents an entity, T j Indicates the timestamp, T ab and W ab Represent the timestamp and weight of the relationship respectively; S62. Define the node set N and edge set E of the public opinion knowledge graph G, and improve it in combination with the specific attributes of the news data; S63. Represent each entity e in the entity set N as a graph node, where the node attributes include entity type, occurrence frequency, related news source, and timestamp: e=(type,freq,sources,timestamp); type = EntityType(e); timestamp=max{T j ∣e=e j }; Among them, type indicates the entity type, freq indicates the frequency of the entity, sources indicates the source of the relevant news, and timestamp indicates the time when the news was released. is the indicator function, when 1 when it is, otherwise 0; S64. Represent each relation r in the relation set E as a graph edge, where the edge attributes include relation type, relation weight and timestamp: r=(type,weight,timestamp); type = RelationType(r); timestamp=max{T ab ∣r=(e a ,e b )}; Among them, type indicates the relationship type, weight indicates the relationship weight, and timestamp indicates the release time of the news corresponding to the relationship; S65. Use time series analysis methods to dynamically update and maintain entities and relationships based on timestamps to ensure the timeliness of the graph; Where T represents the current time, Update(G,T) represents the updated graph, N' and E' represent the updated node and edge sets respectively; S66. Use the graph database system to store entity and relationship data as a graph structure and generate an initial public opinion knowledge graph G: G = (N, E); S67. According to the frequency of occurrence of entities, the weight of relationships and the time of news release, the public opinion knowledge graph G is optimized and streamlined, and noise data and low-relevance nodes and edges are removed to ensure the accuracy and timeliness of the graph; N={e∈N∣freq>θ f and Recency(e,T)<θ t }; E={r∈E∣weight>θ w and Recency(r,T)<θ t }; Among them, θ f ,θ w ,θ t are the thresholds of frequency, weight and time, and Recency(x,T) represents the recency of a node or edge relative to the current time T.

5. According to claim 4, a method for filtering public opinion knowledge graph based on the credibility of news sources is characterized in that: The S8 specifically includes: S81. Perform text sentiment analysis on each piece of news data in the public opinion knowledge graph G to generate a sentiment score set A: <h2 style=";text-align:left;direction:ltr">A = {a1,a2,…,a<h2 style=";text-align:left;direction:ltr"> m <h2 style=";text-align:left;direction:ltr">}; Among them, a j represents the sentiment score of the jth news, and m is the total number of news; S82. Use a deep learning-based sentiment analysis model to classify the sentiment of news texts: Among them, P(c j |d j ) indicates news j Belongs to the emotion category c j The probability of Represents the emotion category c j The parameter, f(d j ) represents news text d j The eigenvector of S83, normalizing the sentiment score set A to generate a normalized sentiment score set A'; S84. Associate the normalized sentiment score A' with the entities and relationships in the public opinion knowledge graph G to generate a public opinion knowledge graph G' with enhanced sentiment tendency: G'=(N,E,A'); Among them, N is the node set, E is the edge set, and A' is the normalized sentiment score set; S85. Analyze the public opinion knowledge graph G' with enhanced sentiment tendency, identify the public opinion attitude and trend towards specific events or topics, and generate a public opinion report R.

6. According to claim 5, a method for filtering public opinion knowledge graph based on the credibility of news sources is characterized in that: The S85 specifically includes: S851. Perform topic modeling on the public opinion knowledge graph G' and extract the main topics using the latent Dirichlet distribution model: i d ~Dirichlet(a); With d,n ~Multinomial(θ d ); Among them, θ d represents the topic distribution of document d, represents the word distribution of topic k, z d,n represents the topic of the nth word in the dth document, w d,n Represents the nth word in the dth document; S852. Based on the results of topic modeling, identify the main sentiment of each topic k : Among them, D k represents the document set containing topic k, a′ d represents the sentiment score of document d; S853. Calculate the public opinion trend of each topic through time series analysis method k : Among them, t i Indicates a point in time, Indicates time point t i The corresponding sentiment score, Slope represents the slope of the calculated sentiment score over time; S854, comprehensive emotional attitude sentiment k and public opinion trends k Generate a public opinion report R, in which each topic contains the corresponding emotional attitude and public opinion trend information.

Images