Network public opinion analysis system based on big data platform

Abstract

The invention discloses a network public opinion analysis system based on a big data platform, and the system comprises a data collection module which is used for collecting network information, and obtaining a network popularity information data set; the data aggregation module is used for performing data similarity aggregation on the network popularity information data set; the data screening module is used for performing word segmentation processing on the network popularity information aggregation data set; the word frequency statistics module is used for carrying out word frequency analysis on the hot word set to obtain a keyword sequence of which the frequency exceeds a set threshold value; the data processing module is used for determining public opinion influence factors of each keyword in the keyword sequence one by one so as to determine public opinion prediction credible data; and the network public opinion prediction module is used for carrying out quantitative analysis on the public opinion prediction credible data by adopting a pre-trained network public opinion prediction model and obtaining a network public opinion prediction result according to a quantitative analysis result. The method has the remarkable effect that the development trend of the network public opinion can be quickly and accurately analyzed and predicted.

Description

TECHNICAL FIELD

[0001] The present application relates to the network public opinion analysis processing technical field, and in particular to a network public opinion analysis system based on a big data platform. BACKGROUND

[0002] With the rapid development of the Internet, network media as a new form of information dissemination has penetrated into people's daily life. Netizen comments are active to an unprecedented degree. Whether it is a major domestic or international event, online public opinion can be formed immediately. It can be said that the Internet has become a hub of ideological and cultural information and an amplifier of social public opinion. Network public opinion has become an important channel for reflecting public sentiment.

[0003] In order to timely grasp the network public opinion dynamics, analyze the development trend of public opinion, and warn potential risks, network public opinion analysis technology has emerged. At present, there are various researches on network public opinion analysis technology. For example, a sentiment and theme mining collaborative analysis model is constructed by applying a cognitive sentiment evaluation model, and a sentiment analysis model based on a user attention mechanism. However, the accuracy of the existing sentiment analysis model for network public opinion sentiment analysis is greatly affected by the quality of the model training sample. When the sample data is small or the noise is too large, the prediction accuracy of the model using this method is low. In addition, the model using the RNN or TextCNN method only considers the output layer, ignores the context relationship between texts, and the prediction accuracy needs to be improved. Moreover, the algorithm cannot be calculated in parallel, and the efficiency is low. In this case, a new method is needed to quickly and accurately analyze and predict the development trend of network public opinion. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application aims to provide a network public opinion analysis system based on a big data platform, which can quickly and accurately analyze and predict the development trend of network public opinion.

[0005] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: A network public opinion analysis system based on a big data platform, the key of which comprises: A data acquisition module for acquiring network information including websites, forums, and social platforms from a big data platform to obtain a network heat information dataset; A data aggregation module for performing data similarity aggregation on the network heat information dataset to obtain a network heat information aggregated data set; A data filtering module for performing word segmentation processing on the network heat information aggregated data set to obtain a hot word set; A word frequency statistical module for performing word frequency analysis on the hot word set and arranging the word frequency in descending order to obtain a keyword sequence with a frequency exceeding a set threshold. a data processing module, configured to determine an influence factor of public opinion for each keyword in the keyword sequence one by one, determine a public opinion prediction confidence of the keyword according to the influence factor of public opinion, and determine public opinion prediction reliable data according to the public opinion prediction confidence; a network public opinion prediction module, configured to quantitatively analyze the public opinion prediction reliable data by using a pre-trained network public opinion prediction model, and obtain a network public opinion prediction result according to a result of the quantitative analysis.

[0006] Further, the data collection module comprises: a big data standard interface unit, configured to provide a standard interface for data interaction with a big data platform; a collection strategy setting unit, configured to set a collection strategy of network information collection; a collection template making unit, configured to make a collection template of network information collection; a public opinion information collection unit, configured to collect network heat information data according to the set collection strategy and collection template; a storage unit, configured to store the collected network heat information data.

[0007] Further, the collection strategy comprises a depth, a range, a frequency and a priority of collected information, and the collection template comprises the following fields: a user name, a title, content, a time, a source, a keyword, a search trend, a geographical range and a population.

[0008] Further, the data aggregation module comprises: a data preprocessing unit, configured to perform information preprocessing on network heat information data in the network heat information data set, so as to realize cleaning, filtering and noise removal of data; a data feature extraction unit, configured to extract data feature quantities of each network heat information data in the preprocessed network heat information data set; a similarity coefficient calculation unit, configured to calculate a similarity coefficient between each two data feature quantities according to all the data feature quantities; a data aggregation unit, configured to aggregate network heat information data in the preprocessed network heat information data set according to the calculated similarity coefficients, so as to obtain a network heat information aggregation data set.

[0009] Further, the data aggregation unit aggregating the network heat information data in the preprocessed network heat information data set comprises: determining a similar aggregation number of the preprocessed network heat information data set; determining a plurality of network heat information aggregation feature data in the preprocessed network heat information data set through the similar aggregation number; Select a data characteristic quantity, obtain the data similarity coefficient corresponding to the data characteristic quantity and each network heat information aggregation characteristic data, and aggregate the network heat information data corresponding to the data characteristic quantity and the network heat information aggregation characteristic data corresponding to the minimum data similarity coefficient into network heat information aggregation data; Repeat the above steps to aggregate the network heat information data corresponding to the remaining data characteristic quantities to obtain the network heat information aggregation data set.

[0010] Further, the data screening module comprises: The word segmentation processing unit is configured to perform word segmentation processing on the network heat information aggregation data. The noun obtaining unit is configured to obtain a set of word data by performing word segmentation processing, convert full-width characters to half-width characters, convert traditional Chinese characters to simplified Chinese characters, convert English uppercase letters to lowercase letters, remove URLs, remove emails, remove @ symbols, and retain emojis, while deleting conjunctions, prepositions, pronouns, and other structural words, removing verbs and adjectives, and only retaining nouns to form a list of noun words. The hot word screening unit is configured to screen network hot words from the list of noun words and form a hot word library according to a network hot word matching library.

[0011] Further, the data processing module determines the public opinion influence factor of each keyword in the keyword sequence, specifically comprising: Select a keyword from the keyword sequence and determine the aggregation degree of the keyword; Calculate the Euclidean distance between the word frequency of the keyword and the threshold value; Determine the public opinion influence factor of the keyword according to the aggregation degree and the Euclidean distance.

[0012] Further, the data processing module determines the public opinion prediction credible data according to the public opinion prediction confidence, specifically comprising: Select a keyword from the keyword sequence, and determine the credible data balance and the credible data fluctuation of the keyword according to the public opinion prediction confidence corresponding to the keyword; Determine the likelihood confidence value of the keyword according to the credible data balance and the credible data fluctuation; Repeat the above steps to determine the likelihood confidence values of the remaining keywords; Determine the public opinion prediction credible data according to all the likelihood confidence values.

[0013] Further, the network public opinion prediction module comprises: The data extraction unit is configured to extract historical public opinion events in the big data platform and capture effective first historical network public opinion data from the historical public opinion events. a data analysis unit configured to obtain second network public opinion data by performing public opinion analysis on the first network public opinion data according to the constructed public opinion development line and using at least one of topic identification and tracking, text tendency analysis, hot spot discovery, and automatic abstract technology; a sample generation unit configured to extract keywords from the second network public opinion data after aggregation, and extract real public opinions at at least two time nodes according to the public opinion development line, so as to form a first training sample set and a second training sample set; a model establishment and training unit configured to establish an initial network public opinion prediction model, and input the first training sample set and the second training sample set into the initial network public opinion prediction model for training, and obtain the network public opinion prediction model when the accuracy of the two times of training meets the requirement.

[0014] The present application has the following remarkable effects: 1. The system disclosed by the present application first obtains a network heat information data set by collecting network information including websites, forums and social platforms from a big data platform through a data collection module; then, the system obtains a network heat information aggregation data set by performing data similarity aggregation on the network heat information data set through a data aggregation module; then, the system obtains a hot word set by performing word segmentation on the network heat information aggregation data set through a data screening module; then, the system obtains a keyword sequence with a frequency exceeding a set threshold by performing word frequency analysis on the hot word set in descending order of word frequency through a word frequency statistical module; then, the system determines an public opinion influence factor of each keyword in the keyword sequence one by one through a data processing module, determines a public opinion prediction confidence of the keyword according to the public opinion influence factor, and determines public opinion prediction credible data according to the public opinion prediction confidence; finally, the system performs quantitative analysis on the public opinion prediction credible data by using a pre-trained network public opinion prediction model through a network public opinion prediction module, and obtains a network public opinion prediction result according to the quantitative analysis result.

[0015] 2. Compared with the existing public opinion analysis system which only contains a single function, the present application can discover public opinions in real time, analyze public opinions accurately, and predict public opinions quickly, thereby greatly improving the efficiency of public opinion supervision and the efficiency of management decision of relevant departments.

[0016] 3. The initial network public opinion prediction model is trained by forming a first training sample set and a second training sample set based on real public opinions at two time nodes, so that the public opinion prediction strategy can be accurately predicted, the accuracy is higher than that of the traditional method of directly calling and obtaining new words, the applicability for subsequent public opinion analysis and prediction is better, and the system migration, update and application cost can be effectively reduced.

[0017] 4、The system can not only obtain useful keywords and reliable data for public opinion prediction, but also effectively reduce data volume and improve operation and prediction efficiency of the system, which helps management departments and researchers quickly understand and master industry public opinion dynamics. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a principle block diagram of the present application; Figure 2 is a principle block diagram of the data collection module; Figure 3 is a principle block diagram of the data aggregation module; Figure 4 is a principle block diagram of the data screening module; Figure 5 is a principle block diagram of the network public opinion prediction module. DETAILED DESCRIPTION

[0019] The specific embodiments and working principles of the present application will be further described in detail below with reference to the accompanying drawings. EMBODIMENT

[0020] As shown in Figure 1 , a network public opinion analysis system based on a big data platform comprises: a data collection module for collecting network information including websites, forums and social platforms from the big data platform to obtain a network heat information dataset; The website can be a policy website such as a government website, which publishes the latest policies and public notices, etc. Such a website data set includes the direction of the government department in this field. The website can also be a news website, including large portal websites and institutional websites. Domestic well-known websites such as Sina, Sohu, NetEase and Tencent belong to large portal websites, which mainly provide consulting and news content, and the website content is relatively comprehensive, including many branch information such as real estate, economy, technology and tourism. In addition, enterprise websites, commercial websites, educational and scientific research institution websites and WeChat media websites also belong to this category, which publish institutional news and information. Large portal websites have a large number of users, and WeChat media covers a wide range. The forum is a comment website such as Weibo, portal website forum, Dianping and Baidu Tieba, which publishes addresses, products and post-use evaluation values. The social platform can be WeChat and QQ, which publishes event opinions, etc.

[0021] a data aggregation module for data similar aggregation of the network heat information dataset to obtain a network heat information aggregated data set; a data screening module for word segmentation processing of the network heat information aggregated data set to obtain a hot word set; a word frequency statistics module configured to perform word frequency analysis on the set of hot words and arrange the hot words in descending order of word frequency to obtain a keyword sequence with a frequency exceeding a set threshold; a data processing module configured to determine an opinion influence factor of each keyword in the keyword sequence one by one, determine an opinion prediction confidence of the keyword according to the opinion influence factor, and determine opinion prediction credible data according to the opinion prediction confidence; a network opinion prediction module configured to perform quantitative analysis on the opinion prediction credible data by using a pre-trained network opinion prediction model, and obtain a network opinion prediction result according to the quantitative analysis result.

[0022] It should be noted that the result of quantitative analysis reflects the probability of occurrence of a certain network opinion, and the greater the quantitative value, the greater the probability of occurrence of the network opinion.

[0023] It can be seen that the system first obtains a network heat information dataset by collecting network information including websites, forums and social platforms from a big data platform through a data collection module; then, the network heat information dataset is subjected to data similarity aggregation by a data aggregation module to obtain a network heat information aggregated data set; then, the network heat information aggregated data set is subjected to word segmentation processing by a data screening module to obtain a set of hot words; then, the set of hot words is subjected to word frequency analysis by a word frequency statistics module, and the hot words are arranged in descending order of word frequency to obtain a keyword sequence with a frequency exceeding a set threshold; then, the data processing module determines an opinion influence factor of each keyword in the keyword sequence one by one, determines an opinion prediction confidence of the keyword according to the opinion influence factor, and determines opinion prediction credible data according to the opinion prediction confidence; finally, the network opinion prediction module performs quantitative analysis on the opinion prediction credible data by using a pre-trained network opinion prediction model, and obtains a network opinion prediction result according to the quantitative analysis result. The system can discover opinions in real time, analyze opinions accurately, and predict opinions quickly, thereby greatly improving the efficiency of opinion supervision and the efficiency of management decisions of relevant departments.

[0024] Referring to the accompanying drawings, Figure 2 the data collection module in the embodiment includes: a big data standard interface unit configured to provide a standard interface for data interaction with a big data platform; a collection strategy setting unit configured to set a collection strategy for network information collection; a collection template making unit configured to make a collection template for network information collection; an opinion information collection unit configured to collect network heat information data according to the set collection strategy and collection template; a storage unit configured to store the collected network heat information data.

[0025] Specifically, the collection strategy includes depth, range, frequency, priority of collected information; and the collection template includes the following fields: username, title, content, time, source, keyword, search trend, geographical range, and crowd.

[0026] Through the collection strategy and the collection template design, the network information collected by the system can be preliminarily screened, so that more useful network information is obtained for accurate public opinion analysis and prediction, and the difficulty of the data processing process of the subsequent modules is reduced.

[0027] Referring to the accompanying drawings Figure 3 The data aggregation module in the embodiment includes: A data preprocessing unit is configured to perform information preprocessing on the network heat information data in the network heat information data set, so as to realize data cleaning, filtering and noise removal. A data feature extraction unit is configured to extract data features of each network heat information data in the preprocessed network heat information data set. A similarity coefficient calculation unit is configured to calculate the similarity coefficient between each two data features according to all data features. A data aggregation unit is configured to aggregate the network heat information data in the preprocessed network heat information data set according to the calculated similarity coefficient, and obtain a network heat information aggregation data set.

[0028] It should be noted that the specific implementation of information preprocessing is as follows: based on the rules prepared in advance, the missing data in the website is automatically completed, the irregular data is regularized, and the irrelevant characters are deleted, etc. At the same time, the noise such as pictures, videos, links irrelevant to content analysis is removed, so as to obtain standard, clean and continuous data, and then the data is segmented and stop word filtered, and the basic method is the same as the data preprocessing method in the platform positioning analysis module. Then, the collected information is displayed in the form of reasonable big data visualization graphics, so as to realize the effect of intuitive expression.

[0029] It should be noted that the data feature in the present application represents a measurement parameter of the influence degree of the corresponding network heat information data on network public opinion. In some embodiments, the data feature can be determined by prior experience or comprehensive evaluation of each network heat information data in the network heat information data set on network public opinion. In other embodiments, other methods can also be used to determine the data feature, which is not limited here.

[0030] The calculation process of the similarity coefficient can be realized by the following steps: Sort all data features in ascending order to obtain a data feature sequence. Select one data feature quantity in the data feature quantity sequence, determine the similarity coefficient of the data feature quantity and other data feature quantities through ratio calculation, and reflect the similarity degree between the corresponding two data feature quantities through the similarity coefficient. Repeat the above steps to determine the similarity coefficients of the remaining data feature quantities in the data feature quantity sequence and other data feature quantities.

[0031] The network heat information data in the network heat information data set is aggregated through the similarity coefficient, which not only greatly reduces the data volume, but also retains as many useful features of each data as possible, thereby avoiding affecting the analysis and prediction results of network public opinion due to data aggregation.

[0032] In some preferred embodiments, the data aggregation unit aggregates the network heat information data in the preprocessed network heat information data set, specifically comprising: Determine the similarity aggregation number of the preprocessed network heat information data set; Determine a plurality of network heat information aggregation feature data in the preprocessed network heat information data set through the similarity aggregation number; Select one data feature quantity, obtain the data similarity coefficient corresponding to the data feature quantity and each network heat information aggregation feature data, and aggregate the network heat information data corresponding to the data feature quantity and the network heat information aggregation feature data corresponding to the minimum data similarity coefficient into network heat information aggregation data; Repeat the above steps to aggregate the network heat information data corresponding to the remaining data feature quantities to obtain the network heat information aggregation data set.

[0033] It should be noted that the similarity aggregation number in the present application represents the number of network heat information data samples. The number of network heat information aggregation feature data in the network heat information data set can be estimated using prior experience, and the similarity aggregation number can be set according to the estimated value. In other embodiments, other methods can be used to determine the similarity aggregation number, which is not limited here.

[0034] Referring to FIG. 2, the data screening module includes: Figure 4 The word segmentation processing unit is configured to perform word segmentation processing on the network heat information aggregation data. The noun acquisition unit is configured to perform word segmentation processing on the word data set obtained by the word segmentation processing unit, convert full-width characters to half-width characters, convert traditional Chinese characters to simplified Chinese characters, convert English capital letters to lowercase letters, remove URLs, remove emails, remove @ symbols, and retain emojis, while deleting structural words such as conjunctions, prepositions, and pronouns, removing words with the verb and adjective properties, and only retaining nouns to form a noun vocabulary list. ​The hot word screening unit is configured to screen network hot words from the list of noun words and form a hot word library according to a network hot word matching library.

[0035] The hot word library is formed by data screening, so that more useful keywords are obtained for network public opinion analysis.

[0036] In the embodiment, the data processing module determines the public opinion influence factor of each keyword in the keyword sequence, and specifically includes the following steps: selecting a keyword from the keyword sequence, and determining the aggregation degree of the keyword; The aggregation degree of the keyword represents a parameter value of the aggregation degree of the keyword in the data aggregation process, and in some embodiments, the mean square error of the similarity of the network heat information data corresponding to the keyword.

[0037] calculating the Euclidean distance between the word frequency of the keyword and the threshold value; determining the public opinion influence factor of the keyword according to the aggregation degree and the Euclidean distance.

[0038] Further, the data processing module determines the public opinion prediction credible data according to the public opinion prediction confidence, and specifically includes the following steps: selecting a keyword from the keyword sequence, and determining the credible data balance and the credible data fluctuation of the keyword according to the public opinion prediction confidence corresponding to the keyword; determining the likelihood confidence value of the keyword according to the credible data balance and the credible data fluctuation; repeating the above steps to determine the likelihood confidence values of the remaining keywords; determining the public opinion prediction credible data according to all the likelihood confidence values.

[0039] It should be noted that the credible data balance and the credible data fluctuation respectively reflect the balance and fluctuation degree of the similarity of the network heat information data corresponding to the keyword. The likelihood confidence value reflects the credibility of each keyword, and the greater the likelihood confidence value, the greater the credibility or usefulness of the keyword in network public opinion analysis.

[0040] Referring to the accompanying drawings, Figure 5 The network public opinion prediction module in the embodiment includes: The data extraction unit is configured to extract historical public opinion events in the big data platform, and grab effective first historical network public opinion data from the historical public opinion events. a data analysis unit configured to obtain second network public opinion data by analyzing the first network public opinion data according to the constructed public opinion development line and using at least one of topic identification and tracking, text tendency analysis, hot spot discovery, and automatic abstract technology; a sample generation unit configured to extract keywords from the second network public opinion data after aggregation, and extract real public opinions at at least two time nodes according to the public opinion development line, thereby forming a first training sample set and a second training sample set; a model establishment and training unit configured to establish an initial network public opinion prediction model, input the first training sample set and the second training sample set into the initial network public opinion prediction model for training, and obtain the network public opinion prediction model when the accuracy of the two times of training meets the requirement.

[0041] The initial network public opinion prediction model is trained by forming the first training sample set and the second training sample set based on the real public opinions at the two time nodes, so that the public opinion prediction strategy can be accurately predicted, the accuracy is higher than that of a traditional method of directly calling new words, the subsequent public opinion analysis and prediction can have better applicability, and the system migration, update and application cost can be effectively reduced.

[0042] In summary, the system can obtain useful keywords and public opinion prediction credible data, effectively reduce the data amount, and improve the system operation and prediction efficiency, and help the management department and the scientific research personnel quickly understand and master the industry public opinion dynamics, by the processes of data aggregation, data screening, word frequency statistics and data processing.

[0043] The technical solutions provided by the present application are described in detail above. The principles and implementation manners of the present application are described by using specific examples, and the above examples are only used to help understand the method and core idea of the present application. It should be noted that, for ordinary skilled persons in the technical field, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A network public opinion analysis system based on a big data platform, characterized by: include: The data collection module is used to collect network information including websites, forums, and social platforms from the big data platform to obtain network popularity information datasets; A data aggregation module is used to perform data similarity aggregation on the network popularity information data set to obtain a network popularity information aggregated data set; A data screening module is used to perform word segmentation processing on the network hot information aggregation data set to obtain a hot word set; A word frequency statistics module is used to perform word frequency analysis on the hot word set and sort them in descending order of word frequency to obtain a keyword sequence whose frequency exceeds a set threshold; A data processing module, configured to determine the public opinion influence factor of each keyword in the keyword sequence one by one, and determine the public opinion prediction confidence of the keyword based on the public opinion influence factor; It is also used to determine the credible data of public opinion prediction based on the confidence level of public opinion prediction; The network public opinion prediction module is used to use a pre-trained network public opinion prediction model to perform quantitative analysis on the public opinion prediction credible data, and obtain the network public opinion prediction results based on the quantitative analysis results.

2. The network public opinion analysis system based on the big data platform according to claim 1 is characterized in that: The data acquisition module includes: Big data standard interface unit, used to provide a standard interface for data interaction with the big data platform; A collection strategy setting unit, used to set a collection strategy for network information collection; A collection template making unit, used for making collection templates for network information collection; The public opinion information collection unit is used to collect network popularity information data according to the set collection strategy and collection template; A storage unit is used to store the collected network popularity information data.

3. The network public opinion analysis system based on the big data platform according to claim 2 is characterized in that: The collection strategy includes the depth, scope, frequency, and priority of the collected information; the collection template includes the following fields: user name, title, content, time, source, keyword, search trend, geographical scope, and population.

4. The network public opinion analysis system based on a big data platform according to claim 1 is characterized in that: The data aggregation module includes: A data preprocessing unit, configured to perform information preprocessing on the network popularity information data in the network popularity information dataset to achieve data cleaning, filtering, and noise removal; A data feature extraction unit is used to extract data features of each network hot information data in the pre-processed network hot information data set; A similarity coefficient calculation unit, configured to calculate a similarity coefficient between every two data feature quantities based on all data feature quantities; The data aggregation unit aggregates the network heat information data in the pre-processed network heat information data set according to the calculated similarity coefficient to obtain a network heat information aggregated data set.

5. The network public opinion analysis system based on the big data platform according to claim 1 is characterized in that: The data aggregation unit aggregates the network popularity information data in the pre-processed network popularity information data set, specifically comprising: Determining the similarity aggregation number of the preprocessed network popularity information dataset; Determining a plurality of network popularity information aggregation feature data in the pre-processed network popularity information dataset according to the similarity aggregation number; Select a data feature, obtain a data similarity coefficient corresponding to the data feature and each network heat information aggregation feature data, and aggregate the network heat information data corresponding to the data feature and the network heat information aggregation feature data corresponding to the minimum data similarity coefficient into network heat information aggregation data; Repeat the above steps to aggregate the network popularity information data corresponding to the remaining data feature quantities to obtain the network popularity information aggregated data set.

6. The network public opinion analysis system based on a big data platform according to claim 1 is characterized in that: The data screening module includes: A word segmentation processing unit, used for performing word segmentation processing on the network popularity information aggregation data; A noun acquisition unit is used to convert the word data set obtained by word segmentation processing to half-width, traditional Chinese to simplified Chinese, uppercase to lowercase English, remove URLs, remove emails, remove @, and retain emojis. It also deletes structural words such as conjunctions, prepositions, and pronouns, removes words with verb and adjective parts of speech, and retains only nouns to form a noun vocabulary list; The hot word screening unit is used to screen out Internet hot words from the noun vocabulary list according to the Internet hot word matching library and form a hot word library.

7. The network public opinion analysis system based on a big data platform according to claim 1 is characterized in that: The data processing module determines the public opinion impact factor of each keyword in the keyword sequence specifically including: Selecting a keyword from the keyword sequence and determining the aggregation degree of the keyword; Calculate the Euclidean distance between the keyword frequency and the set threshold; The public opinion impact factor of the keyword is determined according to the aggregation degree and the Euclidean distance.

8. The network public opinion analysis system based on a big data platform according to claim 1 or 7, characterized in that: The data processing module determines the public opinion prediction credible data according to the public opinion prediction confidence level, specifically including: Select a keyword from the keyword sequence, and determine the trustworthy data balance and trustworthy data fluctuation of the keyword based on the public opinion prediction confidence corresponding to the keyword; Determining a likelihood credibility value of the keyword based on the credibility data balance and the credibility data fluctuation; Repeat the above steps to determine the likelihood credibility values ​​of the remaining keywords; Determine the credible data for public opinion prediction based on all likelihood credibility values.

9. The network public opinion analysis system based on a big data platform according to claim 1 is characterized in that: The network public opinion prediction module includes: A data extraction unit is used to extract historical public opinion events from the big data platform and capture effective first historical network public opinion data from the historical public opinion events; a data analysis unit, configured to perform public opinion analysis on the first network public opinion data based on the constructed public opinion development line and using at least one of topic identification and tracking, text tendency analysis, hotspot discovery, and automatic summarization technology to obtain second network public opinion data; A sample generation unit is configured to extract keywords from the second network public opinion data and extract real public opinions at at least two time points according to the public opinion development line, thereby forming a first training sample set and a second training sample set; The model establishes a training unit, establishes an initial network public opinion prediction model, and inputs the first training sample set and the second training sample set into the initial network public opinion prediction model for training respectively. When the accuracy of the two trainings meets the requirements, the network public opinion prediction module is obtained.

Images