Deep learning and data mining fused public opinion trend prediction method and system

Abstract

The invention discloses a deep learning and data mining fused public opinion trend prediction method and system, and relates to the technical field of public opinion trend prediction, and the method comprises the steps: obtaining various types of public opinion data through social media, and carrying out the preprocessing of the collected public opinion data; constructing an information source scoring mechanism, and scoring through different information sources; weighting the public opinion data based on the credibility, namely adjusting the weight of each data point according to the credibility of the information source; and taking the weighted public opinion data as input, performing sentiment analysis in combination with a deep learning model, and predicting a future public opinion trend. According to the method, an information source scoring mechanism is established, multi-dimensional factors such as the accuracy, the user feedback and the activeness of the information sources are considered, and the credibility of each information source is dynamically evaluated. In this way, unreliable information sources can be effectively eliminated, and interference of low-quality data on prediction results is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of public opinion trend prediction, and particularly to a public opinion trend prediction method and system that integrates deep learning and data mining. Background Art

[0002] With the rapid development of social media and news platforms, public opinion (public opinion) has had a profound impact on social, political, economic and other fields. The prediction of public opinion trends has become an important research topic in academia and industry. Especially in the context of digitalization and informatization, how to accurately and real-time predict public opinion changes has become a technical problem to be solved urgently. At present, public opinion trend prediction methods mainly rely on data mining, sentiment analysis and deep learning technologies to analyze a large amount of social media and news data to identify potential public opinion trends.

[0003] Although existing public opinion analysis technologies have applied technologies such as sentiment analysis and keyword extraction, they still have the following problems: inconsistent information source quality: the reliability and credibility of different information sources (such as social platforms, news media, etc.) vary greatly, resulting in serious impacts on the analysis results; insufficient accuracy of sentiment analysis: although deep learning has good performance in sentiment analysis, there are still great challenges in identifying some complex emotions, sarcasm and multiple emotions. Summary of the Invention

[0004] The purpose of this part is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this part, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this part, the abstract and the title, and such simplifications or omissions shall not be used to limit the scope of the present invention.

[0005] In view of the problems existing in the above-mentioned prior art, the present invention is proposed.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a public opinion trend prediction method that integrates deep learning and data mining, including:

[0007] Obtain various public opinion data through social media, and preprocess the collected public opinion data to provide high-quality data as the basis for subsequent analysis;

[0008] Construct an information source scoring mechanism to score different information sources, that is, evaluate the trustworthiness of each information source;

[0009] Weight the public opinion data based on trustworthiness, that is, adjust the weight of each data point according to the trustworthiness of the information source to adjust the contribution of the public opinion data to the final prediction result;

[0010] Taking the weighted public opinion data as input, combining with a deep learning model for sentiment analysis, after obtaining the sentiment analysis results of each piece of data, predict the future public opinion trend.

[0011] As a preferred embodiment of the public opinion trend prediction method integrating deep learning and data mining according to the present invention, wherein: the information source scoring mechanism includes:

[0012] S101: Determine the evaluation indicators for each information source S i Mainly including: accuracy P i (t), user feedback F i (t), activity A i (t);

[0013] S102: Assign weight coefficients to each evaluation indicator, which are α i , β i , γ i ; and α i + β i + γ i = 1;

[0014] S103: Conduct a weighted evaluation of each information source to obtain the trustworthiness calculation result T i (t) of the information source at time t, and the calculation formula is:

[0015]

[0016] Wherein, α i , β i , γ i Correspond to the weight coefficients of accuracy, user feedback, and activity respectively.

[0017] As a preferred embodiment of the public opinion trend prediction method integrating deep learning and data mining according to the present invention, wherein: eliminate the differences between the trustworthiness calculation results of different information sources, perform normalization processing on the calculated results, and ensure that the trustworthiness scores of all information sources are within a standard range. Then the trustworthiness value of the normalized information source S i is:

[0018]

[0019] Where n represents the number of information sources.

[0020] As a preferred embodiment of the public opinion trend prediction method integrating deep learning and data mining according to the present invention, wherein: the trustworthiness-weighted public opinion data specifically includes:

[0021] S201: Collect relevant public opinion data, and this public opinion data comes from a certain information source S i, where the sentiment score or topic label D of each piece of public opinion data j ;

[0022] S202: Weight the public opinion data according to the calculated information source trust degree T(S i ); reflect the actual influence of each piece of data through the weighted public opinion data, and the formula is as follows:

[0023]

[0024] where, m represents the total number of public opinion data, and W(D j ) represents the importance degree of the weighted public opinion data.

[0025] As a preferred scheme of the public opinion trend prediction method integrating deep learning and data mining according to the present invention, wherein: if D j takes the sentiment score, then its specific representation is as follows: +1 (positive), 0 (neutral), -1 (negative);

[0026] If D j takes the topic label, then the value is taken by means of category encoding. If there are multiple topic categories, each category is encoded as a unique integer value.

[0027] As a preferred scheme of the public opinion trend prediction method integrating deep learning and data mining according to the present invention, wherein: when T(S i ) is greater than or equal to the trust degree threshold, it is considered that the information source makes a greater contribution to the prediction of the public opinion trend. At this time, the weighted public opinion data W(D j ) is amplified, that is, the amplified W(D j ) is equal to: W(D j )×(T(S i ) / trust degree threshold);

[0028] When T(S i ) is less than the trust degree threshold, it is considered that the influence of the information source on the prediction of the public opinion trend is weakened. At this time, the weighted public opinion data W(D j ) is reduced, and the reduced W(D j ) is equal to: W(D j )×(T(S i ) / trust degree threshold).

[0029] As a preferred scheme of the public opinion trend prediction method integrating deep learning and data mining according to the present invention, wherein: based on the amplified or reduced W(D j ), set a weight threshold to determine which data has a significant impact on the prediction of the public opinion trend;

[0030] Only when the amplified or reduced W(Dj ) Only when it is greater than this weight threshold, will this data be marked as important data and enter the final public opinion trend prediction model.

[0031] A prediction system applied to the above public opinion trend prediction method that combines deep learning and data mining. This system includes: a public opinion data collection module, responsible for collecting public opinion data from data sources of different social media platforms; a public opinion data preprocessing module, which performs cleaning, denoising, and formatting preprocessing operations on the collected public opinion data to provide high-quality data for subsequent analysis;

[0032] An information source trustworthiness evaluation module, which scores each information source according to multiple indicators and calculates its trustworthiness; a public opinion data weighting module, which weights the public opinion data according to the trustworthiness of the information source to adjust the contribution degree of each piece of data to the final prediction result; and a sentiment analysis module, which performs sentiment analysis on the weighted public opinion data to analyze the sentiment tendency of each piece of data; a public opinion trend prediction module, which predicts the future public opinion trend based on the weighted public opinion data and the sentiment analysis result, in combination with a deep learning model.

[0033] The present invention also discloses a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above public opinion trend prediction method that combines deep learning and data mining.

[0034] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above public opinion trend prediction method that combines deep learning and data mining.

[0035] Advantages of the present invention:

[0036] 1. By establishing an information source scoring mechanism, the present invention dynamically evaluates the trustworthiness of each information source by considering multi-dimensional factors such as the accuracy of the information source, user feedback, and activity. This can effectively eliminate unreliable information sources and reduce the interference of low-quality data on the prediction result;

[0037] 2. The present invention weights the public opinion data according to the trustworthiness to ensure that information sources with high trustworthiness contribute more to the final result. This weighting method avoids the bias that may be brought by the equal treatment of data in traditional methods and improves the accuracy of analysis; and by combining a deep learning model to perform sentiment analysis on the weighted public opinion data, it can more accurately capture the changing trend of public opinion sentiment, especially enhancing the processing ability for complex emotions and ironic emotions. Description of the Drawings

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. Among them:

[0039] Figure 1 It is a schematic diagram of the overall structure of the public opinion trend prediction method and system that combines deep learning and data mining proposed by the present invention. Specific embodiments

[0040] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention in conjunction with the drawings of the specification.

[0041] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0042] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that excludes other embodiments.

[0043] Refer to Figure 1 , which is an embodiment of the present invention, providing a public opinion trend prediction method and system that combines deep learning and data mining. This method includes:

[0044] Step 1: Obtain various public opinion data through social media, and preprocess the collected public opinion data to provide high-quality data as the basis for subsequent analysis.

[0045] Step 2: Construct an information source scoring mechanism to score different information sources, that is, evaluate the trustworthiness of each information source.

[0046] Specifically, the information source scoring mechanism includes:

[0047] S101: Determine the evaluation indicators for each information source S i , mainly including: accuracy P i (t), user feedback F i (t), activity A i (t);

[0048] Accuracy Pi (t) can be measured by the historical error rate or accuracy rate of the content published by the information source. Specifically, P i (t) = the number of real content / the total number of published content, with a value range of [0, 1]. The closer it is to 1, the more credible the information source is. User feedback F i (t) can be quantified by the ratio of positive and negative comments on the social media platform. Specifically, F i (t) = the number of real content / the total number of published content, with a value range of [0, 1]. The closer it is to 1, it indicates that the audience of the information source is more positive. And the activity A i (t) can be measured by the frequency of the content published by the information source; specifically, A i (t) = N i / T, where N i represents the number of content published within the observation time period T, with a value range of [0, +∞). The larger the value, the higher the publishing activity of the information source.

[0049] S102: Assign weight coefficients to each evaluation index, which are α i , β i , γ i respectively; and α i +β i +γ i = 1;

[0050] S103: Conduct a weighted evaluation on each information source to obtain the calculation result T i (t) of the trust level of the information source at time t. The calculation formula is:

[0051]

[0052] where α i , β i , γ i correspond to the weight coefficients of accuracy, user feedback, and activity respectively.

[0053] Furthermore, to eliminate the differences between the calculation results of the trust levels of different information sources, normalize the calculated results to ensure that the trust level scores of all information sources are within a standard range. Then the trust level value of the normalized information source S i is:

[0054]

[0055] where n represents the number of information sources.

[0056] Step 3: Weight the public opinion data based on the trust level, that is, adjust the weight of each data point according to the trust level of the information source to adjust the contribution of the public opinion data to the final prediction result.

[0057] Specifically, the sentiment data weighted based on trust includes the following:

[0058] S201: Collect relevant sentiment data, which is sourced from an information source S i , where the sentiment score or topic label D of each piece of sentiment data j ;

[0059] If D j takes the sentiment score, then its specific representation is as follows: +1 (positive), 0 (neutral), -1 (negative).

[0060] If D j takes the topic label, then it is valued by means of category encoding. If there are multiple topic categories, each category is encoded as a unique integer value. For example, if the sentiment data involves topic A, it can be encoded as D j = 1, and if it involves topic B, it can be encoded as D j = 2, and so on.

[0061] S202: Weight the sentiment data according to the calculated information source trust degree T(S i ); reflect the actual influence of each piece of data through the weighted sentiment data. The formula is as follows:

[0062]

[0063] where, m represents the total number of sentiment data, and W(D j ) represents the importance degree of the weighted sentiment data.

[0064] When T(S i ) is greater than or equal to the trust degree threshold, it is considered that the information source makes a greater contribution to the prediction of the sentiment trend. At this time, the weighted sentiment data W(D j ) is amplified, that is, the amplified W(D j ) is equal to: W(D j ) × (T(S i ) / trust degree threshold); when T(S i ) is less than the trust degree threshold, it is considered that the influence of the information source on the prediction of the sentiment trend is weakened. At this time, the weighted sentiment data W(D j ) is reduced, and the reduced W(D j ) is equal to: W(D j ) × T(S i ) / trust degree threshold).

[0065] Based on the amplified or reduced W(D j ), set a weight threshold to determine which data has a significant impact on the prediction of the sentiment trend; only when the amplified or reduced W(D j)Only when it is greater than this weight threshold will the data be marked as important data and enter the final public opinion trend prediction model.

[0066] Step 4: Use the weighted public opinion data as input, combine with a deep learning model for sentiment analysis, and after obtaining the sentiment analysis results of each piece of data, predict the future public opinion trend.

[0067] For example, assume there are n pieces of public opinion data, and the features of each piece of data are multi-dimensional, denoted as X j and its corresponding weighted value W(D j ), we represent the data as matrix X and weight vector W, that is: X = [X1, X2,..., X n , W = [W1, W2,..., W n . Among them, X j is the feature vector of the j-th piece of public opinion data, and W(D j ) is the weighted value of this data.

[0068] Select a deep learning model (such as LSTM, GRU) to predict the public opinion trend. This model receives the weighted public opinion data and sentiment analysis results as input and outputs the future public opinion trend. For example: The model structure can be a multi-layer LSTM network, which processes the changes in time series data to predict the public opinion trend. The formula can be expressed as:

[0069]

[0070] Model(X j ) is the prediction model, which outputs the trend prediction for this data based on the public opinion data feature X j , and P(t) represents the public opinion trend prediction result. The meaning of the formula is: The weighted value W(D j ) of each piece of data affects the final public opinion trend P(t), and the future public opinion trend is further predicted through the deep learning model. P(t) > 0 indicates that the public opinion is inclined to be positive; P(t) < 0 indicates that the public opinion is inclined to be negative.

[0071] A prediction system applied to the above public opinion trend prediction method that combines deep learning and data mining. This system includes: a public opinion data collection module, which is responsible for collecting public opinion data from data sources of different social media platforms; a public opinion data preprocessing module, which performs cleaning, denoising, and formatting preprocessing operations on the collected public opinion data to provide high-quality data for subsequent analysis.

[0072] An information source trustworthiness evaluation module, which scores each information source according to multiple indicators and calculates its trustworthiness; a public opinion data weighting module, which weights the public opinion data according to the trustworthiness of the information source and adjusts the contribution degree of each piece of data to the final prediction result.

[0073] and a sentiment analysis module that performs sentiment analysis on the weighted public opinion data to analyze the sentiment tendency of each piece of data; a public opinion trend prediction module that, based on the weighted public opinion data and the sentiment analysis results, combines a deep learning model to predict future public opinion trends.

[0074] This embodiment also provides a computer device applicable to the situation of the public opinion trend prediction method that integrates deep learning and data mining, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the public opinion trend prediction method that integrates deep learning and data mining as proposed in the above embodiment.

[0075] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0076] This embodiment also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, it implements the public opinion trend prediction method and its system that integrate deep learning and data mining as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM for short), Electrically Erasable Programmable Read-Only Memory (EEPROM for short), Erasable Programmable Read Only Memory (EPROM for short), Programmable Red-Only Memory (PROM for short), Read-Only Memory (ROM for short), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0077] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An opinion trend prediction method integrating deep learning and data mining, characterized in that, Including: Obtain various public opinion data through social media, and preprocess the collected public opinion data to provide high-quality data as the basis for subsequent analysis; Construct an information source scoring mechanism, score different information sources, that is, evaluate the trustworthiness of each information source; Weight the public opinion data based on trustworthiness, that is, adjust the weight of each data point according to the trustworthiness of the information source to adjust the contribution of the public opinion data to the final prediction result; Use the weighted public opinion data as input, combine with a deep learning model for sentiment analysis, and predict the future public opinion trend after obtaining the sentiment analysis result of each piece of data.

2. The public opinion trend prediction method integrating deep learning and data mining according to claim 1, characterized in that: The information source scoring mechanism includes: S101: Determine each information source S i 's evaluation metrics, mainly including: accuracy P i (t), user feedback F i (t), activity A i (t); S102: Assign weight coefficients to each evaluation index, which are α i , β i , γ i ; and α i + β i + γ i = 1; S103: Perform weighted evaluation on each information source to obtain the calculation result T i (t) of the trustworthiness of the information source at time t. The calculation formula is as follows: Among them, α i , β i , γ i correspond to the weight coefficients of accuracy, user feedback, and activity respectively.

3. The method for predicting public opinion trends by integrating deep learning and data mining according to claim 2, characterized in that: Eliminate the differences between the trustworthiness calculation results of different information sources, perform normalization processing on the calculated results, ensure that the trustworthiness scores of all information sources are within a standard range, then the trustworthiness value of the normalized information source S i is: Where n represents the number of information sources.

4. The method for predicting public opinion trends by integrating deep learning and data mining according to claim 3, characterized in that: Weighting the public opinion data based on trustworthiness specifically includes: S201: Collect relevant public opinion data, which is sourced from an information source S i , where the sentiment score or topic label D of each piece of public opinion data j ; S202: Weight the public opinion data according to the calculated information source trust value T(S i ); reflect the actual influence of each piece of data through the weighted public opinion data, and the formula is as follows: Among them, m represents the total number of public opinion data, and W(D j ) represents the importance degree of the weighted public opinion data.

5. The method for predicting public opinion trends by integrating deep learning and data mining according to claim 4, characterized in that: If D j takes the sentiment score, then D j specifically represents as: +1 (positive), 0 (neutral), -1 (negative); If D j When taking topic tags, the values are obtained by means of category encoding. If there are multiple topic categories, each category is encoded as a unique integer value.

6. The method for predicting public opinion trends by integrating deep learning and data mining according to claim 5, characterized in that: When T(S i ) is greater than or equal to the confidence threshold, it is considered that the information source makes a greater contribution to the prediction of the public opinion trend. At this time, the weighted public opinion data W(D j ) is amplified, that is, the amplified W(D j ) is equal to: W(D j ) × (T(S i ) / confidence threshold); When T(S i ) is less than the confidence threshold, it is considered that the influence of the information source on the prediction of the public opinion trend is weakened. At this time, the weighted public opinion data W(D j ) is reduced, and the reduced W(D j ) is equal to: W(D j ) × (T(S i ) / confidence threshold).

7. The public opinion trend prediction method integrating deep learning and data mining according to claim 7, characterized in that: Based on the amplified or reduced W(D j ), a weight threshold is set to determine which data has a significant impact on the prediction of public opinion trends; Only when the enlarged or reduced W(D j ) is greater than the weight threshold, will the data be marked as important data and enter the final public opinion trend prediction model.

8. An opinion trend prediction system integrating deep learning and data mining, which is applied to the opinion trend prediction method integrating deep learning and data mining according to any one of the above claims 1-7, and is characterized in that: The system includes: A public opinion data collection module, responsible for collecting public opinion data from data sources on different social media platforms; A public opinion data preprocessing module, which performs cleaning, denoising, and formatting preprocessing operations on the collected public opinion data to provide high-quality data for subsequent analysis; An information source trustworthiness evaluation module, which scores each information source according to multiple indicators and calculates its trustworthiness; A public opinion data weighting module, which weights the public opinion data according to the trustworthiness of the information source to adjust the contribution degree of each piece of data to the final prediction result; And a sentiment analysis module, which performs sentiment analysis on the weighted public opinion data to analyze the sentiment tendency of each piece of data; a public opinion trend prediction module, which predicts the future public opinion trend based on the weighted public opinion data and the sentiment analysis result, combined with a deep learning model.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the public opinion trend prediction method for integrating deep learning and data mining according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the public opinion trend prediction method for integrating deep learning and data mining according to any one of claims 1 to 7.

Images