Digitized new media public opinion analysis and management method and system based on big data

Abstract

The invention relates to the technical field of data analysis and management, in particular to a digital new media public opinion analysis and management method and system based on big data, and the method comprises the steps of demand determination and target setting, data collection, data preprocessing, data analysis, report compiling and data feedback. According to the method, the public opinion data is subjected to detailed type identification, and a targeted preprocessing method is adopted for different types of data, so that the quality and availability of the data are remarkably improved; performing shot segmentation, key frame extraction and feature extraction on the video data; the picture data is subjected to scaling, normalization, noise removal and edge detection; the voice data is subjected to noise suppression, endpoint detection, preloading, framing processing and feature extraction; the preprocessing steps not only enhance the accuracy and reliability of the data, but also provide a better data source for subsequent data analysis, thereby improving the efficiency and accuracy of the whole public opinion analysis process.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis and management, and in particular to a digital new media public opinion analysis management method and system based on big data. Background Art

[0002] With the rapid development of information technology, the amount of global data has shown explosive growth, and the era of big data has arrived. According to relevant data, the amount of global data has continued to rise in recent years. This huge user group generates massive amounts of data on the Internet every day. These data cover social media, news media, online forums and other fields, providing a rich data source for public opinion analysis.

[0003] In the new media environment, the types of public opinion data are becoming increasingly diverse and are no longer limited to traditional text data. Multimedia data such as video, pictures, and voice have become important carriers for the dissemination of public opinion. These multimedia data not only have a large amount of information, but also have rich forms of expression, and can more intuitively reflect the public's emotions and attitudes. However, this has also brought unprecedented challenges to public opinion analysis. How to effectively identify and process these diverse data types and extract valuable information has become a problem that needs to be urgently solved in the field of public opinion analysis.

[0004] Traditional public opinion analysis methods mainly rely on manual reading and judgment, which is incapable of processing large-scale and diverse data. On the one hand, manual processing is inefficient and costly, and it is difficult to meet real-time requirements; on the other hand, due to human subjectivity and limitations, the analysis results may be biased or missed. Therefore, a more efficient, accurate and automated public opinion analysis method is needed to meet the challenges of the new media environment. Summary of the invention

[0005] In order to overcome the above problems, the present invention provides a digital new media public opinion analysis management method and system based on big data.

[0006] The technical solution of the present invention is: a digital new media public opinion analysis and management method based on big data, comprising the following steps:

[0007] S11: Demand clarification and goal setting. First, clarify the specific purpose of public opinion analysis, and set specific public opinion analysis goals based on the purpose of public opinion analysis. Among them, specific public opinion analysis goals include media platforms that need to be monitored, monitoring time range and monitoring keywords;

[0008] S12: Data collection: first determine the data source, identify and determine the data source that needs to be collected, then use data collection technology to automatically capture public opinion data from the selected data source, and store the captured data;

[0009] S13: Data preprocessing: first, identify the types of captured public opinion data, and use targeted data preprocessing methods to preprocess and classify different types of public opinion data, and divide the preprocessed public opinion data into different data sets according to the types of public opinion data;

[0010] S14: Data analysis: using targeted data analysis methods to analyze public opinion data in different data sets, and using data mining algorithms to discover hidden information in the data, where the hidden information in the data includes hidden patterns of the data, association rules of the data, and trend predictions of the data;

[0011] S15: Report preparation: present the analysis results of public opinion data in a visual form and compile the analysis results into a public opinion analysis report, where the content of the public opinion analysis report includes the public opinion overview, main findings, trend forecasts and recommended measures;

[0012] S16: Data feedback, evaluate the effectiveness of public opinion analysis, including the accuracy, timeliness and practicality of the analysis results, and optimize the data preprocessing methods and data analysis methods based on the evaluation results and user feedback.

[0013] As a preferred method, when clarifying the specific purpose of public opinion analysis and setting specific public opinion analysis goals based on the purpose of public opinion analysis, the following steps are included:

[0014] S21: Determine the main purpose of public opinion analysis and break it down into quantifiable indicators;

[0015] S22: According to the main purpose of public opinion analysis, determine the media platforms that need to be monitored and clarify the time span of public opinion analysis. Media platforms include social media platforms, news websites, forums and blogs;

[0016] S23: According to the goal of public opinion analysis and the subject of public opinion analysis, determine keywords related to the subject of public opinion analysis;

[0017] S24: Develop a monitoring plan. Based on the identified media platforms that need to be monitored, the time span of public opinion analysis, and the identified keywords, develop a detailed monitoring plan. The content of the monitoring plan includes the monitoring time, frequency, data source, and data processing method.

[0018] Preferably, when using data collection technology to automatically capture public opinion data from a selected data source and storing the captured data, the following steps are included:

[0019] S31: Select a suitable data collection tool and configure the environment required for the data collection tool to run;

[0020] S32: setting the operating parameters of the data collection tool, and then running the data collection tool to collect public opinion data using the data collection tool;

[0021] S33: Designing a corresponding database table structure according to the data type and structure collected, and building a database, and then storing the collected data in the built database;

[0022] S34: Back up the database regularly to prevent data loss.

[0023] Preferably, when the captured public opinion data is type-identified, and different types of public opinion data are pre-processed and classified using a targeted data pre-processing method, and the pre-processed public opinion data is divided into different data sets according to the type of the public opinion data, the following steps are included:

[0024] S41: type identification, performing format identification on the captured public opinion data, and classifying the public opinion data into different types according to the format of the public opinion data, wherein the public opinion data is classified into video public opinion data, picture public opinion data, voice public opinion data, and text public opinion data according to the type;

[0025] S42: preprocessing different types of public opinion data using different data preprocessing methods to enhance the quality of the public opinion data and obtain the preprocessed public opinion data;

[0026] S43: Establish multiple data sets according to the types of public opinion data divided in step S41, and store different types of public opinion data after preprocessing in different data sets.

[0027] Preferably, when preprocessing the video public opinion data, the preprocessing method used includes the following steps:

[0028] S4211: Shot segmentation, segmenting the video public opinion data into multiple shots, wherein the adopted methods include frame difference, color histogram comparison, block matching and motion detection methods;

[0029] S4212: extracting key frames, extracting key frames representing the content of each shot from each shot, wherein the extracted key frames contain main information of the corresponding shot;

[0030] S4213: Feature extraction: further processing the extracted key frames to extract the main features of the key frames, wherein the main features of the key frames include color features, texture features and shape features.

[0031] Preferably, when preprocessing the image public opinion data, the preprocessing method used includes the following steps:

[0032] S4221: Image scaling, scaling all images in the image public opinion data to the same size;

[0033] S4222: Normalization processing is performed to normalize the pixel values ​​of the images so that the pixel values ​​of all images are distributed within a set range. The principle formula of the normalization processing is:

[0034]

[0035] Among them, x is the pixel value of the image, x' is the new value of the pixel value after normalization, and the value range of x' is [0,1]. max is the maximum value among all pixels with bright red values, x min is the minimum value among all pixel values;

[0036] S4223: Noise removal: removing noise in the image by using an image filtering algorithm, wherein the image filtering algorithm used is:

[0037] Among them, I filtered (x, y) is the pixel value of the filtered image at the coordinate (x, y), σ is the standard deviation of the Gaussian function, k is the size of the filter, i and j are loop variables used to traverse all elements of the filter, I(x+i, y+j) is the pixel value of the input image at the coordinate (x+i, y+j), that is, an element in the filter;

[0038] S4224: Edge detection and segmentation: using edge detection algorithm to extract edge information in the image and segment the image into different areas.

[0039] Preferably, when preprocessing the voice public opinion data, the preprocessing method adopted includes the following steps:

[0040] S4231: Noise suppression, removes background noise from speech signals and improves speech quality;

[0041] S4232: endpoint detection, determining the start and end points of the voice signal and removing the silent part;

[0042] S4233: Pre-emphasis, which enhances the energy of the high frequency part and makes the spectrum flatter;

[0043] S4234: Frame processing, dividing the continuous speech signal into multiple shorter frames for subsequent processing;

[0044] S4235: Feature extraction, extracting feature parameters from each frame of the speech signal segmented into multiple frames, wherein the extracted feature parameters include Mel-frequency cepstral coefficients.

[0045] Preferably, when adopting targeted data analysis methods to analyze public opinion data in different data sets and using data mining algorithms to discover hidden information in the data, the following steps are included:

[0046] S51: Use targeted data analysis methods to analyze public opinion data in different data sets and extract features from public opinion data;

[0047] S52: Use data mining algorithms to discover hidden patterns and associations in public opinion data. The data mining algorithms used include association rule mining algorithms and sequence pattern mining algorithms. When using association rule mining algorithms for data analysis, it includes calculating support, calculating confirmation, and calculating improvement. When calculating support, the mathematical principle formula is:

[0048]

[0049] Among them, count(X) represents the number of times item set X appears, total_transactions represents the total number of transactions, and item set X is a feature in the public opinion data;

[0050] S53: Based on historical data and current trends, use prediction algorithms to predict the future development of public opinion. The prediction algorithms used include time series analysis algorithms and machine learning algorithms.

[0051] S54: Cross-analyze different types of data to gain more comprehensive insights into public opinion.

[0052] Preferably, when adopting targeted data analysis methods to analyze the public opinion data in different data sets and extracting features from the public opinion data, the data analysis methods adopted for the public opinion data in different data sets are:

[0053] A11: For text public opinion data, natural language processing technology is used to identify the emotional tendencies, main discussion topics and hot spots in the text. The natural language technologies used include sentiment analysis, topic clustering and keyword extraction.

[0054] A12: For video public opinion data, use video recognition technology to extract key frames, subtitles, and audio information in the video to perform sentiment analysis and content classification;

[0055] A13: For image public opinion data, use image recognition technology to identify key elements and scenes in the image, and make sentiment judgments and content classification based on the context;

[0056] A14: For voice public opinion data, use speech recognition and voice sentiment analysis technology to convert voice data into text and extract sentiment features for further analysis.

[0057] Preferably, when presenting the analysis results of the public opinion data in a visual form and compiling the analysis results into a public opinion analysis report, the following steps are included:

[0058] S61: Data collation and aggregation: collating the analyzed public opinion data and aggregating them according to one of the dimensions of time, topic and sentiment tendency;

[0059] S62: Visualization design: select appropriate visualization tools and chart types based on the results of public opinion analysis, and design visualization solutions based on the selected visualization tools and chart types;

[0060] S63: Report writing: Write the report introduction, briefly introduce the purpose, background and significance of the public opinion analysis, then summarize the main characteristics and trends of public opinion, describe the main findings in detail, and propose targeted recommended measures based on the results;

[0061] S64: Review and modification: Conduct internal review of the report to ensure the accuracy of the analysis results and the rigor of the report, and modify and improve the report based on the review opinions to ensure that the report content is correct and the logic is clear;

[0062] S65: Customized adjustment, customize the report according to the needs and preferences of different users.

[0063] Digital new media public opinion analysis and management system based on big data, including:

[0064] Data collection module, used to capture public opinion data from various new media platforms;

[0065] Data preprocessing module, used to preprocess the collected raw data to improve the quality and availability of the data;

[0066] A data storage module is used to store the pre-processed data in the system safely and efficiently;

[0067] Data analysis module, responsible for in-depth mining and analysis of stored data;

[0068] The data visualization module is used to present the analysis results of the data analysis module to the user in a visual form.

[0069] Beneficial effects of the present invention:

[0070] 1. Through detailed type identification of public opinion data (such as video, picture, voice and text), and targeted preprocessing methods for different types of data, the quality and availability of data have been significantly improved. For video data, through shot segmentation, key frame extraction and feature extraction, key information in the video is effectively captured; image data is scaled, normalized, noise removed and edge detected to ensure image consistency and clarity; voice data is improved through noise suppression, endpoint detection, pre-emphasis, frame processing and feature extraction to improve the clarity and analyzability of voice signals; these preprocessing steps not only enhance the accuracy and reliability of the data, but also provide a better data source for subsequent data analysis, thereby improving the efficiency and accuracy of the entire public opinion analysis process;

[0071] 2. Through the comprehensive use of targeted data analysis methods and advanced data mining and prediction algorithms, a comprehensive and in-depth insight into the complex and ever-changing public opinion data is achieved; it can not only effectively extract multi-dimensional features in public opinion data, including key information and emotional tendencies in text, video, pictures and voice, but also reveal the deep-level patterns and correlations hidden in public opinion data through association rule mining and sequence pattern mining algorithms; further, combining historical data with current trends, using time series analysis and machine learning algorithms to accurately predict future public opinion developments, enhancing the foresight and scientific nature of decision-making; in addition, through cross-analysis of cross-type data, it promotes the comprehensive integration and in-depth understanding of information, providing strong support for the formulation of accurate and effective public opinion response strategies;

[0072] 3. Comprehensive, accurate and timely monitoring and analysis of new media public opinion are achieved through clear needs and goals, efficient data collection, refined data preprocessing, in-depth data analysis and visual report preparation. This technical solution not only helps to quickly capture public opinion dynamics, mine hidden information and predict future trends, but also provides strong data support for decision-making and improves the scientificity and effectiveness of public opinion management. At the same time, the analysis methods and processes are continuously optimized through the data feedback mechanism to ensure the accuracy and practicality of public opinion analysis, which provides an important guarantee for organizations to gain the initiative and shape a good image in a complex and changing media environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 What is shown is a flow chart of the digital new media public opinion analysis and management method based on big data of the present invention;

[0074] Figure 2 What is shown is a schematic diagram of the structure of the digital new media public opinion analysis and management system based on big data of the present invention. DETAILED DESCRIPTION

[0075] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0076] See also Figure 1 The present invention provides an embodiment: a digital new media public opinion analysis and management method based on big data, comprising the following steps:

[0077] S11: Demand clarification and goal setting. First, clarify the specific purpose of public opinion analysis, and set specific public opinion analysis goals based on the purpose of public opinion analysis. Among them, specific public opinion analysis goals include media platforms that need to be monitored, monitoring time range and monitoring keywords;

[0078] S12: Data collection: first determine the data source, identify and determine the data source that needs to be collected, then use data collection technology to automatically capture public opinion data from the selected data source, and store the captured data;

[0079] S13: Data preprocessing: first, identify the types of captured public opinion data, and use targeted data preprocessing methods to preprocess and classify different types of public opinion data, and divide the preprocessed public opinion data into different data sets according to the types of public opinion data;

[0080] S14: Data analysis: using targeted data analysis methods to analyze public opinion data in different data sets, and using data mining algorithms to discover hidden information in the data, where the hidden information in the data includes hidden patterns of the data, association rules of the data, and trend predictions of the data;

[0081] S15: Report preparation: present the analysis results of public opinion data in a visual form and compile the analysis results into a public opinion analysis report, where the content of the public opinion analysis report includes the public opinion overview, main findings, trend forecasts and recommended measures;

[0082] S16: Data feedback, evaluate the effectiveness of public opinion analysis, including the accuracy, timeliness and practicality of the analysis results, and optimize the data preprocessing methods and data analysis methods based on the evaluation results and user feedback.

[0083] Specifically, as described above, the present invention realizes comprehensive, accurate and timely monitoring and analysis of new media public opinion through clear needs and goals, efficient data collection, refined data preprocessing, in-depth data analysis and visual report preparation; this technical solution not only helps to quickly capture public opinion dynamics, mine hidden information, and predict future trends, but also provides strong data support for decision-making, and improves the scientificity and effectiveness of public opinion management; at the same time, through the data feedback mechanism, the analysis methods and processes are continuously optimized to ensure the accuracy and practicality of public opinion analysis, which provides an important guarantee for organizations to gain the initiative and shape a good image in a complex and changing media environment.

[0084] As a preferred method, when clarifying the specific purpose of public opinion analysis and setting specific public opinion analysis goals based on the purpose of public opinion analysis, the following steps are included:

[0085] S21: Determine the main purpose of public opinion analysis and break it down into quantifiable indicators;

[0086] S22: According to the main purpose of public opinion analysis, determine the media platforms that need to be monitored and clarify the time span of public opinion analysis. Media platforms include social media platforms, news websites, forums and blogs;

[0087] S23: According to the goal of public opinion analysis and the subject of public opinion analysis, determine keywords related to the subject of public opinion analysis;

[0088] S24: Develop a monitoring plan. Based on the identified media platforms that need to be monitored, the time span of public opinion analysis, and the identified keywords, develop a detailed monitoring plan. The content of the monitoring plan includes the monitoring time, frequency, data source, and data processing method.

[0089] Specifically, as described above, the present invention significantly improves the pertinence and effectiveness of public opinion analysis through detailed planning; first, the main purpose is refined into quantifiable indicators to ensure the clarity of the analysis objectives; secondly, the media platform and time span of monitoring are accurately located to achieve comprehensive coverage of key information sources; thirdly, keywords are determined based on the analysis subject to enhance the pertinence and relevance of data capture; finally, a detailed monitoring plan is formulated to clarify the time, frequency, data source and processing method of monitoring, laying a solid foundation for conducting public opinion analysis in an efficient and orderly manner; this series of steps not only improves the accuracy and efficiency of public opinion analysis, but also ensures the comprehensiveness and timeliness of the analysis results, providing strong information support for organizational decision-making.

[0090] Preferably, when using data collection technology to automatically capture public opinion data from a selected data source and storing the captured data, the following steps are included:

[0091] S31: Select a suitable data collection tool and configure the environment required for the data collection tool to run;

[0092] S32: setting the operating parameters of the data collection tool, and then running the data collection tool to collect public opinion data using the data collection tool;

[0093] S33: Designing a corresponding database table structure according to the data type and structure collected, and building a database, and then storing the collected data in the built database;

[0094] S34: Back up the database regularly to prevent data loss.

[0095] Specifically, as described above, the present invention demonstrates efficient and secure data processing capabilities through the implementation of steps S31 to S34; first, a suitable data collection tool is selected and a suitable operating environment is configured to ensure the smooth progress of data collection; secondly, the collection parameters are flexibly set and the tool is run to achieve accurate capture of public opinion data; then, the database table structure is designed and the database is built according to the data type, which effectively organizes the collected data for subsequent analysis and processing; finally, the database is backed up regularly to ensure the security and integrity of the data and prevent potential data loss risks; this series of measures not only improves the efficiency and accuracy of data collection, but also enhances the security and reliability of data storage, laying a solid foundation for subsequent public opinion analysis.

[0096] Preferably, when the captured public opinion data is type-identified, and different types of public opinion data are pre-processed and classified using a targeted data pre-processing method, and the pre-processed public opinion data is divided into different data sets according to the type of the public opinion data, the following steps are included:

[0097] S41: type identification, performing format identification on the captured public opinion data, and classifying the public opinion data into different types according to the format of the public opinion data, wherein the public opinion data is classified into video public opinion data, picture public opinion data, voice public opinion data, and text public opinion data according to the type;

[0098] S42: preprocessing different types of public opinion data using different data preprocessing methods to enhance the quality of the public opinion data and obtain the preprocessed public opinion data;

[0099] S43: Establish multiple data sets according to the types of public opinion data divided in step S41, and store different types of public opinion data after preprocessing in different data sets.

[0100] Preferably, when preprocessing the video public opinion data, the preprocessing method used includes the following steps:

[0101] S4211: Shot segmentation, segmenting the video public opinion data into multiple shots, wherein the adopted methods include frame difference, color histogram comparison, block matching and motion detection methods;

[0102] S4212: extracting key frames, extracting key frames representing the content of each shot from each shot, wherein the extracted key frames contain main information of the corresponding shot;

[0103] S4213: Feature extraction: further processing the extracted key frames to extract the main features of the key frames, wherein the main features of the key frames include color features, texture features and shape features.

[0104] Preferably, when preprocessing the image public opinion data, the preprocessing method used includes the following steps:

[0105] S4221: Image scaling, scaling all images in the image public opinion data to the same size;

[0106] S4222: Normalization processing is performed to normalize the pixel values ​​of the images so that the pixel values ​​of all images are distributed within a set range. The principle formula of the normalization processing is:

[0107]

[0108] Among them, x is the pixel value of the image, x' is the new value of the pixel value after normalization, and the value range of x' is [0,1]. max is the maximum value among all pixels with bright red values, x min is the minimum value among all pixel values;

[0109] S4223: Noise removal: removing noise in the image by using an image filtering algorithm, wherein the image filtering algorithm used is:

[0110] Among them, I filtered (x, y) is the pixel value of the filtered image at the coordinate (x, y), σ is the standard deviation of the Gaussian function, k is the size of the filter, i and j are loop variables used to traverse all elements of the filter, I(x+i, y+j) is the pixel value of the input image at the coordinate (x+i, y+j), that is, an element in the filter;

[0111] S4224: Edge detection and segmentation: using edge detection algorithm to extract edge information in the image and segment the image into different areas.

[0112] Preferably, when preprocessing the voice public opinion data, the preprocessing method adopted includes the following steps:

[0113] S4231: Noise suppression, removes background noise from speech signals and improves speech quality;

[0114] S4232: endpoint detection, determining the start and end points of the voice signal and removing the silent part;

[0115] S4233: Pre-emphasis, which enhances the energy of the high frequency part and makes the spectrum flatter;

[0116] S4234: Frame processing, dividing the continuous speech signal into multiple shorter frames for subsequent processing;

[0117] S4235: Feature extraction, extracting feature parameters from each frame of the speech signal segmented into multiple frames, wherein the extracted feature parameters include Mel-frequency cepstral coefficients.

[0118] Specifically, as described above, the present invention significantly improves the quality and availability of public opinion data by performing detailed type identification (such as video, picture, voice and text) on public opinion data and adopting targeted preprocessing methods for different types of data; for video data, through shot segmentation, key frame extraction and feature extraction, the key information in the video is effectively captured; the picture data is scaled, normalized, denoised and edge detected to ensure the consistency and clarity of the image; the voice data is improved through noise suppression, endpoint detection, pre-emphasis, frame processing and feature extraction to improve the clarity and analyzability of the voice signal; these preprocessing steps not only enhance the accuracy and reliability of the data, but also provide a higher quality data source for subsequent data analysis, thereby improving the efficiency and accuracy of the entire public opinion analysis process.

[0119] Preferably, when adopting targeted data analysis methods to analyze public opinion data in different data sets and using data mining algorithms to discover hidden information in the data, the following steps are included:

[0120] S51: Use targeted data analysis methods to analyze public opinion data in different data sets and extract features from public opinion data;

[0121] S52: Use data mining algorithms to discover hidden patterns and associations in public opinion data. The data mining algorithms used include association rule mining algorithms and sequence pattern mining algorithms. When using association rule mining algorithms for data analysis, it includes calculating support, calculating confirmation, and calculating improvement. When calculating support, the mathematical principle formula is:

[0122]

[0123] Among them, count(X) represents the number of times item set X appears, total_transactions represents the total number of transactions, and item set X is a feature in the public opinion data;

[0124] S53: Based on historical data and current trends, use prediction algorithms to predict the future development of public opinion. The prediction algorithms used include time series analysis algorithms and machine learning algorithms.

[0125] S54: Cross-analyze different types of data to gain more comprehensive insights into public opinion.

[0126] Preferably, when adopting targeted data analysis methods to analyze the public opinion data in different data sets and extracting features from the public opinion data, the data analysis methods adopted for the public opinion data in different data sets are:

[0127] A11: For text public opinion data, natural language processing technology is used to identify the emotional tendencies, main discussion topics and hot spots in the text. The natural language technologies used include sentiment analysis, topic clustering and keyword extraction.

[0128] A12: For video public opinion data, use video recognition technology to extract key frames, subtitles, and audio information in the video to perform sentiment analysis and content classification;

[0129] A13: For image public opinion data, use image recognition technology to identify key elements and scenes in the image, and make sentiment judgments and content classification based on the context;

[0130] A14: For voice public opinion data, use speech recognition and voice sentiment analysis technology to convert voice data into text and extract sentiment features for further analysis.

[0131] Specifically, as described above, the present invention achieves a comprehensive and in-depth insight into complex and changeable public opinion data by comprehensively using targeted data analysis methods and advanced data mining and prediction algorithms; it can not only effectively extract multi-dimensional features in public opinion data, including key information and emotional tendencies in text, video, pictures and voice, but also reveals the deep-level patterns and correlations hidden in public opinion data through association rule mining and sequence pattern mining algorithms; further, combining historical data with current trends, using time series analysis and machine learning algorithms to accurately predict future public opinion developments, thereby enhancing the foresight and scientific nature of decision-making; in addition, through cross-analysis of cross-type data, it promotes the comprehensive integration and in-depth understanding of information, providing strong support for the formulation of accurate and effective public opinion response strategies.

[0132] Preferably, when presenting the analysis results of the public opinion data in a visual form and compiling the analysis results into a public opinion analysis report, the following steps are included:

[0133] S61: Data collation and aggregation: collating the analyzed public opinion data and aggregating them according to one of the dimensions of time, topic and sentiment tendency;

[0134] S62: Visualization design: select appropriate visualization tools and chart types based on the results of public opinion analysis, and design visualization solutions based on the selected visualization tools and chart types;

[0135] S63: Report writing: Write the report introduction, briefly introduce the purpose, background and significance of the public opinion analysis, then summarize the main characteristics and trends of public opinion, describe the main findings in detail, and propose targeted recommended measures based on the results;

[0136] S64: Review and modification: Conduct internal review of the report to ensure the accuracy of the analysis results and the rigor of the report, and modify and improve the report based on the review opinions to ensure that the report content is correct and the logic is clear;

[0137] S65: Customized adjustment, customize the report according to the needs and preferences of different users.

[0138] Specifically, as described above, the present invention converts complex public opinion analysis results into intuitive, easy-to-understand, information-rich visualization charts and in-depth analysis reports through systematic data collation and aggregation, carefully designed visualization presentation and detailed report writing process. This process not only improves the efficiency and accuracy of information communication, but also meets the personalized needs of different users through customized adjustments. Ultimately, the generated public opinion analysis report not only provides decision makers with comprehensive and in-depth insights into public opinion, but also helps them formulate more scientific and reasonable response strategies through targeted recommendations and measures, effectively improving decision-making efficiency and effectiveness.

[0139] See also Figure 2 The present invention provides an embodiment: a digital new media public opinion analysis and management system based on big data, comprising:

[0140] Data collection module, used to capture public opinion data from various new media platforms;

[0141] Data preprocessing module, used to preprocess the collected raw data to improve the quality and availability of the data;

[0142] A data storage module is used to store the pre-processed data in the system safely and efficiently;

[0143] Data analysis module, responsible for in-depth mining and analysis of stored data;

[0144] The data visualization module is used to present the analysis results of the data analysis module to the user in a visual form.

[0145] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of those skilled in the art without departing from the spirit of the present invention.

Claims

1. A digital new media public opinion analysis and management method based on big data; it is characterized by: The following steps are included: S11: Demand clarification and goal setting. First, clarify the specific purpose of public opinion analysis, and set specific public opinion analysis goals based on the purpose of public opinion analysis. Among them, specific public opinion analysis goals include media platforms that need to be monitored, monitoring time range and monitoring keywords; S12: Data collection: first determine the data source, identify and determine the data source that needs to be collected, then use data collection technology to automatically capture public opinion data from the selected data source, and store the captured data; S13: Data preprocessing: first, identify the types of captured public opinion data, and use targeted data preprocessing methods to preprocess and classify different types of public opinion data, and divide the preprocessed public opinion data into different data sets according to the types of public opinion data; S14: Data analysis: using targeted data analysis methods to analyze public opinion data in different data sets, and using data mining algorithms to discover hidden information in the data, where the hidden information in the data includes hidden patterns of the data, association rules of the data, and trend predictions of the data; S15: Report preparation: present the analysis results of public opinion data in a visual form and compile the analysis results into a public opinion analysis report, where the content of the public opinion analysis report includes the public opinion overview, main findings, trend forecasts and recommended measures; S16: Data feedback, evaluate the effectiveness of public opinion analysis, including the accuracy, timeliness and practicality of the analysis results, and optimize the data preprocessing methods and data analysis methods based on the evaluation results and user feedback.

2. The digital new media public opinion analysis and management method based on big data according to claim 1 is characterized by: When clarifying the specific purpose of public opinion analysis and setting specific public opinion analysis goals based on the purpose of public opinion analysis, the following steps are included: S21: Determine the main purpose of public opinion analysis and break it down into quantifiable indicators; S22: According to the main purpose of public opinion analysis, determine the media platforms that need to be monitored and clarify the time span of public opinion analysis. Media platforms include social media platforms, news websites, forums and blogs; S23: According to the goal of public opinion analysis and the subject of public opinion analysis, determine keywords related to the subject of public opinion analysis; S24: Develop a monitoring plan. Based on the identified media platforms that need to be monitored, the time span of public opinion analysis, and the identified keywords, develop a detailed monitoring plan. The content of the monitoring plan includes the monitoring time, frequency, data source, and data processing method.

3. The digital new media public opinion analysis and management method based on big data according to claim 2 is characterized by: When using data collection technology to automatically capture public opinion data from a selected data source and store the captured data, the following steps are included: S31: Select a suitable data collection tool and configure the environment required for the data collection tool to run; S32: setting the operating parameters of the data collection tool, and then running the data collection tool to collect public opinion data using the data collection tool; S33: Designing a corresponding database table structure according to the data type and structure collected, and building a database, and then storing the collected data in the built database; S34: Back up the database regularly to prevent data loss.

4. The digital new media public opinion analysis and management method based on big data according to claim 3 is characterized by: When the captured public opinion data is identified by type, and different types of public opinion data are preprocessed and classified by targeted data preprocessing methods, and the preprocessed public opinion data is divided into different data sets according to the type of public opinion data, the following steps are included: S41: type identification, performing format identification on the captured public opinion data, and classifying the public opinion data into different types according to the format of the public opinion data, wherein the public opinion data is classified into video public opinion data, picture public opinion data, voice public opinion data, and text public opinion data according to the type; S42: preprocessing different types of public opinion data using different data preprocessing methods to obtain preprocessed public opinion data; S43: Establish multiple data sets according to the types of public opinion data divided in step S41, and store different types of public opinion data after preprocessing in different data sets.

5. The digital new media public opinion analysis and management method based on big data according to claim 4 is characterized by: When preprocessing video public opinion data, the preprocessing method used includes the following steps: S4211: Shot segmentation, segmenting the video public opinion data into multiple shots, wherein the adopted methods include frame difference, color histogram comparison, block matching and motion detection methods; S4212: extracting key frames, extracting key frames representing the content of each shot from each shot, wherein the extracted key frames contain main information of the corresponding shot; S4213: Feature extraction: further processing the extracted key frames to extract the main features of the key frames, wherein the main features of the key frames include color features, texture features and shape features.

6. The digital new media public opinion analysis and management method based on big data according to claim 5 is characterized by: When preprocessing the image public opinion data, the preprocessing method used includes the following steps: S4221: Image scaling, scaling all images in the image public opinion data to the same size; S4222: Normalization processing is performed to normalize the pixel values ​​of the images so that the pixel values ​​of all images are distributed within a set range. The principle formula of the normalization processing is: Among them, x is the pixel value of the image, x' is the new value of the pixel value after normalization, and the value range of x' is [0,1]. max is the maximum value among all pixels with bright red values, x min is the minimum value among all pixel values; S4223: Noise removal: removing noise in the image by using an image filtering algorithm, wherein the image filtering algorithm used is: Among them, I filtered (x, y) is the pixel value of the filtered image at the coordinate (x, y), σ is the standard deviation of the Gaussian function, k is the size of the filter, i and j are loop variables used to traverse all elements of the filter, I(x+i, y+j) is the pixel value of the input image at the coordinate (x+i, y+j), that is, an element in the filter; S4224: Edge detection and segmentation: using edge detection algorithm to extract edge information in the image and segment the image into different areas.

7. The digital new media public opinion analysis and management method based on big data according to claim 6 is characterized by: When preprocessing voice public opinion data, the preprocessing method used includes the following steps: S4231: Noise suppression, removes background noise from speech signals and improves speech quality; S4232: endpoint detection, determining the start and end points of the voice signal and removing the silent part; S4233: Pre-emphasis, which enhances the energy of the high frequency part and makes the spectrum flatter; S4234: Frame processing, dividing the continuous speech signal into multiple shorter frames for subsequent processing; S4235: Feature extraction, extracting feature parameters from each frame of the speech signal segmented into multiple frames, wherein the extracted feature parameters include Mel-frequency cepstral coefficients.

8. The digital new media public opinion analysis and management method based on big data according to claim 7 is characterized by: When using targeted data analysis methods to analyze public opinion data in different data sets and using data mining algorithms to discover hidden information in the data, the following steps are included: S51: Use targeted data analysis methods to analyze public opinion data in different data sets and extract features from public opinion data; S52: Use data mining algorithms to discover hidden patterns and associations in public opinion data. The data mining algorithms used include association rule mining algorithms and sequence pattern mining algorithms. When using association rule mining algorithms for data analysis, it includes calculating support, calculating confirmation, and calculating improvement. When calculating support, the mathematical principle formula is: Among them, count(X) represents the number of times item set X appears, total_transactions represents the total number of transactions, and item set X is a feature in the public opinion data; S53: Based on historical data and current trends, use prediction algorithms to predict the future development of public opinion. The prediction algorithms used include time series analysis algorithms and machine learning algorithms. S54: Cross-analyze different types of data to gain more comprehensive insights into public opinion.

9. The digital new media public opinion analysis and management method based on big data according to claim 8 is characterized by: When using targeted data analysis methods to analyze public opinion data in different data sets and extract features from public opinion data, the data analysis methods used for public opinion data in different data sets are: A11: For text public opinion data, natural language processing technology is used to identify the emotional tendencies, main discussion topics and hot spots in the text. The natural language technologies used include sentiment analysis, topic clustering and keyword extraction. A12: For video public opinion data, use video recognition technology to extract key frames, subtitles, and audio information in the video to perform sentiment analysis and content classification; A13: For image public opinion data, use image recognition technology to identify key elements and scenes in the image, and make sentiment judgments and content classification based on the context; A14: For voice public opinion data, use speech recognition and voice sentiment analysis technology to convert voice data into text and extract sentiment features for further analysis.

10. A digital new media public opinion analysis and management system based on big data, characterized by: Included are: Data collection module, used to capture public opinion data from various new media platforms; Data preprocessing module, used to preprocess the collected raw data to improve the quality and availability of the data; A data storage module is used to store the pre-processed data in the system safely and efficiently; Data analysis module, responsible for in-depth mining and analysis of stored data; The data visualization module is used to present the analysis results of the data analysis module to the user in a visual form.

Images