Negative public opinion discovery method and system based on intelligent algorithm

Abstract

The invention relates to the field of information extraction, in particular to a negative public opinion discovery method and system based on an intelligent algorithm. The method comprises the following steps: selecting a social media platform, collecting information data, and processing the information data to obtain processed data; checking the processed data, and performing warehousing processing on the data meeting the standard to obtain warehousing data; performing extraction and emotion recognition on the stored data by using a small model to obtain preliminary negative information; and performing extraction and emotion recognition on the preliminary negative information by using a large model to obtain negative public opinions. According to the method, negative information is screened out through the small model, and the large model further finds information of a specified field from the negative information. Therefore, public opinion information can be extracted more quickly, more accurately and more effectively from massive social data.

Description

Technical Field

[0001] The present invention relates to the field of information extraction, and more specifically, to a method and system for discovering negative public opinion based on intelligent algorithms. Background Art

[0002] With the rapid development of Internet information technology, the Internet, as a tool of public opinion, has the function of responding to and guiding public opinion. In view of this free and disordered online public opinion information, the guiding role of online public opinion index calculation is becoming increasingly powerful. Timely grasping the dynamics of public opinion and actively guiding social public opinion are important measures to maintain social stability. Therefore, the calculation of public opinion index is the basis for studying public opinion monitoring, event deduction and prediction technologies. The study of negative public opinion index calculation has important theoretical and practical significance. The calculation of negative public opinion mainly relies on natural language processing technology and the establishment of some model indicator systems to identify negative information, but there are some defects that cannot be fundamentally solved. Therefore, in recent years, a large number of scholars have combined multi-dimensional data mining, natural language processing, knowledge graphs and other technologies to manage public opinion information, monitor and visualize public opinion, timely discover public opinion hotspots, and early warn, control and guide public opinion information, so as to avoid large-scale public opinion crises and the development of negative public opinion, and achieve the purpose of saving network resources and purifying the network environment. However, current news websites and social media are filled with a large amount of ambiguous, non-normative and unstructured information. At the same time, natural language also contains a large amount of knowledge accumulation and thinking reasoning process. Therefore, the discovery of negative public opinion cannot simultaneously take into account the rapid, accurate and effective extraction of public opinion information from massive social data.

[0003] The prior art discloses a method and system for calculating the negative network public opinion index based on deep learning, which belongs to the technical field of text analysis and natural language processing, and solves the problem in the prior art that a large amount of manually annotated data is required to guide model training by using supervised machine learning technology. The present invention obtains public opinion articles containing keywords; obtains a large number of articles and processes them to obtain a training set, and uses the training set to train the MLSTM model to obtain a negative public opinion recognition model; based on the negative public opinion recognition model, a small number of articles are annotated, and after obtaining the feature vector of each article based on the negative public opinion recognition model, a logistic regression classifier is trained to obtain the dimension k corresponding to the weight |W| with the largest absolute value; based on the negative public opinion recognition model, the negative public opinion score of the public opinion article containing the keyword is calculated, and then the final negative public opinion index of the keyword is obtained according to the negative public opinion score corresponding to the dimension k. And the system corresponding to the method, the present invention is used to calculate the negative network public opinion index. However, this method still cannot extract public opinion information from massive social data quickly, accurately and effectively. Summary of the invention

[0004] The purpose of the present invention is to disclose a negative public opinion discovery method and system based on intelligent algorithms that are applicable to massive social data and are faster, more accurate and more effective.

[0005] In order to achieve the above objectives, the present invention provides a faster, more accurate and more effective method for discovering negative public opinion based on intelligent algorithms that is applicable to massive social data, including:

[0006] S1: Select a social media platform, collect information data, and process the information data to obtain processed data;

[0007] S2: Check the processed data, and process the data that meets the standards into the warehouse to obtain the warehoused data;

[0008] S3: Use a small model to extract and identify sentiment from incoming data to obtain preliminary negative information;

[0009] S4: Use the large model to extract preliminary negative information and identify sentiment to obtain negative public opinion.

[0010] Furthermore, in step S4, it includes:

[0011] Through a large-parameter large model, preliminary negative information is extracted and sentiment is recognized to obtain negative public opinion results, and the text content of the negative public opinion results is directly output; through a multimodal large model, the image content of the negative public opinion results is secondary extracted and sentiment recognized and then output.

[0012] Furthermore, in step S1, collecting information data includes: using hyperlink analysis technology, combining multiple web page data format standards, intelligently parsing multiple web page types, and automatically extracting information data; the web page type includes at least one of the following: static html, dynamic JS, Ajax, ASP, PHP; the information data includes at least one of the following: the title, time, source, author, and text of the article;

[0013] Furthermore, in step S1, processing the information data includes:

[0014] Manual management of source labels: organize information data to form corresponding index hash codes, including media classification and labeling: classification by level and type includes: CCTV media, provincial media, local media; semantic classification by content includes: economic news, government news, financial news, people's livelihood news;

[0015] Fingerprint calculation of similar articles: By extracting the main content of information data and raising the subject words, the subject words are uniformly mapped to a common fingerprint template, and the accurate and raised article content fingerprint is obtained through encryption algorithm and hash mapping calculation;

[0016] Spam identification and filtering: Based on the existing spam filtering mechanism, useless information is automatically cleaned. Useless information includes at least one of the following: advertisements, irrelevant images, hyperlinks, and dynamic Flash;

[0017] Data deduplication and merging: Through the content duplication detection engine, based on the analysis of semantics, it can automatically determine whether there is duplicate information data, as well as the number, title, source and collection time of the duplicate information data, to achieve automatic deduplication of information data; and it can automatically identify and merge multiple web page contents with continuity to improve user reading efficiency.

[0018] Furthermore, in step S2, checking the processed data includes:

[0019] Data consistency verification: set a series of verification rules, strict verification and early warning, including time trend verification of processed data, early warning to find sites with obvious decline; threshold verification rules from the perspective of field rationality, find errors in processed data, and quickly warn people; at the same time, combined with manual inspection strategies, complete offline and online consistency comparison of processed data at key sites;

[0020] Data integrity verification: used to measure whether the processed data is lost or unavailable; standardization, used to measure whether the processed data is stored in a unified format; consistency, used to measure whether the values ​​of the processed data are conflicting in information meaning; accuracy, used to measure whether the processed data is correct or whether the processed data is expired; uniqueness, used to measure whether the processed data is duplicated;

[0021] Data validity verification: including non-empty check: when the field is required to be non-empty, the processed data of the field needs to be checked; primary key duplication: after the same type of processed data in multiple business systems is cleaned, it is necessary to check it when it is saved in a unified manner to ensure the uniqueness of the primary key; illegal code and illegal value cleaning: illegal code problems include illegal code and code inconsistency with data standards, illegal value problems include value errors, format errors, extra characters, and garbled codes, which need to be checked and corrected according to specific circumstances; data format check: the accuracy is measured by checking whether the format of the attribute value in the table is correct, such as time format, currency format, extra characters, and garbled codes; record number check: refers to the total number of data between related data in each system or the fluctuation check of the daily data volume in the data table; business constraint verification: in the process of implementing data processing, the responsible personnel must consider the data verification rules from the perspectives of business correctness, consistency, and effectiveness.

[0022] Furthermore, in step S3, extracting the incoming data using the small model includes:

[0023] The small model is an event transmission chain model, which sets different indicator parameters and key features according to the incubation period, germination period, outbreak period, maturity period and decline period of the event, and focuses on negative information in each period from different angles, especially the information, opinions and comments generated in the outbreak period as the key points to introduce into the sample data of the event transmission chain model;

[0024] During the training process of the small model, humans constantly intervene and mark relevant and characteristic data for the machine to learn by itself;

[0025] The small model automatically identifies concepts with the same meaning and filters out a large amount of irrelevant information, so that the preliminary negative information obtained has a higher accuracy rate.

[0026] Furthermore, it specifically includes:

[0027] Load and prepare the model: Use Python and HuggingFace's transformers library to load the BERT model and tokenizer;

[0028] Process article content: Segment the article content and convert it into a format that the model can understand.

[0029] Get word vectors: Use the model to get the vector representation of each word.

[0030] Calculate vector similarity: Use cosine similarity or other similarity measurement methods to calculate the similarity between word vectors.

[0031] Identify synonyms: Identify synonyms based on the similarity threshold. A threshold can be set in advance, such as 0.8. Word pairs with similarity higher than this threshold can be considered synonyms.

[0032] Furthermore, using the small model to realize sentiment recognition of incoming data includes:

[0033] Chapter-level sentiment judgment: mainly based on massive data training models, combining manual training sets with machine-captured features, such as expressions, sentiment dictionaries, and contexts, using multi-task learning hybrid training and shared parameters;

[0034] Entity-level sentiment judgment: Based on the training of the BERT double-mask sequence model, this design is divided into three layers. The bottom layer is the input layer mask, which shields the analysis entity and improves the generalization ability of the model. The middle layer is the BERT model, which is a large-scale pre-trained model customized for the industry and continuously optimized and adjusted based on manual intervention. The top layer is the output layer mask, which shields the sentence components outside the analysis entity and forces the model to learn to analyze the entity sentiment.

[0035] Fine-grained sentiment judgment: Use intelligent triplet algorithms to identify feature words, identify some wrong characters and incorrect sentences, and extract the subject, features, and sentiment of the stored data for fine-grained analysis;

[0036] The specific steps include:

[0037] Load the BERT model and word segmenter;

[0038] Segment the incoming data;

[0039] Get emotional tendency;

[0040] Sentiment label mapping;

[0041] 0 is very negative; 1 is negative; 2 is neutral; 3 is positive; and 4 is very positive.

[0042] Further, in step S4, the large model includes:

[0043] Training data preparation: preliminary negative information is manually screened to select high-quality ones and added to the training queue, and input prompts and corresponding target outputs are written;

[0044] Prompt writing: Write instruction prompts to clearly tell the big model the tasks to be performed, provide examples for reference, help the big model understand the specific requirements of the tasks, add conditional judgments, and different judgments lead to different output results;

[0045] Model loading: submit data and prompt tasks to the big model and load the trained model weights;

[0046] Model fine-tuning: Fine-tune the model on the preliminary negative information, including: defining training parameters, setting the training batch size, learning rate, and number of training rounds. In each training round, input the preliminary negative information into the model, calculate the loss, and use the optimizer to update the model parameters. Evaluate the model performance on the validation set.

[0047] Model evaluation: Evaluate the performance of the model on an independent test set, including accuracy, recall, and F1 score;

[0048] Model optimization: Based on the evaluation results, adjust model parameters, suggest design or training strategies to further improve model performance.

[0049] Continuous improvement and iteration: Continuously iterate and optimize the model, use new data for training and evaluation to adapt to changing data and task requirements.

[0050] In addition, the present invention provides a negative public opinion discovery system based on an intelligent algorithm, comprising:

[0051] Data collection module: select social media platforms, collect information data, and process the information data to obtain processed data;

[0052] Data processing module: Check the processed data, store the data that meets the standards into the warehouse, and obtain the stored data;

[0053] Preliminary extraction and recognition module: Use a small model to extract and identify sentiment from incoming data to obtain preliminary negative information;

[0054] Secondary extraction and recognition module: Use a large model to extract preliminary negative information and identify sentiment to obtain negative public opinion.

[0055] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0056] This method uses a small model to extract and identify sentiment from the incoming data to obtain preliminary negative information; then uses a large model to extract preliminary negative information and identify sentiment to obtain negative public opinion. The small model filters out negative information, and the large model further finds information in a specified field from the negative information. This allows faster, more accurate, and more effective public opinion information extraction from massive social data. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flow chart of a method for discovering negative public opinion based on an intelligent algorithm as described in Example 1;

[0058] Figure 2 This is a block diagram of a negative public opinion discovery system based on an intelligent algorithm as described in Example 2;

[0059] Figure 3 This is a graph of data collection results described in Example 4;

[0060] Figure 4 The configuration entity words described in the fourth embodiment are used to extract the result graph of names, place names, brands, and products;

[0061] Figure 5 This is a flowchart of the operation of the small model described in Example 4;

[0062] Figure 6 This is a flow chart of the large model operation described in Example 4;

[0063] Figure 7 The result diagram of setting relevant parameters as described in Example 4;

[0064] Figure 8 This is a flow chart of combining a multi-modal large model as described in Example 4; DETAILED DESCRIPTION

[0065] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0066] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0067] Embodiment 1:

[0068] A method for discovering negative public opinion based on intelligent algorithms, comprising:

[0069] S1: Select a social media platform, collect information data, and process the information data to obtain processed data;

[0070] S2: Check the processed data, and process the data that meets the standards into the warehouse to obtain the warehoused data;

[0071] S3: Use a small model to extract and identify sentiment from incoming data to obtain preliminary negative information;

[0072] S4: Use the large model to extract preliminary negative information and identify sentiment to obtain negative public opinion.

[0073] This embodiment uses a small model to extract and identify sentiment from the incoming data to obtain preliminary negative information; then uses a large model to extract and identify sentiment from the preliminary negative information to obtain negative public opinion. The small model filters out negative information, and the large model further finds information in a specified field from the negative information. This enables faster, more accurate, and more effective extraction of public opinion information from massive social data.

[0074] Embodiment 2:

[0075] This embodiment further discloses on the basis of the first embodiment:

[0076] In step S4, it includes:

[0077] Through a large-parameter large model, preliminary negative information is extracted and sentiment is recognized to obtain negative public opinion results, and the text content of the negative public opinion results is directly output; through a multimodal large model, the image content of the negative public opinion results is secondary extracted and sentiment recognized and then output.

[0078] In step S1, collecting information data includes: using hyperlink analysis technology, combining multiple web page data format standards, intelligently parsing multiple web page types, and automatically extracting information data; the web page type includes at least one of the following: static html, dynamic JS, Ajax, ASP, PHP; the information data includes at least one of the following: the title, time, source, author, and text of the article;

[0079] In step S1, processing the information data includes:

[0080] Manual management of source labels: organize information data to form corresponding index hash codes, including media classification and labeling: classification by level and type includes: CCTV media, provincial media, local media; semantic classification by content includes: economic news, government news, financial news, people's livelihood news;

[0081] Fingerprint calculation of similar articles: By extracting the main content of information data and raising the subject words, the subject words are uniformly mapped to a common fingerprint template, and the accurate and raised article content fingerprint is obtained through encryption algorithm and hash mapping calculation;

[0082] Spam identification and filtering: Based on the existing spam filtering mechanism, useless information is automatically cleaned. Useless information includes at least one of the following: advertisements, irrelevant images, hyperlinks, and dynamic Flash;

[0083] Data deduplication and merging: Through the content duplication detection engine, based on the analysis of semantics, it can automatically determine whether there is duplicate information data, as well as the number, title, source and collection time of the duplicate information data, to achieve automatic deduplication of information data; and it can automatically identify and merge multiple web page contents with continuity to improve user reading efficiency.

[0084] In step S2, checking the processed data includes:

[0085] Data consistency verification: set a series of verification rules, strict verification and early warning, including time trend verification of processed data, early warning to find sites with obvious decline; threshold verification rules from the perspective of field rationality, find errors in processed data, and quickly warn people; at the same time, combined with manual inspection strategies, complete offline and online consistency comparison of processed data at key sites;

[0086] Data integrity verification: used to measure whether the processed data is lost or unavailable; standardization, used to measure whether the processed data is stored in a unified format; consistency, used to measure whether the values ​​of the processed data are conflicting in information meaning; accuracy, used to measure whether the processed data is correct or whether the processed data is expired; uniqueness, used to measure whether the processed data is duplicated;

[0087] Data validity verification: including non-empty check: when the field is required to be non-empty, the processed data of the field needs to be checked; primary key duplication: after the same type of processed data in multiple business systems is cleaned, it is necessary to check it when it is saved in a unified manner to ensure the uniqueness of the primary key; illegal code and illegal value cleaning: illegal code problems include illegal code and code inconsistency with data standards, illegal value problems include value errors, format errors, extra characters, and garbled codes, which need to be checked and corrected according to specific circumstances; data format check: the accuracy is measured by checking whether the format of the attribute value in the table is correct, such as time format, currency format, extra characters, and garbled codes; record number check: refers to the total number of data between related data in each system or the fluctuation check of the daily data volume in the data table; business constraint verification: in the process of implementing data processing, the responsible personnel must consider the data verification rules from the perspectives of business correctness, consistency, and effectiveness.

[0088] In step S3, using the small model to extract the incoming data includes:

[0089] The small model is an event transmission chain model, which sets different indicator parameters and key features according to the incubation period, germination period, outbreak period, maturity period and decline period of the event, and focuses on negative information in each period from different angles, especially the information, opinions and comments generated in the outbreak period as the key points to introduce into the sample data of the event transmission chain model;

[0090] During the training process of the small model, humans constantly intervene and mark relevant and characteristic data for the machine to learn by itself;

[0091] The small model automatically identifies concepts with the same meaning and filters out a large amount of irrelevant information, so that the preliminary negative information obtained has a higher accuracy rate.

[0092] Specifically include:

[0093] Load and prepare the model: Use Python and HuggingFace's transformers library to load the BERT model and tokenizer;

[0094] Process article content: Segment the article content and convert it into a format that the model can understand.

[0095] Get word vectors: Use the model to get the vector representation of each word.

[0096] Calculate vector similarity: Use cosine similarity or other similarity measurement methods to calculate the similarity between word vectors.

[0097] Identify synonyms: Identify synonyms based on the similarity threshold. A threshold can be set in advance, such as 0.8. Word pairs with similarity higher than this threshold can be considered synonyms.

[0098] Using a small model to perform sentiment recognition on incoming data includes:

[0099] Chapter-level sentiment judgment: mainly based on massive data training models, combining manual training sets with machine-captured features, such as expressions, sentiment dictionaries, and contexts, using multi-task learning hybrid training and shared parameters;

[0100] Entity-level sentiment judgment: Based on the training of BERT's double mask sequence model, this design is divided into three layers. The bottom layer is the input layer mask, which shields the analysis entity and improves the generalization ability of the model. The middle layer is the BERT model, which is a large-scale pre-trained model customized for the industry and continuously optimized and adjusted based on manual intervention. The top layer is the output layer mask, which shields the sentence components outside the analysis entity and forces the model to learn to analyze the entity sentiment.

[0101] Fine-grained sentiment judgment: Use intelligent triplet algorithms to identify feature words, identify some wrong characters and incorrect sentences, and extract the subject, features, and sentiment of the stored data for fine-grained analysis;

[0102] The specific steps include:

[0103] Load the BERT model and word segmenter;

[0104] Segment the incoming data;

[0105] Get emotional tendency;

[0106] Sentiment label mapping;

[0107] 0 is very negative; 1 is negative; 2 is neutral; 3 is positive; and 4 is very positive.

[0108] In step S4, the large model includes:

[0109] Training data preparation: preliminary negative information is manually screened to select high-quality ones and added to the training queue, and input prompts and corresponding target outputs are written;

[0110] Prompt writing: Write instruction prompts to clearly tell the big model the tasks to be performed, provide examples for reference, help the big model understand the specific requirements of the tasks, add conditional judgments, and different judgments lead to different output results;

[0111] Model loading: submit data and prompt tasks to the big model and load the trained model weights;

[0112] Model fine-tuning: Fine-tune the model on the preliminary negative information, including: defining training parameters, setting the training batch size, learning rate, and number of training rounds. In each training round, input the preliminary negative information into the model, calculate the loss, and use the optimizer to update the model parameters. Evaluate the model performance on the validation set.

[0113] Model evaluation: Evaluate the performance of the model on an independent test set, including accuracy, recall, and F1 score;

[0114] Model optimization: Based on the evaluation results, adjust model parameters, suggest design or training strategies to further improve model performance.

[0115] Continuous improvement and iteration: Continuously iterate and optimize the model, use new data for training and evaluation to adapt to changing data and task requirements.

[0116] This embodiment uses a small model to extract and identify sentiment from the incoming data to obtain preliminary negative information; then uses a large model to extract and identify sentiment from the preliminary negative information to obtain negative public opinion. The small model filters out negative information, and the large model further finds information in a specified field from the negative information. This enables faster, more accurate, and more effective extraction of public opinion information from massive social data.

[0117] Embodiment three:

[0118] This embodiment provides Figure 2 A negative public opinion discovery system based on intelligent algorithms is shown, comprising:

[0119] Data collection module: select social media platforms, collect information data, and process the information data to obtain processed data;

[0120] Data processing module: Check the processed data, store the data that meets the standards into the warehouse, and obtain the stored data;

[0121] Preliminary extraction and recognition module: Use a small model to extract and identify sentiment from incoming data to obtain preliminary negative information;

[0122] Secondary extraction and recognition module: Use a large model to extract preliminary negative information and identify sentiment to obtain negative public opinion.

[0123] This embodiment uses a small model to extract and identify sentiment from the incoming data to obtain preliminary negative information; then uses a large model to extract and identify sentiment from the preliminary negative information to obtain negative public opinion. The small model filters out negative information, and the large model further finds information in a specified field from the negative information. This enables faster, more accurate, and more effective extraction of public opinion information from massive social data.

[0124] Embodiment 4:

[0125] This embodiment provides a method for discovering negative public opinion based on an intelligent algorithm, including:

[0126] Step 1: Collect data related to a certain place or enterprise (taking Guangzhou as an example)

[0127] Select keywords such as place names, landmarks, roads, etc. related to Guangzhou, and collect data from different platforms at home and abroad. The collection platforms include Weibo, Douyin, Toutiao, Xiaohongshu, Bilibili, Kuaishou, news, forums, long videos, overseas, etc. Figure 3 shown.

[0128] After the collection is completed, the relevant information is displayed uniformly by category.

[0129] Step 2: rule denoising to remove pure forwarding, pure image, pure video information, and other invalid data.

[0130] Remove stop words: Remove some common stop words, such as "a", "an", "the", etc. These words have no practical meaning in language processing, but they will affect the effect of feature extraction and classification. Here we use the Natural Language Processing Library (NLTK) to remove stop words.

[0131] Remove missing values ​​and outliers: There may be fields with missing content, meaningless content, simply repeated content, and fragmented text in the text. We will remove this data.

[0132] Data preprocessing: such as word segmentation, stop word removal, stemming, etc.

[0133] Step 3, such as Figure 4 The entity words shown in the configuration are used to extract names, place names, brands, and products.

[0134] Step 4, such as Figure 5 The small inspection model shown is used for data denoising

[0135] After denoising by the small model, the original 1 million data are reduced to about 50,000.

[0136] Step 5, large model configuration preparation - write a suitable prompt

[0137] Step 6: The large model performs fine labeling on the denoised data

[0138] The specific process of large model denoising data is as follows Figure 6 As shown:

[0139] According to the process, you can Figure 7The relevant parameter configuration is shown below:

[0140] Step 7: Big model data performance evaluation

[0141] Performance evaluation mainly includes speed stress testing and data evaluation. The model inference process is monitored through special tools. The evaluation results are shown below.

[0142]

[0143]

[0144]

[0145] Step 8: Large model task training

[0146] The training is carried out according to the following three tasks:

[0147] Pure classification

[0148] Input: text+prompt

[0149] Output: Answer

[0150] Generate Task + CoT

[0151] Input: text+prompt

[0152] Output: Reason first, then answer

[0153] Generate Task + CoT

[0154] Input: text+prompt

[0155] Output: Answer first, then reason

[0156] The training results are shown below:

[0157]

[0158] Step 9: Combine the multimodal large model to output the final negative public opinion data

[0159] According to the characteristics of public opinion data, some negative content will be concentrated in non-text mode. It is necessary to combine multimodal large models with graphic information to increase the accuracy of negative data recognition. The specific process is as follows: Figure 8 As shown:

[0160] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for discovering negative public opinion based on intelligent algorithms, characterized in that: include: S1: Select a social media platform, collect information data, and process the information data to obtain processed data; S2: Check the processed data, and process the data that meets the standards into the warehouse to obtain the warehoused data; S3: Use a small model to extract and identify sentiment from incoming data to obtain preliminary negative information; S4: Use the large model to extract preliminary negative information and identify sentiment to obtain negative public opinion.

2. According to claim 1, a method for discovering negative public opinion based on intelligent algorithm is characterized in that: In step S4, it includes: Through a large-parameter large model, preliminary negative information is extracted and sentiment is recognized to obtain negative public opinion results, and the text content of the negative public opinion results is directly output; through a multimodal large model, the image content of the negative public opinion results is secondary extracted and sentiment recognized and then output.

3. According to claim 1, a method for discovering negative public opinion based on intelligent algorithm is characterized in that: In step S1, collecting information data includes: using hyperlink analysis technology, combining multiple web page data format standards, intelligently parsing multiple web page types, and automatically extracting information data; the web page type includes at least one of the following: static html, dynamic JS, Ajax, ASP, PHP; the information data includes at least one of the following: the title, time, source, author, and text of the article.

4. According to claim 1, a method for discovering negative public opinion based on intelligent algorithm is characterized in that: In step S1, processing the information data includes: Manual management of source labels: organize information data to form corresponding index hash codes, including media classification and labeling: classification by level and type includes: CCTV media, provincial media, local media; semantic classification by content includes: economic news, government news, financial news, people's livelihood news; Fingerprint calculation of similar articles: By extracting the main content of information data and raising the subject words, the subject words are uniformly mapped to a common fingerprint template, and the accurate and raised article content fingerprint is obtained through encryption algorithm and hash mapping calculation; Spam identification and filtering: Based on the existing spam filtering mechanism, useless information is automatically cleaned. Useless information includes at least one of the following: advertisements, irrelevant images, hyperlinks, and dynamic Flash; Data deduplication and merging: Through the content duplication detection engine, based on the analysis of semantics, it can automatically determine whether there is duplicate information data, as well as the number, title, source and collection time of the duplicate information data, to achieve automatic deduplication of information data; and it can automatically identify and merge multiple web page contents with continuity to improve user reading efficiency.

5. According to claim 1, a method for discovering negative public opinion based on intelligent algorithm is characterized in that: In step S2, checking the processed data includes: Data consistency verification: set a series of verification rules, strict verification and early warning, including time trend verification of processed data, early warning to find sites with obvious decline; threshold verification rules from the perspective of field rationality, find errors in processed data, and quickly warn people; at the same time, combined with manual inspection strategies, complete offline and online consistency comparison of processed data at key sites; Data integrity verification: used to measure whether the processed data is lost or unavailable; standardization, used to measure whether the processed data is stored in a unified format; consistency, used to measure whether the values ​​of the processed data are conflicting in information meaning; accuracy, used to measure whether the processed data is correct or whether the processed data is expired; uniqueness, used to measure whether the processed data is duplicated; Data validity verification: including non-empty check: when the field is required to be non-empty, the processed data of the field needs to be checked; primary key duplication: after the same type of processed data in multiple business systems is cleaned, it is necessary to check it when it is saved in a unified manner to ensure the uniqueness of the primary key; illegal code and illegal value cleaning: illegal code problems include illegal code and code inconsistency with data standards, illegal value problems include value errors, format errors, extra characters, and garbled codes, which need to be checked and corrected according to specific circumstances; data format check: the accuracy is measured by checking whether the format of the attribute value in the table is correct, such as time format, currency format, extra characters, and garbled codes; record number check: refers to the total number of data between related data in each system or the fluctuation check of the daily data volume in the data table; business constraint verification: in the process of implementing data processing, the responsible personnel must consider the data verification rules from the perspectives of business correctness, consistency, and effectiveness.

6. According to claim 1, a method for discovering negative public opinion based on intelligent algorithm is characterized in that: In step S3, using the small model to extract the incoming data includes: The small model is an event transmission chain model, which sets different indicator parameters and key features according to the incubation period, germination period, outbreak period, maturity period and decline period of the event, and focuses on negative information in each period from different angles, especially the information, opinions and comments generated in the outbreak period as the key points to introduce into the sample data of the event transmission chain model; During the training process of the small model, humans constantly intervene and mark relevant and characteristic data for the machine to learn by itself; The small model automatically identifies concepts with the same meaning and filters out a large amount of irrelevant information, so that the preliminary negative information obtained has a higher accuracy rate.

7. The method for discovering negative public opinion based on intelligent algorithm according to claim 6, characterized in that: Specifically include: Load and prepare the model: Use Python and HuggingFace's transformers library to load the BERT model and tokenizer; Process the article content: segment the article content and convert it into a format that the model can understand; Get word vectors: Use the model to get the vector representation of each word; Calculate vector similarity: Use cosine similarity or other similarity measurement methods to calculate the similarity between word vectors; Identify synonyms: Identify synonyms based on the similarity threshold. A threshold can be set in advance, such as 0.

8. Word pairs with similarity higher than this threshold can be considered synonyms.

8. According to claim 1, a method for discovering negative public opinion based on intelligent algorithm is characterized in that: Using a small model to perform sentiment recognition on incoming data includes: Chapter-level sentiment judgment: mainly based on massive data training models, combining manual training sets with machine-captured features, such as expressions, sentiment dictionaries, and contexts, using multi-task learning hybrid training and shared parameters; Entity-level sentiment judgment: Based on the training of BERT's double mask sequence model, this design is divided into three layers. The bottom layer is the input layer mask, which shields the analysis entity and improves the generalization ability of the model. The middle layer is the BERT model, which is a large-scale pre-trained model customized for the industry and continuously optimized and adjusted based on manual intervention. The top layer is the output layer mask, which shields the sentence components outside the analysis entity and forces the model to learn to analyze the entity sentiment. Fine-grained sentiment judgment: Use intelligent triplet algorithms to identify feature words, identify some wrong characters and incorrect sentences, and extract the subject, features, and sentiment of the stored data for fine-grained analysis; The specific steps include: Load the BERT model and tokenizer; Segment the incoming data; Get emotional tendency; Sentiment label mapping; 0 is very negative; 1 is negative; 2 is neutral; 3 is positive; and 4 is very positive.

9. According to claim 1, a method for discovering negative public opinion based on intelligent algorithm is characterized in that: In step S4, the large model includes: Training data preparation: preliminary negative information is manually screened to select high-quality ones and added to the training queue, and input prompts and corresponding target outputs are written; Prompt writing: Write instruction prompts to clearly tell the big model the tasks to be performed, provide examples for reference, help the big model understand the specific requirements of the tasks, add conditional judgments, and different judgments lead to different output results; Model loading: submit data and prompt tasks to the big model and load the trained model weights; Model fine-tuning: Fine-tune the model on the preliminary negative information, including: defining training parameters, setting the training batch size, learning rate, and number of training rounds. In each training round, input the preliminary negative information into the model, calculate the loss, and use the optimizer to update the model parameters. Evaluate the model performance on the validation set. Model evaluation: Evaluate the performance of the model on an independent test set, including accuracy, recall, and F1 score; Model optimization: Based on the evaluation results, adjust model parameters, suggest design or training strategies to further improve model performance; Continuous improvement and iteration: Continuously iterate and optimize the model, use new data for training and evaluation to adapt to changing data and task requirements.

10. A negative public opinion discovery system based on intelligent algorithms, characterized in that: include: Data collection module: select social media platforms, collect information data, and process the information data to obtain processed data; Data processing module: Check the processed data, store the data that meets the standards into the warehouse, and obtain the stored data; Preliminary extraction and recognition module: Use a small model to extract and identify sentiment from incoming data to obtain preliminary negative information; Secondary extraction and recognition module: Use a large model to extract preliminary negative information and identify sentiment to obtain negative public opinion.

Images