Opinion analysis system and method based on large model, electronic device and storage medium

Abstract

The application belongs to the technical field of natural language processing, and discloses an opinion analysis system and method based on a large model, an electronic device and a storage medium. The system comprises a data acquisition module, a data processing module, an opinion analysis module, an auxiliary decision-making module and a user interaction module. Social media opinion data is acquired in real time through a distributed crawler; then the data is cleaned, vectorized based on Word2Vec, and key words and themes are extracted using a TF-IDF and LDA model; then a CBA_emotion model that combines a convolutional neural network, a bidirectional long short-term memory network and an attention mechanism is used for accurate sentiment analysis; finally, the analysis results are input into a large language model to automatically generate opinion mitigation strategies. The system solves the problems of existing opinion platforms, such as non-real-time monitoring, single analysis dimension and lack of intelligent decision support, and provides an integrated and intelligent solution from data collection to strategy generation, significantly improving the efficiency and effectiveness of opinion monitoring and management.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing and information retrieval, in particular to an opinion analysis and strategy generation platform system based on large language models and deep learning, which simultaneously covers network crawler technology, sentiment recognition algorithm and topic modeling method. BACKGROUND

[0002] With the rapid development of the Internet, especially mobile Internet, social media represented by microblog has become an important platform for public expression of views and exchange of emotions. The comments, forwarding and other behaviors of massive users on social hot events quickly gather and ferment in the network space, forming a network public opinion with broad influence. Relevant reports show that in recent years, the number of network hot public opinion events in China has increased year by year, and the mobilization ability of network public opinion and the influence on the real society have significantly increased.

[0003] However, the current market opinion monitoring platform or system generally has the following technical bottlenecks: (1) lack of real-time performance: most platforms rely on timed or post-analysis, making it difficult to capture the golden intervention period at the initial stage of public opinion outbreak. (2) Single analysis dimension: traditional systems mostly stay at the keyword matching and word frequency statistics level, and cannot deeply mine the complex emotional tendency behind public comments. (3) Weak decision support capability: existing systems mostly only provide data display, lack of situation judgment and intelligent guidance suggestion output for public opinion events, and cannot form a closed-loop management from monitoring to guidance.

[0004] In addition, traditional opinion analysis methods, such as relying on sentiment dictionary for sentiment judgment, perform poorly in handling the flexibility, diversity and context semantic dependence of network language. In the face of the suddenness of network public opinion outbreak, the rapidity of its spread and the complexity of public emotions, there is an urgent need for an opinion analysis platform that can integrate deep learning, large language models, distributed crawlers and other cutting-edge technical means to realize real-time, efficient and intelligent analysis and decision assistance. SUMMARY

[0005] The present application aims to solve the problems raised in the background art, and provides an opinion analysis system and method based on large models, electronic devices and storage media, which can realize automatic real-time grabbing, deep structured processing, accurate sentiment recognition and multi-dimensional topic modeling of public opinion information on social media platforms, and finally generate intelligent public opinion guidance strategy text.

[0006] In a first aspect, the present application provides an opinion analysis system based on large models, comprising the following modules:

[0007] The data acquisition module is used to acquire raw public opinion data, including trending searches, articles, and comments, from social media platforms in real time based on distributed crawler technology.

[0008] The data processing module, connected to the data acquisition module, is used to perform data cleaning, text segmentation, and feature vectorization on the raw public opinion data to generate structured feature data.

[0009] The public opinion analysis module, connected to the data processing module, is used to perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data based on a preset deep learning model, so as to generate sentiment distribution data and topic keyword data.

[0010] The decision support module is connected to the public opinion analysis module. It has a built-in large language model to receive the sentiment distribution data and topic keyword data, and automatically generate a text-based public opinion guidance strategy in combination with preset event information.

[0011] The user interaction module is connected to the data acquisition module, the public opinion analysis module, and the decision support module, respectively, and is used to visually display the raw public opinion data, sentiment distribution data, topic keyword data, and the public opinion guidance strategy to the user.

[0012] As an optional implementation of the first aspect of this application, the data acquisition module is specifically built using the Scrapy framework. Its internal crawler tasks are executed in a preset hierarchical order, which is as follows: The hot search crawler submodule is executed to extract popularity, keywords, and type fields in JSON format by accessing the hot search interface of the social media platform; based on the obtained hot search keywords, the topic article crawler submodule is executed to crawl article content, repost count, comment count, and like count metadata through URL parsing and extraction using CSS or XPath; based on the obtained articles, the comment crawler submodule is executed to collect comment content, timestamps, and user information in pages by constructing asynchronous interface requests; and the data persistence submodule stores the above three types of data—hot search, articles, and comments—in a unified MySQL database using UTF-8MB4 encoding.

[0013] As an optional implementation of the first aspect of this application, the data processing module specifically includes: a cleaning submodule, which uses the pandas library to perform operations such as removing duplicates, filling missing values, and processing outliers on the original public opinion data; a word vector submodule, which uses the jieba tool to segment the cleaned text and remove stop words to obtain a word list; and based on the Word2Vec model, specifically using the CBOW mode, through neural network training, to generate high-dimensional word vectors containing contextual semantic information from the word list.

[0014] As an optional implementation of the first aspect of this application, the data processing module further includes: a keyword extraction submodule, used to calculate the weight value of each word in the word list using the TF-IDF algorithm, and select the top 10 words with the highest weight values ​​as keywords of the public opinion event; wherein the TF-IDF weight value is calculated by multiplying the term frequency (TF) and the inverse document frequency (IDF); and a topic modeling submodule, used to model the cleaned text using the LDA model, and generate the topic distribution of the document and the word distribution corresponding to the topic through Dirichlet distribution sampling, so as to realize the clustering of topic words and keywords.

[0015] As an optional implementation of the first aspect of this application, the sentiment analysis module employs a CBA_emotion model when performing sentiment polarity classification. The network structure of the CBA_emotion model includes, in sequence according to the data flow direction: an embedding layer, used to map the input word vectors to a low-dimensional dense space; a convolutional layer, connected to the embedding layer, used to extract local region features of the text; an attention mechanism layer, connected to the convolutional layer, used to weight the output of the convolutional layer to highlight the key information that has the greatest impact on sentiment classification, so as to obtain a global perspective; and a bidirectional long short-term memory network layer, connected to the attention mechanism layer, used to capture long-distance contextual dependencies of the text from both forward and backward directions.

[0016] As an optional implementation of the first aspect of this application, in the auxiliary decision-making module, the event information input to the large language model is a multimodal input, specifically including: a text description of the event, an analysis of the development trend of the event, LDA topic keywords generated by the public opinion analysis module, and statistical data on sentiment distribution; the public opinion guidance strategy output by the large language model includes strategy content text and an explanation of the strategy.

[0017] As an optional implementation of the first aspect of this application, the user interaction module is built on the Django framework and provides users with the following functional sub-modules: a registration and login sub-module, used to complete identity authentication through the registration and login page, and the isolation and security of data access are guaranteed by the backend account system; a public opinion data browsing sub-module, used to browse the real-time hot search list, Weibo article summaries and comment data in the homepage module, and supports searching and filtering by keywords or event topics; and a sentiment analysis display sub-module, used to view the sentiment analysis results in a visual manner on the public opinion analysis page, the visualization method including: displaying the sentiment category corresponding to each comment, a bar chart of the overall comment sentiment ratio, and the calculation results of the positive and negative ratio of public opinion.

[0018] Secondly, embodiments of this application provide a public opinion analysis method based on a large model, including:

[0019] Step 1: Based on distributed crawler technology, obtain raw public opinion data, including trending searches, articles, and comments, from social media platforms in real time;

[0020] Step 2: Perform data cleaning, text segmentation, and feature vectorization on the raw public opinion data to generate structured feature data;

[0021] Step 3: Based on the preset deep learning model, perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data to generate sentiment distribution data and topic keyword data;

[0022] Step 4: Using a large language model, receive the sentiment distribution data and topic keyword data, and combine them with preset event information to automatically generate a text-based public opinion guidance strategy;

[0023] Step 5: The user will be presented with a visual representation of the original public opinion data, sentiment distribution data, topic keyword data, and the public opinion guidance strategy.

[0024] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0025] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0026] Compared with existing technologies, the public opinion analysis system, method, electronic device, and storage medium based on large models provided by this invention have the following beneficial effects:

[0027] 1. Significantly improves the real-time performance and comprehensiveness of public opinion monitoring: This invention adopts a distributed Scrapy crawler framework and designs a hierarchical crawling strategy of "hot searches → articles → comments". This solution not only enables second-level response and real-time collection of public opinion information, effectively capturing the "golden intervention period" in the early stages of public opinion fermentation, but also ensures the data integrity and logical correlation from macro hot topics to micro comments, overcoming the problems of high data latency and incomplete information chains that are common in existing technologies.

[0028] 2. Significantly improved accuracy and depth of public opinion analysis: This invention abandons the traditional surface-level analysis method that relies on sentiment dictionaries. On the one hand, the proposed CBA_emotion model deeply integrates convolutional neural networks, bidirectional long short-term memory networks, and attention mechanisms, enabling it to accurately identify the true sentiment tendencies under complex linguistic phenomena such as online slang, irony, and metaphors, with an accuracy far exceeding traditional methods. On the other hand, by combining LDA topic modeling and TF-IDF keyword extraction, it can automatically mine the core issues and multi-dimensional focuses of public opinion from massive amounts of text, achieving a deep insight into the public opinion landscape.

[0029] 3. Achieving a leapfrog breakthrough from "data analysis" to "intelligent decision-making": The greatest innovation of this invention lies in the introduction of a large language model to construct an auxiliary decision-making module. It uses multi-dimensional analysis results such as sentiment distribution and topic keywords as input, and can automatically generate logical, targeted, and interpretable public opinion guidance strategies. This completely changes the passive situation of traditional public opinion systems that can only display data and cannot provide decision support, upgrading the system from a "monitoring tool" to an "intelligent advisor," providing users with proactive and efficient decision-making assistance, and forming an intelligent management closed loop of "monitoring-analysis-decision-making."

[0030] 4. An integrated and automated end-to-end solution has been built: This invention seamlessly integrates data acquisition, processing, analysis, and decision suggestion generation modules into a unified platform based on the Django framework. The entire process is highly automated, greatly reducing the cost of manual intervention and operational complexity. Users can complete complex public opinion analysis and management tasks through a user-friendly web interface, significantly improving work efficiency, lowering the barrier to entry, and demonstrating high practicality and deployability. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the structure of the public opinion analysis system based on a large model in an embodiment of the present invention;

[0032] Figure 2 This is a logic block diagram of the CBOW model used in the embodiments of the present invention;

[0033] Figure 3 This is a flowchart of the LDA topic model used in the embodiments of the present invention;

[0034] Figure 4 This is a diagram showing the overall network structure of the CBA_emotion model proposed in this embodiment of the invention.

[0035] Figure 5 This is a diagram of the long short-term memory neural network architecture used in the embodiments of the present invention;

[0036] Figure 6This is a flowchart of a public opinion analysis method based on a large model in an embodiment of the present invention.

[0037] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0040] Example 1

[0041] Please see Figure 1 This is a schematic diagram of the structure of a large-model-based public opinion analysis system provided in an embodiment of the present invention. The system's operation flow and the specific implementation methods of each module are as follows:

[0042] 1. Data acquisition module 100 is used to acquire raw public opinion data, including trending searches, articles and comments, from social media platforms in real time based on distributed crawler technology;

[0043] The data acquisition module 100 is responsible for collecting public opinion data in real time. In this embodiment, the data acquisition module 100 uses Python and the Scrapy framework to build a distributed crawler system. To ensure data integrity and logical consistency, the crawler tasks are executed strictly in the following hierarchical order:

[0044] The hot search crawler module 110 is executed to extract the popularity, keywords, and type fields in JSON format by accessing the hot search interface of social media platforms.

[0045] Based on the obtained trending search terms, the topic article crawler module 120 is executed to extract article content, repost count, comment count, and like count metadata by parsing the URL and using CSS or XPath extraction methods.

[0046] Based on the obtained articles, the comment crawler submodule 130 is executed to collect comment content, timestamps and user information in pages by constructing asynchronous interface requests;

[0047] The data persistence submodule 140 stores the aforementioned three types of data—hot searches, articles, and comments—into a MySQL database using UTF-8MB4 encoding.

[0048] 2. A data processing module 200, connected to the data acquisition module 100, is used to perform data cleaning, text segmentation, and feature vectorization on the raw public opinion data to generate structured feature data;

[0049] The data processing module 200 is responsible for preprocessing the raw data, laying the foundation for subsequent modeling and analysis.

[0050] Specifically, the data processing module 200 includes:

[0051] The cleaning submodule 210 uses the pandas library to perform operations on the original public opinion data, including removing duplicates, filling in missing values, and handling outliers.

[0052] The word vector submodule 220 uses the jieba tool to segment the cleaned text and remove stop words to obtain a word list; it is based on the Word2Vec model, specifically using the CBOW mode (e.g., Figure 2 As shown, through neural network training, the word list is used to generate high-dimensional word vectors containing contextual semantic information.

[0053] Furthermore, the data processing module 200 also specifically includes:

[0054] The keyword extraction submodule 230 is used to calculate the weight value of each word in the word list using the TF-IDF algorithm, and select the top 10 words with the highest weight values ​​as keywords for the public opinion event; wherein, the TF-IDF weight value is calculated by multiplying the term frequency (TF) and the inverse document frequency (IDF);

[0055] Term frequency (TF) refers to the list of words obtained after data cleaning. Specific topic words The frequency is calculated as follows:

[0056]

[0057] in This represents the total number of words in the article. This represents the total number of words in the document.

[0058] Inverse document frequency (IDF) represents the ratio of the number of texts to the number of times a specified word appears in a text set, calculated as follows:

[0059]

[0060] in, Represents the total number of documents in the corpus. The corpus contains words The total number of documents.

[0061] Therefore, the TF-IDF weight values ​​are calculated as follows:

[0062]

[0063] The topic modeling submodule 240 is used to model the cleaned text using an LDA model, such as... Figure 3 As shown, the topic distribution and corresponding word distribution of documents are generated by sampling using the Dirichlet distribution to achieve clustering of topic words and keywords. The method is as follows:

[0064] First, based on a fixed vocabulary list, each cleaned document... From a parameter Sampling from the Dirichlet distribution to generate documents Theme distribution This distribution determines the probability of each topic appearing in the document, as follows:

[0065]

[0066] in, Represents a probability vector. The parameter vector representing the Dirichlet distribution. Indicates that under given parameters Below, probability vector The probability density, It is a dimension, namely the total number of topics. Represents the parameter vector The One portion, Indicates in Under the component, the document belongs to the first... k The probability of each topic, express Dirichlet distribution, The normalization coefficient is called the Dirichlet distribution.

[0067]

[0068] in Let represent the integral. Since the integral of the Dirichlet distribution is 1, we can obtain:

[0069]

[0070] Then from the polynomial distribution of the topic Sampling to generate the first The topic of the word, then from the Dirichlet distribution Mid-sampling generates topics Corresponding word distribution Finally, from the polynomial distribution of words Mid-sampling ultimately generates words .

[0071] 3. The public opinion analysis module 300, connected to the data processing module 200, is used to perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data based on a preset deep learning model, so as to generate sentiment distribution data and topic keyword data;

[0072] The core of the public opinion analysis module 300 is sentiment polarity classification. This embodiment proposes a deep learning model called CBA_emotion. Figure 4 As shown, the public opinion analysis module 300 is implemented based on the Keras framework, and its network structure integrates the advantages of multiple components.

[0073] Specifically, the network structure of the CBA_emotion model includes the following components in order of data flow direction:

[0074] Embedding layer 310 is used to map the input word vectors to a low-dimensional dense space;

[0075] Convolutional layer 320, connected to the embedding layer 310, is used to extract local region features of the text;

[0076] Attention mechanism layer 330, connected to convolutional layer 320, is used to weight the output of convolutional layer 320 to highlight the key information that has the greatest impact on sentiment classification in order to obtain a global perspective;

[0077] Bidirectional Long Short-Term Memory network with 340 layers, such as Figure 5 As shown, it is connected to the attention mechanism layer 330 and is used to capture long-distance contextual dependencies of text from both forward and backward directions;

[0078] It should be noted that the CBA_emotion model is a machine learning model for sentiment analysis based on the Keras framework. It uses the Naive Bayes method to estimate the probability of text sentiment and uses the sentiment ratio to measure the polarity of public opinion. Internally, CBA_emotion mainly consists of an embedding layer 310, a convolutional layer 320, an attention mechanism layer 330, and a bidirectional long short-term memory network layer 340. The convolutional layer 320 assists the bidirectional long short-term memory network layer 340 in feature extraction, and the attention mechanism layer 330 improves the short-sightedness problem of the RNN-based bidirectional long short-term memory network layer 340, helping to obtain contextual information and gain a global perspective. Furthermore, the CBA_emotion model proposes a text classification algorithm based on a long short-term memory neural network, which includes the following steps:

[0079] Step 1: Input a variable-length sequence of comment text and set a dropout retention ratio to prevent overfitting;

[0080] Step 2: Read the pre-trained vocabulary and word vector matrix, and convert the input text sequence into a word vector representation;

[0081] Step 3: The word vector representations are sequentially fed into the embedding layer 310, convolutional layer 320, attention mechanism layer 330, and bidirectional long short-term memory network layer 340 for processing;

[0082] Step 4: Connect the output of the attention mechanism layer 330 to the fully connected layer 350, and perform a nonlinear transformation through an activation function;

[0083] Step 5: Obtain the output vector representing the predicted category, and output the positive, negative, or neutral sentiment prediction category according to the preset dictionary vector correspondence table.

[0084] 4. The decision support module 400 is connected to the public opinion analysis module 300. It has a built-in large language model to receive the sentiment distribution data and topic keyword data, and automatically generate a text-based public opinion guidance strategy in combination with preset event information.

[0085] The decision support module 400 is the core innovation of this invention. This module integrates a Large Language Model (LLM). When a public opinion event is analyzed, the system submits comprehensive information about the event, including its textual description, development trend, keywords extracted through LDA, and sentiment distribution data (such as the ratio of positive to negative comments) obtained from the CBA_emotion model, as input (Prompt) to the Large Language Model. Based on its powerful natural language understanding and generation capabilities, the Large Language Model automatically generates a suggested text for guiding public opinion in response to the current situation. This text may include suggestions on response guidelines, information disclosure guidelines, and directions for public opinion guidance, along with explanations of the strategies.

[0086] 5. User interaction module 500, which is connected to the data acquisition module 100, the public opinion analysis module 300 and the decision support module 400 respectively, is used to visually display the original public opinion data, sentiment distribution data, topic keyword data and the public opinion guidance strategy to the user.

[0087] User interaction module 500 serves as a bridge for communication between the system and the user. Built on the Django framework, it provides the following functional sub-modules to the user:

[0088] The registration and login submodule 510 allows users to complete identity authentication through the platform's registration and login pages. The front-end page includes user input fields and an authentication mechanism. The platform supports multiple user registrations and uses the account system within the Django framework for backend verification, ensuring the isolation and security of data access permissions.

[0089] The public opinion data browsing submodule 520 is accessible to users after logging in. The page displays: a real-time hot search list; retrieved Weibo articles (title, author, publication time, and content summary); comment data (number of likes, comment time, and comment content); and supports searching and filtering by specific keywords or event topics. All data comes from content obtained and stored by Scrapy crawlers.

[0090] The sentiment analysis display submodule 530, located on the "Public Opinion Analysis" page, allows users to view the platform's sentiment identification results for comment data. The displayed content includes: the sentiment category (positive, neutral, negative) for each comment; a bar chart showing the overall sentiment ratio of comments; and the calculation results of the positive / negative sentiment ratio.

[0091] The large-model-based public opinion analysis system in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application embodiment does not impose specific limitations.

[0092] The large-model-based public opinion analysis system in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0093] The public opinion analysis system based on a large model provided in this application can achieve... Figure 6 The various processes implemented by the large-model-based public opinion analysis method in the method embodiment will not be repeated here to avoid duplication.

[0094] Example 2

[0095] Please see Figure 6 The diagram shows a flowchart of a public opinion analysis method based on a large model proposed in the second embodiment of this application. The method includes the following steps:

[0096] Step 1: Based on distributed crawler technology, obtain raw public opinion data, including trending searches, articles, and comments, from social media platforms in real time;

[0097] Step 2: Perform data cleaning, text segmentation, and feature vectorization on the raw public opinion data to generate structured feature data;

[0098] Step 3: Based on the preset deep learning model, perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data to generate sentiment distribution data and topic keyword data;

[0099] Step 4: Using a large language model, receive the sentiment distribution data and topic keyword data, and combine them with preset event information to automatically generate a text-based public opinion guidance strategy;

[0100] Step 5: The user will be presented with a visual representation of the original public opinion data, sentiment distribution data, topic keyword data, and the public opinion guidance strategy.

[0101] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiments of the public opinion analysis method based on a large model and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0102] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiments of the public opinion analysis method based on a large model and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0103] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0104] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0106] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A public opinion analysis system based on a large model, characterized in that, include: The data acquisition module is used to acquire raw public opinion data, including trending searches, articles, and comments, from social media platforms in real time based on distributed crawler technology. The data processing module, connected to the data acquisition module, is used to perform data cleaning, text segmentation, and feature vectorization on the raw public opinion data to generate structured feature data. The public opinion analysis module, connected to the data processing module, is used to perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data based on a preset deep learning model, in order to generate sentiment distribution data and topic keyword data. When performing sentiment polarity classification, the public opinion analysis module uses a CBA_emotion model. The network structure of the CBA_emotion model, in the order of data flow, includes: Embedding layers are used to map the input word vectors to a low-dimensional dense space; A convolutional layer, connected to the embedding layer, is used to extract local region features of the text; An attention mechanism layer, connected to the convolutional layer, is used to weight the output of the convolutional layer, highlighting the key information that has the greatest impact on sentiment classification, in order to obtain a global perspective. A bidirectional long short-term memory network layer, connected to the attention mechanism layer, is used to capture long-distance contextual dependencies of text in both forward and backward directions; The CBA_emotion model proposes a text classification algorithm based on a long short-term memory neural network: It takes a variable-length comment text sequence as input and sets a dropout retention ratio to prevent overfitting; it reads a pre-trained vocabulary and word vector matrix, and converts the input text sequence into word vector representations; the word vector representations are then sequentially fed into the embedding layer, convolutional layer, attention mechanism layer, and bidirectional long short-term memory network layer for processing; the output of the attention mechanism layer is connected to a fully connected layer and non-linearly transformed using an activation function; finally, an output vector representing the predicted category is obtained, and based on a pre-defined dictionary vector correspondence table, the predicted sentiment category is output as positive, negative, or neutral. The decision support module is connected to the public opinion analysis module. It has a built-in large language model to receive the sentiment distribution data and topic keyword data, and automatically generate a text-based public opinion guidance strategy in combination with preset event information. The user interaction module is connected to the data acquisition module, the public opinion analysis module, and the decision support module, respectively, and is used to visually display the raw public opinion data, sentiment distribution data, topic keyword data, and the public opinion guidance strategy to the user.

2. The public opinion analysis system based on a large model according to claim 1, characterized in that, The data acquisition module is specifically built using the Scrapy framework, and its internal crawler tasks are executed in a preset hierarchical order, which is as follows: The hot search crawler module is executed to extract the popularity, keywords, and type fields in JSON format by accessing the hot search interface of social media platforms; Based on the obtained trending search terms, the topic article crawler module is executed to extract article content, repost count, comment count, and like count metadata by parsing the URL and using CSS or XPath extraction methods. Based on the retrieved articles, the comment crawler submodule is executed to collect comment content, timestamps, and user information in pages by constructing asynchronous interface requests. The data persistence submodule stores the three types of data mentioned above—trending searches, articles, and comments—into a MySQL database using UTF-8MB4 encoding.

3. The public opinion analysis system based on a large model according to claim 1, characterized in that, The data processing module specifically includes: The cleaning submodule uses the pandas library to perform operations on the raw public opinion data, such as removing duplicates, filling in missing values, and handling outliers. The word vector submodule uses the jieba tool to segment the cleaned text and remove stop words to obtain a word list. Based on the Word2Vec model, specifically using the CBOW mode, the word list is trained through neural network to generate high-dimensional word vectors containing contextual semantic information.

4. The public opinion analysis system based on a large model according to claim 3, characterized in that, The data processing module further includes: The keyword extraction submodule is used to calculate the weight value of each word in the word list using the TF-IDF algorithm, and select the top 10 words with the highest weight values ​​as keywords for the public opinion event; wherein, the weight value is calculated by multiplying the term frequency (TF) and the inverse document frequency (IDF); The topic modeling submodule is used to model the cleaned text using the LDA model. It generates the topic distribution of the document and the word distribution corresponding to the topic by sampling through the Dirichlet distribution, so as to achieve the clustering of topic words and keywords.

5. The public opinion analysis system based on a large model according to claim 1, characterized in that, In the auxiliary decision-making module, the event information input to the large language model is a multimodal input, specifically including: textual description of the event, analysis of the development trend of the event, LDA topic keywords generated by the public opinion analysis module, and statistical data on sentiment distribution; the public opinion guidance strategy output by the large language model includes strategy content text and explanation of the strategy.

6. The system according to claim 1, characterized in that, The user interaction module is built on the Django framework and provides users with the following functional sub-modules: The registration and login submodule is used to complete identity authentication through the registration and login pages, and the backend account system ensures the isolation and security of data access; The public opinion data browsing submodule is used to browse the real-time hot search list, Weibo article summaries and comment data in the homepage module, and supports searching and filtering by keywords or event topics; The sentiment analysis display submodule is used to view the sentiment analysis results in a visual manner on the public opinion analysis page. The visualization method includes: displaying the sentiment category corresponding to each comment, a bar chart of the overall sentiment ratio of comments, and the calculation results of the positive and negative ratio of public opinion.

7. A public opinion analysis method based on a large model, applied to a public opinion analysis system based on a large model as described in any one of claims 1-6, characterized in that, include: Step 1: Based on distributed crawler technology, obtain raw public opinion data, including trending searches, articles, and comments, from social media platforms in real time; Step 2: Perform data cleaning, text segmentation, and feature vectorization on the raw public opinion data to generate structured feature data; Step 3: Based on the preset deep learning model, perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data to generate sentiment distribution data and topic keyword data; Step 4: Using a large language model, receive the sentiment distribution data and topic keyword data, and combine them with preset event information to automatically generate a text-based public opinion guidance strategy; Step 5: Visually display the raw public opinion data, sentiment distribution data, topic keyword data, and public opinion guidance strategies to users.

8. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the public opinion analysis method based on a large model as described in claim 7.

9. A readable storage medium, characterized in that, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by the processor, they implement the steps of the public opinion analysis method based on a large model as described in claim 7.

Images