Public opinion analysis system and method based on large model, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of natural language processing, and discloses a public opinion analysis system and method based on a large model, electronic equipment and a storage medium, and the system comprises a data acquisition module, a data processing module, a public opinion analysis module, an auxiliary decision making module and a user interaction module. Social media public opinion data are acquired in real time through a distributed crawler; then cleaning the data, and based on Word2Vec vectorization, adopting TF-IDF (term frequency-inverse document frequency) and LDA (linear discriminant analysis) models to extract keywords and topics; a CBAemotion model fusing a convolutional neural network, a bidirectional long-short-term memory network and an attention mechanism is adopted to carry out precise sentiment analysis; and finally, inputting an analysis result into a large language model, and automatically generating a public opinion grooming strategy. According to the system, the problems that an existing public opinion platform is not real-time in monitoring, single in analysis dimension and lack of intelligent decision support are solved, an integrated and intelligent solution from data collection to strategy generation is provided, and the public opinion monitoring and management efficiency and effect are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing and information retrieval, in particular to an opinion analysis and strategy generation platform system based on large language models and deep learning, which simultaneously covers network crawler technology, sentiment recognition algorithm and topic modeling method. BACKGROUND

[0002] With the rapid development of the Internet, especially mobile Internet, social media represented by microblog has become an important platform for public expression of views and exchange of emotions. The comments, forwarding and other behaviors of massive users on social hot events quickly gather and ferment in the network space, forming a network public opinion with broad influence. Relevant reports show that in recent years, the number of network hot public opinion events in China has increased year by year, and the mobilization ability of network public opinion and the influence on the real society have significantly increased.

[0003] However, the current market opinion monitoring platform or system generally has the following technical bottlenecks: (1) lack of real-time performance: most platforms rely on timed or post-analysis, making it difficult to capture the golden intervention period at the initial stage of public opinion outbreak. (2) Single analysis dimension: traditional systems mostly stay at the level of keyword matching and word frequency statistics, and cannot deeply mine the complex emotional tendencies behind public comments. (3) Weak decision support capability: existing systems mostly only provide data display, lack of situation judgment and intelligent guidance suggestion output for public opinion events, and cannot form a closed-loop management from monitoring to guidance.

[0004] In addition, traditional opinion analysis methods, such as relying on sentiment dictionary for sentiment judgment, perform poorly in handling the flexibility, diversity and context semantic dependence of network language. In the face of the suddenness of network public opinion outbreak, the rapidity of its spread and the complexity of public emotions, there is an urgent need for an opinion analysis platform that can integrate deep learning, large language models, distributed crawlers and other cutting-edge technical means to realize real-time, efficient and intelligent analysis and decision assistance. SUMMARY

[0005] The present application aims to solve the problems raised in the background art, and provides an opinion analysis system, method, electronic device and storage medium based on large models, which can realize automatic real-time grabbing, deep structured processing, accurate sentiment recognition and multi-dimensional topic modeling of public opinion information on social media platforms, and finally generate intelligent public opinion guidance strategy text.

[0006] In a first aspect, the present application provides an opinion analysis system based on large models, comprising the following modules: A data acquisition module is configured to acquire opinion raw data including hot searches, articles and comments from social media platforms in real time based on distributed crawler technology; The data processing module is connected with the data acquisition module, and is used for data cleaning, text segmentation and feature vectorization on the public opinion original data to generate structured feature data. The public opinion analysis module is connected with the data processing module, and is used for multi-dimensional sentiment polarity classification and topic modeling on the structured feature data based on a preset deep learning model to generate sentiment distribution data and topic keyword data. The auxiliary decision module is connected with the public opinion analysis module, and is internally provided with a large language model, and is used for receiving the sentiment distribution data and the topic keyword data and automatically generating a public opinion dissuasion strategy in a text form in combination with preset event information. The user interaction module is connected with the data acquisition module, the public opinion analysis module and the auxiliary decision module respectively, and is used for visualizing and displaying the public opinion original data, the sentiment distribution data, the topic keyword data and the public opinion dissuasion strategy to a user.

[0007] As an optional implementation manner of the first aspect of the application, the data acquisition module is specifically built by using a Scrapy framework, and internal crawler tasks are executed in a preset hierarchical order, which is: executing a hot search crawler sub-module, accessing a hot search interface of a social media platform to extract hotness, keyword and type fields in a JSON format; based on the obtained hot search keywords, executing a topic article crawler sub-module to extract article content, forwarding quantity, comment quantity and like quantity meta information by URL analysis and using a CSS or Xpath extraction method; based on the obtained articles, executing a comment crawler sub-module to collect comment content, timestamp and user information by page through construction of an asynchronous interface request; and through a data persistence sub-module, the above-mentioned hot search, article and comment data are uniformly stored in a MySQL database in a UTF-8MB4 encoding format.

[0008] As an optional implementation manner of the first aspect of the application, the data processing module specifically includes: a cleaning sub-module that uses a pandas library to perform operations of removing duplicates, filling missing values and processing abnormal values on the public opinion original data; a word vector sub-module that uses a jieba tool to perform text segmentation and stop word removal on the cleaned text to obtain a word list; based on a Word2Vec model and specifically using a CBOW mode, the word list is generated into a high-dimensional word vector containing context semantic information through neural network training.

[0009] As an optional implementation of the first aspect of the application, the data processing module further comprises: a keyword extraction submodule, configured to calculate a weight value of each word in the word list by using a TF-IDF algorithm, and select the top 10 words in the weight value as keywords of the public opinion event; wherein the TF-IDF weight value is calculated by the product of term frequency TF and inverse document frequency IDF; and a topic modeling submodule, configured to model the cleaned text by using an LDA model, and generate a topic distribution of the document and a word distribution corresponding to the topic by Dirichlet distribution sampling, so as to realize clustering of topic words and keywords.

[0010] As an optional implementation of the first aspect of the application, when the public opinion analysis module performs sentiment polarity classification, a CBA emotion model is used; the network structure of the CBA emotion model comprises, in sequence in the direction of data flow: an embedding layer, configured to map an input word vector to a low-dimensional dense space; a convolution layer, connected with the embedding layer, configured to extract local region features of the text; an attention mechanism layer, connected with the convolution layer, configured to weight the output of the convolution layer, highlight the key information that has the greatest impact on sentiment classification, and obtain a global view; and a bidirectional long short-term memory network layer, connected with the attention mechanism layer, configured to capture long-distance context dependency of the text from two directions of forward and backward.

[0011] As an optional implementation of the first aspect of the application, in the auxiliary decision-making module, the event information input to the large language model is a multi-modal input, specifically comprising: a text description of the event, an analysis of the development trend of the event, LDA topic keywords generated by the public opinion analysis module, and statistical data of sentiment distribution; and the public opinion mitigation strategy output by the large language model comprises a strategy content text and an explanation of the strategy.

[0012] As an optional implementation of the first aspect of the application, the user interaction module is built based on a Django framework, and provides the following function submodules for the user: a registration and login submodule, configured to complete identity authentication through a registration and login page, and ensure data access isolation and security by a background account system; a public opinion data browsing submodule, configured to browse a real-time hot search list, a microblog article abstract and comment data in a home page module, and support keyword or event topic-based retrieval and filtering; and a sentiment analysis display submodule, configured to view sentiment analysis results in a visual manner in a public opinion analysis page, the visual manner comprising: displaying a sentiment classification corresponding to each comment, a column chart of an overall comment sentiment proportion, and a calculation result of a positive-to-negative ratio of public opinion.

[0013] In a second aspect, the embodiments of the application provide a public opinion analysis method based on a large model, comprising: Step one: based on distributed crawler technology, real-time acquisition of public opinion original data including hot search, articles and comments from social media platforms; Step two: data cleaning, text segmentation and feature vectorization are performed on the public opinion original data to generate structured feature data; Step three: based on a pre-set deep learning model, multi-dimensional sentiment polarity classification and topic modeling are performed on the structured feature data to generate sentiment distribution data and topic keyword data; Step four: using a large language model, receiving the sentiment distribution data and topic keyword data, and combining pre-set event information, automatically generating public opinion mitigation strategies in text form; Step five: visualizing the public opinion original data, sentiment distribution data, topic keyword data and public opinion mitigation strategies to the user.

[0014] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the method of the first aspect.

[0015] In a fourth aspect, a readable storage medium is provided, which stores a program or instructions, and the program or instructions, when executed by a processor, implement the steps of the method of the first aspect.

[0016] Compared with the prior art, the public opinion analysis system, method, electronic device and storage medium based on a large model provided by the present application have the following beneficial effects: 1. Significantly improve the real-time and comprehensiveness of public opinion monitoring: the present application adopts a distributed Scrapy crawler framework and designs a hierarchical order grabbing strategy of "hot search -> article -> comment". This scheme not only realizes second-level response and real-time collection of public opinion information, effectively capturing the "golden intervention period" in the early stage of public opinion fermentation, but also ensures the data integrity and logical correlation from macro hot spots to micro comments, overcoming the problems of high data delay and incomplete information chain in the prior art.

[0017] 2. Greatly improve the accuracy and depth of public opinion analysis: the present application abandons the traditional surface analysis method relying on sentiment dictionary. On the one hand, the proposed CBA_emotion model deeply integrates convolutional neural network, bidirectional long short-term memory network and attention mechanism, which can accurately identify the real sentiment tendency under complex language phenomena such as network language, irony and metaphor, with accuracy far exceeding traditional methods. On the other hand, combined with LDA topic model and TF-IDF keyword extraction, it can automatically mine the core issues and multi-dimensional focus of public opinion from massive texts, realizing deep insight into public opinion situation.

[0018] 3. Achieved a leap from “data analysis” to “intelligent decision-making”: The biggest innovation of the present application is to introduce a large language model to build an auxiliary decision-making module. It takes multi-dimensional analysis results such as sentiment distribution and theme keywords as input, and can automatically generate public opinion guiding strategies with logic, pertinence and interpretability. This completely changes the passive situation of traditional public opinion systems that can only display data and cannot provide decision support, upgrades the system from a “monitoring tool” to an “intelligent adviser”, provides active and efficient decision support for users, and forms an intelligent management closed loop of “monitoring-analysis-decision”.

[0019] 4. Built an integrated and automated full-process solution: The present application seamlessly integrates data acquisition, processing, analysis and decision suggestion generation and other functional modules in a unified platform based on the Django framework. The whole process is highly automated, greatly reducing the cost and complexity of manual intervention. Users can complete complex public opinion analysis and management tasks through a friendly Web interface, significantly improving work efficiency and reducing the use threshold, and has high practicality and deployability. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 FIG. 1 is a structural schematic diagram of the public opinion analysis system based on a large model in an embodiment of the present application; Figure 2 FIG. 2 is a logic diagram of the CBOW model used in an embodiment of the present application; Figure 3 FIG. 3 is a flowchart of the LDA topic model used in an embodiment of the present application; Figure 4 FIG. 4 is a network overall structure diagram of the CBA_emotion model proposed in an embodiment of the present application; Figure 5 FIG. 5 is an architecture diagram of the long short-term memory neural network used in an embodiment of the present application; Figure 6 FIG. 6 is a flowchart of the public opinion analysis method based on a large model in an embodiment of the present application.

[0021] The following specific embodiments will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in a "or" relationship. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0024] Embodiment 1 Please refer to Figure 1 The structure diagram of the public opinion analysis system based on a large model provided by the embodiment of the present application. The operation flow and specific implementation mode of each module of the system are as follows: 1. The data acquisition module 100 is used to acquire public opinion raw data including hot search, article and comment from social media platforms in real time based on distributed crawler technology; The data acquisition module 100 is responsible for real-time collection of public opinion data. In this embodiment, the data acquisition module 100 uses Python language and Scrapy framework to build a distributed crawler system. In order to ensure the integrity and logical correlation of the data, the crawler task is strictly executed according to the following hierarchical order, specifically: The hot search crawler submodule 110 is executed to extract the hotness, keyword and type fields in JSON format by accessing the hot search interface of the social media platform; Based on the obtained hot search keywords, the theme article crawler submodule 120 is executed to extract article content, forwarding quantity, comment quantity and like quantity meta information by URL analysis and using CSS or Xpath extraction method; Based on the obtained article, the comment crawler submodule 130 is executed to collect comment content, timestamp and user information by constructing an asynchronous interface request and paging; The above three types of data of hot search, article and comment are uniformly stored in the MySQL database using UTF-8MB4 encoding format through the data persistence submodule 140.

[0025] 2. The data processing module 200 is connected with the data acquisition module 100 and is used to perform data cleaning, text segmentation and feature vectorization on the public opinion raw data to generate structured feature data; The data processing module 200 is responsible for preprocessing the original data to lay a foundation for subsequent modeling analysis.

[0026] Specifically, the data processing module 200 specifically includes: The cleaning submodule 210 uses the pandas library to perform the operations of removing duplicates, filling missing values, and processing abnormal values on the public opinion raw data; The word vector submodule 220 uses the jieba tool to perform word segmentation and remove stop words on the cleaned text to obtain a word list; based on the Word2Vec model and specifically using the CBOW mode (as shown in Figure 2 ), the word list is generated into a high-dimensional word vector containing context semantic information through neural network training.

[0027] Further, the data processing module 200 further specifically includes: The keyword extraction submodule 230 is configured to calculate the weight value of each word in the word list using the TF-IDF algorithm, and select the top 10 words with the highest weight values as the keywords of the public opinion event; wherein the TF-IDF weight value is calculated by the product of term frequency TF and inverse document frequency IDF. The term frequency (TF) refers to the frequency of a specific topic word in the word list obtained after data cleaning, and is calculated as follows: wherein represents the total number of words in the article, represents the total number of words in the document.

[0028] The inverse document frequency (IDF) represents the ratio of the number of texts and the number of times a specified word appears on the text set, and is calculated as follows: wherein, represents the total number of all documents in the corpus, represents the total number of documents in the corpus containing the word .

[0029] Therefore, the TF-IDF weight value is calculated as follows: The topic modeling submodule 240 is configured to model the cleaned text using the LDA model, as shown in Figure 3 , the topic distribution of the document and the word distribution corresponding to the topic are generated by Dirichlet distribution sampling to realize the clustering of topic words and keywords, in the following manner: First, based on a fixed vocabulary, the topic distribution of each cleaned document is sampled from a Dirichlet distribution with a parameter to generate the topic distribution of the document . ​​This distribution determines the probability of each topic appearing in a document, which is given by: where, denotes the probability vector, denotes the parameter vector of the Dirichlet distribution, denotes the probability density of the probability vector given the parameter is the dimension, i.e., the total number of topics, denotes the th component of the parameter vector denotes the probability that the document belongs to the th topic given the k th component, denotes the Dirichlet distribution of The normalization coefficient of the Dirichlet distribution is called the Dirichlet distribution: where denotes the integral. Because the integral of the Dirichlet distribution is 1, we have: Then, the topic of the th word is generated by sampling from the multinomial distribution of the topic, followed by sampling the topic from the Dirichlet distribution to generate the word distribution corresponding to the topic, and finally sampling the word from the multinomial distribution of the word to generate the word.

[0030] 3. The public opinion analysis module 300 is connected with the data processing module 200, and is used for performing multi-dimensional sentiment polarity classification and topic modeling on the structured feature data based on a preset deep learning model, to generate sentiment distribution data and topic keyword data. The core of the public opinion analysis module 300 is sentiment polarity classification. The embodiment proposes a deep learning model named CBA emotion. As shown in Figure 4 , the public opinion analysis module 300 is implemented based on the keras framework, and the network structure thereof integrates the advantages of multiple components.

[0031] Specifically, the network structure of the CBA emotion model includes, in sequence in the data flow direction: An embedding layer 310 is used to map the input word vector to a low-dimensional dense space. ​​​The convolution layer 320 is connected to the embedding layer 310 and is used to extract local area features of the text; An attention mechanism layer 330 is connected to the convolutional layer 320 and is used to weight the output of the convolutional layer 320 to highlight the key information that has the greatest impact on sentiment classification, so as to obtain a global perspective; Bidirectional long short-term memory network layer 340, such as Figure 5 As shown, it is connected to the attention mechanism layer 330 to capture the long-range contextual dependencies of the text from both forward and backward directions; It should be noted that the CBA_emotion model is a machine learning model for sentiment analysis based on the Keras framework. It uses the Naive Bayes method to estimate the probability of text sentiment and uses the sentiment ratio to measure the polarity of public opinion. CBA_emotion mainly includes an embedding layer 310, a convolutional layer 320, an attention mechanism layer 330, and a bidirectional long short-term memory network layer 340. The convolutional layer 320 can help the bidirectional long short-term memory network layer 340 to extract features, and the attention mechanism layer 330 can improve the short-sightedness problem of the RNN-based bidirectional long short-term memory network layer 340, helping to obtain contextual information and gain a global perspective. In addition, the CBA_emotion model proposes a text classification algorithm based on a long short-term memory neural network. The algorithm includes the following steps: Step 1: Input a variable-length comment text sequence and set the dropout retention ratio to prevent overfitting; Step 2: Read the pre-trained vocabulary and word vector matrix, and convert the input text sequence into a word vector representation; Step 3: Send the word vector representation to the embedding layer 310, convolution layer 320, attention mechanism layer 330 and bidirectional long short-term memory network layer 340 for processing; Step 4: Connect the output of the attention mechanism layer 330 to the fully connected layer 350 and perform nonlinear transformation through the activation function; Step 5: Obtain the output vector representing the predicted category, and output the positive, negative, or neutral sentiment prediction category based on the preset dictionary vector correspondence table.

[0032] 4. A decision support module 400, connected to the public opinion analysis module 300, has a built-in large language model for receiving the sentiment distribution data and topic keyword data, and automatically generating a text-based public opinion guidance strategy based on preset event information; The auxiliary decision-making module 400 is the core innovation of the present application. The auxiliary decision-making module 400 integrates a large language model (LLM). When an opinion event is analyzed, the system will submit the comprehensive information of the event, including the text description of the event, the development trend, the theme keywords extracted by LDA, and the sentiment distribution data (such as the ratio of positive / negative comments) obtained by the CBA emotion model analysis, as input (Prompt) to the large language model. Based on its powerful natural language understanding and generation capabilities, the large language model will automatically generate a text of a guiding strategy suggestion for the current public opinion situation, which may include response range suggestions, information disclosure guidelines, public opinion guidance directions, etc., and is accompanied by an explanation of the strategy.

[0033] 5. The user interaction module 500 is connected with the data acquisition module 100, the public opinion analysis module 300 and the auxiliary decision-making module 400 respectively, and is used for visualizing and displaying the public opinion original data, the sentiment distribution data, the theme keyword data and the public opinion guiding strategy to the user.

[0034] The user interaction module 500 is a bridge for the system to communicate with the user. The user interaction module 500 is built based on the Django framework and provides the following functional submodules for the user: The registration and login submodule 510 is used for identity authentication. The front-end page includes a user input box and an identity verification mechanism. The platform supports multi-user registration and uses the account system in the Django framework for background verification to ensure the isolation and security of data access permissions.

[0035] The public opinion data browsing submodule 520 is used for accessing the platform home page module after the user logs in. The page displays the following: real-time hot search list; microblog articles (title, author, publication time, content abstract) obtained; comment data (like number, comment time, comment content); support for searching and filtering specific keywords or event themes; all data are obtained by the Scrapy crawler and stored in the database.

[0036] The sentiment analysis display submodule 530 is used for viewing the sentiment recognition results of the comment data by the platform in the “public opinion analysis” page. The display content includes: the sentiment classification (positive, neutral, negative) corresponding to each comment; the overall comment sentiment proportion bar chart; the public opinion positive / negative ratio calculation result.

[0037] The big model-based public opinion analysis system in the embodiments of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an Ultra-mobile Personal Computer (UMPC), a netbook, or a Personal Digital Assistant (PDA), etc., and the non-mobile electronic device can be a server, a Network Attached Storage (NAS), a Personal Computer (PC), etc., and the embodiments of the present application are not limited specifically.

[0038] The big model-based public opinion analysis system in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an IOS operating system, or other possible operating systems, and the embodiments of the present application are not limited specifically.

[0039] The big model-based public opinion analysis system provided in the embodiments of the present application can realize Figure 6 The processes realized by the big model-based public opinion analysis method in the method embodiment are not repeated here to avoid repetition.

[0040] Embodiment 2 Please refer to Figure 6 , which is a flowchart of a big model-based public opinion analysis method proposed in the second embodiment of the present application. The method includes the following steps: Step one: based on a distributed crawler technology, real-time acquisition of public opinion original data including hot searches, articles, and comments from social media platforms; Step two: data cleaning, text segmentation, and feature vectorization are performed on the public opinion original data to generate structured feature data; Step three: based on a preset deep learning model, multi-dimensional sentiment polarity classification and topic modeling are performed on the structured feature data to generate sentiment distribution data and topic keyword data; Step four: using a large language model, the sentiment distribution data and the topic keyword data are received, and combined with preset event information, a text-form public opinion mitigation strategy is automatically generated; Step five: the public opinion original data, the sentiment distribution data, the topic keyword data, and the public opinion mitigation strategy are visually displayed to a user.

[0041] Optionally, the embodiment of the present application further provides an electronic device, comprising a processor, a memory, a program or instructions stored in the memory and executable on the processor, which, when executed by the processor, implement each process of the above-mentioned method for public opinion analysis based on a large model and achieve the same technical effects. To avoid repetition, details are not described here.

[0042] The embodiment of the present application further provides a readable storage medium, which stores a program or instructions, which, when executed by a processor, implement each process of the above-mentioned method for public opinion analysis based on a large model and achieve the same technical effects. To avoid repetition, details are not described here.

[0043] The processor is the processor of the electronic device in the above-mentioned embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0044] It should be noted that in this paper, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to the order of performing the functions shown or discussed, but also includes performing the functions in a substantially simultaneous manner or in the opposite order according to the functions involved, for example, the described method can be performed in an order different from the described order, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0045] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, also can be through hardware, but many cases the former is the better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of the contribution to the prior art can be embodied in the form of software products, the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), including a number of instructions to make a terminal (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0046] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the above-mentioned specific embodiments, the above-mentioned specific embodiments are only illustrative, but not limited, those skilled in the art can make many forms without departing from the purpose of the present application and the scope of the claims under the inspiration of the present application, all belong to the protection of the present application.

Claims

1. A public opinion analysis system based on a large model, characterized by: include: The data acquisition module is used to obtain the original public opinion data including hot searches, articles and comments from social media platforms in real time based on distributed crawler technology; A data processing module, connected to the data acquisition module, is used to perform data cleaning, text segmentation and feature vectorization on the original public opinion data to generate structured feature data; A public opinion analysis module, connected to the data processing module, is used to perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data based on a preset deep learning model to generate sentiment distribution data and topic keyword data; The auxiliary decision-making module is connected to the public opinion analysis module and has a built-in large language model for receiving the sentiment distribution data and topic keyword data, and automatically generating a text-based public opinion guidance strategy based on preset event information; The user interaction module is connected to the data acquisition module, the public opinion analysis module and the auxiliary decision module respectively, and is used to visually display the public opinion original data, sentiment distribution data, topic keyword data and the public opinion guidance strategy to the user.

2. The public opinion analysis system based on a large model according to claim 1 is characterized in that: The data acquisition module is specifically constructed using the Scrapy framework, and its internal crawler tasks are executed in a preset hierarchical order, which is: Execute the trending search crawler submodule to access the trending search interface of the social media platform and extract the popularity, term, and type fields in JSON format; Based on the obtained hot search terms, the topic article crawler module is executed to crawl the article content, forwarding volume, comment volume, and like volume meta information through URL parsing and CSS or Xpath extraction; Based on the obtained articles, the comment crawler module is executed to collect comment content, timestamps and user information in pages by constructing asynchronous interface requests; Through the data persistence sub-module, the above three types of data, hot searches, articles, and comments, are uniformly stored in a MySQL database using the UTF-8MB4 encoding format.

3. The public opinion analysis system based on a large model according to claim 1 is characterized in that: The data processing module specifically includes: The cleaning submodule uses the pandas library to remove duplicates, fill missing values, and process outliers on the original public opinion data; The word vector submodule uses the jieba tool to segment the cleaned text and remove stop words to obtain a word list; based on the Word2Vec model, and specifically adopting the CBOW mode, through neural network training, the word list is converted into a high-dimensional word vector containing contextual semantic information.

4. The public opinion analysis system based on a large model according to claim 3 is characterized in that: The data processing module further specifically includes: A keyword extraction submodule is used to calculate the weight value of each word in the word list using the TF-IDF algorithm, and select the top 10 words with the highest weight value as the keywords of the public opinion event; wherein the TF-IDF weight value is calculated by multiplying the term frequency TF and the inverse document frequency IDF; The topic modeling submodule is used to model the cleaned text using the LDA model, and to generate the topic distribution of the document and the word distribution corresponding to the topic through Dirichlet distribution sampling to achieve clustering of topic words and keywords.

5. The public opinion analysis system based on a large model according to claim 1 is characterized in that: When the public opinion analysis module performs sentiment polarity classification, a CBA_emotion model is used; the network structure of the CBA_emotion model includes the following in the data flow direction: Embedding layer, used to map the input word vector to a low-dimensional dense space; A convolutional layer, connected to the embedding layer, for extracting local area features of the text; An attention mechanism layer, connected to the convolutional layer, is used to weight the output of the convolutional layer to highlight the key information that has the greatest impact on sentiment classification, so as to obtain a global perspective; The bidirectional long short-term memory network layer is connected to the attention mechanism layer to capture the long-range contextual dependencies of the text in both forward and backward directions.

6. The public opinion analysis system based on a large model according to claim 1 is characterized in that: In the auxiliary decision-making module, the event information input into the large language model is multimodal input, specifically including: a text description of the event, an analysis of the development trend of the event, LDA topic keywords generated by the public opinion analysis module, and statistical data on sentiment distribution; the public opinion guidance strategy output by the large language model includes the strategy content text and an explanation of the strategy.

7. The system according to claim 1, wherein: The user interaction module is built based on the Django framework and provides users with the following functional submodules: The registration and login submodule is used to complete identity authentication through the registration and login pages, and the background account system ensures the isolation and security of data access; The public opinion data browsing submodule is used to browse real-time hot search lists, Weibo article summaries and comment data on the homepage module, and supports searching and filtering by keywords or event topics; The sentiment analysis display submodule is used to view the sentiment analysis results in a visual manner on the public opinion analysis page. The visualization method includes: displaying the sentiment classification corresponding to each comment, a bar chart of the overall comment sentiment ratio, and the calculation results of the positive and negative ratio of public opinion.

8. A public opinion analysis method based on a large model, characterized in that: include: Step 1: Using distributed crawler technology, obtain real-time public opinion data including trending searches, articles, and comments from social media platforms; Step 2: Perform data cleaning, text segmentation, and feature vectorization on the original public opinion data to generate structured feature data; Step 3: Based on a preset deep learning model, perform multi-dimensional sentiment polarity classification and topic modeling on the structured feature data to generate sentiment distribution data and topic keyword data; Step 4: Using a large language model, receiving the sentiment distribution data and topic keyword data, and combining it with preset event information, automatically generating a text-based public opinion guidance strategy; Step 5: Visually display the public opinion original data, sentiment distribution data, topic keyword data and the public opinion guidance strategy to the user.

9. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of a public opinion analysis method based on a large model as described in claim 8 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the public opinion analysis method based on a large model as claimed in claim 8 are implemented.

Images