Public opinion data analysis method, device, electronic device and readable storage medium

Abstract

The present application relates to the field of data processing technology, and provides a method, device, electronic device and readable storage medium for analyzing public opinion data. The method includes: obtaining public opinion data, and performing streaming clustering on the public opinion data to obtain the cluster to which the public opinion data belongs; obtaining the attention index value of the cluster based on the attribute feature data of each target text of the cluster; performing public opinion analysis on the public opinion events corresponding to the cluster based on the large language model to obtain the public opinion summary corresponding to the cluster; judging and generating the event category corresponding to the cluster based on the large language model; when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster are sent to the terminal device. The method solves the problems of low generalization and inaccurate analysis of public opinion data in the prior art, and improves the efficiency and accuracy of public opinion monitoring and management.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a public opinion data analysis method, device, electronic device and readable storage medium. Background Art

[0002] By collecting and analyzing information from multiple channels such as online public opinion, media reports, and social media, we can understand the public's views and attitudes on various public opinion issues in real time, which helps to more accurately analyze various current events that trigger social public opinion and prepare solutions to the events in a timely manner. The public opinion analysis in existing technologies is inefficient and has a narrow coverage. The analysis of various public opinion data requires targeted customization, and the generalization and accuracy need to be improved. It is slow to update and costly, and cannot meet actual usage needs. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a public opinion data analysis method, device, electronic device and readable storage medium to solve the problems in the prior art that public opinion data analysis has low generalization and inaccurate analysis.

[0004] According to a first aspect of an embodiment of the present application, a method for analyzing public opinion data is provided, comprising: obtaining public opinion data, and formatting and extracting the public opinion data to obtain an initial text; vectorizing the initial text to obtain a feature vector of the initial text; streaming clustering the feature vector of the initial text to obtain a cluster to which the initial text belongs; determining at least one target text of the cluster according to the distance between each initial text in the cluster and the cluster center of the cluster; determining the attention index value of each target text of the cluster based on the attribute feature data of each target text of the cluster, and summing the attention index values ​​of each target text of the cluster to obtain the attention index value of the cluster; inputting each target text in the cluster into a large language model, performing public opinion analysis on the public opinion events corresponding to the cluster, and obtaining a public opinion summary corresponding to the cluster; inputting the public opinion summary corresponding to the cluster and multiple preset categories into the large language model, performing category matching on the public opinion events corresponding to the cluster, and generating an event category corresponding to the cluster; when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, sending the event category corresponding to the cluster, the attention index value of the cluster, and the public opinion summary corresponding to the cluster to a terminal device.

[0005] According to a second aspect of an embodiment of the present application, a public opinion data analysis device is provided, including: an acquisition module for acquiring public opinion data and formatting and extracting the public opinion data to obtain an initial text; a vector representation module for performing vector representation on the initial text to obtain a feature vector of the initial text; a clustering module for performing streaming clustering on the feature vector of the initial text to obtain a cluster to which the initial text belongs; a first determination module for determining at least one target text of a cluster based on the distance between each initial text in the cluster and the cluster center of the cluster; a second determination module for determining the attention index value of each target text of the cluster based on the attribute feature data of each target text of the cluster, and The attention index values ​​of each target text in the cluster are summed up to obtain the attention index value of the cluster; the analysis module is used to input each target text in the cluster into the large language model, perform public opinion analysis on the public opinion events corresponding to the cluster, and obtain the public opinion summary corresponding to the cluster; the matching module is used to input the public opinion summary corresponding to the cluster and multiple preset categories into the large language model, perform category matching on the public opinion events corresponding to the cluster, and generate the event category corresponding to the cluster; the sending module is used to send the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster to the terminal device when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present application, a readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0008] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by obtaining public opinion data from various channels and performing formatted extraction, the raw data is converted into initial text that is convenient for subsequent processing, which helps to unify the format of the data and provide a standardized and normalized data basis for subsequent analysis. The initial text is converted into a numerical form that can be understood by the computer through vector representation, that is, the feature vector of the initial text, capturing the semantic information in the text, so that subsequent clustering, classification and other operations are more accurate. The feature vector of the initial text is clustered in a streaming manner, and a large amount of data is processed in real time, and similar texts are clustered together to form different clusters. According to the distance between each initial text in the cluster and the cluster center, a representative target text is selected. The target text can better reflect the theme or core content of the cluster, and provide key information for the subsequent numerical calculation of the attention index and public opinion analysis. Based on the attribute feature data of the target text, the attention index value of each cluster is calculated, and the degree of attention of different public opinion events is quantitatively evaluated, providing an important reference for candidate decision analysis. The target text is input into a large language model for public opinion analysis, and a public opinion summary is generated, which helps to quickly understand the main content of the public opinion event. The public opinion summary and multiple preset categories corresponding to the cluster are input into the large language model, and the category matching is performed through the large language model to determine the category of the public opinion event. When the attention index value of a cluster reaches or exceeds the preset threshold, the event category, attention index value and public opinion summary of the cluster are sent to the terminal device, and important public opinion information is promptly conveyed to relevant personnel so that they can respond and make decisions quickly. The public opinion data analysis method proposed in this application realizes the automatic collection, processing, analysis and classification of public opinion data through a series of steps, and prioritizes and pushes important public opinion events in combination with attention index values, realizing a comprehensive and in-depth analysis of public opinion data, which can provide important reference information for decision makers. It solves the problems of low generalization and inaccurate analysis of public opinion data in the prior art, and improves the efficiency and accuracy of public opinion monitoring and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 is a schematic diagram of an application scenario of an embodiment of the present application;

[0011] Figure 2 It is a flowchart of a method for analyzing public opinion data provided by an embodiment of the present application;

[0012] Figure 3 It is a flowchart of another public opinion data analysis method provided in an embodiment of the present application;

[0013] Figure 4 It is a flowchart of another method for analyzing public opinion data provided by an embodiment of the present application;

[0014] Figure 5 It is a flowchart of another method for analyzing public opinion data provided by an embodiment of the present application;

[0015] Figure 6 It is a structural schematic diagram of a public opinion data analysis device provided in an embodiment of the present application;

[0016] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0018] A method and device for analyzing public opinion data according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0019] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario may include terminal devices 1, 2 and 3, a server 4 and a network 5.

[0020] Terminal devices 1, 2 and 3 can be hardware or software. When terminal devices 1, 2 and 3 are hardware, they can be various electronic devices with display screens and supporting communication with server 4, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers, etc.; when terminal devices 1, 2 and 3 are software, they can be installed in the above electronic devices. Terminal devices 1, 2 and 3 can be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of the present application are not limited to this. Furthermore, various applications can be installed on terminal devices 1, 2 and 3, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0021] The server 4 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 4 may be a single server, a server cluster composed of several servers, or a cloud computing service center, which is not limited in the present application.

[0022] It should be noted that the server 4 can be hardware or software. When the server 4 is hardware, it can be various electronic devices that provide various services for the terminal devices 1, 2, and 3. When the server 4 is software, it can be multiple software or software modules that provide various services for the terminal devices 1, 2, and 3, or it can be a single software or software module that provides various services for the terminal devices 1, 2, and 3, and the embodiments of the present application are not limited to this.

[0023] The network 5 can be a wired network connected by coaxial cable, twisted pair and optical fiber, or it can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present application.

[0024] The user can establish a communication connection with the server 4 via the network 5 through the terminal devices 1, 2 and 3 to receive or send information, etc. Specifically, the server 4 obtains public opinion data, formats and extracts the public opinion data, and obtains the initial text; vectorizes the initial text to obtain the feature vector of the initial text; performs streaming clustering on the feature vector of the initial text to obtain the cluster to which the initial text belongs; determines at least one target text of the cluster according to the distance between each initial text in the cluster and the cluster center of the cluster; determines the attention index value of each target text of the cluster based on the attribute feature data of each target text of the cluster, and sums the attention index values ​​of each target text of the cluster to obtain the attention index value of the cluster; inputs each target text in the cluster into the large language model, performs public opinion analysis on the public opinion events corresponding to the cluster, and obtains the public opinion summary corresponding to the cluster; inputs the public opinion summary corresponding to the cluster and multiple preset categories into the large language model, performs category matching on the public opinion events corresponding to the cluster, and generates the event category corresponding to the cluster; when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster, and the public opinion summary corresponding to the cluster are sent to the terminal device.

[0025] It should be noted that the specific types, quantities and combinations of the terminal devices 1, 2 and 3, the server 4 and the network 5 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present application do not limit this.

[0026] Figure 2 It is a flow chart of a public opinion data analysis method provided in an embodiment of the present application. Figure 2 The public opinion data analysis method can be Figure 1 The server executes. Figure 2 As shown, the public opinion data analysis method includes:

[0027] Step 201, obtain public opinion data, and format and extract the public opinion data to obtain an initial text.

[0028] In some embodiments, the content published can be captured from a designated platform (such as social media, news websites, forums, etc.) by crawlers or other technical means to obtain initial public opinion data, providing rich materials for subsequent analysis. The initial public opinion data contains some information that is not related to public opinion, which can be advertising marketing, business activities, entertainment and funny content. It can be screened by text matching and rule filtering to remove invalid data and obtain public opinion data, which helps to improve the accuracy and efficiency of subsequent analysis. There are problems such as inconsistent formats and special characters in public opinion data. The public opinion data can be formatted and converted into a unified format to facilitate subsequent analysis and comparison to obtain the initial text. The formatting process can include removing punctuation marks, unifying fonts and font sizes, and converting to a unified encoding format. The initial text obtained in a unified standardized format is convenient for subsequent cluster analysis and improves the accuracy of public opinion analysis. Through the method proposed in this application, complex public opinion data is gradually converted into a clearly structured initial text, providing a high-quality data basis for further public opinion analysis.

[0029] Step 202: Perform vector representation on the initial text to obtain a feature vector of the initial text.

[0030] In some embodiments, the initial text input semantic embedding big model can be vectorized, which helps to convert text data into a numerical form that can be understood and processed by a computer, and obtains the feature vector of the initial text, so as to facilitate subsequent text analysis and processing. The above-mentioned semantic embedding big model can be a bidirectional encoder representation (Bidirectional Encoder Representation from Transformer, BERT) model based on a converter, a generative pre-training (Generative Pre-Training, GPT) model, etc., which is pre-trained on large-scale text data by a deep learning method, so that the deep semantic meaning of the text can be understood. The initial text input semantic embedding big model, based on the language knowledge and pattern learned by the semantic embedding big model, the initial text is converted into a vector representation in a high-dimensional vector space, i.e., the feature vector of the initial text. The key semantic information in the initial text is contained in the feature vector of the initial text, so that similar initial texts have a close distance in the vector space, so as to facilitate subsequent operations such as clustering.

[0031] Step 203: Perform streaming clustering on the feature vector of the initial text to obtain the cluster to which the initial text belongs.

[0032] In some embodiments, a streaming clustering algorithm is used to process feature vectors of multiple initial texts in real time, and the feature vectors of the initial texts are divided into different clusters in real time or near real time. The feature vectors of the initial texts are divided into different clusters according to their intrinsic distribution structure, and the feature vectors of the initial texts are classified. Each cluster represents a set of texts representing a public opinion event, which helps to better understand and analyze the structure and content of public opinion data. Streaming clustering can achieve real-time public opinion analysis, which helps to quickly respond to and manage public opinion. When public opinion changes or new events occur, streaming clustering can quickly adapt to changes, and timely update the classification of public opinion to reflect the trend of public opinion evolution. Streaming clustering is to cluster initial texts with similar semantic features together, which can automatically discover and distinguish different public opinion events, and help users quickly lock in the direction of public opinion that they are focusing on.

[0033] Step 204: determining at least one target text of the cluster according to the distance between each initial text in the cluster and the cluster center of the cluster.

[0034] In some embodiments, the cluster center is the representative point of each cluster after clustering, which can reflect the core features of the text in the cluster. The above cluster center can be calculated by weighted average of the feature vectors of all the initial texts in the cluster. For each initial text in each cluster, the distance between their respective feature vectors and the cluster center is calculated. The smaller the distance, the more similar the text is to the cluster center, that is, it has a higher semantic similarity with other texts in the cluster. Select multiple initial texts with the smallest distance from the cluster center as target texts. In public opinion analysis, the above target texts can contain the most important viewpoints, the most important discussion content or the most influential remarks of the public opinion event represented by the cluster. By screening out representative target texts, it is possible to avoid analyzing a large number of redundant or similar texts and improve analysis efficiency. The target text contains the core information and key content of the cluster. In-depth analysis based on multiple target texts can more accurately analyze the theme, emotional tendency, etc. of the public opinion event.

[0035] Step 205 , based on the attribute feature data of each target text of the cluster, determine the attention index value of each target text of the cluster, and sum the attention index values ​​of each target text of the cluster to obtain the attention index value of the cluster.

[0036] In some embodiments, the attribute feature data of the target text may include the basic score of the target text, the weight corresponding to the target text, the interactive amount of the target text, the decay constant, and the time interval from the target text going online to the current moment. The basic score of the target text is an artificially set value, which can be used to reflect the credibility of the source of the target text. The target texts from different sources, such as mainstream media, social media, forums, etc., may have different basic scores, and the target texts published by authoritative media often have higher basic scores. The weight corresponding to the target text is related to the distance between the target text and the cluster center, and is related to the content quality of the target text. A higher weight can be set for high-quality target texts, and high-quality target texts can be texts that contain in-depth analysis, exclusive reports, or cause extensive discussion. The interactive amount of the target text includes the number of likes, comments, and forwardings of the target text. The higher the interactive amount, the more popular the target text is. As time goes by, the value of the target text's attention index gradually decays. To reflect the decay effect, the decay constant and the time interval from the target text's launch to the current moment are introduced. The decay constant is related to the decay speed. Through the decay constant and the time interval from the target text's launch to the current moment, the newer target text can occupy a greater weight in the calculation of the attention index value, while the older target text gradually reduces its influence. This application can calculate the attention index value of each cluster by the following formula: ∑(w*P+I)**e -λt

[0037] Among them, P is the basic score of the target text, w is the weight of the target text, I is the interaction volume of the target text, λ is the decay constant, and t is the time interval from the launch of the target text to the current moment.

[0038] According to the attribute feature data of multiple target texts corresponding to the cluster, the attention index value of each target text is determined, and the attention index values ​​of each target text in the cluster are summed to obtain the attention index value of the cluster. The attention index value of the cluster represents the overall attention level of the public opinion event represented by the cluster, and is a basis for quantitatively evaluating the importance of public opinion events. Calculating the attention index value of the cluster and converting the attention level of the public opinion event into a specific value will help make the comparison between different events more intuitive and accurate. By real-time monitoring of the attention index value of the cluster, the development trend and changes of the public opinion event can be discovered in time, providing users with timely warnings and reminders.

[0039] Step 206: Input each target text in the cluster into the large language model, perform public opinion analysis on the public opinion events corresponding to the cluster, and obtain a public opinion summary corresponding to the cluster.

[0040] In some embodiments, the above-mentioned large language model has powerful language understanding and generation capabilities, and can generate a pre-trained transformer (Chat Generative Pre-trained Transformer, ChatGPT) for chat. Generate the prompt text prompt corresponding to the cluster based on each target text in the cluster, and input the prompt into the large language model, and use the natural language processing capabilities of the large language model to deeply analyze and understand each target text. After training with a large amount of data, the large language model can capture the key information in the input text, understand the semantics and context, and generate accurate and coherent text output. This application can effectively perform public opinion analysis on the target text in the cluster through the large language model, extract key information related to the public opinion event, and obtain a comprehensive and objective public opinion summary. The public opinion summary may include the basic situation of the public opinion event, the development context, and the public's views, attitudes, emotional tendencies, etc. on the event. The comprehensive analysis based on the large prediction model helps to understand the nature and impact of the public opinion event more deeply, and provide strong support for subsequent decision-making and response. This application makes full use of the advantages of the large language model to extract and condense the important information of public opinion events and generate a high-quality overview of public opinion events, that is, the public opinion summary corresponding to the cluster, which is helpful for the subsequent monitoring and management of public opinion.

[0041] Step 207: input the public opinion summary and multiple preset categories corresponding to the cluster into the large language model, perform category matching on the public opinion events corresponding to the cluster, and generate event categories corresponding to the cluster.

[0042] In some embodiments, the public opinion summary is a high-level summary of multiple target texts, which contains rich information, such as the theme of the public opinion event, the main entities involved, the public's emotional tendencies, etc., which is crucial for determining the category of the public opinion event. Multiple preset categories can be event categories set based on historical experience, professional knowledge or specific needs, which are a series of possible public opinion event types. The preset categories have clear definitions and boundaries and can be used to guide the large language model to perform category matching. The public opinion summary corresponding to the cluster and multiple preset categories are input into the large language model, and the generated public opinion summary is compared and analyzed with each preset category. Based on the text understanding and reasoning ability of the large language model, it is determined which category of the multiple preset categories the public opinion event corresponding to the cluster best conforms to, and the event category corresponding to the cluster is obtained. The event category can be the withholding of wages for migrant workers, violent law enforcement by urban management, etc. The public opinion data analysis method proposed in this application improves the speed and scale effect of public opinion data analysis and processing through automated category matching.

[0043] Step 208, when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster are sent to the terminal device.

[0044] In some embodiments, each event category has a corresponding attention index value threshold value, and the attention index value threshold value corresponding to each event category can be set based on historical data, business needs or expert advice, and is used to determine when the attention of a certain event category reaches the level that requires an alarm. When the attention index value of a cluster reaches or exceeds the attention index value threshold value of the corresponding event category, the sending mechanism is triggered to alarm. After the sending mechanism is triggered, the event category corresponding to the cluster, the attention index value of the cluster, and the public opinion summary corresponding to the cluster are sent to the designated terminal device. The terminal device can be a mobile phone, a computer or other device that can receive and display this information. By sending the event category corresponding to the cluster, the attention index value of the cluster, and the public opinion summary corresponding to the cluster to the terminal device, users can obtain detailed information on high-attention public opinion events in a timely and accurate manner, providing strong support for users to make decisions on the public opinion event.

[0045] Through the public opinion data processing method of the present application, public opinion data from various channels are obtained, and formatted extraction is performed to convert the original data into initial text that is convenient for subsequent processing, which helps to unify the format of the data and provide a standardized and normalized data basis for subsequent analysis. The initial text is converted into a numerical form that can be understood by the computer through vector representation, that is, the feature vector of the initial text, capturing the semantic information in the text, so that subsequent clustering, classification and other operations are more accurate. The feature vector of the initial text is stream clustered, a large amount of data is processed in real time, and similar texts are clustered together to form different clusters. According to the distance between each initial text in the cluster and the center of the cluster, a representative target text is selected. The target text can better reflect the theme or core content of the cluster, and provide key information for the subsequent numerical calculation of the attention index and public opinion analysis. Based on the attribute feature data of the target text, the attention index value of each cluster is calculated, and the degree of attention of different public opinion events is quantitatively evaluated, providing an important reference for candidate decision analysis. The target text is input into a large language model for public opinion analysis, and a public opinion summary is generated, which helps to quickly understand the main content of the public opinion event. The public opinion summary and multiple preset categories corresponding to the cluster are input into the large language model, and the category matching is performed through the large language model to determine the category of the public opinion event. When the attention index value of a cluster reaches or exceeds the preset threshold, the event category, attention index value and public opinion summary of the cluster are sent to the terminal device, and important public opinion information is promptly conveyed to relevant personnel so that they can respond and make decisions quickly. The public opinion data analysis method proposed in this application realizes the automatic collection, processing, analysis and classification of public opinion data through a series of steps, and prioritizes and pushes important public opinion events in combination with attention index values, realizing a comprehensive and in-depth analysis of public opinion data, which can provide important reference information for decision makers. It solves the problems of low generalization and inaccurate analysis of public opinion data in the prior art, and improves the efficiency and accuracy of public opinion monitoring and management.

[0046] In some embodiments, obtaining public opinion data includes: crawling data on published content on a preset platform to obtain initial public opinion data; and screening the initial public opinion data to obtain public opinion data.

[0047] In some embodiments, the preset platform may include social media, news websites, forums, etc. The preset platform is an important channel for publishing public opinion information. A large amount of initial public opinion data can be obtained through crawlers or other API interfaces to provide a basis for subsequent analysis. The initial public opinion data can be screened through text matching and rule filtering to remove content in the initial public opinion data that is not related to news information and opinions, such as advertising marketing, business activities, and entertainment and comedy, and obtain public opinion data related to public opinion events, which can improve the accuracy and efficiency of subsequent analysis.

[0048] In some embodiments, the initial text includes a title text and a main body text, and a vector representation is performed on the initial text to obtain a feature vector of the initial text, including: inputting the title text into a semantic embedding large model for embedding processing to obtain a feature vector of the title text; inputting the main body text into a semantic embedding large model for embedding processing to obtain a feature vector of the main body text; and performing weighted summation of the feature vector of the title text and the feature vector of the main body text to obtain a feature vector of the initial text.

[0049] In some embodiments, the initial text consists of a title text and a body text, the title text succinctly and clearly summarizes the theme of the text, and the body text elaborates or describes the specific content in detail. The title text and the body text are respectively input into the semantic embedding macro model, which can understand the semantics of the text and convert it into a feature vector in a high-dimensional vector space, thereby obtaining a feature vector of the title text and a feature vector of the body text. The feature vector of the title text and the feature vector of the body text are weighted and summed to obtain a feature vector of the initial text that integrates the title and body information. The title text and the body text are vectorized by the semantic embedding macro model, converted into a computable numerical form, and a more comprehensive feature vector representation is obtained by fusing the information of the title and the body, namely, the feature vector of the initial text, which provides an important data basis for subsequent public opinion analysis.

[0050] In some embodiments, streaming clustering is performed on the feature vector of the initial text to obtain the cluster to which the initial text belongs, including: performing inverse product quantization on the feature vector of the preset text in the storage database based on the feature vector of the initial text, and determining the feature vectors of multiple corresponding first candidate texts from the feature vector of the preset text in the storage database; obtaining the first candidate cluster to which the feature vector of each first candidate text belongs through the feature vector of each first candidate text; determining the first candidate cluster whose average intra-cluster distance of the first candidate cluster is less than the first preset distance as the second candidate cluster; calculating the distance between the cluster center of each second candidate cluster and the feature vector of the initial text; if there is a distance less than the second preset distance, determining the cluster to which the initial text belongs based on each distance; if each distance is greater than or equal to the second preset distance, generating a new cluster and determining the new cluster as the cluster to which the initial text belongs.

[0051] In some embodiments, multiple preset texts similar to the initial text are quickly found from a large storage database through inverted product quantization to obtain a set of feature vectors of preset texts that are most similar to the feature vector of the initial text. The feature vectors of the set of preset texts obtained by screening are called the feature vectors of the first candidate text, which can prepare for subsequent more sophisticated clustering operations. The feature vector of each preset text has been classified into a cluster in the previous clustering process. After determining the feature vectors of each first candidate text, the first candidate cluster to which the feature vector of each first candidate text belongs is obtained. For all first candidate clusters, based on the index of average distance within the cluster, those clusters with high internal consistency, that is, the average distance within the cluster is less than the first preset distance, are screened as objects for further consideration, that is, second candidate clusters. The distance between the feature vector of the initial text and the cluster center of each second candidate cluster is calculated. If there is a feature vector of the initial text and the cluster center distance of a second candidate cluster is less than the second preset distance, the second candidate cluster with the smallest distance between the cluster center and the feature vector of the initial text is selected as the cluster to which the initial text should belong, which meets the clustering requirements. If the distances between all cluster centers and the feature vector of the initial text are greater than or equal to the second preset distance, it means that the initial text may belong to a new category. At this time, a new cluster is generated and the initial text is classified into the new cluster. The above streaming clustering method can process a large amount of text data in real time and maintain the accuracy and efficiency of clustering.

[0052] In some embodiments, automated clustering is an important step in the public opinion data analysis process. A large amount of public opinion data can be grouped based on certain similarities or associations to form different clusters, each cluster representing a public opinion event. However, due to the diversity and complexity of public opinion events, there are great differences in the granularity required for clustering and the size of the cluster. In the clustering process, the hyperparameters of the clustering algorithm can be adjusted. The adjusted hyperparameters of the clustering algorithm can be the average distance threshold within the cluster and the threshold of the distance between the feature vector of the initial text and the cluster center. The average distance threshold within the cluster affects the closeness between the texts within the cluster, while the threshold of the distance between the feature vector of the initial text and the cluster center affects the standard of which cluster the new initial text is classified into. The average distance threshold within the cluster and the threshold of the distance between the feature vector of the initial text and the cluster center can be adjusted according to actual needs, so that the clustering results are more in line with the actual situation. If a public opinion event requires a finer granularity division, the average distance threshold within the cluster can be appropriately reduced to make the cluster smaller and more specific. On the contrary, the average distance threshold within the cluster can be increased. In addition, for clusters that have been clustered together, if it is found that the division of some clusters is not accurate or does not meet the actual needs, manual division can be performed. Manual segmentation can be performed based on more specific business rules or expert knowledge, which helps make the clustering more accurate and meaningful.

[0053] In some embodiments, when the attention index value of a cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, before sending the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster to the terminal device, it also includes: inputting each target text in the cluster into a large language model, matching the emotion label category of each target text, and obtaining the emotion label category of each target text; determining the proportion of each type of emotion label target text according to the number of each type of emotion label target text and the number of target texts; determining the emotion label category corresponding to the cluster according to the proportion of each type of emotion label target text; when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the emotion label category corresponding to the cluster, sending the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster and the proportion of each type of emotion label target text corresponding to the cluster to the terminal device.

[0054] In some embodiments, each target text in the cluster is input into the large language model, and the sentiment analysis capability of the large language model is used to match the sentiment label for each target text. The sentiment label category may include positive, negative, neutral, etc., which are used to describe the sentiment tendency expressed by the target text. The matching of sentiment labels helps to understand the sentiment tendency of the target text more deeply. For example, by matching the sentiment label of each text, the overall sentiment tendency of the public towards a certain public opinion event can be understood. The number of target texts under each sentiment label category is determined, and their respective proportions of the total number of target texts are calculated. The proportion of each type of sentiment label can reflect the distribution of different sentiment tendencies in the overall target text. According to the proportion of each type of sentiment label, the sentiment label category of the cluster is determined. By comparing the proportions of different sentiment labels in the cluster, the sentiment label with the highest proportion is selected as the sentiment label category of the cluster. Each cluster of each sentiment label category has a corresponding attention index value threshold. When the attention index value of the cluster exceeds the attention threshold set for the corresponding sentiment label category, the relevant information of the cluster, including the event category corresponding to the cluster, the attention index value corresponding to the cluster, the public opinion summary of the cluster, and the proportion of target texts of various sentiment labels are sent to the terminal device. Ensure that important information can be conveyed to relevant personnel in a timely and accurate manner. Through terminal devices, relevant personnel can understand public opinion events with high attention and obvious emotional tendencies in real time, so as to make timely responses and decisions.

[0055] In some embodiments, when the attention index value of a cluster is greater than or equal to the attention index value threshold corresponding to the emotional label category corresponding to the cluster, before sending the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, and the target text ratios of various types of emotional labels corresponding to the cluster to the terminal device, it also includes: inputting the public opinion summary corresponding to the cluster into a large language model, performing sensitive label category judgment on the public opinion events corresponding to the cluster, and obtaining the sensitive label category corresponding to the cluster; when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the sensitive label category corresponding to the cluster, sending the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratios of various types of emotional labels corresponding to the cluster, and the sensitive label category corresponding to the cluster to the terminal device.

[0056] In some embodiments, the public opinion summary is a summary and conclusion of multiple target text information in the cluster, reflecting the main content of the public opinion event represented by the cluster. The public opinion summary is input into the large language model, and the model's ability to understand and analyze text is used to determine the sensitive label category of the public opinion event corresponding to the cluster, and the sensitivity of the public opinion event is evaluated to obtain the sensitive label category corresponding to the cluster. The sensitive label category may include insensitive, generally sensitive, and very sensitive. Each cluster of sensitive label category has a corresponding attention index value threshold. When the attention index value of the cluster exceeds the attention threshold corresponding to the pre-set sensitive label category, it indicates that the public opinion event has a specific sensitive nature and has also caused certain attention and influence in society. It is necessary to send the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster to the terminal device, which helps decision makers to quickly grasp the public opinion dynamics and social hot spots and take corresponding countermeasures.

[0057] In some embodiments, after the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the proportion of target texts of various types of emotional labels corresponding to the cluster, and the sensitive label category corresponding to the cluster are sent to the terminal device, it also includes: receiving a task instruction input by the user, and configuring a corresponding target report generation task based on the task instruction; based on the target report generation task, inputting each target text in all newly generated clusters within a preset time into a large language model to obtain the number of events generated within a preset time, the attention index value corresponding to each event, the number of events corresponding to various event categories, the number of events with various sensitive labels, and the number of events with various emotional labels; based on a preset summary template and the number of events generated within a preset time, the attention index value corresponding to each event, the number of events corresponding to various event categories, the number of events with various sensitive labels, and the number of events with various emotional labels, generate a corresponding target report.

[0058] In some embodiments, a task instruction input by a user is received, such as generating a public opinion analysis report within a specific time period (preset time). Based on the user's task instruction, a special target report generation task is configured to clarify the data range to be captured, the indicator dimensions of concern (such as the number of events, attention, event category, sensitive label, emotional label, etc.) and the presentation form of the final report. Within the preset time range, new public opinion data is continuously monitored and collected. The above public opinion data is clustered into different clusters, and each cluster can represent an independent public opinion event. Each target text in these clusters is analyzed, and the key information is understood and refined using a large language model to obtain: the total number of events generated within the preset time, the value of the attention index of each event, the number of events of different event categories, the number of events with various sensitive labels (insensitive, generally sensitive, very sensitive), and the number of events with various emotional labels (such as positive, negative, neutral). According to the preset summary template, the various statistical results obtained from the above analysis are filled into the corresponding positions in the template to automatically generate a target report with a clear structure and detailed content. By receiving task instructions input by users, configuring target report generation tasks, and processing data to generate reports based on a large language model, the efficiency and accuracy of summary report generation can be improved. Through data statistics and text generation technology, large amounts of complex data can be converted into easy-to-understand text reports, thereby helping decision makers or analysts to better understand and respond to events occurring within a specific time window, and realize real-time monitoring and early warning functions.

[0059] In some embodiments, for public opinion events with high popularity or high sensitivity or negative nature or specific labels, efficient and targeted response measures can be generated through a large language model. The public opinion summary of public opinion events with high popularity or high sensitivity or negative nature or specific labels and the corresponding multiple target texts are input into the large language model. Based on its powerful semantic understanding and reasoning capabilities, the causal relationship, social impact, public opinion orientation and potential risk points behind the event are analyzed to obtain analysis results, and based on the analysis results, specific response measures for each event are generated to improve the efficiency and pertinence of the execution of corresponding solutions. By using a large model, various public opinion events can be monitored in real time and responded to quickly, and artificial intelligence can be used to assist decision-making to enhance the scientificity and effectiveness of public opinion data processing.

[0060] In some embodiments, reference Figure 3 , the public opinion data analysis method proposed in this application includes:

[0061] Step 310, obtaining public opinion data, and formatting and extracting the public opinion data to obtain an initial text;

[0062] Step 320, performing vector representation on the initial text to obtain a feature vector of the initial text;

[0063] Step 330, performing streaming clustering on the feature vector of the initial text to obtain the cluster to which the initial text belongs, and obtaining a clustering result;

[0064] Step 340, performing clustering post-processing to obtain a processing result;

[0065] Step 350, based on the processing result, the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster are sent to the terminal device;

[0066] Step 360, receiving a task instruction input by a user, configuring a corresponding target report generation task based on the task instruction; based on the target report generation task, inputting each target text in all newly generated clusters within a preset time into a large language model and using a preset summary template to generate a corresponding target report;

[0067] Step 370, for public opinion events with high popularity or high sensitivity or negative nature or specific labels, generate efficient and targeted response measures through a large language model.

[0068] refer to Figure 4 , step 340, i.e., performing clustering post-processing to obtain processing results, including:

[0069] Step 401, based on the attribute feature data of each target text of the cluster, determine the attention index value of each target text of the cluster, and sum the attention index values ​​of each target text of the cluster to obtain the attention index value of the cluster;

[0070] Step 402, input each target text in the cluster into the large language model, perform public opinion analysis on the public opinion events corresponding to the cluster, and obtain a public opinion summary corresponding to the cluster;

[0071] Step 403, inputting the public opinion summary and multiple preset categories corresponding to the cluster into the large language model, performing category matching on the public opinion events corresponding to the cluster, and generating event categories corresponding to the cluster;

[0072] Step 404, input each target text in the cluster into the large language model, perform sentiment label category matching on each target text, and obtain the sentiment label category of each target text; determine the proportion of each sentiment label target text according to the number of each sentiment label target text and the number of target texts; determine the sentiment label category corresponding to the cluster according to the proportion of each sentiment label target text;

[0073] Step 405: input the public opinion summary corresponding to the cluster into the large language model, perform sensitive label category determination on the public opinion event corresponding to the cluster, and obtain the sensitive label category corresponding to the cluster.

[0074] refer to Figure 5 Step 350, based on the processing result, sends the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster to the terminal device, including:

[0075] Step 501, when the attention index value of a cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster are sent to the terminal device;

[0076] Step 502, when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the emotion tag category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each emotion tag corresponding to the cluster, and the sensitive tag category corresponding to the cluster are sent to the terminal device;

[0077] Step 503, when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the sensitive label category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster are sent to the terminal device.

[0078] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0079] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0080] Figure 6 Schematic diagram of a public opinion data analysis device provided in an embodiment of the present application. Figure 4 As shown, the public opinion data analysis device includes:

[0081] The acquisition module 601 is used to acquire public opinion data and format and extract the public opinion data to obtain an initial text;

[0082] A vector representation module 602 is used to perform vector representation on the initial text to obtain a feature vector of the initial text;

[0083] A clustering module 603 is used to perform streaming clustering on the feature vector of the initial text to obtain the cluster to which the initial text belongs;

[0084] A first determination module 604, configured to determine at least one target text of a cluster according to the distance between each initial text in the cluster and the cluster center of the cluster;

[0085] The second determination module 605 is used to determine the attention index value of each target text in the cluster based on the attribute feature data of each target text in the cluster, and sum the attention index values ​​of each target text in the cluster to obtain the attention index value of the cluster;

[0086] Analysis module 606, used to input each target text in the cluster into the large language model, perform public opinion analysis on the public opinion events corresponding to the cluster, and obtain the public opinion summary corresponding to the cluster;

[0087] Matching module 607, used to input the public opinion summary and multiple preset categories corresponding to the cluster into the large language model, perform category matching on the public opinion events corresponding to the cluster, and generate event categories corresponding to the cluster;

[0088] The sending module 608 is used to send the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster to the terminal device when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster.

[0089] According to the technical solution provided by the embodiment of the present application, by obtaining public opinion data from various channels and performing formatted extraction, the original data is converted into an initial text that is convenient for subsequent processing, which helps to unify the format of the data and provide a standardized and normalized data basis for subsequent analysis. The initial text is converted into a numerical form that can be understood by the computer through vector representation, that is, the feature vector of the initial text, capturing the semantic information in the text, so that subsequent clustering, classification and other operations are more accurate. The feature vector of the initial text is stream clustered, a large amount of data is processed in real time, and similar texts are clustered together to form different clusters. According to the distance between each initial text in the cluster and the cluster center, a representative target text is selected. The target text can better reflect the theme or core content of the cluster, and provide key information for the subsequent numerical calculation of the attention index and public opinion analysis. Based on the attribute feature data of the target text, the attention index value of each cluster is calculated, and the degree of attention of different public opinion events is quantitatively evaluated, providing an important reference for candidate decision analysis. The target text is input into a large language model for public opinion analysis, and a public opinion summary is generated, which helps to quickly understand the main content of the public opinion event. The public opinion summary and multiple preset categories corresponding to the cluster are input into the large language model, and the category matching is performed through the large language model to determine the category of the public opinion event. When the attention index value of a cluster reaches or exceeds the preset threshold, the event category, attention index value and public opinion summary of the cluster are sent to the terminal device, and important public opinion information is promptly conveyed to relevant personnel so that they can respond and make decisions quickly. The public opinion data analysis method proposed in this application realizes the automatic collection, processing, analysis and classification of public opinion data through a series of steps, and prioritizes and pushes important public opinion events in combination with attention index values, realizing a comprehensive and in-depth analysis of public opinion data, which can provide important reference information for decision makers. It solves the problems of low generalization and inaccurate analysis of public opinion data in the prior art, and improves the efficiency and accuracy of public opinion monitoring and management.

[0090] In some embodiments, the acquisition module 601 is configured to capture data on the published content of a preset platform to obtain initial public opinion data; and to filter the initial public opinion data to obtain public opinion data.

[0091] In some embodiments, the initial text includes title text and body text, and the vector representation module 602 is configured to embed the title text into a semantic embedding model to obtain a feature vector of the title text; embed the body text into a semantic embedding model to obtain a feature vector of the body text; and perform weighted summation of the feature vector of the title text and the feature vector of the body text to obtain a feature vector of the initial text.

[0092] In some embodiments, the clustering module 603 is configured to perform inverse product quantization on the feature vector of the preset text in the storage database based on the feature vector of the initial text, determine the feature vectors of the corresponding multiple first candidate texts from the feature vectors of the preset text in the storage database; obtain the first candidate cluster to which the feature vector of each first candidate text belongs through the feature vector of each first candidate text; determine the first candidate cluster whose average intra-cluster distance of the first candidate cluster is less than the first preset distance as the second candidate cluster; calculate the distance between the cluster center of each second candidate cluster and the feature vector of the initial text; if there is a distance less than the second preset distance, determine the cluster to which the initial text belongs based on each distance; if each distance is greater than or equal to the second preset distance, generate a new cluster and determine the new cluster as the cluster to which the initial text belongs.

[0093] In some embodiments, when the attention index value of a cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, before sending the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster to the terminal device, the public opinion data analysis device is configured to input each target text in the cluster into a large language model, match the emotion label category of each target text, and obtain the emotion label category of each target text; determine the proportion of each type of emotion label target text according to the number of each type of emotion label target text and the number of target texts; determine the emotion label category corresponding to the cluster according to the proportion of each type of emotion label target text; when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the emotion label category corresponding to the cluster, send the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster and the proportion of each type of emotion label target text corresponding to the cluster to the terminal device.

[0094] In some embodiments, when the attention index value of a cluster is greater than or equal to the attention index value threshold corresponding to the emotional label category corresponding to the cluster, before sending the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, and the target text ratios of various types of emotional labels corresponding to the cluster to the terminal device, the public opinion data analysis device is configured to input the public opinion summary corresponding to the cluster into the large language model, perform sensitive label category judgment on the public opinion events corresponding to the cluster, and obtain the sensitive label category corresponding to the cluster; when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the sensitive label category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratios of various types of emotional labels corresponding to the cluster, and the sensitive label category corresponding to the cluster are sent to the terminal device.

[0095] In some embodiments, after the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the proportion of target texts of various types of emotional labels corresponding to the cluster, and the sensitive label category corresponding to the cluster are sent to the terminal device, the public opinion data analysis device is configured to receive a task instruction input by the user, and configure the corresponding target report generation task based on the task instruction; based on the target report generation task, input each target text in all newly generated clusters within a preset time into the large language model to obtain the number of events generated within the preset time, the attention index value corresponding to each event, the number of events corresponding to various event categories, the number of events with various sensitive labels, and the number of events with various emotional labels; based on the preset summary template and the number of events generated within the preset time, the attention index value corresponding to each event, the number of events corresponding to various event categories, the number of events with various sensitive labels, and the number of events with various emotional labels, generate a corresponding target report.

[0096] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0097] Figure 7 is a schematic diagram of an electronic device 700 provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 of this embodiment includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program 703, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 701 executes the computer program 703, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0098] The electronic device 700 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 700 may include, but is not limited to, a processor 701 and a memory 702. Those skilled in the art will appreciate that Figure 7 The electronic device 700 is merely an example and does not limit the electronic device 700 . The electronic device 700 may include more or fewer components than those shown in the figure, or different components.

[0099] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0100] The memory 702 may be an internal storage unit of the electronic device 700, for example, a hard disk or memory of the electronic device 700. The memory 702 may also be an external storage device of the electronic device 700, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 700. The memory 702 may also include both an internal storage unit of the electronic device 700 and an external storage device. The memory 702 is used to store computer programs and other programs and data required by the electronic device.

[0101] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0102] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. Computer-readable storage media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0103] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for analyzing public opinion data, characterized in that: include: Obtaining public opinion data, and formatting and extracting the public opinion data to obtain an initial text; Performing vector representation on the initial text to obtain a feature vector of the initial text; Performing streaming clustering on the feature vector of the initial text to obtain the cluster to which the initial text belongs; Determining at least one target text of the cluster according to the distance between each initial text in the cluster and the cluster center of the cluster; Determine the attention index value of each target text of the cluster based on the attribute feature data of each target text of the cluster, and sum the attention index values ​​of each target text of the cluster to obtain the attention index value of the cluster; Input each of the target texts in the cluster into a large language model, perform public opinion analysis on the public opinion events corresponding to the cluster, and obtain a public opinion summary corresponding to the cluster; Inputting the public opinion summary and multiple preset categories corresponding to the cluster into the large language model, performing category matching on the public opinion events corresponding to the cluster, and generating event categories corresponding to the cluster; Input each of the target texts in the cluster into the large language model, perform sentiment label category matching on each of the target texts, and obtain the sentiment label category of each of the target texts; Determine the target text ratio of each type of emotional label according to the number of target texts of each type of emotional label and the number of target texts; Determine the emotional label category corresponding to the cluster according to the target text ratio of each emotional label; When the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the emotional tag category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, and the target text ratio of each type of emotional tag corresponding to the cluster are sent to the terminal device; Inputting the public opinion summary corresponding to the cluster into the large language model, performing sensitive label category determination on the public opinion events corresponding to the cluster, and obtaining the sensitive label category corresponding to the cluster; When the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the sensitive label category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster are sent to the terminal device; When the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster, the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster are sent to the terminal device.

2. The method according to claim 1, characterized in that The obtaining of public opinion data includes: Capture data on the content published on the preset platform to obtain initial public opinion data; The initial public opinion data is screened to obtain the public opinion data.

3. The method according to claim 1, characterized in that The initial text includes a title text and a body text, and the vector representation of the initial text to obtain a feature vector of the initial text includes: Inputting the title text into a semantic embedding model for embedding processing to obtain a feature vector of the title text; Inputting the main text into the semantic embedding model for embedding processing to obtain a feature vector of the main text; The feature vector of the title text and the feature vector of the body text are weightedly summed to obtain the feature vector of the initial text.

4. The method according to claim 1, characterized in that: The step of performing streaming clustering on the feature vector of the initial text to obtain the cluster to which the initial text belongs includes: Based on the feature vector of the initial text, the feature vector of the preset text in the storage database is subjected to inverse product quantization, and the feature vectors of the corresponding plurality of first candidate texts are determined from the feature vectors of the preset text in the storage database; Obtaining, by using the feature vectors of each of the first candidate texts, the first candidate cluster to which the feature vectors of each of the first candidate texts belong; Determine the first candidate cluster whose average intra-cluster distance is less than a first preset distance as a second candidate cluster; Calculating the distance between the cluster center of each of the second candidate clusters and the feature vector of the initial text; If there is a distance less than a second preset distance, determining the cluster to which the initial text belongs based on each of the distances; If all the distances are greater than or equal to the second preset distance, a new cluster is generated and the new cluster is determined as the cluster to which the initial text belongs.

5. The method according to claim 1, characterized in that After sending the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster to the terminal device, the method further includes: Receive a task instruction input by a user, and configure a corresponding target report generation task based on the task instruction; Based on the target report generation task, each target text in all newly generated clusters within a preset time is input into the large language model to obtain the number of events generated within the preset time, the attention index value corresponding to each event, the number of events corresponding to various event categories, the number of events with various sensitive labels, and the number of events with various emotional labels; Based on the preset summary template and the number of events generated within the preset time, the attention index value corresponding to each event, the number of events corresponding to various event categories, the number of events with various sensitive labels, and the number of events with various emotional labels, the corresponding target report is generated.

6. A public opinion data analysis device, characterized in that: include: An acquisition module is used to acquire public opinion data and format and extract the public opinion data to obtain an initial text; A vector representation module, used for performing vector representation on the initial text to obtain a feature vector of the initial text; A clustering module, used for performing streaming clustering on the feature vector of the initial text to obtain the cluster to which the initial text belongs; A first determination module, configured to determine at least one target text of the cluster according to the distance between each initial text in the cluster and the cluster center of the cluster; A second determination module is used to determine the attention index value of each target text of the cluster based on the attribute feature data of each target text of the cluster, and sum the attention index values ​​of each target text of the cluster to obtain the attention index value of the cluster; An analysis module, used to input each of the target texts in the cluster into a large language model, perform public opinion analysis on the public opinion events corresponding to the cluster, and obtain a public opinion summary corresponding to the cluster; A matching module, used to input the public opinion summary and multiple preset categories corresponding to the cluster into the large language model, perform category matching on the public opinion events corresponding to the cluster, and generate event categories corresponding to the cluster; An emotion label category matching module is used to input each of the target texts in the cluster into the large language model, perform emotion label category matching on each of the target texts, and obtain the emotion label category of each of the target texts; A ratio determination module is used to determine the ratio of target texts of various emotion labels according to the number of target texts of various emotion labels and the number of target texts; The emotion tag category determination module is used to determine the emotion tag category corresponding to the cluster according to the target text ratio of each emotion tag; The second sending module is used to send the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, and the target text ratio of each type of emotional label corresponding to the cluster to the terminal device when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the emotional label category corresponding to the cluster; A sensitive label category determination module is used to input the public opinion summary corresponding to the cluster into the large language model, perform sensitive label category determination on the public opinion event corresponding to the cluster, and obtain the sensitive label category corresponding to the cluster; A third sending module is used to send the event category corresponding to the cluster, the attention index value of the cluster, the public opinion summary corresponding to the cluster, the target text ratio of each type of emotional label corresponding to the cluster, and the sensitive label category corresponding to the cluster to the terminal device when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the sensitive label category corresponding to the cluster; The sending module is used to send the event category corresponding to the cluster, the attention index value of the cluster and the public opinion summary corresponding to the cluster to the terminal device when the attention index value of the cluster is greater than or equal to the attention index value threshold corresponding to the event category corresponding to the cluster.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images