Public opinion analysis workflow system and method

Abstract

The application discloses an opinion analysis workflow system, comprising: a data analysis function module, which comprises N data analysis modules capable of analyzing opinion data; a workflow establishment module, which selects multiple data analysis modules from the data analysis function module according to opinion analysis requirements, connects them in sequence, establishes a workflow for analyzing a specific task, establishes multiple workflows for analyzing multiple opinion analysis tasks according to opinion analysis requirements of different analysis angles for the same event; a workflow management module, which performs data analysis and calculation on the established workflow and views the calculation results through a visual workflow diagram; and an event management module, which manages multiple opinion analysis tasks for the same event and builds a connection between different tasks through opinion analysis data. The application further provides an opinion analysis workflow method. The system and method can obtain opinion information from different levels and different angles according to opinion analysis requirements.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of workflow management and data analysis. More particularly, the present application relates to an opinion analysis workflow system and method thereof. BACKGROUND

[0002] Social media has extensive participation, and data mining technology based on social media provides technical support for feature analysis, profiling, tracking of netizens, and finding of initiators and guides of public opinion. In addition, in the business community, using social media big data to divide user communities, operate in blocks, convert KOLs, and guide public opinion, etc. brings new space for improvement for traditional public relations and contributes to the interests of the company.

[0003] However, at the same time, many business analysts are skilled in analyzing and identifying data reports, but lack sufficient programming ability, and cannot quickly implement complex data acquisition, preprocessing, calculation, storage, and display processes, etc. to efficiently discover key elements of social media, which seriously hinders the effective development of the business, and it is urgent to empower the traditional analysis process. SUMMARY

[0004] An object of the present application is to solve at least the above problems and / or deficiencies, and to provide at least the advantages described later.

[0005] Another object of the present application is to provide an opinion analysis workflow system that can obtain opinion information from different levels and different angles according to opinion analysis requirements.

[0006] To achieve these objects and other advantages of the present application, in a first aspect, the present application provides an opinion analysis workflow system, comprising:

[0007] a data analysis function module comprising N data analysis modules capable of analyzing opinion data, N >= 2;

[0008] a workflow establishment module that selects multiple data analysis modules from the data analysis function module according to opinion analysis requirements and connects them in order to establish a workflow capable of analyzing specific tasks, wherein for opinion analysis requirements of different analysis angles of the same event, multiple workflows are established to facilitate analysis of multiple opinion analysis tasks;

[0009] a workflow management module for data analysis and calculation of the established workflow, and viewing the calculation results through a visual workflow diagram;

[0010] an event management module for managing multiple opinion analysis tasks of the same event and establishing connections between different tasks through opinion analysis data.

[0011] Preferably, the public opinion analysis workflow system, the workflow establishment module further comprises a workflow model module, which is a preset workflow model stored in a workflow model database, wherein when one of the workflow models is selected, the data analysis module in the workflow model can be modified according to the public opinion analysis requirement.

[0012] Preferably, the public opinion analysis workflow system, the workflow management module specifically comprises:

[0013] An opinion data acquisition module for acquiring different heterogeneous data sources, the opinion data acquisition module comprising: a data acquisition interface module for connecting a social media network or connecting a database storing social media data; a data query module capable of fuzzy query and structured accurate search, wherein the opinion data acquisition module is connected with the first data analysis module in the established workflow;

[0014] An opinion data preprocessing module for constructing a forwarding network relationship and preprocessing social network text on the acquired public opinion data;

[0015] An opinion data calculation module comprising:

[0016] A data analysis algorithm module comprising various data analysis algorithms;

[0017] An algorithm classification module for classifying various data analysis algorithms into two types of analysis algorithms, namely graph and text;

[0018] An algorithm switching module for switching the data analysis algorithms in the connected data analysis modules in the established workflow in a hierarchical nested mode.

[0019] Preferably, the public opinion analysis workflow system, when a data analysis module is selected during the establishment of the workflow, the parameters of the data analysis module need to be set;

[0020] The data analysis function module comprises a community division module, a propagation trend analysis module, a text classifier calculation module, a sensitivity analysis module and an LDA module.

[0021] Preferably, the public opinion analysis workflow system, the workflow management module further comprises an opinion data storage module, which is stored in the form of hierarchical hash index.

[0022] Preferably, the public opinion analysis workflow system, the construction of the forwarding network relationship specifically comprises: establishing the forwarding network relationship according to the forwarding relationship, IP relationship and nickname of the social media network;

[0023] The preprocessing of the social network text specifically comprises: data judgment according to a regular expression, a stop word table and a social media special language word table, and combining a word segmentation method, performing word segmentation, de-duplication, de-stop word, de-special language and de-punctuation operation on the social media text.

[0024] Preferably, the public opinion analysis workflow system, the parameters of the public opinion data acquisition module need to be set are keywords and time range concerned by public opinion analysis.

[0025] In the second aspect, the present application provides a public opinion analysis workflow method applied to the public opinion analysis workflow system, and the method comprises:

[0026] According to the public opinion analysis requirement, a plurality of data analysis modules are selected from the data analysis function module and are connected in sequence to establish a workflow, or a workflow model is selected from the workflow model module and the data analysis module is modified, wherein the parameters of each data analysis module in the workflow need to be set;

[0027] For the public opinion analysis requirement of different analysis angles of the same event, a plurality of workflows are established;

[0028] The social media data processed by the public opinion data preprocessing module in the workflow management module is used to instantiate the workflow, a calculation task is generated, and the public opinion is analyzed, wherein an event is selected or newly created from the event library, and the calculation task is attributed to the event;

[0029] According to the calculation process, data calculation is completed, and the calculation results of each data analysis module are viewed through the visual workflow diagram.

[0030] In the third aspect, the present application provides an electronic device comprising: at least one processor, and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to make the at least one processor execute the public opinion analysis workflow system.

[0031] In the fourth aspect, the present application provides a storage medium having a computer program stored thereon, and the program is executed by a processor to realize the public opinion analysis workflow system.

[0032] The present application at least has the following beneficial effects:

[0033] I. Since the data analysis function module of the application comprises a plurality of data analysis modules capable of public opinion data analysis, when establishing a workflow, a plurality of data analysis modules can be freely selected according to a specific public opinion analysis requirement, and connected in sequence to form a multi-level analysis module, so that the established workflow can complete a specific public opinion analysis task. Compared with the prior art, the application has great freedom and can establish a workflow that meets any requirement. In addition, through the multi-level analysis module connected freely, the data passing through different data analysis modules can be assigned a label of a different analysis module, thereby providing a hierarchical and labeled result for business personnel, constructing a multi-level and multi-label classification system of analysis tasks, and enabling a new task to be started on the analysis result for further analysis. Through the freely connected data analysis module, greater flexibility, expandability and ease of use are provided than the prior art, and business personnel do not need to understand the composition of different types of data, but only need to freely connect different data analysis modules to obtain the required multi-level analysis result, and the visual display mode enables them to freely select the analysis result of different levels, and the interactivity is stronger than the prior art.

[0034] II. The application can establish a plurality of workflows from different analysis angles for the same event, thereby analyzing a plurality of public opinion analysis tasks, so that public opinion information can be obtained from different levels and different angles. The event management module can manage the plurality of public opinion analysis tasks, and can also build the relationship between different tasks through public opinion analysis data, that is, the public opinion analysis result obtained through one workflow calculation analysis can be used as input data of another workflow data acquisition module, so that the event management module realizes different data analysis methods for the same data source, and can continue further analysis on the analysis result.

[0035] III. Since the workflow management module specifically comprises a public opinion data acquisition module, a public opinion data preprocessing module, a public opinion data calculation module and a public opinion data storage module, in the workflow management module of the application, a unified data acquisition, data preprocessing, data calculation and data storage interface is designed by using high-level abstraction and a plurality of design patterns, covering the interconnection of different data sources, structured data and unstructured data, fuzzy query and structured accurate search. Algorithm prototypes are provided in each type of algorithm, and a formed analysis algorithm can be introduced at a low cost, and each item of system content makes a distributed and pluggable system design plan for software and hardware, and can provide efficient and accurate development and debugging in an agile development mode for business requirements that are not met.

[0036] Fourth, the application can be freely connected to the data analysis module of different heterogeneous data such as social network graph and social text data, and the analysis algorithm of each kind of data can be extended, and the conversion work between different data input and output can be completed with a small amount of work, which makes the application provide more flexible and rich analysis means than the existing system, for example, business personnel can find the group forwarding the same type of sensitive information, and further analyze the group portrait, group portrait, and geographical tracing of this part of the crowd. The application is aimed at this point, through the multi-level analysis module based on free connection, different analysis modules can be assigned to the data passing through different data analysis modules, thereby providing hierarchical and tagged results for business personnel, constructing a multi-level and multi-label classification system of analysis tasks, and starting a new task on the analysis result for further analysis. The application provides greater flexibility, scalability and ease of use than the existing system through the workflow model module, business personnel do not need to understand the composition of different types of data, they only need to freely connect different analysis modules to obtain the required multi-level analysis result, and the visual display method allows them to freely select different levels of analysis results, and the interactivity is stronger than the existing system.

[0037] Fifth, the application can track the public opinion of hot events on social media, discover potential factors of social instability early, ensure the social stability and safety of the country from the perspective of network information space security, and has important research significance; using the forwarding network relationship graph of social media, potential opinion leaders and public opinion fermentation communities are found, thereby assisting business personnel in accurately breaking the information cocoon house and grasping the core crowd of public opinion; a simple, efficient and expandable workflow system is provided, which has potential beyond public opinion analysis, can enable data analysis for business personnel in different industries on the basis of a small amount of development, improve their data insight, and better assist business implementation.

[0038] Other advantages, objects, and features of the application will be apparent from the following description, and will be understood by those skilled in the art. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The structure relationship diagram of the public opinion analysis workflow system in one embodiment of the application is shown in the figure.

[0040] Figure 2 The public opinion analysis workflow method flowchart in one embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0041] The application will be further described in detail below with reference to the accompanying drawings, so that those skilled in the art can implement it according to the description.

[0042] In the embodiment of the present application, an event is defined as a set of tasks contained for a specific analysis requirement, specifically, the event can refer to the input of a keyword, a label, a set of data sources within a given time range. A task is defined as a specific instantiation of a workflow, the task can belong to an event or be generated independently to meet the different needs of business personnel for overall event analysis and quick analysis of a specific content.

[0043] As shown in Figure 1 The embodiment of the present application provides an opinion analysis workflow system, which comprises: a data analysis function module comprising N data analysis modules capable of analyzing opinion data, N >= 2; a workflow establishment module which selects a plurality of data analysis modules from the data analysis function module according to opinion analysis requirements and connects them in sequence to establish a workflow capable of analyzing specific tasks, wherein, for opinion analysis requirements of different analysis angles of the same event, a plurality of workflows are established to analyze a plurality of opinion analysis tasks; a workflow management module for data analysis calculation of the established workflow and viewing the calculation results through a visual workflow diagram; and an event management module for managing a plurality of opinion analysis tasks of the same event and establishing the relationship between different tasks through opinion analysis data.

[0044] The data analysis function module comprises a community division module, a propagation trend analysis module, a text classifier calculation module, a sensitivity analysis module, an LDA (Latent Dirichlet Allocation) module, etc. When the above data analysis modules are selected and connected to establish a workflow, a free-dragging connection type generation can be used to shield the business personnel from the complex data input format, the sequence relationship and other details, thereby assisting the business personnel to quickly construct the opinion analysis flowchart in their minds. It should be noted that the types of data analysis modules in the data analysis function module are not limited to the several types listed in the embodiment, and can be specifically set according to requirements.

[0045] When a data analysis module is selected during the establishment of the workflow, the parameters of the data analysis module need to be set, and each data analysis module is provided with some parameters for data analysis. For example, when the community division module is selected, the algorithm for community division is set; when the sensitivity analysis module is selected, the threshold for sensitive information discrimination is set, and the higher the threshold, the more accurate the discrimination result.

[0046] In the above embodiment, since the data analysis function module includes multiple data analysis modules capable of public opinion data analysis, when establishing the workflow, multiple data analysis modules can be freely selected according to a specific public opinion analysis requirement, and are sequentially connected to form multiple-level analysis modules, so that the established workflow can complete a specific public opinion analysis task.

[0047] The embodiment can establish multiple workflows from different analysis angles for the same event, thereby analyzing multiple public opinion analysis tasks, and thereby obtaining public opinion information from different levels and different angles. The event management module can manage the multiple public opinion analysis tasks, and can also build relationships between different tasks through public opinion analysis data, that is, the public opinion analysis result obtained through a workflow calculation analysis can be used as input data of another workflow data acquisition module. For example, three workflows A, B and C are established, a public opinion analysis task is calculated by workflow A, the hierarchical public opinion analysis result obtained by workflow A can be used as input data of workflow B, and the hierarchical public opinion analysis result obtained by workflow B can also be used as input data of workflow C, thereby building relationships between different public opinion analysis tasks and obtaining public opinion analysis results from a deeper level. Therefore, the event management module realizes different data analysis methods for the same data source, and can further analyze the analysis result. In the prior art, an event management module for unified management of different tasks for a group of keywords is not constructed.

[0048] In the embodiment, the calculation result can be viewed through a visual workflow diagram. Since the workflow is a hierarchical analysis module, the embodiment can view the hierarchical calculation result through the visual workflow diagram.

[0049] In one specific embodiment, the public opinion analysis workflow system, the workflow establishment module further includes a workflow model module, which is a preset workflow model stored in a workflow model database. When one of the workflow models is selected, the data analysis modules in the workflow model can be modified according to the public opinion analysis requirement.

[0050] In the above embodiment, when one of the workflow models is selected, the front-back relationship and specific parameters between different data analysis modules can be visually adjusted, so that business personnel can easily build a hierarchical and diversified workflow.

[0051] In one specific embodiment, the public opinion analysis workflow system, the workflow management module specifically includes:

[0052] An opinion data acquisition module for acquiring different heterogeneous data sources, the opinion data acquisition module comprising: a data acquisition interface module for connecting social media networks or connecting databases storing social media data; a data query module capable of fuzzy query and structured precise search, wherein the opinion data acquisition module is connected with the first data analysis module in the established workflow;

[0053] An opinion data preprocessing module for constructing a forwarding network relationship and preprocessing social network text on the acquired opinion data;

[0054] An opinion data calculation module comprising: a data analysis algorithm module comprising various data analysis algorithms; an algorithm classification module for classifying various data analysis algorithms into two types of analysis algorithms, namely graph and text; an algorithm switching module for switching the data analysis algorithms in the connected data analysis modules in the established workflow in a hierarchical nested mode.

[0055] In the above embodiment, the opinion data acquisition module designs a unified data acquisition interface through high-level abstraction, including the acquisition methods of various heterogeneous data sources, covering structured data and unstructured data. Therefore, the opinion data acquisition module can acquire different heterogeneous data sources, and the data acquisition module can directly acquire opinion data from social media networks and also can acquire opinion data from databases storing social media data. Retrieval is required when acquiring data, and the data query module can perform fuzzy query and structured precise search. Since the opinion data acquisition module is connected with the first data analysis module in the established workflow, it provides data input for the workflow. The parameters required to be set by the opinion data acquisition module are keywords and time range concerned by opinion analysis. Therefore, according to the keywords and time range, the original microblog, tweet and other social media text, time, IP address and other types of data can be obtained from unstructured databases through fuzzy search. According to the predefined storage process, structured data such as calculation results, workflow templates and task parameters can be acquired from structured databases through structured query.

[0056] The public opinion data preprocessing module can carry the input of different heterogeneous data sources in the public opinion data acquisition module through effective interface design, and provides a variety of effective data preprocessing modes. According to the forwarding relationship, IP relationship and nickname of the social media network, the forwarding network relationship is established, which provides a solid data foundation for further public opinion data analysis such as identification of social media community relationship, community detection and KOL discovery; according to regular expressions, stop word tables, social media special word tables and other data judgment, combined with word segmentation means, the social media text is segmented, de-duplicated, de-stopped, de-specialized, de-punctuation and other different operation modes, and can be freely selected, and the social network text is preprocessed. Through the efficient algorithm switching module, the front and rear sequence data analysis algorithms are executed in a hierarchical nested mode, such as "community 1-> topic 1-> positive emotion", so as to show the different levels and different dimensions of the data to the business personnel, thereby assisting the business personnel to accurately control the public opinion from coarse granularity to fine granularity. Compared with the existing system, the public opinion data calculation module provides input and output conversion between heterogeneous data sources through high degree of freedom and high expansibility of free connection, and gives the source data multiple levels and multiple types of labels, which is different from the single data source (such as text) analysis mode of the existing system, and provides more possibilities for business personnel.

[0057] In one embodiment, the public opinion analysis workflow system further comprises a public opinion data storage module which stores data through hierarchical hash index.

[0058] In the above embodiment, the data storage module innovatively constructs a hierarchical hash index of data calculation results, so that the hierarchical calculation results can be efficiently extracted from the storage database and fed back to the front-end visualization workflow diagram for real-time rendering, while taking into account the access efficiency, greatly saving the storage resources, and playing an important role in real-time display of massive data. This part is consistent with the multi-level and multi-type results of data calculation, and the structured storage method of the existing system. The public opinion data storage module of the embodiment avoids the difficulty of maintaining tree structure through hierarchical hash index, retains the hierarchical characteristics and efficient search of tree structure, and uses the combination of unstructured index and structured query to achieve faster and more accurate search than the existing system with more filtering conditions.

[0059] Overall, the workflow management module divides the process of public opinion data analysis into the four modules of the public opinion data acquisition module, the public opinion data preprocessing module, the public opinion data calculation module and the public opinion data storage module by splitting and high-level abstraction of the process of public opinion data analysis, and defines the basic architecture of different data analysis modules, so as to facilitate the coupling between multiple different algorithms and provide extensibility for future development. The embodiment of the application constructs an extensible free connection public opinion analysis workflow system taking tasks as the main calculation mode and events as the main analysis feature by taking the workflow as the core and the foundation.

[0060] The embodiment of the application also provides a public opinion analysis workflow method applied to the public opinion analysis workflow system. Figure 2 As shown in the figure, the method comprises the following steps.

[0061] Step 1: selecting multiple data analysis modules from the data analysis function modules according to the public opinion analysis requirements, connecting them in sequence, establishing a workflow, or selecting a workflow model from the workflow model module and modifying the data analysis modules, wherein the parameters of each data analysis module in the workflow need to be set;

[0062] Step 2: for the public opinion analysis requirements of different analysis angles of the same event, multiple workflows are established.

[0063] Step 3: using the social media data processed by the public opinion data preprocessing module in the workflow management module to instantiate the workflow, generating a calculation task, and analyzing the public opinion, wherein an event is selected from the event library or newly created, and the calculation task is attributed to the event.

[0064] Step 4: completing data calculation according to the calculation process, and viewing the calculation results of each data analysis module through the visual workflow diagram.

[0065] The embodiment of the application also provides an electronic device, comprising at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the public opinion analysis workflow system.

[0066] The embodiment of the application also provides a storage medium having a computer program stored thereon, and the program is executed by a processor to implement the public opinion analysis workflow system.

[0067] The following will be described through a specific example.

[0068] Preceding step S0, constructing a customized workflow, the user can select the data analysis module provided by the system, customize the specific process, and establish the workflow of the specific analysis sequence. For example, the specific construction process of the constructed workflow "community division → topic model text clustering → text sentiment classification → sensitive information identification" is as follows:

[0069] 1. Select the public opinion data acquisition module, set the key words and time range concerned in this analysis.

[0070] 2. Select the community division module, set the algorithm for completing community division. Connect the public opinion data acquisition module and the community division module.

[0071] 3. Select the topic model text clustering module, set the number of topics to be calculated and returned, and the related parameters in the topic calculation inference. Connect the community division module and the topic model text clustering module.

[0072] 4. Select the text sentiment classification module, select the classification method used. Connect the topic model text clustering module and the text sentiment classification module.

[0073] 5. Select the sensitive information identification module, set the sensitive information discrimination threshold (the higher the threshold, the more accurate the discrimination result). Connect the text sentiment classification module and the sensitive information module.

[0074] At this point, the creation of an example workflow is completed, and the following will execute this workflow.

[0075] Step S1, from the microblog stream database, according to the key words and time range set in the public opinion data acquisition module of the customized workflow, the selected number of analysis texts K, filter in the microblog stream database according to the above conditions, and obtain the K microblogs most relevant to the key words;

[0076] Elasticsearch is used as the stream database. Elasticsearch is a distributed, highly scalable, and highly real-time search and data analysis engine. It can easily enable a large amount of data to have search, analysis, and exploration capabilities. In the system of the present application, the microblog data obtained through the microblog platform API interface can be submitted to the Elasticsearch database, and the corresponding sentences can be segmented by the segmentation controller. The weight and segmentation results are stored together, and when retrieval is needed, the results are ranked and scored according to the weight, and the returned results are sent to step S2 for processing.

[0077] Step S2, according to the microblog texts related to the set key words and time range obtained in step S1, construct a microblog forwarding network; according to the related microblog texts, obtain the text representation of the microblog after text cleaning, so as to facilitate direct use of data analysis in the following steps.

[0078] The construction of the microblog forwarding network is mainly completed by adding forwarding relationships. For example, the microblog data stream has a "root_mid" field, which represents the ID of the original microblog forwarded by the microblog. In this way, the forwarding relationship of "this microblog -> original microblog" can be added in the microblog forwarding network. It should be noted that in actual situations, a large number of microblog texts contain long forwarding chains, such as "xxx / / @B:xxx / / @C:xxx / / @D:xxx / / @X". In this case, if the forwarding relationship of "this microblog -> original microblog" is directly added, the microblog forwarding network constructed finally will be biased, thereby affecting the accuracy of subsequent algorithms. Based on this, the system constructs a username matching algorithm in the forwarding chain. Through the algorithm, the related usernames in the long chain are obtained, the user ID is matched according to the username, the corresponding original microblog content is searched, and the forwarding relationship is added into the microblog forwarding network. Therefore, the final forwarding network will be added with the forwarding relationship of "this microblog -> B -> C -> D -> original microblog".

[0079] The node size of the microblog forwarding network. In order to consider the efficiency of calculation, the system of the application selects the open source graph computing toolkit Networkit. The toolkit is written based on C++ language, and it is an open source toolkit for large-scale network analysis that is constantly developing and growing. It realizes efficient graph algorithms, many of which are parallel, because they can well utilize the multi-core architecture for network analysis tasks. These standard metrics for network analysis, such as degree sequence, clustering coefficient and centrality measure. The system uses the graph class defined in the Networkit toolkit to store the forwarding network graph of the microblog.

[0080] After the construction of the forwarding network is completed, the preprocessing of the microblog text is performed.

[0081] Generally, there are three types of microblogs: original, forwarding and comment. Obviously, the forwarding microblog may contain text content published by a user other than the microblog user. In other words, it can be understood that part of the text does not represent the intention of the microblog publisher, and may be far from the emotion or opinion expressed in the forwarded text, or even the opposite.

[0082] Specifically, the forwarding microblog generally contains the content written by the user after forwarding and the original microblog content, for example, "I think we should support / / @Shaping certain user: I really don't know what this is to come to each other's way". These contents unrelated to the user's published text are likely to reduce the accuracy of subsequent sensitive information identification, text sentiment classification and text topic model. Therefore, before obtaining the feature representation of the microblog text, it is necessary to remove the content unrelated to the user's published text in the forwarding and comment microblog using regular expressions.

[0083] Step S3, according to the microblog text and microblog forwarding network graph processed in step S2, the user-defined workflow is instantiated and calculated, and the results of each step are generated for the user to view and reference;

[0084] We take the workflow defined around the microblog text related to specific instances, "community division → topic model text clustering → text sentiment classification → sensitive information identification" as an example.

[0085] 1. Through the community division task, the selected N microblog texts are divided into K communities (K≤N), and the microblog forwarding network is drawn on the display stand. The nodes in different communities are rendered using different colors. Move the mouse cursor over the node to display the ID and original text content of the microblog.

[0086] 2. Through the topic model text clustering task, the K communities divided in the previous task can be calculated by topic model. Set T topics, and finally obtain (K×T) topic pairs. The user can select a community to view the topics of its members' speeches, which can roughly understand the views and tendencies of the community. Special attention should be paid to the text content of the community center node.

[0087] 3. Through the text sentiment classification task, the K community T topics, i.e. K×T text subsets, are classified by sentiment polarity. The user can select "community k-topic t" to view the sentiment tendency of the text under a specific discussion topic in a specific community. The system exhibit will also show the proportion of positive and negative emotions.

[0088] 4. Through the sensitive information identification task, K communities, T topics, and 2 emotions, i.e. K×T×2 texts, are identified for sensitive information. The user can select "community k-topic t-positive" to view whether the text expressing positive emotions in a specific community under the tth topic involves sensitive information. The system exhibit will display the text ID and text content involved in and not involved in sensitive information, giving the user sufficient reference.

[0089] The specific algorithm modules involved in the above workflow execution process are described as follows:

[0090] The first task, community division, uses the microblog forwarding network processed in step S2 and relies on the FastUnfolding algorithm to complete the community division work:

[0091] This algorithm can be roughly divided into two stages and can be iterated until convergence. Suppose the microblog forwarding network returned in step S2 has N communities.

[0092] 1. Assign a community index to each node, so there are N communities in the network at this time. This is the initial state

[0093] 2. For each node i, consider its adjacent node j; let i's community become j's, and see how this action affects the value of modularity. If the change ΔQ is positive, accept the change, otherwise keep the original allocation.

[0094] 3. Repeat the above process until the community division result converges.

[0095] Where ΔQ represents the impact of microblog node i joining the community C on the overall modularity value, and the calculation method is as follows:

[0096]

[0097] Using the Networkit open source toolkit to implement this algorithm, the community division of 100,000 microblog nodes can be completed in 0.14 seconds, which can accurately and efficiently complete the task.

[0098] The second block task is topic model text clustering. The system uses the LDA model to complete this task, and uses the jGibbLDA open source toolkit to implement it. Based on the microblog text data completed by the word segmentation in step S2, the following modeling is performed:

[0099] Assuming that the topic distribution of each document i is θ i The word distribution of word j in document i obeys a multinomial distribution, and the word distribution of word j in document i Finally, the word w i，j , can be written as follows:

[0100]

[0101] Based on this, the maximum likelihood estimation of the word distribution in a document can be given:

[0102]

[0103] Then the parameters in the model can be estimated by Gibbs sampling. Therefore, the topic distribution of each document and the word distribution of each topic can be obtained. The system takes the highest probability topic in each document as the topic of the document, and clusters the documents according to different topics. At the same time, the top 100 high-frequency words in the topic are selected as representative words of the topic for interface display.

[0104] The third block task is a text sentiment classification task, since the data returned by step S2 is mostly short text, a sentiment dictionary and a BERT-based fine-tuning classification model voting method are used to complete the text sentiment classification task.

[0105] The sentiment dictionary is mainly for words expressing affirmation or negation and words related to emotions, and only the field of Chinese grammar is used to determine the sentiment polarity of the text. This method can have relatively high classification accuracy for relatively straightforward text, but it cannot well judge text with deep meanings, such as common sarcastic semantics. Therefore, the system of the present application additionally uses 200,000 labeled microblog texts to train a deep sentiment classification model with fine-tuned representation layer parameters. After voting based on the discrimination results of the two models, the final sentiment discrimination value is returned.

[0106] The fourth block task is a text sensitive information identification task, according to the data returned by step S2, the deep semantic representation of the microblog text is obtained through the pre-trained Chinese language model ERNIE, and then based on the open source tool library Faiss, the vector index is constructed according to the selected sensitive seed and normal seed text, and then the task of sensitive information identification can be realized. Finally, the sensitive information filtered out will be displayed in the workbench.

[0107] Step S4, based on the analysis purpose of practical examples, the related analysis of specific examples can classify the above calculated tasks into instance events, if the event has not been created, a new instance event can be created using the task workbench function to classify the task. The user can view the calculation results of the completed tasks in the event.

[0108] Although the embodiments of the present application have been disclosed as above, it is not limited to the application listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present application. Additional modifications can be easily realized by those skilled in the art. Therefore, the present application is not limited to specific details and the figures shown and described herein, without departing from the general concept defined by the claims and the equivalent scope.

Claims

1. An opinion analysis workflow system, characterized by, Comprise: Data analysis function module, it includes N data analysis module that can carry out public opinion data analysis, N >= 2; Workflow establishment module, according to the public opinion analysis demand from the data analysis function module Multiple data analysis modules are selected and connected in order to establish a workflow that can analyze specific tasks, wherein multiple workflows are established for different analysis angles of the same event Public opinion analysis requirements, so as to be able to analyze multiple public opinion analysis tasks; Workflow management module for data analysis and calculation of the established workflow, and view the calculation results through the visual workflow diagram; Event management module for managing multiple public opinion analysis tasks for the same event, and building connections between different tasks through public opinion analysis data; Wherein, the workflow management module specifically includes: Public opinion data acquisition module for acquiring different heterogeneous data sources, the public opinion data acquisition module includes: data acquisition interface module for connecting social media networks or connecting databases storing social media data; Data query module can carry out fuzzy query and structured accurate search, wherein the public opinion data acquisition module is connected with the first data analysis module in the established workflow; Public opinion data preprocessing module for constructing forwarding network relationship and preprocessing social network text for the acquired public opinion data; Public opinion data calculation module, including: Data analysis algorithm module, including various data analysis algorithms; Algorithm classification module for classifying various data analysis algorithms into two types of analysis algorithms, graph and text; Algorithm switching module for data analysis algorithms in the connected data analysis modules in the established workflow in a hierarchical nested mode; When establishing a workflow, the parameters of the data analysis module need to be set after selecting a data analysis module;The data analysis function module includes community division module, propagation trend analysis module, text classifier calculation module, sensitivity analysis module and LDA module.

2. The sentiment analysis workflow system of claim 1, wherein, The workflow establishment module also includes a workflow model module, which is a pre-set workflow model stored in a workflow model database, wherein when selecting one of the workflow models, the data analysis modules in the workflow model can be modified according to the public opinion analysis requirements.

3. The sentiment analysis workflow system of claim 1, wherein, The workflow management module also includes a public opinion data storage module, which is stored in the form of hierarchical hash index.

4. The public opinion analysis workflow system of claim 1, wherein: The constructed forwarding network relationship specifically includes: establishing forwarding network relationship according to the forwarding relationship of social media network, IP relationship and nickname; The preprocessing of social network text specifically includes: data judgment according to regular expression, stop word table, social media special word table, combined with word segmentation means, social media text is segmented, de-duplication, de-stop word, de-special language, and de-punctuation operation.

5. The sentiment analysis workflow system of claim 1, wherein, The public opinion data acquisition module needs to set the keywords and time range concerned by the public opinion analysis.

6. The public opinion analysis workflow method applied to the public opinion analysis workflow system of any one of claims 1-5, characterized in that, The method comprises: According to the public opinion analysis requirement, multiple data analysis modules are selected from the data analysis function module and connected in sequence to establish a workflow, or a workflow model is selected from the workflow model module and the data analysis modules are modified, wherein parameters of each data analysis module in the workflow are set; For public opinion analysis requirements of different analysis angles of the same event, multiple workflows are established; Social media data processed by the public opinion data preprocessing module in the workflow management module is used to instantiate the workflow, a calculation task is generated, and public opinion is analyzed, wherein an event is selected from an event library or newly created, and the calculation task is attributed to the event; According to the calculation process, data calculation is completed, and the calculation results of each data analysis module are viewed through a visual workflow diagram.

7. An electronic device, comprising: The system comprises at least one processor and a memory connected in communication with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of claims 1-5. The program is executed by the processor to implement the system of any one of claims 1-5.

8. A storage medium having stored thereon a computer program, characterized in that ​

Images