A network public opinion service intelligent diagnosis system
The intelligent diagnostic system for online public opinion, utilizing deep learning and collaborative filtering algorithms, enables rapid identification and processing of online public opinion, solving the problem of low efficiency in existing public opinion processing technologies and improving the efficiency and quality of public opinion management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA TELECOM DIGITAL INTELLIGENCE TECH CO LTD
- Filing Date
- 2023-09-22
- Publication Date
- 2026-06-05
AI Technical Summary
In the current technology, online public opinion handling personnel can only solve a few public opinion problems after short-term training, resulting in low processing efficiency and difficulty in coping with the complex and ever-changing online public opinion life cycle.
Design an intelligent diagnostic system for online public opinion monitoring. The system uses a multi-mode service device for data preprocessing, feature extraction, model matching, and recommendation. It utilizes deep learning and collaborative filtering algorithms to identify public opinion types in real time and provide solutions.
It improves the efficiency and quality of public opinion handling, can quickly identify and recommend suitable handling methods, is applicable to various public opinion scenarios, and enhances the application effect of public opinion management.
Smart Images

Figure CN117194796B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of Internet technology, and in particular to an intelligent diagnostic system for online public opinion. Background Technology
[0002] The life cycle of online public opinion has two meanings: the development stage of public opinion and the development cycle of public opinion.
[0003] As time progresses and events unfold, public opinion generally goes through five stages: the occurrence stage, the development stage, the peak stage, the fluctuation stage, and the decline stage, each with its own certain regularity. The development cycle of public opinion can be roughly divided into the incubation period, the formation period, the fluctuation period, and the decline period.
[0004] With the increasing popularity of the Internet, the volume of online public opinion services is also constantly rising. Different clients often have different problems with different public opinion products, and public opinion handling personnel who have only received short-term training can often only solve one or a few problems caused by public opinion, resulting in low processing efficiency. Summary of the Invention
[0005] The purpose of this invention is to at least address one of the aforementioned technical deficiencies.
[0006] Therefore, the purpose of this invention is to propose an intelligent diagnostic system for online public opinion services, which can match corresponding public opinion processing data for users based on the public opinion issues raised by users, and recommend public opinion processing methods suitable for users, so that public opinion managers can make selective recommendations during communication.
[0007] To achieve the above objectives, embodiments of the present invention provide an intelligent diagnostic system for online public opinion services, comprising: a multi-mode service device, an input device, and an output device, wherein the multi-mode service device is connected to the input device and the output device respectively;
[0008] The input device is used to input data into the multi-mode service device through manual import and batch database import methods;
[0009] The multi-modal service device includes: a data preprocessing module, a database module, an ETL module, a processing module, and a result output module, wherein...
[0010] The input end of the data preprocessing module is connected to the output end of the input device, and is used to receive data from the input device, and send the preprocessed input data to the ETL module and the database module respectively.
[0011] The input end of the database module is connected to the output end of the data preprocessing module, and the output end of the database module is connected to the input end of the processing module. It stores data for training the short text classification model, including: rule table, similar data and public opinion data.
[0012] The input of the ETL module is connected to the output of the data preprocessing module, and is used to extract and transform the preprocessed data for data feature extraction.
[0013] The input of the processing module is connected to the output of the ETL module, and is used to perform feature rule model matching, similar data analysis, and public opinion data recommendation. This includes: performing feature rule model matching on the extracted data; training a short text classification model to obtain a scene recognition model; inputting the data to be identified into the trained scene recognition model for scene recognition; and identifying the public opinion type result in real time. It also involves performing similarity analysis of single text data against historical data and outputting similar historical text information, which is used to provide reference for public opinion management personnel in business operations and processing status. Finally, based on single text data and public opinion data...
[0014] Generate public opinion data recommendation results;
[0015] The input end of the result processing module is connected to the output end of the processing module, and is used to output the processed public opinion type results, similar historical text information and public opinion recommendation results to the output device;
[0016] The output device outputs the received public opinion type results, similar historical text information, and public opinion recommendation results in JSON format and provides them to public opinion management personnel.
[0017] Furthermore, the input device includes a manual import module and a Kafka module, which are respectively connected to the data preprocessing module.
[0018] The manual import module is used to output the manually imported data to the database module after preprocessing by the data preprocessing module.
[0019] The Kafka module is used to import data in batches and output it to the ETL module after preprocessing by the data preprocessing module.
[0020] Furthermore, the data preprocessing module includes: an import preprocessing unit and a Flink unit, wherein,
[0021] The import preprocessing unit is connected to the manual import module and is used to preprocess the manually imported data to obtain public opinion data, which is then sent to the database module.
[0022] The Flink unit is connected to the Kafka module and is used to process the imported data in batches using a pipeline approach, and then send it to the ETL module.
[0023] Furthermore, the ETL module performs data feature extraction, including: extracting data features from short text data under a unified knowledge graph through syntactic analysis, Chinese word segmentation, and proper noun extraction according to preset feature extraction rules.
[0024] Furthermore, the processing module calculates the cross-entropy between the training results of the short text classification model and the pre-labeled standard classification results, calculates the average Euclidean distance and uses it as the loss value, and then feeds it back to their respective neural networks for repeated training until the model converges, finally obtaining a complete scene recognition model. By comparing the data with the preset feature rules and feature models, different models are selected for classification to obtain the public opinion type result.
[0025] Furthermore, the feature rules include: hot information rules, sensitive information rules, and biased information rules; the feature models include: hot information model, sensitive information model, and biased model.
[0026] Furthermore, the hotspot information is obtained using an unsupervised clustering algorithm, while the sensitive information and bias information are obtained using a supervised algorithm.
[0027] Furthermore, the processing module uses the Word2Vec model to perform similarity analysis of historical data on a single text data entry.
[0028] Furthermore, the processing module employs a collaborative filtering recommendation algorithm to analyze individual text data and public opinion data in order to achieve public opinion data recommendation.
[0029] Furthermore, the input device also includes: an HTTP request module, used to provide a query interface for user public opinion management;
[0030] The multi-mode service device further includes an Nginx module and a Flask module. The Nginx module is used to process and respond to multiple HTTP requests from the HTTP request module simultaneously. The Flask module responds to multiple HTTP request data sent by the Nginx module at once through the Werkzeug function library.
[0031] The intelligent diagnostic system for online public opinion services according to embodiments of the present invention has the following beneficial effects:
[0032] (1) Classify public opinion information on the Internet according to the client's public opinion business needs, match the classified information with the corresponding public opinion handling personnel, identify the client's public opinion business problems, and send the corresponding solutions or handling suggestions to the public opinion handling personnel in real time, thereby improving the efficiency of public opinion response.
[0033] (2) The Word2Vec model is used to perform similarity analysis of historical data on a single text data to provide a reference for public opinion managers to conduct business and processing status.
[0034] (3) By setting up a collaborative filtering recommendation algorithm, the system can match the user with corresponding public opinion processing data based on the public opinion issues raised by the user, so as to recommend a suitable public opinion processing method for the user, which is convenient for public opinion management personnel to make selective recommendations during communication.
[0035] (4) It can significantly improve the speed and quality of users' public opinion processing, improve the application effect of public opinion management, and has broad application prospects and huge market demand.
[0036] (5) It is widely used in various public opinion scenarios and is suitable for rapid handling of sudden public opinion events.
[0037] (6) It can be applied to China Telecom’s integrated management products or applications such as Tianyi Cloud and self-developed building platform, thereby enhancing the core competitiveness of the products and promoting the expansion of China Telecom’s cloud business and government and enterprise customer market.
[0038] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0039] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:
[0040] Figure 1 This is a structural diagram of the intelligent diagnostic system for online public opinion services according to an embodiment of the present invention;
[0041] Figure 2 This is a schematic diagram of an intelligent diagnostic system for online public opinion services according to an embodiment of the present invention;
[0042] Figure 3 This is a structural block diagram of ETL processing in the intelligent diagnostic system for online public opinion services according to an embodiment of the present invention;
[0043] Figure 4 This is a flowchart of the processing module in the intelligent diagnostic system for online public opinion services according to an embodiment of the present invention.
[0044] Figure 5 This is a flowchart of the intelligent diagnostic system for online public opinion services according to an embodiment of the present invention. Detailed Implementation
[0045] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0046] like Figure 1 As shown, the intelligent diagnostic system for online public opinion services according to an embodiment of the present invention includes: a multi-mode service device 200, an input device 100, and an output device 300. The multi-mode service device 200 is connected to both the input device 100 and the output device 300.
[0047] Specifically, the input device 100 is used to input data into the multi-mode service device 200 through manual import and database batch import methods.
[0048] refer to Figure 2 The input device 100 includes a manual import module 120 and a Kafka module 110, which are respectively connected to the data preprocessing module 210.
[0049] The manual import module 120 is used to output the preprocessed data from the manually imported data to the database module 220 after preprocessing by the data preprocessing module 210.
[0050] Kafka module 110 connects to the public opinion information collection system to input data into the database in real time in batches, and outputs the preprocessed data to ETL module 230 after preprocessing by data preprocessing module 210. Kafka is an open-source stream processing platform developed by the Apache Software Foundation and written in Scala and Java.
[0051] The multi-mode service device 200 includes: a data preprocessing module 210, a database module 220, an ETL module 230, a processing module 240, and a result output module 250.
[0052] The input terminal of the data preprocessing module 210 is connected to the output terminal of the input device 100, and is used to receive data from the input device 100 and send the preprocessed input data to the ETL module 230 and the database module 220 respectively.
[0053] Data preprocessing module 210 includes: import preprocessing unit 212 and Filter unit 211.
[0054] The import preprocessing unit 212 is connected to the manual import module 120 and is used to preprocess the manually imported data to obtain public opinion data, which is then sent to the database module 220.
[0055] Flink unit 211 connects to Kafka module 110 and is used to batch process data imported into Kafka module 110 in a pipelined manner, then send the processed data to ETL module 230. Flink is an open-source stream processing framework developed by the Apache Software Foundation. Its core is a distributed streaming data stream engine written in Java and Scala. Flink executes arbitrary streaming data programs in a data-parallel and pipelined manner. Flink's pipelined runtime system can execute both batch and stream processing programs. After execution, Flink programs are mapped to streaming data streams. Each Flink data stream begins with one or more sources (data inputs, such as message queues or file systems) and ends with one or more receivers (data outputs, such as message queues, file systems, or databases). Furthermore, Flink's runtime itself also supports the execution of iterative algorithms.
[0056] The input of database module 220 is connected to the output of data preprocessing module 210, and the output of database module 220 is connected to the input of processing module 240. It stores data used to train the short text classification model, including: rule tables, similarity data, and public opinion data. Database module 220 generates public opinion data based on the data from import preprocessing unit 212. (Reference) Figure 2 As shown, the rule table and similar data are pre-stored in the database, while the public opinion data is obtained by importing the preprocessing unit 212 to process the input data. Figure 2 The public opinion database in the system is used to store public opinion information that is currently of concern.
[0057] The input of ETL module 230 is connected to the output of data preprocessing module 210, and is used to extract and transform the preprocessed data for data feature extraction.
[0058] Specifically, such as Figure 3 As shown, the ETL module 230 performs data feature extraction, including: extracting data features from short text data under a unified knowledge graph through syntactic analysis, Chinese word segmentation, and proper noun extraction according to preset feature extraction rules.
[0059] In embodiments of the present invention, common features include word frequency, part-of-speech tagging, and named entities. Syntactic analysis is primarily performed using HanLP, Chinese word segmentation is primarily performed using Jieba, and proper nouns are primarily processed using regular expressions.
[0060] The input of processing module 240 is connected to the output of ETL module 230, and is used to perform feature rule model matching, similar data analysis, and public opinion data recommendation. (See reference...) Figure 4 As shown.
[0061] (1) Feature rule model matching
[0062] After feature extraction, the data undergoes feature rule model matching to train a short text classification model, resulting in a scene recognition model. The data to be identified is then input into the trained scene recognition model for scene identification, providing real-time results for identifying the type of public opinion. In other words, by utilizing deep learning neural networks, data features from various types of information can be automatically acquired, and model matching can be performed on these features. The feature rule matching process involves selecting and processing different models based on the business requirements for the feature data, and providing diagnostic results based on the load and degree of bias to quickly and accurately determine the type of public opinion issue.
[0063] Specifically, during the feature rule model matching process of the data after feature extraction by the ETL module 230, the processing module 240 compares the data with preset feature rules and feature models. The processing module 240 is the core of this invention; it classifies the feature data identified by the ETL module 230 according to business (hotspot information, sensitive information, and trend information) and selects different models. After model selection, a diagnostic result is given based on the degree of similarity between the classification models.
[0064] In embodiments of the present invention, the feature rules include: hot information rules, sensitive information rules, and biased information rules; the feature models include: hot information model, sensitive information model, and biased model.
[0065] Hot topics information is analyzed using an unsupervised clustering algorithm. The specific steps include hot word generation, abstract extraction, consultation clustering, and manual screening. The core algorithms used include TF-IDF, LDA, and TextRank.
[0066] Sensitive and biased information were classified using supervised algorithms, both of which are multi-text classification algorithms.
[0067] Sensitive information can be divided into 5 categories (according to GB / T 22240-2020 "Information Technology Security Network Security Level Protection Classification Guide", it can be divided into 1-National Security / Public Rights / Personal Privacy / Corporate Legitimate Rights, 2-Personal Privacy / Corporate Legitimate Rights, 3-Personal Privacy / Corporate Legitimate Rights / Public Rights, 4-Personal Privacy / Corporate Legitimate Rights / Public Rights, 5-Public Rights / National Security), and biased information is divided into three categories (positive, negative, and neutral).
[0068] The processing module 240 calculates the cross-entropy between the training results of the short text classification model and the pre-labeled standard classification results, calculates the average Euclidean distance and uses it as the loss value, and then feeds it back to their respective neural networks for repeated training until the model converges, finally obtaining a complete scene recognition model. The data is compared with the feature model through preset feature rules to select different models for classification and obtain the public opinion type result.
[0069] Before training a classification model, a dataset needs to be prepared and labeled. Then, machine learning algorithms such as Naive Bayes and Support Vector Machines can be used to train the model, or a deep learning model with higher classification accuracy can be used to obtain a classification model. After obtaining the classification model, it is used to monitor sensitive and biased information in new text data. Specifically, new text data can be input into the model for classification and filtering to obtain the recognition results.
[0070] In embodiments of the present invention, deep learning models with higher classification accuracy can be TextCNN, TextRNN, and BERT language models, etc.
[0071] (2) Similar data analysis
[0072] This process involves performing a similarity analysis on individual text data within historical data sets, and outputting similar historical text information. This information is provided to public opinion management personnel for reference in business operations and processing status. It should be noted that the individual text data used in this similarity analysis is "data to be identified."
[0073] In an embodiment of the present invention, the processing module 240 employs the Word2Vec model to perform similarity analysis of a single text data entry within historical data. Word2Vec can be efficiently trained on dictionaries of millions and datasets of hundreds of millions. Furthermore, the training results obtained by this tool—word embeddings—can effectively measure the similarity between words.
[0074] (3) Public opinion data recommendation
[0075] Based on individual text data and public opinion data, a public opinion data recommendation result is generated, suggesting possible solutions to public opinion issues so that public opinion managers can make selective recommendations during communication. It should be noted that the individual text data recommended here is "data to be identified".
[0076] In an embodiment of the present invention, the processing module 240 uses a collaborative filtering recommendation algorithm to analyze single text data and public opinion data in order to achieve public opinion data recommendation.
[0077] Collaborative filtering, simply put, uses the preferences of a group of like-minded individuals with shared experiences to recommend information that might interest a user. Collaborative filtering recommendation is rapidly becoming a popular technique in information filtering and information systems. Unlike traditional content filtering, which directly analyzes content for recommendations, collaborative filtering analyzes user interests, finds similar users (with shared interests) within a user group, and synthesizes their evaluations of specific information to predict the system's likelihood of that user's preference for that particular information.
[0078] The input end of the result processing module 240 is connected to the output end of the processing module 240, and is used to output the processed public opinion type results, similar historical text information and public opinion recommendation results to the output device 300.
[0079] The output device 300 outputs the received public opinion type results, similar historical text information, and public opinion recommendation results in JSON format and provides them to public opinion managers, thereby providing them with historical solutions and processing suggestions and improving their service efficiency.
[0080] In addition, the input device 100 also includes an HTTP request module 130, which provides a query interface for user public opinion management.
[0081] The multi-mode service device 200 also includes: Nginx module 260 and Flask module 270.
[0082] Nginx module 260 handles simultaneous responses to multiple HTTP requests from HTTP request module 130. Nginx is a powerful, high-performance web and reverse proxy service with many superior features: it's a good alternative to Apache in high-concurrency scenarios; it can support up to 50,000 concurrent connections; it functions as a load balancer; and it can directly support Rails and PHP programs internally, as well as act as an HTTP proxy. Written in C, Nginx is significantly more efficient than Perlbal in terms of system resource consumption and CPU usage.
[0083] Flask module 270 responds to multiple HTTP requests sent by Nginx module 260 in one go using the Werkzeug function library.
[0084] Flask is a popular web framework that uses the Python programming language to implement its functionality. A key feature of Flask is its relatively simple core structure, coupled with strong extensibility and compatibility. Flask primarily consists of two core libraries: Werkzeug and Jinja2, which handle business logic and security functions, respectively. The Werkzeug library supports URL routing request integration, allowing it to respond to multiple user requests simultaneously; it supports cookies and session management, establishing persistent connections through identity caching and improving user access speed; it supports interactive JavaScript debugging, enhancing the user experience; and it can handle basic HTTP transactions, quickly responding to client-pushed requests. The Jinja2 library supports automatic HTML redirection, effectively controlling script attacks from external hackers.
[0085] The following is combined Figure 5 The processing flow of the intelligent diagnostic system for online public opinion services according to an embodiment of the present invention will be described.
[0086] S1. Perform Chinese word segmentation on the short text;
[0087] S2. Input the short text segments and corresponding tags of different businesses into their respective transformer models for training;
[0088] S3. Extract features from short texts;
[0089] S4. Train the short text classification model;
[0090] S5. Calculate the cross-entropy between the output in S4 and the pre-labeled standard classification results, calculate the average Euclidean distance and use it as the loss value, and then feed it back to their respective neural networks. Repeat the training until the model converges and finally obtains a complete scene recognition model.
[0091] S6. Input the short text to be identified into the trained scene recognition model, perform scene recognition, and provide the results in real time directly to the public opinion processing personnel for business processing.
[0092] In summary, the intelligent public opinion diagnosis system provided by this invention utilizes deep learning neural networks to automatically acquire data features from short texts sent by users and perform model matching on these features to quickly and accurately determine the type of public opinion issue the user is experiencing. Based on this, the system then directs the user to the appropriate public opinion management personnel for assistance. After model matching, the system can provide historical solutions and processing suggestions to public opinion management personnel based on the issue, thereby improving the service efficiency of public opinion management personnel.
[0093] It should be noted that the public opinion business intelligent diagnosis system of the present invention supports the processing of short text data. If it is a long text, it can be processed into short text data through abstract extraction and other methods before applying the public opinion business intelligent diagnosis system of the present invention.
[0094] The intelligent diagnostic system for online public opinion services according to embodiments of the present invention has the following beneficial effects:
[0095] (1) Classify public opinion information on the Internet according to the client's public opinion business needs, match the classified information with the corresponding public opinion handling personnel, identify the client's public opinion business problems, and send the corresponding solutions or handling suggestions to the public opinion handling personnel in real time, thereby improving the efficiency of public opinion response.
[0096] (2) The Word2Vec model is used to perform similarity analysis of historical data on a single text data to provide a reference for public opinion managers to conduct business and processing status.
[0097] (3) By setting up a collaborative filtering recommendation algorithm, the system can match the user with corresponding public opinion processing data based on the public opinion issues raised by the user, so as to recommend a suitable public opinion processing method for the user, which is convenient for public opinion management personnel to make selective recommendations during communication.
[0098] (4) It can significantly improve the speed and quality of users' public opinion processing, improve the application effect of public opinion management, and has broad application prospects and huge market demand.
[0099] (5) It is widely used in various public opinion scenarios and is suitable for rapid handling of sudden public opinion events.
[0100] (6) It can be applied to China Telecom’s integrated management products or applications such as Tianyi Cloud and self-developed building platform, thereby enhancing the core competitiveness of the products and promoting the expansion of China Telecom’s cloud business and government and enterprise customer market.
[0101] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0102] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the invention. The scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A network public opinion business intelligent diagnostic system, characterized in that, include: A multi-mode service device, an input device, and an output device, wherein the multi-mode service device is connected to the input device and the output device, respectively; The input device is used to input data into the multi-mode service device through manual import and batch database import methods; The multi-modal service device includes: a data preprocessing module, a database module, an ETL module, a processing module, and a result output module, wherein... The input end of the data preprocessing module is connected to the output end of the input device, and is used to receive data from the input device, and send the preprocessed input data to the ETL module and the database module respectively; the data preprocessing module includes: an import preprocessing unit and a Flink unit, wherein the import preprocessing unit is connected to the manual import module, and is used to preprocess the manually imported data to obtain public opinion data, and send it to the database module; the Flink unit is connected to the Kafka module, and is used to batch process the batch-imported data in a pipeline manner, and send it to the ETL module; The input end of the database module is connected to the output end of the data preprocessing module, and the output end of the database module is connected to the input end of the processing module. It stores data used to train the short text classification model, including: a rule table, similar data, and public opinion data. The database module generates public opinion data based on the data from the import preprocessing unit. The rule table and similar data are pre-stored in the database, and the public opinion data is obtained after the import preprocessing unit processes the input data. The public opinion database stores currently relevant public opinion information. The input of the ETL module is connected to the output of the data preprocessing module, and is used to extract and transform the preprocessed data for data feature extraction. The input of the processing module is connected to the output of the ETL module, and is used to perform feature rule model matching, similar data analysis, and public opinion data recommendation. This includes: performing feature rule model matching on the data after feature extraction; training a short text classification model to obtain a scene recognition model; inputting the data to be identified into the trained scene recognition model for scene recognition; and identifying the public opinion type result in real time; performing similarity analysis on single text data within historical data and outputting similar historical text information, which is used to provide reference for public opinion management personnel in business operations and processing status; generating public opinion data recommendation results based on single text data and public opinion data; and using a collaborative filtering recommendation algorithm to analyze single text data and public opinion data to achieve public opinion data recommendation. In the process of feature rule model matching of the data after feature extraction by the ETL module, the processing module compares the data with the preset feature rules and feature models; performs classification selection of different models, and obtains the public opinion type result. The feature rules include: hot topic information rules, sensitive information rules, and biased information rules; the feature models include: hot topic information model, sensitive information model, and biased information model. The hotspot information is obtained using an unsupervised clustering algorithm, while the sensitive information and the bias information are obtained using a supervised algorithm; both of these belong to multi-text classification algorithms. The sensitive information is divided into 5 categories, including: Category 1: National security, public rights and interests, personal privacy, and legitimate corporate rights and interests; The second category: personal privacy and the legitimate rights and interests of enterprises; The third category includes: personal privacy, corporate legal rights and interests, and public rights and interests. The fourth category includes: personal privacy, corporate legal rights, and public rights. Category 5: Public rights and national security; The bias information is divided into three categories: positive, negative, and neutral. Before training the short text classification model, prepare the dataset and label the data; use machine learning algorithms or deep learning models to train the data to obtain the classification model; after obtaining the classification model, use the classification model to monitor sensitive information and bias information in new text data, input the new text data into the model for classification and filtering, and thus obtain the recognition result; wherein, the deep learning model is TextCNN, TextRNN or BERT language model; The input end of the result processing module is connected to the output end of the processing module, and is used to output the processed public opinion type results, similar historical text information and public opinion recommendation results to the output device. The output device outputs the received public opinion type results, similar historical text information, and public opinion recommendation results in JSON format and provides them to public opinion management personnel.
2. The intelligent diagnostic system for online public opinion services as described in claim 1, characterized in that, The input device includes a manual import module and a Kafka module, which are respectively connected to the data preprocessing module. The manual import module is used to output the manually imported data to the database module after preprocessing by the data preprocessing module. The Kafka module is used to import data in batches and output it to the ETL module after preprocessing by the data preprocessing module.
3. The intelligent diagnostic system for online public opinion services as described in claim 1, characterized in that, The ETL module performs data feature extraction, including: extracting data features from short text data under a unified knowledge graph through syntactic analysis, Chinese word segmentation, and proper noun extraction according to preset feature extraction rules.
4. The intelligent diagnostic system for online public opinion services as described in claim 1, characterized in that, The processing module calculates the cross-entropy between the training results of the short text classification model and the pre-labeled standard classification results, calculates the average Euclidean distance and uses it as the loss value, and then feeds it back to their respective neural networks for repeated training until the model converges, finally obtaining a complete scene recognition model. The data is compared with the feature model through preset feature rules to select different models for classification and obtain the public opinion type result.
5. The intelligent diagnostic system for online public opinion services as described in claim 1, characterized in that, The processing module uses the Word2Vec model to perform similarity analysis of a single text data entry within historical data.
6. The intelligent diagnostic system for online public opinion services as described in claim 1, characterized in that, The input device further includes: an HTTP request module, used to provide a query interface for user public opinion management; The multi-mode service device further includes an Nginx module and a Flask module. The Nginx module is used to process and respond to multiple HTTP requests from the HTTP request module simultaneously. The Flask module responds to multiple HTTP request data sent by the Nginx module at once through the Werkzeug function library.