Text monitoring method and device based on mixed word vector coding

Abstract

The invention discloses a text monitoring method and device based on mixed word vector coding. The method comprises the following steps: crawling a public opinion text; encoding the public opinion text by using a BERT model to obtain a word vector corresponding to each word; performing word segmentation on the public opinion text to obtain a plurality of words; searching a first word vector corresponding to the word from a preset dictionary; the first word vector is a word vector which is obtained by carrying out word vector clustering on the initial word vector according to categories by utilizing a linear discriminant analysis method and is improved in spatial anisotropy; obtaining a second word vector corresponding to the word according to the word vector, and merging the first word vector and the second word vector corresponding to the word to obtain an improved word vector; and public opinion monitoring is carried out based on the improved word vector. The problems that in the prior art, when a BERT model is used for word vector coding, the word vector coding space is narrow, and anisotropy is obvious are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic data processing, and in particular relates to a text monitoring method and device based on hybrid word vector encoding. Background Art

[0002] Social media is the main battlefield for online public opinion supervision. In the context of big data, most public opinion analysis platforms use natural language analysis algorithms combined with deep learning algorithms to analyze text, which facilitates efficient and timely analysis of large sample data. The most critical step is the feature extraction of natural language text. From static word vectors (Word2Vec, Glove) to the BERT text encoding model based on the Transformer structure, the extraction of text features has undergone a series of developments and improvements.

[0003] Before a neural network analyzes a text, it needs to represent the text as a word vector that the neural network can understand. The text feature extraction and summarization capabilities of the word vector, namely word embedding, also directly affect the accuracy and speed of downstream tasks. The concept of word embedding was first introduced by Xu, Rudnicky, and Bengio in the Neural Network Language Model (NNLM), which believed that words are more likely to have similar meanings when they are in similar contexts.

[0004] With the development of word embedding, the representation of word vectors no longer depends on neural networks and embedding layers, but uses word context matrices to obtain word vector representations. In 2013, Mikolov proposed the Word2Vector model, which includes two frameworks: Skip-Gram and (Continuous Bag-of-Words) CBOW. These two frameworks are very popular in NLP (natural language processing) tasks and can be regarded as a two-layer neural network language model. In 2014, Pennington et al. proposed the Glove model. Word embedding based on the context matrix is ​​a dense, distributed, fixed-length word vector, which is constructed using word co-occurrence statistics based on the distributed assumption. However, word embedding based on the context matrix cannot capture the semantic features of the long sentences and texts in which the words are located. It was not until 2018 that the transformer model came out and refreshed many NLP downstream tasks. The BERT model is a bidirectional encoder based on the Transformer model. This model is different from all previous encoding models. The BERT model is a context-based embedding model, while other popular embedding models such as word2vec are context-free.

[0005] Therefore, most word vector encodings with large amounts of data today will give priority to the BERT model. However, due to its mask mechanism, the word vector space encoded by the BERT model is anisotropic, which affects the feature extraction of social media texts and thus affects related downstream tasks of public opinion analysis.

[0006] Specifically, when studying BERT's word vector space, Timothee Mickus and other scholars found that BERT showed a consistent trend, but it did not fully meet the natural expectation of the semantic vector space, that is, similar words should be represented by similar vectors. Sara Rajaee and other scholars conducted a detailed study on the degradation problem of the BERT model and found that in the anisotropic embedding space, word vectors are distributed in a narrow cone, and even unrelated words are considered to have high cosine similarity. This unwelcome property hinders the representativeness of the embedding space and limits the diversity of encoded knowledge. In order to improve the degradation problem of contextual representation word vectors in BERT, some people proposed a method based on local clustering, which clustered the embeddings using PCA (principal component analysis) to find the principal component PC. The principal component represents the dominant direction of each cluster. Clearing the PC will make the vector space more isotropic. However, clearing the PC (principal component) may cause the semantic representation of the word vector to be destroyed. Based on the idea of ​​clearing PC, Yuxin Liang proposed to use a set of learnable weights to remove several dominant directions of BERT embedding, and indirectly adjust the geometric characteristics of the word vector distribution in the vector space by adjusting the similarity of words, so that it develops in the isotropic direction. However, the existing PCA method will reduce the representational significance of word vectors by clearing the principal components after clustering. In addition, social media texts are often short and colloquial, and the characteristics of low-frequency words caused by the mask mechanism being squeezed into a narrower cone and pushed away from the origin will be more obvious. Summary of the invention

[0007] In order to solve the above problems existing in the prior art, the present invention provides a text monitoring method and device based on hybrid word vector encoding.

[0008] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0009] A text monitoring method based on hybrid word vector encoding, comprising:

[0010] Crawl public opinion texts;

[0011] Encode the public opinion text using the BERT model to obtain a word vector corresponding to each word in the public opinion text;

[0012] Segmenting the public opinion text to obtain multiple words;

[0013] From a preset dictionary, searching for a first word vector corresponding to each of the words; wherein the first word vector is a word vector obtained by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using a linear discriminant analysis method and improved in spatial anisotropy; the initial word vector is a word vector obtained by encoding the words in the dictionary using a WOBERT model;

[0014] According to each of the word vectors, a second word vector corresponding to each word in the public opinion text is obtained, and the first word vector corresponding to each word in the public opinion text is merged with the second word vector to obtain an improved word vector corresponding to the word;

[0015] Public opinion monitoring is performed based on the improved word vectors corresponding to the words in the public opinion text.

[0016] Optionally, the method further comprises:

[0017] After crawling the public opinion text, a relational database or a distributed database is selected to store the public opinion text according to the data source of the public opinion text, so as to subsequently obtain the public opinion text from the database and encode and segment it.

[0018] Optionally, the method further includes: after crawling the public opinion text, preprocessing the public opinion text, and subsequently encoding and segmenting the preprocessed public opinion text; the preprocessing at least includes: text cleaning.

[0019] Optionally, after segmenting the public opinion text, the method further includes:

[0020] Detecting new words appearing in the public opinion text;

[0021] Determining the sentiment attribute of the new word according to the context of the new word, and using the sentiment attribute as the category of the new word;

[0022] The new words and their categories are updated in the dictionary, and the linear discriminant analysis method is reused to cluster the initial word vectors corresponding to all the words in the dictionary according to the categories.

[0023] Optionally, detecting new words appearing in the public opinion text includes:

[0024] A new word detection tool is used to detect new words in a preset proportion of all public opinion texts.

[0025] Optionally, the public opinion monitoring includes: media report analysis, public opinion time heat analysis, public opinion regional heat analysis and / or public opinion sentiment situation analysis.

[0026] Optionally, the method further includes: displaying the content of public opinion monitoring, and / or, issuing public opinion warnings based on the content of public opinion monitoring, and / or, processing public opinion based on the content of public opinion monitoring.

[0027] The present invention also provides a text monitoring device based on hybrid word vector encoding, comprising:

[0028] Text crawling module, used to crawl public opinion texts;

[0029] An encoding module, used to encode the public opinion text using a BERT model to obtain a word vector corresponding to each word in the public opinion text;

[0030] A word segmentation module, used to segment the public opinion text to obtain multiple words;

[0031] A search module, used for searching for a first word vector corresponding to each of the words in a preset dictionary; wherein the first word vector is a word vector obtained by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using a linear discriminant analysis method and improved in spatial anisotropy; the initial word vector is a word vector obtained by encoding the words in the dictionary using a WOBERT model;

[0032] A vector merging module, used to obtain a second word vector corresponding to each word in the public opinion text according to each of the word vectors, and merge the first word vector and the second word vector corresponding to each word in the public opinion text to obtain an improved word vector corresponding to the word;

[0033] The public opinion monitoring module is used to monitor public opinion based on the improved word vectors corresponding to the words in the public opinion text.

[0034] Optionally, the device further comprises:

[0035] The storage module is used to select a relational database or a distributed database to store the public opinion text according to the data source of the public opinion text after the crawling module crawls the public opinion text, so as to subsequently obtain the public opinion text from the database and encode and segment it.

[0036] Optionally, the device further comprises: a new word detection module and a dictionary update module;

[0037] The new word detection module is used to detect new words appearing in the public opinion text; determine the sentiment attribute of the new word according to the context of the new word, and use the sentiment attribute as the category of the new word;

[0038] The dictionary updating module is used to update the new words and their categories into the dictionary, and re-use the linear discriminant analysis method to cluster the initial word vectors corresponding to all the words in the dictionary according to the categories.

[0039] In the text monitoring method based on hybrid word vector encoding provided by the present invention, the public opinion text is segmented, and the first word vector corresponding to each word is searched from the preset dictionary. The first word vector is a word vector with improved spatial anisotropy obtained by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using the linear discriminant analysis method, thereby enhancing the classification distinction of the word vectors, and the spatial distance of the word vectors of the same category is closer, which improves the problem that the original BERT word vector has insufficient cosine value discrimination and anisotropy of spatial features; at the same time, the present invention uses the BERT model to encode the public opinion text, which well preserves the contextual semantic features. Therefore, the improved word vector obtained by merging the second word vector composed of the word vectors encoded by the BERT model and the first word vector after the spatial anisotropy is improved not only retains the representation meaning of the BERT word vector but also improves the crowded vector space, and then when public opinion monitoring is performed based on the improved word vector, different public opinions can be better classified and semantically recognized, thereby improving the monitoring effect.

[0040] The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flowchart of a text monitoring method based on hybrid word vector encoding provided by an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of the structure of a dictionary preset in an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of improving the initial word vector using a nonlinear discriminant analysis method in an embodiment of the present invention;

[0044] Figure 4 It is a schematic diagram of using a diversion mechanism to store public opinion texts in an embodiment of the present invention;

[0045] Figure 5 This is a public opinion monitoring process schematically shown in an embodiment of the present invention;

[0046] Figure 6 The dimensionality reduction effect diagram of the initial word vector before improvement obtained by the PCA algorithm is shown;

[0047] Figure 7 The figure shows the dimensionality reduction effect of the improved word vector obtained through the PCA algorithm. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0049] The BERT model has the problem of anisotropy in the encoding word vector space. There are a large number of colloquialisms and Internet terms in social media language. These low-frequency and rare words will cause the BERT encoding space to present a narrow cone. The specific manifestation is that the similarity of similar words is at the same level as the similarity of non-similar words, resulting in reduced accuracy and performance of downstream tasks of public opinion analysis.

[0050] In order to solve the above problems, the embodiment of the present invention provides a text monitoring method based on hybrid word vector encoding, such as Figure 1 As shown, the method comprises the following steps:

[0051] S10. Crawl public opinion texts.

[0052] Specifically, network ideological public opinion security incidents are different from ordinary public opinion incidents. They often cause influential media and official media to follow up and publicize the handling measures. Therefore, the scope of public opinion event data monitoring should be larger, and public opinion event data can be tracked from a variety of different public opinion sources, such as social media platforms, government and enterprise news websites, and mainstream media websites. Among them, social media platforms mainly involve the scope of text social platforms. Such platforms have a high degree of posting autonomy, a relatively loose review mechanism, a high degree of participation from young people, and public opinion is easy to ferment.

[0053] In step S10, the crawling of public opinion text is mainly achieved through the web page interface. The crawled content can be crawled according to keywords, and one or more keyword search results are continuously obtained, and the text containing the specified keywords in the crawled text is searched. At the same time, in order to ensure the timeliness and accuracy of the crawled content, the search time range can be specified.

[0054] Optionally, after crawling the public opinion text, the public opinion text can be preprocessed, and then the preprocessed public opinion text can be encoded and segmented; the preprocessing mentioned here at least includes: text cleaning, of course, but is not limited to this.

[0055] S20. Use the BERT model to encode the public opinion text to obtain the word vector corresponding to each word in the public opinion text.

[0056] Specifically, the public opinion text is input into the BERT model, and the BERT model is directly used to encode the public opinion text, and the word vector corresponding to each word in the public opinion text is output. Here, the BERT model is used to encode the public opinion text, which better retains the advantages of the BERT model encoding: the word meaning is fully expressed and the semantic features of the context are better preserved.

[0057] S30, segment the public opinion text to obtain multiple words.

[0058] Here, the word segmentation of public opinion text can be implemented using the JIEBA algorithm. The JIEBA algorithm is a Chinese word segmentation algorithm based on statistical dictionaries and hidden Markov models. It scans and counts text by establishing dictionaries and probability models to achieve word segmentation and annotation. JIEBA word segmentation is efficient, accurate, and flexible, and is widely used in Chinese natural language processing.

[0059] S40. Search for the first word vector corresponding to each word in a preset dictionary; wherein the first word vector is a word vector improved in spatial anisotropy by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using the linear discriminant analysis method; the initial word vector is a word vector obtained by encoding the words in the dictionary using the WOBERT model.

[0060] It is understandable that the dictionary contains a large number of words, each of which has a corresponding category and first word vector. These first word vectors are obtained by improving the word vectors through linear discriminant analysis. Specifically, the WOBERT model is used to encode the words in the dictionary to obtain the initial word vector corresponding to each word in the dictionary; then the linear discriminant analysis method is used to cluster the word vectors of all the initial word vectors according to the category of the words they correspond to, and the first word vector corresponding to each word in the dictionary is obtained.

[0061] In the embodiment of the present invention, the dictionary can be as follows Figure 2 The structure shown is used for classification and storage. Each classification word library is made into a csv file, similar to EXCEL, and the file is divided into three columns, which contain words, parts of speech, and the first word vector. Part of speech refers to the identification of each classification word, for example, noun identification is 1, verb identification is 2, quantifier identification is 3, and so on.

[0062] Linear Discriminant Analysis (LDA) is a classic pattern recognition and machine learning method used to find the direction in feature space that best distinguishes different categories. It is often used for dimensionality reduction and classification tasks. The goal of linear discriminant analysis is to find a projection direction given a class label so that samples of the same class are as close as possible and samples of different classes are as far apart as possible. This projection direction is determined by maximizing the inter-class distance and minimizing the intra-class distance.

[0063] In practice, the steps to improve the initial word vector by using linear discriminant analysis are as follows: calculate the mean vector of each category; calculate the within-class scatter matrix; calculate the between-class scatter matrix; solve a generalized eigenvalue problem to obtain the optimal projection direction (linear discriminant vector). The word vector obtained in the projection direction is the word vector after dimensionality reduction, that is, the first word vector, which is essentially a subspace of the BERT word vector. This part of the word vector enhances the classification distinction of the BERT word vector, and the spatial distance of the word vectors of the same category is closer. It improves the problem that the cosine value of the original BERT word vector is not discriminative enough and the spatial features are isotropic.

[0064] In addition, after the initial word vector is processed by linear discriminant analysis, the processed word vector can be filled with 0 and dimension increased (such as Figure 3 ), so that the word vector merging operation can be performed later.

[0065] S50. According to each word vector, a second word vector corresponding to each word in the public opinion text is obtained, and the first word vector and the second word vector corresponding to each word in the public opinion text are merged to obtain an improved word vector corresponding to the word.

[0066] Specifically, for each word in the public opinion text, the word vectors corresponding to the characters contained in it are merged into a second word vector, and then the second word vector is merged with the first word vector corresponding to the word to obtain the improved word vector corresponding to the word.

[0067] In an embodiment of the present invention, in order to solve the problem of isotropy in the encoding space of the BERT encoding model, it is proposed to use the idea of ​​a linear discriminant analysis method to improve the problem that BERT word vectors of the same category are far apart and the space is anisotropic. The improved word vector includes a second word vector based on the original encoding of the BERT model and an improved first word vector. Therefore, the improved word vector not only retains the representational meaning of the BERT word vector, but also improves the crowded vector space.

[0068] S60. Perform public opinion monitoring based on the improved word vectors corresponding to the words in the public opinion text.

[0069] Specifically, based on the improved word vectors corresponding to the words in the public opinion text, the text analysis algorithm is used to monitor the public opinion. Here, the text analysis algorithm includes word frequency analysis, semantic analysis, topic analysis, sentiment analysis, entity recognition, relationship analysis, time series analysis, text classification and text clustering.

[0070] Exemplarily, the above-mentioned public opinion monitoring may include: media report analysis, public opinion time heat analysis, public opinion regional heat analysis and / or public opinion sentiment situation analysis.

[0071] Figure 5 The figure shows by way of example a flow chart of public opinion monitoring based on improved word vectors corresponding to words in public opinion texts.

[0072] Among them, for public opinion data, we can combine the downstream text analysis algorithm, use keyword analysis and the frequency of related hot words to obtain the public opinion word cloud; extract the time and place of text release to obtain the public opinion time heat change line chart and regional heat map; analyze the emotional situation of the public and obtain the public's emotional attitude towards this public opinion event.

[0073] The time heat map of public opinion can be generated according to the following process: convert the crawled timestamps into a usable time format (such as hours, days, weeks) for time normalization; count the number of related content by time unit (such as hours, days), so as to calculate the heat data in different time units; then, use visualization tools to visualize the calculated public opinion heat data using line charts or bar charts to represent the relationship between time and public opinion heat. Here, visualization tools include Python data visualization tools such as Tableau and Power BI.

[0074] The regional heat map of public opinion can be generated according to the following process: the number of relevant contents is counted by region, and the heat value of each region is generated. Then, the above-mentioned visualization tools are used to display the heat of public opinion in different regions.

[0075] For mainstream media reports, the sentiment trend of mainstream media can be analyzed, and the popularity of mainstream media reports can be obtained based on the number and frequency of media reports. Among them, sentiment trend analysis can be achieved by sending the improved word vector into LSTM (Long Short-Term Memory Network) for sentiment analysis.

[0076] For authoritative official media reports, we can use the same emotional analysis method as mainstream media reports, or we can combine it with backend manual analysis to analyze emotional trends. In addition, since authoritative official media reports generally have less content and are more important, we can crawl the public opinion text completely and display it on the front-end page.

[0077] From the above, it can be seen that in the text monitoring method based on hybrid word vector encoding provided by the embodiment of the present invention, the public opinion text is segmented, and the first word vector corresponding to each word is searched from the preset dictionary. The first word vector is a word vector with improved spatial anisotropy obtained by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using the linear discriminant analysis method, thereby enhancing the classification distinction of word vectors, and the spatial distance of word vectors of the same category is closer, which improves the problem that the original BERT word vector has insufficient cosine value discrimination and anisotropy of spatial features; at the same time, the embodiment of the present invention uses the BERT model to encode the public opinion text, which well preserves the contextual semantic features. Therefore, the improved word vector obtained by merging the second word vector composed of the word vectors encoded by the BERT model and the first word vector after the spatial anisotropy is improved not only retains the representation meaning of the BERT word vector but also improves the crowded vector space, and then when public opinion monitoring is performed based on the improved word vector, different public opinions can be better classified and semantically recognized, thereby improving the monitoring effect.

[0078] Optionally, in one implementation, the text monitoring method of hybrid word vector encoding provided by an embodiment of the present invention may also include: after crawling the public opinion text, according to the data source of the public opinion text, selecting a relational database or a distributed database to store the public opinion text, so as to subsequently obtain the public opinion text from the database and encode and segment it.

[0079] For details, see Figure 4 , the official media response content released by government and enterprise websites is stored in traditional databases, the public opinion content is stored in distributed databases, and the data of mainstream news websites are divided into small public opinion events and large public opinion events according to the scope and data volume of the public opinion event. Among them, the news data website data of small public opinion events is often small in scale and can be stored in a relational database. When the data scale of the public opinion event affects the data reading speed of the database (for example, the data capacity limit can be set to 4GB), it can be determined as a non-small public opinion event, and this part of the data is stored in a distributed database.

[0080] It is understandable that official media's comments and treatment measures often have less text, more client calls, and higher security and confidentiality requirements. They require structured data, high security, and reliability. When analyzing public opinion, they are called more frequently and the research is more detailed. Relational databases are suitable for their fast reading speed, high call frequency, and small text volume. Therefore, relational databases are a better choice. For scenarios that require processing a large amount of unstructured data such as social media language, high concurrent access, and high performance, non-relational databases, or distributed databases, are more suitable.

[0081] Therefore, the embodiment of the present invention stores the captured data into different databases according to different data sources, and adopts a diversion mode to mix the traditional relational database with the combined distributed database, which helps to improve the subsequent flexible calling of different public opinion texts.

[0082] Optionally, in one implementation, the hybrid word vector encoding text monitoring method provided by the embodiment of the present invention may also include: detecting new words appearing in the public opinion text; judging the sentiment attribute of the new words according to the context of the new words, and using the sentiment attribute as the category of the new words; updating the new words and their categories to the dictionary, and reusing the linear discriminant analysis method to cluster the initial word vectors corresponding to all the words in the dictionary according to the category. Figure 5 In the process, 10% to 20% of public opinion texts are randomly selected for new word detection and identification of their sentiment attributes, and then they are all included in the dictionary.

[0083] Among them, new word detection can be achieved using a new word detection tool, such as NLPIR developed by the Chinese Academy of Sciences, but is not limited to this.

[0084] It is understandable that the main force of social network users are young people, who are good at using online language to express their comments on hot current events. The emergence of new online buzzwords is often time-sensitive, and their semantics may be very different from the surface meaning of the words. The emotional meaning of new buzzwords triggered by a single public opinion event needs to be judged based on the semantics of the text. Judging only by the surface meaning of the word will increase the probability of misjudgment.

[0085] In order to solve the above problems, in an embodiment of the present invention, the sentiment attribute of a word is judged according to the context of the word, specifically, the sentiment tendency of the word, i.e., commendatory, derogatory or neutral, is judged by analyzing the usage of the word in a specific context. This method is widely used in the fields of natural language processing, text sentiment classification, etc. In natural language processing, sentiment discrimination based on corpus is a common method. By mining information on a large-scale corpus and analyzing the grammatical characteristics and usage context of a word, its sentiment attribute can be judged.

[0086] In addition, when detecting new words appearing in public opinion texts, new word detection tools can be used to perform new word detection on all public opinion texts; or, new word detection tools can be used to perform new word detection on a preset proportion of all public opinion texts, for example, extracting 10% to 20% from all public opinion texts for new word detection.

[0087] It is understandable that when a new situational word with a strong emotional tendency appears in a related public opinion event, it often has the characteristics of high frequency of occurrence and strong emotional expression in the sentence. The traditional new word judgment is a refined all-round analysis, which wastes too many resources. The existing new word recognition algorithm mainly relies on statistical methods such as word frequency statistics, which requires statistics of a large amount of text content. The efficiency of new word recognition is low, and the professionalism and accuracy of new word recognition for specific public opinion events are insufficient. It is impossible to quickly analyze the properties of new words for the purpose of downstream tasks, such as emotional tendencies and parts of speech, to help quickly perform text analysis. In comparison, the embodiment of the present invention only detects public opinion texts, and randomly extracts a certain proportion of public opinion texts for new word detection, thereby being able to detect new words related to downstream text analysis and processing in a targeted manner with higher efficiency.

[0088] In an embodiment of the present invention, taking into account the frequent appearance of new words in the real network environment, the dictionary is updated by detecting new words in the public opinion text and judging the new words in different tasks in real time, so as to facilitate the implementation of downstream tasks of public opinion analysis and enhance the accuracy of text processing tasks such as text sentiment judgment.

[0089] Optionally, in one implementation, the text monitoring method of hybrid word vector encoding provided by an embodiment of the present invention may also include: displaying the content of public opinion monitoring, and / or, issuing public opinion warnings based on the content of public opinion monitoring, and / or, performing public opinion processing based on the content of public opinion monitoring.

[0090] For details, see Figure 5 , the public opinion time heat map, public opinion regional heat map, public sentiment situation map, official media reports, and handling measures in official media reports can be displayed on the front end. In addition, based on the analysis of mainstream media reports and official media reports, the scope of influence and spread of public opinion events can be judged in combination with mainstream media and public public opinion heat, so as to determine whether to give a public opinion warning. If no warning is given, continue monitoring. If a warning is given, relevant governance opinions can be given through reports from mainstream media and official media, combined with relevant national laws and regulations, that is, public opinion processing can be carried out. Here, the giving of governance opinions can be achieved by identifying content related to "suggestions" through topic modeling (such as LDA). Subject modeling can be achieved by using the topic model of Latent Dirichlet Allocation (LDA).

[0091] In a specific example, based on the vocab.txt in the BERT model, a dictionary is compiled, including a dictionary of thousands of words with two parts of speech classification: positive and negative. The BERT model is used to encode the words in the dictionary and integrate them into the initial word vector and save them in the dictionary. In addition, the Wordpiece algorithm is used to segment a single sentence and organize the words into a dictionary. The words in the dictionary are encoded and saved as an indexed dictionary. The linear discriminant analysis method is used to achieve two goals: minimize the intra-class scatter matrix and maximize the inter-class scatter matrix, so as to improve the initial word vector. Figure 6 The dimensionality reduction effect diagram of the initial word vector before improvement obtained by the PCA algorithm is shown. Figure 7 The figure shows the dimensionality reduction effect of the improved word vector obtained by the PCA algorithm, where the ordinate and abscissa refer to the projection of the multidimensional vector on the two-dimensional plane. Figure 7 The dots of the same color refer to similar emotional words.

[0092] The specific manifestation of the spatial isotropy of word vectors is that the spatial distance between similar words and non-similar words is not very different, and they do not have the ability to distinguish word classes. Figure 6 As shown in the figure, the classification effect of the original word vector encoded by the BERT model is poor, with almost no classification ability and a classification accuracy of less than 50%. The classification effect of the improved word vector is greatly improved, as shown in the figure. Figure 7 As shown in the figure, the improved word vectors of the same type are closer in the word vector encoding space (the same color is more clustered), and the accuracy is finally fixed at about 83%. This shows that the new hybrid word vector obtained based on the idea of ​​linear discriminant analysis has greatly improved the spatial anisotropy problem of the initial word vector.

[0093] The manifestation of the anisotropy of the encoding space is that the distance between similar words in the encoding vector space does not conform to the actual meaning, and the distance between words with similar meanings is not closer. Figure 7 It can be seen that the distance between similar words in the improved word vector is closer, which effectively improves the spatial anisotropy and improves the word vector encoding space while ensuring that the word vector representation meaning does not change much. At the same time, the improved word vector contains a large proportion of the initial word vector, which greatly retains the original characteristics of the BERT word vector. This can effectively avoid the problem of the existing PCA method removing the principal component after clustering, which leads to a decrease in the representation meaning of the word vector.

[0094] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0095] Based on the same inventive concept, an embodiment of the present invention also provides a text monitoring device based on hybrid word vector encoding, including: a text crawling module, an encoding module, a word segmentation module, a search module, a vector merging module and a public opinion monitoring module.

[0096] Among them, the text crawling module is used to crawl public opinion texts;

[0097] The encoding module is used to encode the public opinion text using the BERT model to obtain the word vector corresponding to each word in the public opinion text;

[0098] The word segmentation module is used to segment the public opinion text into multiple words;

[0099] A search module, used to search for the first word vector corresponding to each of the above words from a preset dictionary; wherein the first word vector is a word vector obtained by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using a linear discriminant analysis method and improved in spatial anisotropy; the initial word vector is a word vector obtained by encoding the words in the dictionary using a WOBERT model;

[0100] A vector merging module is used to obtain a second word vector corresponding to each word in the public opinion text according to each word vector, and merge the first word vector and the second word vector corresponding to each word in the public opinion text to obtain an improved word vector corresponding to the word;

[0101] The public opinion monitoring module is used to monitor public opinion based on the improved word vectors corresponding to the words in the public opinion text.

[0102] Optionally, the above device further includes:

[0103] The storage module is used to select a relational database or a distributed database to store the public opinion text according to the data source of the public opinion text after the crawling module crawls the public opinion text, so as to subsequently obtain the public opinion text from the database and encode and segment it.

[0104] Optionally, the above device further comprises: a new word detection module and a dictionary update module;

[0105] The new word detection module is used to detect new words appearing in public opinion texts; the sentiment attribute of the new words is determined according to the context of the new words, and the sentiment attribute is used as the category of the new words;

[0106] The dictionary update module is used to update new words and their categories into the dictionary, and re-use the linear discriminant analysis method to cluster the initial word vectors corresponding to all the words in the dictionary according to their categories.

[0107] The text monitoring device based on hybrid word vector encoding provided by the embodiment of the present invention can be a software system. The overall architecture of the system is a distributed architecture, which is divided into five levels according to the functionality of each part: data acquisition layer, data storage layer, text encoding layer, business layer and display layer. The display layer mainly displays the system functions through the front-end page design. It mainly includes two parts: the front-end display of the business done by the business layer and the processing warning results and processing content of the public opinion event.

[0108] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment, and the embodiment of the present invention will not be described in detail.

[0109] In summary, the embodiment of the present invention proposes a complete public opinion monitoring technology framework based on ideological security, and embeds an improved text encoding algorithm in the monitoring architecture design, providing a solid and implementable technical framework template for the gap in this field. It solves the problems of low efficiency and insufficient purpose of new network word recognition; narrow word vector encoding space and obvious anisotropy; and lack of monitoring technology framework templates for security public opinion events. It builds a text monitoring technology framework based on big data scenarios, and provides technical template samples for monitoring sudden ideological public opinion events on social media under the background of big data.

[0110] It should be noted that the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0111] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0112] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art can understand and implement other changes to the disclosed embodiments by viewing the drawings and the disclosed content. In the description of the present invention, the term "comprising" does not exclude other components or steps, "one" or "an" does not exclude multiple situations, and the meaning of "multiple" is two or more, unless otherwise clearly and specifically limited. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0113] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware, which are collectively referred to as "modules" or "systems" herein. Moreover, the present application may adopt the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes. The computer program is stored / distributed in a suitable medium, provided together with other hardware or as a part of hardware, or may adopt other distribution forms, such as by Internet or other wired or wireless telecommunication systems.

[0114] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses (devices) and computer program products of the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A text monitoring method based on hybrid word vector encoding, characterized in that: include: Crawl public opinion texts; Encode the public opinion text using the BERT model to obtain a word vector corresponding to each word in the public opinion text; Segmenting the public opinion text to obtain multiple words; From a preset dictionary, searching for a first word vector corresponding to each of the words; wherein the first word vector is a word vector obtained by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using a linear discriminant analysis method and improved in spatial anisotropy; the initial word vector is a word vector obtained by encoding the words in the dictionary using a WOBERT model; According to each of the word vectors, a second word vector corresponding to each word in the public opinion text is obtained, and the first word vector corresponding to each word in the public opinion text is merged with the second word vector to obtain an improved word vector corresponding to the word; Public opinion monitoring is performed based on the improved word vectors corresponding to the words in the public opinion text.

2. The text monitoring method based on hybrid word vector encoding according to claim 1, characterized in that: The method further comprises: After crawling the public opinion text, a relational database or a distributed database is selected to store the public opinion text according to the data source of the public opinion text, so as to subsequently obtain the public opinion text from the database and encode and segment it.

3. The text monitoring method based on hybrid word vector encoding according to claim 1, characterized in that: The method further includes: after crawling the public opinion text, preprocessing the public opinion text, and subsequently encoding and segmenting the preprocessed public opinion text; the preprocessing at least includes: text cleaning.

4. The text monitoring method based on hybrid word vector encoding according to claim 1, characterized in that: After segmenting the public opinion text, the method further includes: Detecting new words appearing in the public opinion text; Determining the sentiment attribute of the new word according to the context of the new word, and using the sentiment attribute as the category of the new word; The new words and their categories are updated in the dictionary, and the linear discriminant analysis method is reused to cluster the initial word vectors corresponding to all the words in the dictionary according to the categories.

5. The text monitoring method based on hybrid word vector encoding according to claim 4 is characterized in that: Detect new words appearing in the public opinion text, including: A new word detection tool is used to detect new words in a preset proportion of all public opinion texts.

6. The text monitoring method based on hybrid word vector encoding according to claim 1, characterized in that: The public opinion monitoring includes: media report analysis, public opinion time heat analysis, public opinion regional heat analysis and / or public opinion sentiment situation analysis.

7. The text monitoring method based on hybrid word vector encoding according to claim 6 is characterized in that: The method also includes: displaying the content of public opinion monitoring, and / or, issuing public opinion warnings based on the content of public opinion monitoring, and / or, processing public opinion based on the content of public opinion monitoring.

8. A text monitoring device based on hybrid word vector encoding, characterized in that: include: Text crawling module, used to crawl public opinion texts; An encoding module, used to encode the public opinion text using a BERT model to obtain a word vector corresponding to each word in the public opinion text; A word segmentation module, used to segment the public opinion text to obtain multiple words; A search module, used for searching for a first word vector corresponding to each of the words in a preset dictionary; wherein the first word vector is a word vector obtained by clustering the initial word vectors corresponding to all the words in the dictionary according to categories using a linear discriminant analysis method and improved in spatial anisotropy; the initial word vector is a word vector obtained by encoding the words in the dictionary using a WOBERT model; A vector merging module, used to obtain a second word vector corresponding to each word in the public opinion text according to each of the word vectors, and merge the first word vector and the second word vector corresponding to each word in the public opinion text to obtain an improved word vector corresponding to the word; The public opinion monitoring module is used to monitor public opinion based on the improved word vectors corresponding to the words in the public opinion text.

9. The text monitoring device based on hybrid word vector encoding according to claim 8, characterized in that: The device also includes: The storage module is used to select a relational database or a distributed database to store the public opinion text according to the data source of the public opinion text after the crawling module crawls the public opinion text, so as to subsequently obtain the public opinion text from the database and encode and segment it.

10. The text monitoring device based on hybrid word vector encoding according to claim 8 is characterized in that: The device also includes: a new word detection module and a dictionary update module; The new word detection module is used to detect new words appearing in the public opinion text; determine the sentiment attribute of the new word according to the context of the new word, and use the sentiment attribute as the category of the new word; The dictionary updating module is used to update the new words and their categories into the dictionary, and re-use the linear discriminant analysis method to cluster the initial word vectors corresponding to all the words in the dictionary according to the categories.

Images