Long text sentiment analysis method based on modern Chinese grammar economic security field

Abstract

The invention discloses a long text sentiment analysis method based on the field of economy and security of modern Chinese grammar. The long text sentiment analysis method comprises the following steps: acquiring network public opinion data; preprocessing the data; classifying the texts, and balancing the sample proportion; performing vocabulary-level word segmentation by using a proper noun dictionary in the field of economy and safety; building a model; randomly initializing a word embedding matrix and obtaining a static word vector; defining a position encoder layer by adopting relative position encoding; carrying out feature extraction by utilizing a bidirectional Transform model; semantic representation is enhanced through a multi-head self-attention mechanism; carrying out segment processing on the super-long text and establishing a relationship between paragraphs through an attention mechanism; and converting the features into probability distribution of sentiment classification through a full connection layer and a Softmax function. According to the method, a special dictionary in the field of economy and safety is used, a traditional mode of segmenting words through single characters is replaced, relative position coding is adopted to replace absolute position coding of traditional Transform, and a model learns and adjusts parameters in a network by himself / herself.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a long text sentiment analysis method based on the field of modern Chinese grammar, economy and security. Background Art

[0002] Text sentiment analysis, also known as opinion mining or tendency analysis, is a complex process of in-depth analysis, processing, induction and logical reasoning of subjective texts containing emotions. In the Internet space, especially blogs, forums and social service network platforms such as Dianping, users generate a large number of participatory comments. These contents are aimed at topics such as people, events, and products, and contain rich emotional expressions and tendencies, such as joy, anger, sorrow, and happiness, criticism and praise, etc.

[0003] Based on this data, text sentiment analysis aims to reveal and quantify these emotional colors, providing potential users with a way to gain insight into public opinion on specific events or products by examining subjective comments. This analysis method not only helps to understand the dynamic changes in public sentiment, but also provides market intelligence for companies to help them optimize decision-making, improve user experience, and ultimately promote the development of business strategies.

[0004] Rule-based sentiment analysis usually refers to opinion extraction. Opinion extraction is a view or attitude towards things from a certain standpoint or perspective. It is a statement that expresses feelings, views, and beliefs. The emotional tendency of an opinion is also called polarity, which can be positive (positive), negative (negative), or neutral (neutral). Each different emotional tendency also has different intensities. For example, "perfect" is more positive than "good". Many adverbs also have the function of strengthening or weakening emotional tendencies. The opinion extraction method formulates specific rules, that is, organizes specific keywords for precise matching. If these words appear in the text, the text is judged to be of the corresponding emotional type.

[0005] Feature extraction: Feature extraction of text is performed through machine learning methods such as CountVectorizer, which converts the text into word frequency vectors so that computers can understand and process it.

[0006] Training classifier: After obtaining the feature matrix, use the naive Bayes classifier to train the model based on the feature matrix and the corresponding category labels. The core of the machine learning model based on the naive Bayes classifier is to calculate the probability of each category appearing under given sample features and classify the sample into the category with the highest probability.

[0007] Predict new samples: Use the trained machine learning model to classify and predict new text data and output the category to which it belongs.

[0008] Disadvantages include: In order to calculate the posterior probability, the prior probability must be known or estimated, which may not be easy to obtain in practice. Since machine learning models assume that attributes are independent of each other, this assumption rarely holds true in practice, and thus classification errors may occur. Machine learning models are sensitive to the form of input data, and different feature extraction methods may affect the results. Due to the assumption of sample attribute independence, naive Bayesian machine learning models do not work well when there is a correlation between attributes.

[0009] Recurrent Neural Network (RNN) is a neural network used to process sequence data. The characteristic of sequence data is that the following data is related to the previous data, such as time series data, text sequence, etc.

[0010] The main disadvantages of RNN are: when the input sequence is too long, the transmitted hidden state information is difficult to capture the information at the front end of the sequence; parallel calculation is impossible, and the input of the next time step depends on the output of the previous time step.

[0011] Long Short-Term Memory (LSTM) is a time-recurrent neural network designed to solve the long-term dependency problem of general RNNs. All RNNs have a chain form of repeated neural network modules. In standard RNNs, this repeated structural module has only a very simple structure, such as a tanh layer.

[0012] The disadvantage of LSTM is that it introduces a gate mechanism, which increases the RNN's ability to capture long-term dependencies to about 150 time steps.

[0013] The attention-based Transformer is an encoder-decoder network structure, which is mainly built by the attention mechanism, including the self-attention mechanism, the multi-head attention mechanism, and the encoder-decoder attention mechanism. It has made great improvements in feature extraction and data parallel computing.

[0014] The main disadvantages of the Transformer based on the attention mechanism are: it still cannot solve the problem of long-term dependency; there is the problem of context fragmentation; for very long texts, the ability of feature extraction will be weakened and the calculation time will increase.

[0015] The current mainstream model also has the following disadvantages:

[0016] (1) The current mainstream word segmentation method is to model long texts by segmenting single words, without integrating professional domain vocabulary;

[0017] (2) The current mainstream models, LSTM and attention-based Transformer, are unable to capture long-term dependency problems. Long-term dependency problems refer to texts with a length of more than one thousand Chinese characters or English words, such as very long sequences with a word count between 500 and 20,000.

[0018] (3) The current mainstream models take a long time to model long texts and require high computing power when the amount of data is large. Summary of the invention

[0019] The technical problem to be solved by the present invention is to overcome the above technical defects and provide a long text sentiment analysis method based on the field of modern Chinese grammar, economy and security.

[0020] In order to solve the above problems, the technical solution of the present invention comprises the following steps:

[0021] S1. Obtain online public opinion data;

[0022] S2, data preprocessing;

[0023] S2.1. Clean the acquired data;

[0024] S2.2, classify the text and balance the sample ratio;

[0025] S2.3, use the dictionary of special terms in the field of economic security to perform lexical segmentation;

[0026] S3, model building;

[0027] S3.1, randomly initialize the word embedding matrix and obtain the static word vector;

[0028] S3.2, define the position encoder layer using relative position encoding;

[0029] S3.3, feature extraction using bidirectional Transformer model;

[0030] S3.4, enhancing semantic representation through multi-head self-attention mechanism;

[0031] S3.5, process very long texts in segments and establish relationships between paragraphs through the attention mechanism;

[0032] S3.6, convert the features into probability distribution of sentiment classification through the fully connected layer and Softmax function;

[0033] S4, model training;

[0034] S5. Model evaluation.

[0035] Furthermore, the text in step S2.2 is classified into positive, neutral and negative.

[0036] Furthermore, in step S3.3, the Transformer model is used to connect the contextual semantic relationship to capture the potential implicit semantic representation in the sentence.

[0037] Furthermore, the segmentation processing and attention mechanism in step S3.5 enable multiple paragraphs of text to have a relationship between the upper and lower paragraphs, and then pass it to the feedforward neural network layer.

[0038] Further, in step S5, steps S1-S4 are repeated for unsatisfactory data in the model evaluation, and in step S5, ideal data in the model evaluation is saved.

[0039] Furthermore, the network public opinion data in step S1 is derived from corporate public opinion acquired by web crawlers, especially corporate and bank reports related to economic security.

[0040] Furthermore, the dictionary of proper nouns in the field of economic security in step S2.3 is compiled based on the entries in the field of law in the "Encyclopedia of China", and the proper nouns are segmented at the "lexical level".

[0041] Furthermore, in step S3.4, the multi-head self-attention mechanism is used to normalize the layer organization residual connection structure sub-layer.

[0042] Furthermore, in step S4, the model training uses millions of online public opinion data, 10,000 pieces of data are used for cross-validation, and 10,000 pieces of data are used for final testing.

[0043] The advantages of the present invention compared with the prior art are:

[0044] 1. The present invention provides a long text sentiment analysis method in the field of economic security based on modern Chinese grammar. When doing Chinese word segmentation, a special dictionary in the field of economic security is used to replace the traditional word segmentation based on single words, so that the word segmentation results are more suitable for tasks in the field of economic security;

[0045] 2. The present invention provides a long text sentiment analysis method in the field of economic security based on modern Chinese grammar. The training corpus uses corporate public opinions obtained through network-oriented acquisition (combining self-developed web crawlers and purchased data). Each piece of public opinion is a report related to enterprises and banks, which is very suitable for scenarios in the field of economic security.

[0046] 3. The present invention provides a long text sentiment analysis method based on the field of modern Chinese grammar, economic security, and in feature extraction, relative position coding is used to replace the absolute position coding of the traditional Transformer, so that the model can learn by itself and adjust the parameters in the network;

[0047] 4. The present invention provides a long text sentiment analysis method based on the field of modern Chinese grammar economic security. When the model effect is not good, a loop mechanism (the process from data acquisition to model training and model evaluation) is adopted to model the ultra-long text, so that the characteristics of the ultra-long text in the high-dimensional vector space can be represented. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a workflow diagram of a long text sentiment analysis method based on the field of modern Chinese grammar, economy and security of the present invention. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0050] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] A long text sentiment analysis method based on the field of modern Chinese grammar economic security includes the following steps:

[0052] S1. Obtain online public opinion data;

[0053] S2, data preprocessing;

[0054] S2.1. Clean the acquired data;

[0055] S2.2, classify the text and balance the sample ratio;

[0056] S2.3, use the dictionary of special terms in the field of economic security to perform lexical segmentation;

[0057] S3, model building;

[0058] S3.1, randomly initialize the word embedding matrix and obtain the static word vector;

[0059] S3.2, define the position encoder layer using relative position encoding;

[0060] S3.3, feature extraction using bidirectional Transformer model;

[0061] S3.4, enhancing semantic representation through multi-head self-attention mechanism;

[0062] S3.5, process very long texts in segments and establish relationships between paragraphs through the attention mechanism;

[0063] S3.6, convert the features into probability distribution of sentiment classification through the fully connected layer and Softmax function;

[0064] S4, model training;

[0065] S5. Model evaluation.

[0066] Furthermore, the online public opinion data in step S1 is derived from corporate public opinion acquired by web crawlers, especially corporate and bank reports related to economic security.

[0067] Furthermore, in step S2.2, the text is classified into positive, neutral and negative.

[0068] Furthermore, in step S2.3, the dictionary of proper nouns in the field of economic security is compiled based on the entries in the field of law in the Encyclopedia of China, and the proper nouns are segmented at the "lexical level".

[0069] Furthermore, in step S3.3, the Transformer model is used to connect the contextual semantic relations to capture the potential implicit semantic representation in the sentence.

[0070] Furthermore, in step S3.4, the multi-head self-attention mechanism is used to normalize the layer organization residual connection structure sub-layer.

[0071] Furthermore, the segmentation processing and attention mechanism in step S3.5 enable multiple paragraphs of text to have a relationship between the upper and lower paragraphs, and then pass it to the feedforward neural network layer.

[0072] Furthermore, in step S4, the model training uses millions of online public opinion data, 10,000 pieces of data are used for cross-validation, and 10,000 pieces of data are used for final testing.

[0073] Further, in step S5, steps S1-S4 are repeated for unsatisfactory data in the model evaluation, and in step S5, ideal data in the model evaluation is saved.

[0074] In specific use, according to Figure 1As shown in the figure, first, we use web crawler technology to collect reports on companies and banks related to economic security from major news websites, financial forums, social media platforms and other channels. These reports cover various economic events, policy changes, market fluctuations, etc., and are an important data source for analyzing sentiment trends in the field of economic security. The crawler program is configured to execute regularly to ensure that the acquired data is up-to-date and comprehensive.

[0075] The raw data obtained often contains noise, such as advertisements, irrelevant links, duplicate content, etc. We use tools such as regular expressions and HTML parsers to clean the text, remove these non-text information, and retain the pure text content.

[0076] The cleaned text is divided into three categories according to sentiment tendency: positive, neutral, and negative. Since the distribution of categories in actual data may be uneven, we adjust the sample ratio through oversampling or undersampling technology to ensure the balance of data in each category during training.

[0077] The text is segmented at the lexical level using a dictionary of economic security terms compiled based on the legal field entries in the Encyclopedia of China, combined with general segmentation tools (such as Ji Eba). This step pays special attention to the professional terms in the economic security field to improve the accuracy and pertinence of the segmentation.

[0078] The word embedding matrix is ​​generated by random initialization, mapping each word in the text to a vector in a high-dimensional space. These vectors will be used as the input of the model.

[0079] To handle the sequential information in the text, we define the positional encoder layer using relative position encoding, which can capture the relative position relationship between words and is crucial for understanding the overall meaning of a sentence.

[0080] The bidirectional Transformer model is used for feature extraction. The model captures the contextual semantic relationship in the sentence through the self-attention mechanism and generates the implicit semantic representation of each word. Bidirectionality means that the model can consider the information before and after each word at the same time when processing it.

[0081] A multi-head self-attention mechanism is introduced into the Transformer model to enhance the semantic representation ability of the model. The multi-head mechanism allows the model to capture the association between words from different subspaces, while the self-attention mechanism enables the model to focus on important information in the entire sentence.

[0082] For very long texts, we adopt a segmented processing method. After each paragraph of text is processed by the Transformer model, the relationship between paragraphs is established through the attention mechanism. This processing method not only retains the local features of the text, but also takes into account the global context information.

[0083] Finally, the features are converted into probability distribution of sentiment classification through a fully connected layer and a Softmax function. The output layer contains three neurons, corresponding to the positive, neutral, and negative categories respectively.

[0084] The model was trained using millions of online public opinion data. During the training process, we used cross-validation to evaluate the performance of the model and adjusted the model parameters based on the validation results. Finally, we used another 10,000 pieces of data for testing to verify the generalization ability of the model.

[0085] The model output on the test set is evaluated, and the accuracy, recall, F1 score and other indicators are calculated as shown in Table 1. If the evaluation result is not ideal, return to steps S1 to S4 to reprocess the data and retrain the model. If the evaluation result is ideal, save the model parameters and architecture for subsequent use.

[0086]

[0087] Table 1

[0088] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0089] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0090] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A long text sentiment analysis method based on the field of modern Chinese grammar, economic security, characterized by: The following steps are involved: S1. Obtain online public opinion data; S2, data preprocessing; S2.

1. Clean the acquired data; S2.2, classify the text and balance the sample ratio; S2.3, use the dictionary of special terms in the field of economic security to perform lexical segmentation; S3, model building; S3.1, randomly initialize the word embedding matrix and obtain the static word vector; S3.2, define the position encoder layer using relative position encoding; S3.3, feature extraction using bidirectional Transformer model; S3.4, enhancing semantic representation through multi-head self-attention mechanism; S3.5, process very long texts in segments and establish relationships between paragraphs through the attention mechanism; S3.6, convert the features into probability distribution of sentiment classification through the fully connected layer and Softmax function; S4, model training; S5. Model evaluation.

2. According to claim 1, a long text sentiment analysis method based on the field of modern Chinese grammar, economic security, is characterized by: The network public opinion data in step S1 is derived from corporate public opinion acquired by web crawlers, especially corporate and bank reports related to economic security.

3. The long text sentiment analysis method based on the field of modern Chinese grammar, economy and security according to claim 1 is characterized by: The text in step S2.2 is classified into positive, neutral and negative.

4. The long text sentiment analysis method based on the field of modern Chinese grammar, economy and security according to claim 1 is characterized by: The dictionary of proper nouns in the field of economic security in step S2.3 is compiled based on the entries in the field of law in the "Encyclopedia of China", and the proper nouns are segmented at the "lexical level".

5. The long text sentiment analysis method based on the field of modern Chinese grammar, economy and security according to claim 1 is characterized by: The Transformer model in step S3.3 is used to connect the contextual semantic relationship to capture the potential implicit semantic representation in the sentence.

6. The long text sentiment analysis method based on the field of modern Chinese grammar, economy and security according to claim 1 is characterized by: In step S3.4, the multi-head self-attention mechanism is used to normalize the layer organization residual connection structure sub-layer.

7. The long text sentiment analysis method based on modern Chinese grammar, economic security field according to claim 1 is characterized by: The segmentation processing and attention mechanism in step S3.5 enable multiple paragraphs of text to have a relationship between the upper and lower paragraphs, and then pass it to the feedforward neural network layer.

8. The long text sentiment analysis method based on the field of modern Chinese grammar, economy and security according to claim 1 is characterized by: In step S4, the model training uses millions of online public opinion data, 10,000 pieces of data are used for cross-validation, and 10,000 pieces of data are used for final testing.

9. The long text sentiment analysis method based on modern Chinese grammar, economic security field according to claim 1 is characterized by: In the step S5, steps S1-S4 are repeated for the unsatisfactory data in the model evaluation, and in the step S5, the ideal data in the model evaluation is saved.

Images