A method for analyzing multilingual text data

By adopting the architecture of CNN and LSTM in sentiment analysis, the language limitations of multilingual text data analysis in the prior art are solved, cross-language sentiment analysis and high-accurate text sentiment prediction are realized, which is suitable for a variety of Locale.

CN113076741BActive Publication Date: 2024-12-03SHANXI SANYOUHUO INTELLIGENCE INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110257644.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-09
Publication Date
2024-12-03
Estimated Expiration
2041-03-09

AI Technical Summary

Technical Problem

Most of the existing sentiment analysis methods are only applicable to specific languages, which limits the effective analysis and application of multilingual text data, making it difficult to achieve accurate emotion recognition in different language environments.

Method used

Using a convolutional neural network (CNN) and long and short-term memory network (LSTM), features are extracted through word embedding layers, combined with the maximum pooling layer, full connection layer, Dropout layer and batch standardization layer, a processable process is constructed. Sentiment analysis model of multilingual text data ensures language-independent feature extraction and high accuracy.

Benefits of technology

Cross-language sentiment analysis is realized, and can predict the emotions of text in different language environments with high accuracy. It is highly extensible and is suitable for text sentiment analysis in multiple languages, avoiding the dependence of language-specific tools.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113076741B_ABST
    Figure CN113076741B_ABST
Patent Text Reader

Abstract

The present application relates to the field of natural language processing, and in particular to a method for analyzing multilingual text data, comprising the following modules: a text data acquisition module, a data processing module, a model construction module and a model storage module, wherein the text data acquisition module is used to acquire text data, and to label the text data categories, so as to complete the data set construction of the model; the data processing module is used to convert the terms in the text into vectors through an embedding layer to facilitate entry into the architecture; the model construction module is used to construct a training model based on a convolutional neural network (CNN) and a long short-term memory network (LSTM), extract the features of the model that are independent of language through the convolutional neural network, and use the LSTM to capture long-term dependencies for subsequent data processing; the model storage module is used to save the model when the loss function of the model no longer decreases; the present invention enables an application to utilize the results obtained by sentiment analysis in a language-forgetting or language-independent manner.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method for analyzing multilingual text data. Background Art

[0002] The amount of text generated by social media platforms and online forums is increasing rapidly, causing companies, government agencies and media organizations to try to perform sentiment analysis on this rich text data. The results of these analyses are used to adjust marketing strategies, customize products, security and various other decisions, but most methods are only applicable to a specific field or a specific language.

[0003] Most approaches to sentiment analysis only work on texts written in a specific language, which greatly limits their applicability to specific demographics and geographic regions. Summary of the invention

[0004] Based on this, the present invention completes the general multilingual text data sentiment analysis by developing a convolutional neural network (CNN)-based and long-term memory (LSTM)-based architecture. First, the word embedding layer is mapped into a vector for subsequent feature extraction, and then one path uses CNN to extract n-gram features followed by a maximum pooling layer, while the other path uses two layers of LSTM to capture long-term dependencies, and finally the two paths are fully connected together. Layer, Dropout layer, batch normalization and Softmax layer to prevent overfitting and improve accuracy. Such an architecture can achieve language-independent feature extraction while ensuring a higher accuracy rate.

[0005] This application discloses the following technical solutions:

[0006] The present application discloses a method for analyzing multilingual text data, comprising the following modules: a text data acquisition module, a data processing module, a model building module and a model saving module.

[0007] The text data collection module is used to collect text data and mark the text data categories to complete the data set construction of the model;

[0008] The data processing module is used to convert the terms in the text into vectors through the embedding layer to facilitate entry into the architecture;

[0009] The model building module is used to build a training model based on a convolutional neural network (CNN) and a long short-term memory (LSTM) network, extract features of a model that is language-independent through the convolutional neural network, and use the LSTM to capture long-term dependencies for subsequent data processing;

[0010] The model saving module is used to save the model when the loss function of the model no longer decreases.

[0011] Furthermore, the text data collection module specifically includes the following modules:

[0012] In the text data collection module, the data set used may consist of text content in multiple languages ​​and be manually annotated as positive, neutral, negative or irrelevant.

[0013] Furthermore, the data processing module specifically includes the following modules:

[0014] In the data processing module, the data set is divided into standard training set and test set and then input into the model. The model performs word embedding for the given input text, that is, mapping each word to a d-dimensional vector for subsequent learning of feature representation. The feature is forwarded to a small classifier for final prediction.

[0015] Furthermore, the model saving module specifically includes the following submodules:

[0016] CNN submodule: CNN is a convolutional neural network, which is used to extract features of language-independent models;

[0017] LSTM submodule: LSTM stands for long short-term memory network, which uses two layers of LSTM to capture long-term dependencies for subsequent data processing;

[0018] Maximum pooling layer submodule: used to extract features and reduce dimensionality;

[0019] Fully connected layer submodule: Integrate the local information with category distinction in the convolution layer or pooling layer to improve the performance of the entire network;

[0020] Dropout layer submodule: Adding a Dropout layer can prevent the model from overfitting. Dropout means that during the training process of the deep learning network, a part of the neural network units are temporarily discarded from the network according to a certain probability, which is equivalent to finding a more concise network from the original network.

[0021] Batch Normalization submodule: Through normalization, the distribution of activation input values ​​of any neuron in each layer of the neural network before nonlinear transformation is forcibly pulled back to the standard normal distribution with a mean of 0 and a variance of 1;

[0022] Softmax layer submodule: that is, the Softmax regression model, which is used to solve the classification problem in the model.

[0023] This application has the following beneficial effects:

[0024] This proposal proposes a universal approach to sentiment analysis on multilingual text data. This enables all applications to leverage the results of sentiment analysis in a language-forgetful or language-independent manner. The proposed model focuses on predicting the sentiment of a text while ignoring the language of the text and does not contain any language-specific tools. Therefore, it can be easily extended to language-independent sentiment analysis or to more languages. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain and illustrate the present application, but should not be construed as limiting the scope of protection of the present application.

[0026] Figure 1 It is a flow chart of the main steps disclosed in this application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present application more clear, the following will be combined with the appended examples of the present application. Figure 1 , the technical solutions in the embodiments of the present application are described in more detail.

[0028] This application discloses a method for analyzing multilingual text data, such as Figure 1 As shown, it includes the following modules: text data collection module, data processing module, model building module and model saving module.

[0029] The text data collection module is used to collect text data and mark the text data categories to complete the data set construction of the model;

[0030] The data processing module is used to convert the terms in the text into vectors through the embedding layer to facilitate entry into the architecture;

[0031] The model building module is used to build a training model based on a convolutional neural network (CNN) and a long short-term memory (LSTM) network, extract features through the convolutional neural network, and use LSTM to capture long-term dependencies for language-independent models for subsequent data processing;

[0032] The model saving module is used to save the model when the loss function of the model no longer decreases.

[0033] In the text data collection module, the text data required by the model is collected and its category is annotated to complete the data set construction required by the model. The data set used in this solution can be composed of text content in multiple languages ​​and manually annotated as positive, neutral, negative or irrelevant.

[0034] In the data processing module, the embedding layer converts the words in the text into vectors to facilitate entry into the architecture. After the data set is split into standard training and test sets, it is input into the model. For a given input text, the model first maps each word to a d-dimensional vector, called a word embedding. Word embedding is used to learn feature representations later. These features are forwarded to a small classifier for final prediction. More formally, if a text T has t words (T=<w1,w2,w3,…,wt> ), where each word is defined by a d-dimensional vector randomly initialized from a uniform distribution.

[0035] In the model building module,

[0036] The goal is to predict the sentiment y∈{positive, neutral, negative} of T. To achieve this goal, a convolutional neural network (CNN) and long-term memory (LSTM) based architecture is proposed. The choice of CNN is to learn n-gram features from the input text, because using n-gram features is useful for language-independent models. However, these existing architectures ignore the fact that encoding long-term dependencies in the text is also important for learning robust feature representations. So two layers of LSTM are used to capture long-term dependencies, while CNN is used to learn n-gram features. A max pooling layer is used on the representation learned by CNN, which outputs the maximum value from each feature map learned by a particular filter. The output of the second LSTM and the global max pooling layer is then forwarded to a small feed-forward network for the final prediction. This feed-forward network consists of a fully connected layer, followed by a dropout layer and a batch normalization layer to avoid overfitting. We use the “relu” activation function for the cnn and fully connected layers, which is defined by σ(X)=max(x,0). Finally, a “Softmax” layer is used to calculate the probability of each class and assign a label based on the maximum probability.

[0037] It includes the following sub-modules:

[0038] CNN submodule: Convolutional neural network is a feedforward neural network with deep structure and convolutional calculation. Convolutional neural network has the ability of representation learning and can classify input information in a translation-invariant manner according to its hierarchical structure. Therefore, it is also called "translation-invariant artificial neural network". The reason why CNN is used in the architecture is that it is effective in extracting features of language-independent models.

[0039] LSTM submodule: Long short-term memory network is a time recurrent neural network, which is specially designed to solve the long-term dependency problem of general RNN (recurrent neural network). All RNNs have a chain form of repeated neural network modules. The framework uses two layers of LSTM to capture long-term dependencies for subsequent data processing;

[0040] Max pooling layer submodule: A processing module frequently used in the max pooling layer. It is widely cited for preserving local information of word combinations in texts. Its main function is to extract features and reduce dimensionality.

[0041] Fully connected layer submodule: Each node of the fully connected layer is connected to all nodes of the previous layer to integrate the features extracted previously. Due to its fully connected characteristics, it is named fully connected layer. The fully connected layer can integrate the local information with category distinction in the convolutional layer or pooling layer, thereby improving the performance of the entire network;

[0042] Dropout layer submodule: Dropout means that during the training process of the deep learning network, a part of the neural network units are temporarily discarded from the network according to a certain probability, which is equivalent to finding a more concise network from the original network. Adding the Dropout layer is mainly to prevent the model from overfitting;

[0043] Batch Normalization submodule: Batch Normalization. Because the distribution of the activation input values ​​of the deep neural network before nonlinear transformation gradually shifts or changes as the network depth deepens or during the training process, the reason why the training converges slowly is generally that the overall distribution gradually approaches the upper and lower limits of the value range of the nonlinear function. Therefore, this causes the gradient of the low-level neural network to disappear during back propagation. This is the essential reason why the training of deep neural networks converges more and more slowly. Batch Normalization uses certain normalization methods to forcibly pull the distribution of the input value of any neuron in each layer of the neural network back to a standard normal distribution with a mean of 0 and a variance of 1. In this way, a small change in the input will lead to a large change in the loss function. Its significance is to make the gradient larger and avoid the problem of gradient vanishing. Moreover, a larger gradient means that the learning converges quickly, which can greatly speed up the training.

[0044] Softmax layer submodule: The Softmax regression model is a generalization of the logistic regression model for multi-classification problems. In multi-classification problems, the Softmax regression model is mainly used to solve classification problems.

[0045] In the module model saving module, when the loss function is no longer decreasing, the training model is saved.

[0046] All of the above modules can be encapsulated into one application and collaboratively complete technical functions based on multilingual text data analysis by calling each other's interfaces.

[0047] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for analyzing multilingual text data, characterized in that: include: Text data collection module, data processing module, model building module and model saving module, The text data collection module is used to collect text data and mark the text data categories to complete the data set construction of the model; The data processing module is used to convert the terms in the text into vectors through the embedding layer to facilitate entry into the architecture; The model building module is used to build a training model based on a convolutional neural network (CNN) and a long short-term memory (LSTM) network. One path uses CNN to extract n-gram features followed by a maximum pooling layer, while the other path uses two layers of LSTM to capture long-term dependencies. Finally, the two paths are combined with a full connection layer, a Dropout layer, a batch normalization layer, and a Softmax layer to prevent overfitting and improve accuracy for subsequent data processing. The model saving module is used to save the model when the loss function of the model no longer decreases; In the text data collection module, the data set used consists of text content in multiple languages ​​and is manually annotated as positive, neutral, negative or irrelevant; In the data processing module, the words in the text are converted into vectors through the embedding layer to facilitate entry into the architecture. The data set is divided into standard training sets and test sets and then input into the model. The model embeds words for the given input text, that is, maps each word to a d-dimensional vector for subsequent learning of feature representation. The features are forwarded to a small classifier for final prediction; The model building module includes the following submodules: CNN submodule: The choice of CNN is to learn n-gram features from the input text. CNN is a convolutional neural network. CNN is used to extract features of the model that is language-independent. LSTM submodule: LSTM stands for long short-term memory network, which uses two layers of LSTM to capture long-term dependencies for subsequent data processing; Maximum pooling layer submodule: used to extract features and reduce dimensionality; Fully connected layer submodule: Integrate the local information with category distinction in the convolution layer or pooling layer to improve the performance of the entire network; Dropout layer submodule: Add the Dropout layer to prevent the model from overfitting. Dropout means that during the training process of the deep learning network, a part of the neural network units are temporarily discarded from the network according to a certain probability, which is equivalent to finding a more concise network from the original network. Batch Normalization submodule: Through normalization, the distribution of activation input values ​​of any neuron in each layer of the neural network before nonlinear transformation is forcibly pulled back to the standard normal distribution with a mean of 0 and a variance of 1; Softmax layer submodule: that is, the Softmax regression model, which is used to solve the classification problem in the model.

2. A method for analyzing multilingual text data as claimed in claim 1, characterized in that: The text data acquisition module is connected to the data processing module, the data processing module is connected to the model construction module, and the model construction module is connected to the model storage module.