A method, device, electronic device and storage medium for sentiment classification
By training the combination of natural language pre-trained model and text classification model, using BERT and TextCNN models to optimize parameters, the problem of inaccurate fine-grained sentiment analysis in the existing technology is solved, and fine-grained sentiment analysis of text and improving classification efficiency is achieved. .
Patent Information
- Application Number
- CN201911199692.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-11-29
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2040-06-08
AI Technical Summary
Existing sentiment analysis methods cannot accurately perform fine-grained sentiment analysis and cannot effectively express the objects users want to know.
The serial natural language pre-trained model and text classification model are trained through the training set, and the emotion classification combination model is obtained by combining it. The loss function is calculated using the BERT model and the TextCNN model, and the parameters are optimized to achieve fine-grained emotion classification.
The fine-grained sentiment analysis of the text is realized, the accuracy and speed of emotion classification are improved, and the performance of existing TD-LSTM, TC-LSTM and other models are surpassed.
Smart Images
Figure CN112883188B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a sentiment classification method, device, electronic device and storage medium. Background Art
[0002] With the rise of social networks, more and more people like to express their opinions on the Internet. This kind of text with personal subjective attitudes has great mining value. Due to the increasing user demand, sentence-level sentiment analysis can only give the sentiment value of a whole sentence and cannot accurately express the object that the user wants to know, so the granularity of sentiment analysis needs to be more refined. However, the existing sentiment analysis methods cannot accurately perform fine-grained sentiment analysis on text. Summary of the invention
[0003] In view of this, the purpose of the present invention is to propose a sentiment classification method, device, electronic device and storage medium to solve the technical problem that fine-grained sentiment classification is not accurate enough.
[0004] Based on the above purpose, according to a first aspect of the present invention, a sentiment classification method is provided, comprising:
[0005] A sentiment classification combination model is obtained by training a series of natural language pre-training models and text classification models through a training set; wherein the training set includes a plurality of training samples, each training sample includes a text, an aspect word in the text, and a classification label of the aspect word;
[0006] The text to be predicted and the aspect words in the text to be predicted are input into the sentiment classification combination model to output classification labels of the aspect words in the text to be predicted.
[0007] In some embodiments of the present invention, a sentiment classification combination model is obtained by training a serial natural language pre-training model and a text classification model through a training set, including:
[0008] Inputting the training set into a natural language pre-training model to obtain a first semantic feature vector output by the natural language pre-training model;
[0009] Inputting the first semantic feature vector output by the natural language pre-training model into a text classification model to obtain a second semantic feature vector output by the text classification model;
[0010] The parameters of the natural language pre-training model and the parameters of the text classification model are optimized according to the second semantic feature vector output by the text classification model, thereby obtaining a sentiment classification combination model.
[0011] In some embodiments of the present invention, the parameters of the natural language pre-training model and the parameters of the text classification model are optimized according to the second semantic feature vector output by the text classification model, thereby obtaining a sentiment classification combination model, including:
[0012] Calculating a loss function based on a second semantic feature vector output by the text classification model, parameters of the natural language pre-training model, and parameters of the text classification model;
[0013] By minimizing the loss function, the optimal parameters of the natural language pre-training model and the optimal parameters of the text classification model are obtained, thereby obtaining a sentiment classification combination model.
[0014] In some embodiments of the present invention, the loss function is a cross entropy loss function.
[0015] In some embodiments of the present invention, the natural language pre-training model is a BERT model, and / or the text classification model is a TextCNN model.
[0016] According to a second aspect of the present invention, there is provided a sentiment classification device, comprising:
[0017] The training module is configured to: train a serial natural language pre-training model and a text classification model through a training set to obtain a sentiment classification combination model; wherein the training set includes a plurality of training samples, each training sample includes a text, an aspect word in the text, and a classification label of the aspect word;
[0018] The prediction module is configured to: input the text to be predicted and the aspect words in the text to be predicted into the sentiment classification combination model to output the classification labels of the aspect words in the text to be predicted.
[0019] In some embodiments of the present invention, the training module is further used to:
[0020] Inputting the training set into a natural language pre-training model to obtain a first semantic feature vector output by the natural language pre-training model;
[0021] Inputting the first semantic feature vector output by the natural language pre-training model into a text classification model to obtain a second semantic feature vector output by the text classification model;
[0022] The parameters of the natural language pre-training model and the parameters of the text classification model are optimized according to the second semantic feature vector output by the text classification model, thereby obtaining a sentiment classification combination model.
[0023] In some embodiments of the present invention, the training module is further used to:
[0024] Calculating a loss function based on a second semantic feature vector output by the text classification model, parameters of the natural language pre-training model, and parameters of the text classification model;
[0025] By minimizing the loss function, the optimal parameters of the natural language pre-training model and the optimal parameters of the text classification model are obtained, thereby obtaining a sentiment classification combination model.
[0026] In some embodiments of the present invention, the loss function is a cross entropy loss function.
[0027] In some embodiments of the present invention, the natural language pre-training model is a BERT model, and / or the text classification model is a TextCNN model.
[0028] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of the above embodiments is implemented.
[0029] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided, characterized in that the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any one of the above embodiments.
[0030] From the above description, it can be seen that the sentiment classification method, device, electronic device and storage medium provided in the embodiments of the present invention serially combine the natural language pre-training model and the text classification model, and train these two models through a training set. The trained combined model can realize fine-grained sentiment classification based on aspect words, thereby accurately performing fine-grained sentiment analysis on the text. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 A schematic diagram of a flow chart of a sentiment classification method according to an embodiment of the present invention;
[0033] Figure 2 A schematic diagram of the structure of a sentiment classification combination model according to an embodiment of the present invention;
[0034] Figure 3A schematic diagram of a flow chart of a sentiment classification method according to another embodiment of the present invention;
[0035] Figure 4 Schematic diagram showing input and output of the sentiment classification combination model according to an embodiment of the present invention;
[0036] Figure 5 is a schematic diagram of the structure of a sentiment classification device according to an embodiment of the present invention;
[0037] Figure 6 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0038] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.
[0039] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0040] In one embodiment of the present invention, Figure 1 As shown, an embodiment of the present invention provides a sentiment classification method, the method comprising:
[0041] Step 101, training a serial natural language pre-training model and a text classification model through a training set to obtain a sentiment classification combination model.
[0042] In this step, the serial natural language pre-training model and the text classification model are trained by the training set, thereby obtaining a sentiment classification combination model. The embodiment of the present invention combines the natural language pre-training model and the text classification model, and trains the combination model by the training set. The trained combination model can achieve fine-grained sentiment classification to a certain extent.
[0043] Optionally, the training set includes a plurality of training samples, each of which includes a text, an aspect word in the text, and a classification label of the aspect word. It should be noted that, in an embodiment of the present invention, an aspect word refers to a word that a user pays attention to, that is, an object that a user wants to know. In order to ensure the model training effect, each training sample in the training set can be manually annotated, such as obtaining each aspect word in the text for each text, and marking classification labels for each aspect word respectively. Optionally, the classification label can be configured according to actual needs, and can be divided into two, three, five, ten or twenty categories, etc., and these classification labels are used to mark the classification of each aspect word in the text. For example, the label can be divided into three categories, i.e., a positive label, a negative label and a neutral label. Optionally, in order to save the step of artificial standard, the training sample can also be obtained from the COAE2014 data set or the COAE2015 data set as the training set.
[0044] Optionally, step 101 may include: inputting the training set into a natural language pre-training model to obtain a first semantic feature vector output by the natural language pre-training model; inputting the first semantic feature vector output by the natural language pre-training model into a text classification model to obtain a second semantic feature vector output by the text classification model; and optimizing the parameters of the natural language pre-training model and the parameters of the text classification model according to the second semantic feature vector output by the text classification model, thereby obtaining a sentiment classification combination model.
[0045] In order to train the sentiment classification combination model, the training set is first input into the natural language pre-training model, and then the first semantic feature vector output by the natural language pre-training model is input into the text classification model, and the text classification model outputs the second semantic feature vector, and finally the parameters of the natural language pre-training model and the parameters of the text classification model are optimized according to the second semantic feature vector output by the text classification model, so as to obtain the sentiment classification combination model. In the process of training the sentiment classification combination model, the parameters of the natural language pre-training model and the parameters of the text classification model are optimized through multiple iterative training to obtain the optimal model parameters, so as to obtain the sentiment classification combination model.
[0046] Optionally, the parameters of the natural language pre-training model and the parameters of the text classification model are optimized according to the second semantic feature vector output by the text classification model to obtain a sentiment classification combination model, including: calculating a loss function based on the second semantic feature vector output by the text classification model, the parameters of the natural language pre-training model and the parameters of the text classification model; and obtaining the optimal parameters of the natural language pre-training model and the optimal parameters of the text classification model by minimizing the loss function to obtain the sentiment classification combination model.
[0047] The loss function is calculated by using the second semantic feature vector output by the text classification model, the parameters of the natural language pre-training model and the parameters of the text classification model, and the gradient descent and back-propagation algorithms are used to minimize the loss function to obtain the optimal parameters of the natural language pre-training model and the optimal parameters of the text classification model, thereby obtaining a sentiment classification combination model.
[0048] Optionally, the loss function is a cross entropy loss function, and the cross entropy loss function can be used to better optimize model parameters.
[0049] Optionally, the natural language pre-training model is a BERT model. Optionally, the text classification model is a TextCNN model.
[0050] The full name of the BERT model is Bidirectional Encoder Representations from Transformers. It is a new type of language model that trains pre-trained deep bidirectional representations by jointly adjusting the bidirectional Transformers in all layers, which can more accurately analyze emotions in a fine-grained manner. And because TextCNN has a good convergence effect, the number of iterative training of the model is significantly reduced, so the training speed of the BERT+TextCNN sentiment classification combination model is faster than that of the BERT model. Figure 2 FIG. 1 is a schematic diagram of the structure of the sentiment classification combination model according to an embodiment of the present invention. Figure 2 As shown, the sentiment classification combined model includes a serial BERT model and a TextCNN model. The text and the aspect words in the text are input into the BERT model through the input end, and the classification labels of the aspect words are input into the BERT model through the output end.
[0051] Step 102: input the text to be predicted and the aspect words in the text to be predicted into the sentiment classification combination model to output the classification labels of the aspect words in the text to be predicted.
[0052] First, various aspects of words are obtained from the text to be predicted, and then the text to be predicted and the various aspects of words obtained from the text to be predicted are input into the trained sentiment classification combination model. The sentiment classification combination model will output the classification labels corresponding to the various aspects of words in the text to be predicted, thereby realizing fine-grained sentiment classification of the text.
[0053] Therefore, the sentiment classification method provided in the embodiment of the present invention combines the natural language pre-training model and the text classification model in series, and trains the two models through the training set. The trained combined model can realize fine-grained sentiment classification based on aspect words, thereby accurately performing fine-grained sentiment analysis on the text.
[0054] As another embodiment of the present invention, Figure 3 As shown, taking the BERT model and the TextCNN model as examples, the sentiment classification method may include:
[0055] Step 301: obtain a training set and preprocess each training sample in the training set.
[0056] The training set includes multiple training samples, each of which includes text, aspect words in the text, and classification labels of the aspect words. Optionally, training samples can also be obtained from two public Chinese microblog sentiment tendency datasets, COAE2014 and COAE2015, as the training set. The text, aspect words, and classification labels are separated by TAB, as shown below:
[0057] The touch experience is very good! [TAB]Touch [TAB]1
[0058] Optionally, the labeled data format of the training samples is D = ([q1,...,qn], [d1,...,dn], yi), and each training sample consists of a text sequence [q1,...,qn], an aspect word sequence [d1,...,dn], and an aspect word classification label yi. The training sample is then represented as a single packaged sequence x, then:
[0059] x=([CLS],d1,...,dn,[SEP],q1,...,qm,[SEP])
[0060] Among them, [CLS] and [SEP] are special symbols for the classification output of the BERT model, and [SEP] is a special symbol for separating continuous tag sequences (here referring to text sequences and aspect word sequences).
[0061] Step 302: Train the serial natural language pre-training model and text classification model through the training set to obtain the BERT+TextCNN model.
[0062] Optionally, the packaged sequence x corresponding to each training sample in the training set is input into the BERT model to obtain the first semantic feature vector output by the BERT model; the first semantic feature vector is input into the TextCNN model to obtain the second semantic feature vector output by the TextCNN model; the parameters of the BERT model and the parameters of the TextCNN model are optimized according to the second semantic feature vector, thereby obtaining a BERT+TextCNN sentiment classification combination model. Specifically, the aspect word sequence in the packaged sequence x is embedded using A, and the text sequence is embedded using B. The input and output of the BERT model are represented as follows: Figure 4 shown.
[0063] In order to improve the performance and convergence speed of the model, the embodiment of the present invention feeds the first semantic feature vector output by the BERT model into a TextCNN model. Optionally, TextCNN uses 128 filters, each with three convolution kernel sizes of 2, 3, and 4, to further extract n-gram features of the text. Since the BERT model has extracted high-dimensional feature vectors for the combined model, fine-grained sentiment classification can be achieved by combining the strong and fast text classification capabilities of the TextCNN model. It should be pointed out that the original BERT model finally uses a single-layer neural network for classification, and this method is far inferior to the TextCNN model in terms of classification accuracy and speed.
[0064] Generally speaking, there are two types of BERT model sizes:
[0065] BERT BASE :L=12,H=768,A=12,TotalParameters=110M;
[0066] BERT LARGE :L=12,H=768,A=12,TotalParameters=340M.
[0067] Among them, L is the number of Transformer layers, H is the hidden layer size, and A is the number of self-attention heads. For ease of use, you can use the Chinese version of BERT BASE .
[0068] In an embodiment of the present invention, the hyperparameters of the two models may be selected as follows:
[0069] BERT model hyperparameter settings: select the training batch size (batch size) as 32, the learning rate (learning rate) of the Adam optimization algorithm is set to 5e-5, and the number of iterations (epochs) is 9;
[0070] TextCNN model hyperparameter settings: select the number of filters as 128, the convolution kernel size as [2, 3, 4], and the dropout rate as 0.1.
[0071] In the process of training the model: load the pre-trained weights of the BERT model, input each training sample into the Chinese version of the pre-trained model BERT, obtain the first semantic feature vector output by the BERT model, input the first semantic feature vector into the TextCNN model, perform convolution operation on the output of the BERT model through the text convolution structure, further extract the local features of the text and obtain the output result after the maximum pooling, and obtain the probability of each category of the classification output through the softmax layer. During the training process, the pre-trained weights of the BERT model and the parameters of the TextCNN model are continuously fine-tuned by maximizing the logarithmic probability of the correct label, so that the pre-trained BERT model is gradually migrated to a model for processing fine-grained sentiment classification tasks, and finally a BERT+TextCNN sentiment classification combination model is obtained.
[0072] The first semantic feature output by BERT can be expressed as:
[0073] h 0 =XW t +W s +W p
[0074] h i = Transformer(h i-1 ),i∈[1,L]
[0075] Among them, W t is the word embedding matrix, W p is the position embedding matrix, W s Sentence embedding matrix. h i is the hidden layer vector output by BERT, that is, the output of the i-th layer Transformer network.
[0076] The second semantic feature output by TextCNN can be expressed as:
[0077] h c =Conv(W c h l +b c )
[0078] h d =MaxPooling(h c )
[0079] Where W c is the convolutional layer matrix, h l It is the last hidden layer vector of BERT.
[0080] Optionally, the prediction probability of each category can be expressed as:
[0081]
[0082] Among them, h d is the second semantic feature vector output by the TextCNN model, W is the parameter matrix of the sentiment classification combination model, and b is the bias value.
[0083] Then, the cross entropy loss function is used as the loss function of the sentiment classification combination model:
[0084]
[0085] Where k is the number of classification labels. In this embodiment, k=2. They are the real sentiment label and the predicted sentiment label respectively. By minimizing the loss function, all parameters of the sentiment classification combination model are jointly fine-tuned to maximize the logarithmic probability of the correct label, that is, minimize the loss, and thus obtain the optimal parameters.
[0086] Step 303: input the text to be predicted and the aspect words in the text to be predicted into the BERT+TextCNN model to output the classification labels of the aspect words in the text to be predicted.
[0087] After the BERT+TextCNN model is trained, various aspects of the words are obtained from the text to be predicted, and then the text to be predicted and the various aspects of the words obtained from the text to be predicted are input into the trained BERT+TextCNN model. The BERT+TextCNN model will output the classification labels corresponding to the various aspects of the words in the text to be predicted, thereby realizing fine-grained sentiment classification of the text.
[0088] In order to verify the prediction accuracy of the BERT+TextCNN model, the embodiment of the present invention respectively tests the sentiment polarity classification effects of the BERT model, the BERT+TextCNN model, and the existing TD-LSTM model, the TC-LSTM model, the TD_TextCNN model, and the TC_TextCNN model on the COAE2014 dataset and the COAE2015 data. The COAE2014 dataset and the COAE2015 data are online product evaluation datasets. Each product evaluation has a corresponding aspect word and a corresponding polarity label (pos / neg). The data distribution of positive and negative polarities is shown in Table 1:
[0089] Table 1 Experimental data
[0090] polarity Training set Validation set Test Set COAE2014 Yang / Yin 4163 / 3136 417 / 318 775 / 994 COAE2015 Yang / Yin 6528 / 2893 887 / 316 1000 / 1000
[0091] In the experiment, each model was trained and validated using the same training set and validation set. During the test, the same fixed number of product evaluation test data (1769 items in the COAE2014 test data set and 2000 items in the COAE2015 test data set) was input. Each model outputs the sentiment polarity of each test data, and the accuracy Acc and F1 value of the sentiment polarity results output by each model are calculated to compare the performance between the models. The results are shown in Table 2.
[0092] Table 2 Experimental results of various models on two datasets
[0093]
[0094]
[0095] Experimental results show that in terms of accuracy and recall, the sentiment polarity classification results of the BERT+TextCNN model are significantly improved over the current TD-LSTM, TC-LSTM, ATAE-LSTM and other models.
[0096] Therefore, the embodiment of the present invention combines the BERT model and the TextCNN model in series, and trains the two models through a training set. The trained BERT+TextCNN model can realize fine-grained sentiment classification based on aspect words, thereby accurately performing fine-grained sentiment analysis on the text; and the use of TextCNN accelerates the fine-tuning convergence speed of the BERT model, so that the BERT+TextCNN model can be trained faster.
[0097] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the described method.
[0098] like Figure 5As shown, the embodiment of the present invention further provides a sentiment analysis device, including a training module 501 and a prediction module 502. The training module 501 is configured to: train a serial natural language pre-training model and a text classification model through a training set to obtain a sentiment classification combination model; wherein the training set includes a plurality of training samples, each training sample includes a text, an aspect word in the text, and a classification label of the aspect word; the prediction module 502 is configured to: input the text to be predicted and the aspect word in the text to be predicted into the sentiment classification combination model to output the classification label of the aspect word in the text to be predicted.
[0099] In some embodiments of the present invention, the training module 501 is further used to:
[0100] Inputting the training set into a natural language pre-training model to obtain a first semantic feature vector output by the natural language pre-training model;
[0101] Inputting the first semantic feature vector output by the natural language pre-training model into a text classification model to obtain a second semantic feature vector output by the text classification model;
[0102] The parameters of the natural language pre-training model and the parameters of the text classification model are optimized according to the second semantic feature vector output by the text classification model, thereby obtaining a sentiment classification combination model.
[0103] In some embodiments of the present invention, the training module 501 is further used to:
[0104] Calculating a loss function based on a second semantic feature vector output by the text classification model, parameters of the natural language pre-training model, and parameters of the text classification model;
[0105] By minimizing the loss function, the optimal parameters of the natural language pre-training model and the optimal parameters of the text classification model are obtained, thereby obtaining a sentiment classification combination model.
[0106] In some embodiments of the present invention, the loss function is a cross entropy loss function.
[0107] In some embodiments of the present invention, the natural language pre-training model is a BERT model, and / or the text classification model is a TextCNN model.
[0108] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0109] Figure 6A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 610, a memory 620, an input / output interface 630, a communication interface 640, and a bus 650. The processor 610, the memory 620, the input / output interface 630, and the communication interface 640 are connected to each other through the bus 650 in the device.
[0110] The processor 610 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0111] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 620 and are called and executed by the processor 610.
[0112] The input / output interface 630 is used to connect the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0113] The communication interface 640 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0114] The bus 650 comprises a pathway for transmitting information between the various components of the device (eg, the processor 610, the memory 620, the input / output interface 630, and the communication interface 640).
[0115] It should be noted that, although the above device only shows the processor 610, the memory 620, the input / output interface 630, the communication interface 640 and the bus 650, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.
[0116] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0117] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present invention, the above embodiments or technical features in different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity.
[0118] In addition, to simplify the description and discussion, and in order not to obscure the present invention, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the devices may be shown in the form of block diagrams to avoid obscuring the present invention, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the present invention will be implemented (i.e., these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present invention, it will be apparent to those skilled in the art that the present invention may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0119] Although the invention has been described in conjunction with specific embodiments of the invention, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
[0120] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A sentiment classification method, characterized in that: include: A sentiment classification combination model is obtained by training a series of natural language pre-training models and text classification models through a training set; wherein the training set includes a plurality of training samples, each training sample includes a text, an aspect word in the text, and a classification label of the aspect word, the natural language pre-training model is a BERT model, and / or the text classification model is a TextCNN model; Inputting the text to be predicted and the aspect words in the text to be predicted into the sentiment classification combination model to output classification labels of the aspect words in the text to be predicted; Among them, the serial natural language pre-training model and text classification model are trained by the training set to obtain the sentiment classification combination model, including: The training set is input into the natural language pre-training model to obtain a first semantic feature vector output by the natural language pre-training model, where the first semantic feature vector is expressed as: Among them, W t is the word embedding matrix, W p is the position embedding matrix, W s Sentence embedding matrix, h i is the hidden layer vector output by BERT, i.e., the output of the i-th layer Transformer network; The first semantic feature vector output by the natural language pre-training model is input into the text classification model to obtain the second semantic feature vector output by the text classification model, wherein the second semantic feature vector is represented as: c =Conv(W c h l +b c ), h d =MaxPooling(h c ), where W c is the convolutional layer matrix, h l It is the last hidden layer vector of BERT; The parameters of the natural language pre-training model and the parameters of the text classification model are optimized according to the second semantic feature vector output by the text classification model, so as to obtain a sentiment classification combination model. The prediction result probability of each category can be expressed as: Among them, h d is the second semantic feature vector output by the TextCNN model, W is the parameter matrix of the sentiment classification combination model, and b is the bias value.
2. The sentiment classification method according to claim 1, characterized in that: Optimizing the parameters of the natural language pre-training model and the parameters of the text classification model according to the second semantic feature vector output by the text classification model, thereby obtaining a sentiment classification combination model, including: Calculating a loss function based on a second semantic feature vector output by the text classification model, parameters of the natural language pre-training model, and parameters of the text classification model; By minimizing the loss function, the optimal parameters of the natural language pre-training model and the optimal parameters of the text classification model are obtained, thereby obtaining a sentiment classification combination model.
3. The sentiment classification method according to claim 2, characterized in that: The loss function is a cross entropy loss function.
4. A sentiment classification device, characterized in that: include: The training module is configured to: train a series of natural language pre-training models and text classification models through a training set to obtain a sentiment classification combination model; wherein the training set includes a plurality of training samples, each training sample includes a text, an aspect word in the text, and a classification label of the aspect word, the natural language pre-training model is a BERT model, and / or the text classification model is a TextCNN model; The prediction module is configured to: input the text to be predicted and the aspect words in the text to be predicted into the sentiment classification combination model to output the classification labels of the aspect words in the text to be predicted; The training module is further configured to: input the training set into the natural language pre-training model to obtain a first semantic feature vector output by the natural language pre-training model, wherein the first semantic feature vector is expressed as: Among them, W t is the word embedding matrix, W p is the position embedding matrix, W s Sentence embedding matrix, h i is the hidden layer vector output by BERT, that is, the output of the i-th layer Transformer network; the first semantic feature vector output by the natural language pre-training model is input into the text classification model to obtain the second semantic feature vector output by the text classification model, and the second semantic feature vector is expressed as: c =Conv(W c h l +b c ), h d =MaxPooling(h c ), where W c is the convolutional layer matrix, h l is the last hidden layer vector of BERT; according to the second semantic feature vector output by the text classification model, the parameters of the natural language pre-training model and the parameters of the text classification model are optimized to obtain a sentiment classification combination model, and the prediction result probability of each category can be expressed as: Among them, h d is the second semantic feature vector output by the TextCNN model, W is the parameter matrix of the sentiment classification combination model, and b is the bias value.
5. The device according to claim 4, characterized in that The training module is further configured to: Calculating a loss function based on a second semantic feature vector output by the text classification model, parameters of the natural language pre-training model, and parameters of the text classification model; By minimizing the loss function, the optimal parameters of the natural language pre-training model and the optimal parameters of the text classification model are obtained, thereby obtaining a sentiment classification combination model.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 3 is implemented.
7. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 3.