A method, device, computer equipment and storage medium for expanding sentiment dictionary

Abstract

The present invention relates to a method, device, computer equipment and storage medium for expanding a sentiment dictionary. The method comprises the following steps: obtaining text data, performing segmentation processing on the text data and determining target segmentations of the text data; taking the first N target segmentations with the largest word frequency as a word set; taking the intersection of the word set and a pre-constructed basic sentiment dictionary to obtain a seed sentiment dictionary composed of seed words; taking the target segmentations with a tf_idf value greater than a preset threshold as new words; determining the sentiment polarity of the new words based on the sentiment vector representation of the new words, the part of speech of the new words, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity; and determining the polarity value of the new words. The method does not require manual annotation of words, and each word can be assigned a different sentiment polarity value, so the polarity expression ability is stronger.

Description

Technical Field

[0001] The present invention relates to the technical field of sentiment analysis, and in particular to a method, device, computer equipment and storage medium for expanding a sentiment dictionary. Background Art

[0002] Articles, sentences, comments and other texts can usually reflect the emotional tendencies of the publisher, such as attitudes towards an event, preferences for an object or a person, etc. Words commonly used to express opinions, views and attitudes often have certain emotional tendencies. The emotional dictionary constructed based on these words is of great significance and can be used for personal image maintenance, corporate marketing management and brand promotion, and business product improvement.

[0003] The current methods for constructing sentiment dictionaries are:

[0004] 1. Expert annotation method

[0005] Expert annotation is the most direct method, that is, experts manually summarize words with sentimental tendencies by reading a large amount of relevant corpus or using existing dictionaries, annotate their sentiment polarity or intensity, and form a dictionary, such as the existing WordNet, General Inquirer (GI) dictionary and HowNet sentiment analysis word set.

[0006] shortcoming:

[0007] It requires a lot of manual labeling work, which is inefficient and susceptible to subjectivity;

[0008] The granularity and accuracy of sentiment intensity annotation cannot be guaranteed.

[0009] 2. Corpus-based methods

[0010] The most representative corpus-based method is SO-PMI (Semantic Orientation Pointwise Mutual Information). The SO-PMI algorithm consists of two parts: SO-PMI and PMI. The overall idea of ​​the algorithm is very simple. It determines the probability of the word P (word) to be judged and the base word P (base) appearing at the same time. If the probability of appearing with positive words is higher, then it is judged as a positive word. If the probability of appearing with negative words is higher, then it is judged as a negative word. If the probability of appearing with positive and negative words is the same, then it is judged as a neutral word.

[0011] shortcoming:

[0012] If the probability of a word appearing is relatively small, the correct sentiment tendency of the word cannot be obtained;

[0013] The co-occurrence window is difficult to select;

[0014] The semantics of the words are not considered;

[0015] There is only one polarity category obtained, but usually, many words have different polarities in different contexts, so some words have multiple polarities. Summary of the invention

[0016] In view of the above problems, the present invention is proposed to provide a method, apparatus, computer device and storage medium for expanding a sentiment dictionary that overcomes the above problems or at least partially solves the above problems.

[0017] In a first aspect, an embodiment of the present invention provides a method, the method comprising the following steps:

[0018] Acquire text data, perform word segmentation processing on the text data, and determine target word segmentation of the text data according to the obtained word segmentation;

[0019] Counting the word frequencies of the target word segments, and taking the top N target word segments with the largest word frequencies as the word set;

[0020] Taking the intersection of the word set and the pre-constructed basic sentiment dictionary, a seed sentiment dictionary consisting of seed words is obtained; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary;

[0021] Calculate the tf_idf value of the target word respectively, and take the target word with a tf_idf value greater than a preset threshold as a new word in the basic sentiment dictionary;

[0022] Determine the sentiment polarity of the new word based on the sentiment vector representation of the new word, the part of speech of the new word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity;

[0023] The polarity value of the newly added word is determined based on the sentiment vector representation of the newly added word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word.

[0024] Optionally, determining a target word segmentation of the text data according to the obtained word segmentation includes:

[0025] Remove the stop words from the obtained segmentations and use the remaining segmentations as the target segmentations.

[0026] Optionally, the determining the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity includes:

[0027] The new word and seed word are respectively input into the word vector model for training to obtain the new word vector and seed word vector. Based on the cosine similarity between the new word vector and various subword vectors, the similarity vector of the new word is determined.

[0028] The sentiment vector representation of the new word is determined based on the similarity vector of the new word, the polarity values ​​of various subwords, and the tf_idf value of the new word.

[0029] Optionally, the determining the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity further includes:

[0030] Determine the similarity vector of the seed word based on the cosine similarity between the seed word vector and various subword vectors;

[0031] Determine the sentiment vector representation of the seed word based on the similarity vector of the seed word, the polarity values ​​of various subwords, and the tf_idf value of the seed word;

[0032] The sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity is determined based on the sentiment vector representation of the seed word.

[0033] Optionally, the determining the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity further includes:

[0034] The polarity value of the seed word is determined based on the cosine similarity between the seed word vector and the center vector of the seed sentiment dictionary in which it is located.

[0035] In a second aspect, an embodiment of the present invention provides a device for expanding a sentiment dictionary, the device comprising:

[0036] A target word segmentation determination module is used to obtain text data, perform word segmentation processing on the text data, and determine the target word segmentation of the text data according to the obtained word segmentation;

[0037] A word set determination module is used to count the word frequencies of the target word segments and take the top N target word segments with the largest word frequencies as the word set;

[0038] A seed sentiment dictionary determination module is used to take the intersection of a word set and a pre-constructed basic sentiment dictionary to obtain a seed sentiment dictionary composed of seed words; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary;

[0039] A new word determination module is used to calculate the tf_idf value of the target word respectively, and take the target word with a tf_idf value greater than a preset threshold as a new word in the basic sentiment dictionary;

[0040] A new word sentiment polarity determination module is used to determine the sentiment polarity of the new word based on the sentiment vector representation of the new word, the part of speech of the new word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity;

[0041] The newly added word polarity value determination module is used to determine the polarity value of the newly added word based on the sentiment vector representation of the newly added word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word.

[0042] Optionally, the target word segmentation determination module further includes:

[0043] The stop word removal submodule is used to remove the stop words in the obtained segmented words and use the remaining segmented words as the target segmented words.

[0044] Optionally, the new word sentiment polarity determination module includes:

[0045] A new word similarity vector determination submodule is used to input the new word and the seed word into the word vector model for training to obtain the new word vector and the seed word vector, and determine the new word similarity vector based on the cosine similarity between the new word vector and various subword vectors;

[0046] The new word sentiment vector representation determination submodule is used to determine the sentiment vector representation of the new word based on the similarity vector of the new word, the polarity values ​​of various subwords, and the tf_idf value of the new word.

[0047] Optionally, the newly added word sentiment polarity determination module further includes:

[0048] A seed word similarity vector determination submodule, used to determine the similarity vector of the seed word based on the cosine similarity between the seed word vector and various sub-word vectors;

[0049] A seed word sentiment vector representation determination submodule is used to determine the sentiment vector representation of the seed word based on the similarity vector of the seed word, the polarity values ​​of various sub-words, and the tf_idf value of the seed word;

[0050] The seed sentiment dictionary sentiment center vector representation determination submodule is used to determine the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity based on the sentiment vector representation of the seed word.

[0051] Optionally, the newly added word sentiment polarity determination module further includes:

[0052] The seed word polarity value determination submodule is used to determine the polarity value of the seed word based on the cosine similarity between the seed word vector and the central vector of the seed sentiment dictionary in which it is located.

[0053] In a third aspect, an embodiment of the present invention provides a computer device, the computer device comprising:

[0054] one or more processors;

[0055] A memory for storing one or more programs;

[0056] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for expanding the sentiment dictionary as described in any one of the first aspects.

[0057] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium.

[0058] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for expanding the sentiment dictionary as described in any one of the first aspects is implemented.

[0059] In this embodiment, text data is obtained, word segmentation is performed on the text data, and target word segmentations of the text data are determined according to the obtained word segmentations; the word frequencies of the target word segmentations are counted, and the top N target word segmentations with the largest word frequencies are taken as a word set; the word set is intersected with a pre-constructed basic sentiment dictionary to obtain a seed sentiment dictionary composed of seed words; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary; the tf_idf values ​​of the target word segmentations are calculated respectively, and the target word segmentations with tf_idf values ​​greater than a preset threshold are taken as new words in the basic sentiment dictionary; the sentiment polarity of the new word is determined based on the sentiment vector representation of the new word, the part of speech of the new word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity; the polarity value of the new word is determined based on the sentiment vector representation of the new word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the new word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the new word. The sentiment dictionary is expanded based on representation learning. There is no need to manually label words. Each word can be assigned a different sentiment polarity value, which has stronger polarity expression ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0061] Figure 1 A flowchart of a method for expanding a sentiment dictionary provided in Embodiment 1 of the present invention;

[0062] Figure 2 A schematic diagram of the structure of a device for expanding a sentiment dictionary provided in Embodiment 2 of the present invention;

[0063] Figure 3 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0065] Embodiment 1

[0066] Figure 1 A flowchart of a method for extending a sentiment dictionary provided in the first embodiment of the present invention is provided. The method can be performed by a device for extending a sentiment dictionary. The device for extending a sentiment dictionary can be implemented by software and / or hardware and can be configured in a computer device, such as a server, a personal computer, etc. The method specifically includes the following steps:

[0067] Step 101: Acquire text data, perform word segmentation on the text data, and determine target word segments of the text data based on the obtained word segments.

[0068] Articles, sentences, comments and other texts can usually reflect the emotional tendencies of the publisher, such as attitudes towards an event, preferences for an object or a person, etc. Words commonly used to express opinions, views and attitudes often have certain emotional tendencies. The emotional dictionary constructed based on these words is of great significance and can be used for personal image maintenance, corporate marketing management and brand promotion, and business product improvement.

[0069] The above text data can be obtained from one or more application scenarios, for example, from social platforms, live broadcast platforms, commodity trading platforms and other platforms. This embodiment does not limit the specific source of the text to be identified and the method of obtaining it.

[0070] In one implementation, after the text data is segmented, stop words in the obtained segmented words are removed, and the remaining segmented words are used as target segmented words.

[0071] Stop words refer to certain words or phrases that are automatically filtered out before or after processing natural language data (or text) in information retrieval in order to save storage space and improve search efficiency. These words or phrases are called Stop Words. For a given purpose, any type of words can be selected as stop words. For the purpose of emotion recognition, stop words refer to words that have no actual meaning in emotion recognition, such as "is", "in", "this", "which", "up", "down", "one", "at", "is", "a", etc., so stop words can be removed.

[0072] Step 102: Count the word frequencies of the target word segments, and take the top N target word segments with the largest word frequencies as a word set.

[0073] The word frequency can be used to evaluate the repetition degree of a word in the text data. The importance of a word is proportional to the number of times it appears in the text data, so the top N target participles with the largest word frequency are selected as the word set. For example, the top 1000 target participles with the largest word frequency can be selected as the word set.

[0074] Step 103: Take the intersection of the word set and the pre-constructed basic sentiment dictionary to obtain a seed sentiment dictionary composed of seed words; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary.

[0075] The basic sentiment dictionary can be a sentiment dictionary that has been constructed by others, such as the Chinese Sentiment Polarity Dictionary of National Taiwan University, the Chinese Sentiment Dictionary of Dalian University of Technology, the HowNet Sentiment Dictionary, etc. It can also be a sentiment dictionary after deduplication by integrating multiple sentiment dictionaries. This embodiment does not limit the specific source and construction method of the sentiment dictionary.

[0076] The basic sentiment dictionary can be divided into four parts according to sentiment polarity: a positive basic sentiment sub-dictionary, a negative basic sentiment sub-dictionary, a degree adverb basic sentiment sub-dictionary, and a negative adverb basic sentiment sub-dictionary. The word set and the basic sentiment dictionary are intersected to obtain a seed sentiment dictionary composed of seed words. Similarly, the seed sentiment dictionary can be divided into four parts according to sentiment polarity: a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary.

[0077] Step 104: Calculate the tf_idf value of the target word respectively, and use the target word with a tf_idf value greater than a preset threshold as a new word in the basic sentiment dictionary.

[0078] TF-IDF (term frequency–inverse document frequency) is a commonly used weighting technique for information retrieval and data mining, which can be used to evaluate the importance of a word to text data. The larger the tf_idf value of a word in an article, the more important the word will be in the article in general. Therefore, a target word segmentation with a tf_idf value greater than a preset threshold is selected as a new word in the basic sentiment dictionary, such as selecting a target word segmentation with a tf_idf value greater than 0.4 as a new word in the basic sentiment dictionary. Of course, the preset threshold can also select other values ​​according to actual conditions, and this embodiment does not limit the specific value of the preset threshold.

[0079] Step 105 : Determine the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity.

[0080] For any new word c i , respectively calculate their sentiment vector representation The sentiment center vector representation POS′ of the positive seed sentiment dictionary centre , the sentiment center vector of the negative seed sentiment dictionary is represented by NEG′ centre , the sentiment center vector of the negative adverb seed sentiment dictionary is represented by NA′ centre , Sentiment Center Vector Representation of Degree Adverb Seed Sentiment Dictionary The cosine similarity of the new word c i The sentiment vector representation The cosine similarity between the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity and the newly added word c i Part of speech, determine the new word c i emotional polarity.

[0081] For example, the new word c i The emotional polarity can be judged by the following rules:

[0082]

[0083]

[0084] It should be noted that in the above-mentioned embodiments, the newly added word c i The above is only an exemplary description and does not limit this specification. In practical applications, other emotion polarity judgment rules can also be used.

[0085] In one embodiment, step 105 includes:

[0086] Sub-step 1051, input the new word and seed word into the word vector model training to obtain the new word vector and seed word vector, and determine the similarity vector of the new word based on the cosine similarity between the new word vector and various sub-word vectors.

[0087] Input the target word segmentation into the word2vec model for training, and you can get the word vector model vector.w2v of the set dimension, such as the 128-dimensional word vector model vector.w2v. Input the new word and seed word into the word vector model for training to get the new word vector and seed word vector.

[0088] For any new word c i , respectively calculate the cosine similarity between the new word vector and various subword vectors, and you can get the similarity vector of the new word

[0089] SIM i =(sim i1 ,sim i2 ,…,sim ik ,)

[0090] Where k represents the number of seed words.

[0091] Sub-step 1052: Determine the sentiment vector representation of the new word based on the similarity vector of the new word, the polarity values ​​of various sub-words, and the tf_idf value of the new word.

[0092] For any new word c i , its sentiment vector representation can be calculated using the following method:

[0093]

[0094] Among them, tf-idf i For the new word c i The tf-idf value of w i Seed word i The polarity value of .

[0095] The polarity value of a seed word can be determined based on the cosine similarity between the seed word vector and the center vector of the seed sentiment dictionary in which it is located.

[0096] For example, the center vector of the positive seed sentiment dictionary can be calculated as follows:

[0097] POS centre =(POS centre1 , POS centre2 ,…,POS centreD )

[0098]

[0099] Among them, D is the dimension of the seed word vector, k pos represents the number of seed words in the positive seed sentiment dictionary, x ij Represents the vector value of the jth dimension of the word vector of the i-th word in the positive seed sentiment dictionary.

[0100] Using the same method, calculate the center vector NEG of the negative seed sentiment dictionary centre , the central vector of the degree adverb seed sentiment dictionary NA centre , the central vector DA of the negative adverb seed sentiment dictionary centre .

[0101] Calculate the cosine similarity between the seed word vector and the center vector of the seed sentiment dictionary in which it is located. If the polarity of the seed word is positive or negative adverb, assign the cosine similarity to its polarity value. If the polarity of the seed word is negative or degree adverb, assign the opposite of the cosine similarity to its polarity value.

[0102] Seed word set SW = {(s1,w1),(s2,w2),…(s k ,w k )}, where w i Seed word i The polarity value of .

[0103] In one embodiment, step 105 further includes:

[0104] Sub-step 1053: Determine the similarity vector of the seed word based on the cosine similarity between the seed word vector and various sub-word vectors.

[0105] Sub-step 1054, determining the sentiment vector representation of the seed word based on the similarity vector of the seed word, the polarity values ​​of various sub-words, and the tf_idf value of the seed word.

[0106] For any seed word s i , its sentiment vector representation can be calculated using the following method:

[0107]

[0108] Among them, tf﹣idf′ i Seed word i The tf-idf value of w i Seed word i The polarity value of .

[0109] Sub-step 1055: Determine the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity based on the sentiment vector representation of the seed word.

[0110] For example, the sentiment center vector representation of the positive seed sentiment dictionary can be calculated as follows:

[0111]

[0112]

[0113] Among them, k pos represents the number of seed words in the positive seed sentiment dictionary, x ij Represents the vector value of the jth dimension of the word vector of the i-th word in the positive seed sentiment dictionary.

[0114] Using the same method, the sentiment center vector NEG′ of the negative seed sentiment dictionary is calculated centre , the sentiment center vector representation of the degree adverb seed sentiment dictionary NA′ centre , the sentiment center vector representation DA′ of the negative adverb seed sentiment dictionary centre .

[0115] Step 106: determine the polarity value of the newly added word based on the sentiment vector representation of the newly added word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word.

[0116] For any new word c i , its positive polarity value can be calculated using the following method:

[0117]

[0118] in, Indicates the newly added word c i The cosine similarity between the sentiment vector representation of and the sentiment center vector representation of the positive seed sentiment dictionary, Represents the average polarity of the positive seed sentiment lexicon.

[0119] The same method is used to calculate the negative polarity value, negation polarity value and degree polarity value of the newly added word ci.

[0120] If the newly added word ci has only one polarity, it will be assigned a unique sentiment polarity value. If the newly added word ci has two or more polarities, it will be assigned different sentiment polarity values ​​in the seed sentiment dictionary corresponding to the sentiment polarity.

[0121] In this embodiment, text data is obtained, word segmentation is performed on the text data, and target word segmentations of the text data are determined according to the obtained word segmentations; the word frequencies of the target word segmentations are counted, and the top N target word segmentations with the largest word frequencies are taken as a word set; the word set is intersected with a pre-constructed basic sentiment dictionary to obtain a seed sentiment dictionary composed of seed words; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary; the tf_idf values ​​of the target word segmentations are calculated respectively, and the target word segmentations with tf_idf values ​​greater than a preset threshold are taken as new words in the basic sentiment dictionary; the sentiment polarity of the new word is determined based on the sentiment vector representation of the new word, the part of speech of the new word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity; the polarity value of the new word is determined based on the sentiment vector representation of the new word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the new word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the new word. The sentiment dictionary is expanded based on representation learning. There is no need to manually label words. Each word can be assigned a different sentiment polarity value, which has stronger polarity expression ability.

[0122] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0123] Embodiment 2

[0124] Figure 2 This is a schematic diagram of the structure of a device for extending a sentiment dictionary provided in Embodiment 2 of the present invention. The device for extending a sentiment dictionary may specifically include the following modules:

[0125] The target word segmentation determination module 201 is used to obtain text data, perform word segmentation processing on the text data, and determine the target word segmentation of the text data according to the obtained word segmentation;

[0126] A word set determination module 202 is used to count the word frequencies of the target word segments and take the top N target word segments with the largest word frequencies as the word set;

[0127] The seed sentiment dictionary determination module 203 is used to take the intersection of the word set and the pre-constructed basic sentiment dictionary to obtain a seed sentiment dictionary composed of seed words; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary;

[0128] A new word determination module 204 is used to calculate the tf_idf value of the target word respectively, and take the target word with a tf_idf value greater than a preset threshold as a new word in the basic sentiment dictionary;

[0129] A newly added word sentiment polarity determination module 205 is used to determine the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity;

[0130] The newly added word polarity value determination module 206 is used to determine the polarity value of the newly added word based on the sentiment vector representation of the newly added word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word.

[0131] In one implementation, the target word segmentation determination module 201 further includes:

[0132] The stop word removal submodule is used to remove the stop words in the obtained segmented words and use the remaining segmented words as the target segmented words.

[0133] In one implementation, the newly added word sentiment polarity determination module 205 includes:

[0134] A new word similarity vector determination submodule is used to input the new word and the seed word into the word vector model for training to obtain the new word vector and the seed word vector, and determine the new word similarity vector based on the cosine similarity between the new word vector and various subword vectors;

[0135] The new word sentiment vector representation determination submodule is used to determine the sentiment vector representation of the new word based on the similarity vector of the new word, the polarity values ​​of various subwords, and the tf_idf value of the new word.

[0136] In one implementation, the newly added word sentiment polarity determination module 205 further includes:

[0137] A seed word similarity vector determination submodule, used to determine the similarity vector of the seed word based on the cosine similarity between the seed word vector and various sub-word vectors;

[0138] A seed word sentiment vector representation determination submodule is used to determine the sentiment vector representation of the seed word based on the similarity vector of the seed word, the polarity values ​​of various sub-words, and the tf_idf value of the seed word;

[0139] The seed sentiment dictionary sentiment center vector representation determination submodule is used to determine the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity based on the sentiment vector representation of the seed word.

[0140] In one implementation, the newly added word sentiment polarity determination module 205 further includes:

[0141] The seed word polarity value determination submodule is used to determine the polarity value of the seed word based on the cosine similarity between the seed word vector and the central vector of the seed sentiment dictionary in which it is located.

[0142] The device for expanding the sentiment dictionary provided in the embodiment of the present invention can execute the method for expanding the sentiment dictionary provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0143] Embodiment 3

[0144] Figure 3 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Figure 3 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 3 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0145] like Figure 3 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0146] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0147] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0148] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 3 not shown, usually called a "hard drive"). Although Figure 3Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0149] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0150] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the computer device 12, and / or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0151] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the method for expanding the sentiment dictionary provided in the embodiment of the present invention.

[0152] Embodiment 4

[0153] Embodiment 4 of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the method for expanding the emotional dictionary described above are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0154] Among them, computer-readable storage media may include, for example, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0155] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for expanding a sentiment dictionary, characterized in that: The method comprises the following steps: Acquire text data, perform word segmentation processing on the text data, and determine target word segmentation of the text data according to the obtained word segmentation; Counting the word frequencies of the target word segments, and taking the top N target word segments with the largest word frequencies as the word set; Taking the intersection of the word set and the pre-constructed basic sentiment dictionary, a seed sentiment dictionary consisting of seed words is obtained; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary; Calculate the tf_idf value of the target word respectively, and take the target word with a tf_idf value greater than a preset threshold as a new word in the basic sentiment dictionary; Determine the sentiment polarity of the new word based on the sentiment vector representation of the new word, the part of speech of the new word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity; The polarity value of the newly added word is determined based on the sentiment vector representation of the newly added word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word.

2. The method according to claim 1, characterized in that Determining the target word segmentation of the text data according to the obtained word segmentation includes: Remove the stop words from the obtained segmentations and use the remaining segmentations as the target segmentations.

3. The method according to claim 1, characterized in that The method of determining the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity includes: The new word and seed word are respectively input into the word vector model for training to obtain the new word vector and seed word vector. Based on the cosine similarity between the new word vector and various subword vectors, the similarity vector of the new word is determined. The sentiment vector representation of the new word is determined based on the similarity vector of the new word, the polarity values ​​of various subwords, and the tf_idf value of the new word.

4. The method according to claim 3, characterized in that The method of determining the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity also includes: Determine the similarity vector of the seed word based on the cosine similarity between the seed word vector and various subword vectors; Determine the sentiment vector representation of the seed word based on the similarity vector of the seed word, the polarity values ​​of various subwords, and the tf_idf value of the seed word; The sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity is determined based on the sentiment vector representation of the seed word.

5. The method according to claim 3 or 4, characterized in that: The method of determining the sentiment polarity of the newly added word based on the sentiment vector representation of the newly added word, the part of speech of the newly added word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity also includes: The polarity value of the seed word is determined based on the cosine similarity between the seed word vector and the center vector of the seed sentiment dictionary in which it is located.

6. A device for expanding a sentiment dictionary, characterized in that: The device comprises: A target word segmentation determination module is used to obtain text data, perform word segmentation processing on the text data, and determine the target word segmentation of the text data according to the obtained word segmentation; A word set determination module is used to count the word frequencies of the target word segments and take the top N target word segments with the largest word frequencies as the word set; A seed sentiment dictionary determination module is used to take the intersection of a word set and a pre-constructed basic sentiment dictionary to obtain a seed sentiment dictionary composed of seed words; the seed sentiment dictionary includes a positive seed sentiment dictionary, a negative seed sentiment dictionary, a negative adverb seed sentiment dictionary, and a degree adverb seed sentiment dictionary; A new word determination module is used to calculate the tf_idf value of the target word respectively, and take the target word with a tf_idf value greater than a preset threshold as a new word in the basic sentiment dictionary; A new word sentiment polarity determination module is used to determine the sentiment polarity of the new word based on the sentiment vector representation of the new word, the part of speech of the new word, and the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity; The newly added word polarity value determination module is used to determine the polarity value of the newly added word based on the sentiment vector representation of the newly added word, the sentiment center vector representation of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word, and the average polarity value of the seed sentiment dictionary corresponding to the sentiment polarity of the newly added word.

7. The device according to claim 6, characterized in that The new word sentiment polarity determination module includes: A new word similarity vector determination submodule is used to input the new word and the seed word into the word vector model for training to obtain the new word vector and the seed word vector, and determine the new word similarity vector based on the cosine similarity between the new word vector and various subword vectors; The new word sentiment vector representation determination submodule is used to determine the sentiment vector representation of the new word based on the similarity vector of the new word, the polarity values ​​of various subwords, and the tf_idf value of the new word.

8. The device according to claim 7, characterized in that The newly added word sentiment polarity determination module also includes: A seed word similarity vector determination submodule, used to determine the similarity vector of the seed word based on the cosine similarity between the seed word vector and various sub-word vectors; A seed word sentiment vector representation determination submodule is used to determine the sentiment vector representation of the seed word based on the similarity vector of the seed word, the polarity values ​​of various sub-words, and the tf_idf value of the seed word; The seed sentiment dictionary sentiment center vector representation determination submodule is used to determine the sentiment center vector representation of the seed sentiment dictionary of each sentiment polarity based on the sentiment vector representation of the seed word.

9. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for expanding the sentiment dictionary according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for expanding the sentiment dictionary according to any one of claims 1 to 5 is implemented.

Images