A word representation method, device, equipment and computer-readable storage medium

Abstract

The embodiments of the present application disclose a method, apparatus, device and computer-readable storage medium for representing a word, wherein the method comprises: obtaining the root of the word to be represented; when there is no vector representation corresponding to the root of the word to be represented in the trained vector model, decomposing the word to be represented with simple words as the decomposition granularity to obtain a set of simple words; obtaining the vector representation of each simple word in the set of simple words according to the vector model; and determining the word vector representation of the word to be represented according to the vector representation of each simple word.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and relate to but are not limited to a word representation method, apparatus, device, and computer-readable storage medium. Background Art

[0002] Word embedding is a feature representation of words. Each word is represented as a vector of fixed length through semantic dictionaries or machine learning techniques. At present, word embedding models play an increasingly important role in the fields of semantic web, information retrieval, data mining, information integration, knowledge management, text classification, speech recognition, word sense disambiguation, etc.

[0003] Rare words, also known as rare words or uncommon words, refer to words that are rarely used or appear infrequently. In the prior art, the vector representation of words is usually trained by word vector algorithms or the word vector of rare words is inferred with the help of an external semantic dictionary (WordNet). Among them, in the training of words by word vector algorithms, the context word set of a word in the position where it appears in the corpus is used as the input of the neural network, and the vector expression of the central word is obtained through continuous iteration. However, for rare words, due to the low frequency of occurrence in the corpus, it is difficult to obtain sufficient training, so that the accuracy of the semantic expression of rare words is greatly reduced compared with non-rare words, and even the word vector of rare words cannot be obtained; and when the word vector of rare words is inferred by WordNet, it is limited by the scope of WordNet. Once the rare word does not appear in WordNet, the method of inferring the word vector of rare words by using WordNet will no longer work, and WordNet itself contains common words in general fields, and some field-specific words are not included. The prior art does not have a good method for obtaining the word vector of rare words. Even if the vector representation of rare words can be obtained, the accuracy of its semantic expression is low. Summary of the invention

[0004] In view of this, embodiments of the present application provide a word representation method, apparatus, word representation device, and computer-readable storage medium.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for representing a word, the method comprising:

[0007] Get the root of the word to be represented;

[0008] When there is no vector representation corresponding to the root of the word to be represented in the trained vector model, the word to be represented is decomposed with simple words as the decomposition granularity to obtain a simple word set;

[0009] Obtaining a vector representation of each simple word in the simple word set according to the vector model;

[0010] According to the vector representation of each simple word, the word vector representation of the word to be represented is determined.

[0011] In a second aspect, an embodiment of the present application provides a word representation device, including:

[0012] A first acquisition module is used to acquire the root of the word to be represented;

[0013] A decomposition module, used for decomposing the word to be represented with simple words as the decomposition granularity to obtain a simple word set when there is no vector representation corresponding to the root of the word to be represented in the trained vector model;

[0014] A second acquisition module, used for acquiring a vector representation of each simple word in the simple word set according to the vector model;

[0015] A determination module is used to determine the word vector representation of the word to be represented based on the vector representation of each simple word.

[0016] In a third aspect, an embodiment of the present application provides a word representation device, the device comprising:

[0017] Processor; and

[0018] a memory for storing a computer program executable on the processor;

[0019] Among them, when the computer program is executed by the processor, the steps of the above-mentioned word representation method are implemented.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are configured to execute the steps of the word representation method described above.

[0021] The word representation method, apparatus, device and computer-readable storage medium provided by the embodiments of the present application, when the root of the word to be represented (such as a rare word) is not in the trained vector model, the word to be represented is decomposed into a set of simple words, and the word to be represented is determined by obtaining the vector representation of each simple word in the simple word set in the vector model. Because a simple word has only one meaning, the vector representation of the word to be represented is obtained by the vector representation corresponding to the simple word, thereby achieving a more accurate expression of the semantics of the word vector of the word to be represented. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In the drawings, which are not necessarily drawn to scale, like reference numerals may describe similar components in different views.The drawings generally illustrate, by way of example and not limitation, various embodiments discussed herein.

[0023] Figure 1 It is a structural diagram of the CBOW model provided in the related art;

[0024] Figure 2 It is a structural diagram of the Skip-gram model provided in the related art;

[0025] Figure 3 A flowchart of a word representation method provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of the improved CBOW model structure provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of a process for obtaining a vector representation of each simple word in a simple word set provided in an embodiment of the present application;

[0028] Figure 6 A flowchart of a word representation method provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the composition structure of a word representation device provided in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of the composition structure of a word representation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0032] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0033] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0035] In order to better understand the word representation method, apparatus, device and computer-readable storage medium provided in the embodiments of the present application, first, the implementation methods and existing problems of the word representation method in the related art are analyzed and explained.

[0036] There are two main methods for representing word vectors: one-hot representation and distributed representation. Distributed representation is currently the most commonly used method to represent word features.

[0037] Among the distributed representation word vector models, the most well-known is the word to vector (word2vec, word tovector) model. Word2vec is an open source learning tool based on deep learning launched by Google. Word2vec is a tool that converts words into vector form by training a model on a given corpus. Word vectors have good semantic properties and are a common way to represent word features. The value of each dimension of the word vector represents a feature with a certain semantic and grammatical interpretation. Through word vector training, the processing of text content can be simplified to vector operations in the vector space, and the correlation between words in the vector space can be calculated to represent the semantic correlation of words.

[0038] Word2vec includes two training models, namely the Continuous Bag-Of-Words (CBOW) model and the Skip-gram model. Figure 1 It is a structural diagram of the CBOW model in the related technology, such as Figure 1 As shown, the model includes an input layer 101, a mapping layer 102 and an output layer 103. The CBOW model predicts the current word through the context, while the Skip-gram model predicts the context through the current word. Figure 1 As shown, the input layer 101 of the CBOW model is the context of the current word wt, including the word vectors of w(t-2), w(t-1), w(t+1), and w(t+2). The input of the mapping layer 102 is the cumulative sum of the word vectors of the context, and the word vector of wt is output in the output layer 103.

[0039] Figure 2 It is a structural diagram of the Skip-gram model in the related art, such as Figure 2 As shown, the model includes: an input layer 201, a mapping layer 202 and an output layer 203. The input layer 201 of the Skip-gram model is wt, and the mapping layer 202 predicts the context of wt, thereby outputting its context in the output layer 203: w(t-2), w(t-1), w(t+1), w(t+2), that is, the mapping method of the Skip-gram model is opposite to that of the CBOW model. Among them, the CBOW model and the Skip-gram model are both based on the neural network model.

[0040] Take the CBOW model as an example:

[0041] The objective function of CBOW model optimization is:

[0042]

[0043] Where C is the corpus, Context(w) represents the context of word w, that is, the set of words around w. Taking the NEG model as an example, the context Context(w) of word w is known, and w needs to be predicted. For a given Context(w), word w is a positive sample, and other words are negative samples. For a given sample (Context(w), w), maximization processing is performed:

[0044]

[0045] Among them, NEG(w) is a negative sample set, the positive sample label is 1, and the negative sample label is 0. Its probability is calculated as follows:

[0046]

[0047] Among them, x w The sum of the word vectors of each word representing Context(w), θ u ∈R m Represents an auxiliary vector of word u, that is, the parameter to be trained by the model. The σ(x) function is the sigmoid function, which is defined as:

[0048]

[0049] Substituting formula (3) into formula (1), the final objective function is:

[0050]

[0051]

[0052] Using the stochastic gradient ascent method to solve (6) we get:

[0053]

[0054]

[0055] So, the parameter θ u The update formula for the context word vector is:

[0056]

[0057]

[0058] When using the CBOW model to train the distributed representation of words, the context word set of all occurrences of a word in the corpus is used as the input of the neural network, and the vectorized representation of the central word is obtained through continuous iteration. However, for rare words, due to their low frequency of occurrence in the corpus, it is difficult to obtain sufficient training, which makes the accuracy of the semantic representation of rare words significantly lower than that of non-rare words, and even the word vector of rare words cannot be obtained. Taking the word2vec algorithm as an example, words with a frequency greater than or equal to 5 are usually used to train word vectors. Therefore, neither the CBOW model nor the Skip-gram model contains the distributed representation of words with a frequency less than 5.

[0059] To address this problem, a word vector and text classification model (FastText) with enhanced subword information is proposed in the related art. The FastText model decomposes each word into multiple Chinese language model (N-Gram) strings, and simultaneously trains the N-Gram vector and word vector, and then sums the word vector and the vectors corresponding to all N-Grams that appear in the word as the final word vector. Taking the 2-Gram FastText model as an example, the word vector of "book" is the superposition of all 2-Gram corresponding vectors in the 2-Gram set {"bo", "oo", "ok"} and the word vector of "book" in the model. For a rare word, it is first decomposed into multiple subwords, and then the subword vectors corresponding to the FastText model are obtained, and then the sum of the subword vectors is used as the word vector of the rare word. This method can enhance the word vector representation of rare words with the help of subword vector information.

[0060] Another method is to use WordNet to infer the word vectors of rare words. For a rare word, first search for the corresponding semantic information in WordNet, including word meaning interpretation information, example sentence information, and words that are semantically related to the rare word (synonyms, hypernyms, hyponyms, etc.), and then superimpose the word vectors of the words contained in these semantic information to obtain a distributed representation of the rare word.

[0061] In the FastText method, the word vector of rare words is realized by superimposing subword vectors. However, there is no correlation between some simple words and their subwords, such as the word "choke" and the subword "hok". This situation makes the accuracy of the semantic expression of rare words significantly lower than that of non-rare words. Moreover, many rare words contain subwords that are also rare words, and these subwords are not included in the trained subword vector model. For example, the subwords of some domain-specific vocabulary are also rare words, such as iight, scaldabiberon and other words, so the word vector of rare words cannot be obtained.

[0062] The method of inferring the word vector of rare words with the help of WordNet is limited by the scope of the semantic dictionary WordNet. Once the rare word does not appear in WordNet, the method of inferring the word vector of rare words with the help of WordNet will not work. WordNet contains common words in general fields, and does not include some field-specific words. In this case, the vector of rare words cannot be obtained.

[0063] Based on the problems existing in the related art, the embodiment of the present application provides a method for representing a word. The method provided by the embodiment can be implemented by a computer program. When the computer program is executed, each step in the method provided by the embodiment is completed. In some embodiments, the computer program can be executed by a processor in a word representation device.

[0064] Figure 3 A flow chart of a word representation method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the method is applied to a word representation device, and the method comprises:

[0065] Step S301, obtaining the stem of the word to be represented.

[0066] In an embodiment of the present application, the word to be represented can be a rare word or a common word. Taking rare words as an example, rare words are also called rare words or uncommon words, which refer to words that are less used or do not appear frequently. The rare words can be one of compound words, inflected words and domain-specific words. In an embodiment of the present application, it is usually possible to determine whether a word is a rare word by the word frequency of the word in the corpus, and the word frequency is the number of times it appears in the corpus. Exemplarily, words with a word frequency less than 5 in the corpus can be set as rare words, and words with a word frequency greater than or equal to 5 are common words.

[0067] In the embodiment of the present application, when a word to be represented is obtained, the word representation device can extract the root of the word to be represented.

[0068] Taking rare words in English as an example, the core part of English word formation lies in the root, and the meaning of a word is mainly reflected by the root that makes up the word. Roots can form words alone or be combined with each other to change the word's part of speech and meaning through prefixes and suffixes. In other words, a word generally consists of three parts: root, prefix and suffix. The root determines the meaning of the word, the prefix changes the meaning of the word, and the suffix determines the word's part of speech. For example: refrigerating extracts its root "refriger".

[0069] In some embodiments, the root of the word to be represented is still the word itself, for example, scaldabiberon is still the word itself after the root is extracted.

[0070] In some embodiments, before step S301, the word representation device is further used to receive a word to be represented.

[0071] Step S302: When there is no vector representation corresponding to the root of the word to be represented in the trained vector model, the word to be represented is decomposed with simple words as the decomposition granularity to obtain a simple word set.

[0072] In the embodiment of the present application, the trained vector model includes word vectors and root vectors. In the embodiment of the present application, the vector model is searched to see whether there is a vector representation corresponding to the extracted root. For example, the root extracted by refrigerating is "refriger". At this time, it can be searched whether there is a vector representation of "refriger" in the vector model.

[0073] In the embodiment of the present application, the simple word is composed of one morpheme, that is, a simple word can only express one meaning and cannot be separated. Exemplarily, take, come, back, man, make, etc. are all simple words. In the embodiment of the present application, the decomposition granularity is the smallest unit for decomposing the word to be represented. When the word to be represented is decomposed with simple words as the decomposition granularity, it can be decomposed into one or more simple words, thereby obtaining a simple word set. In other words, the simple word set obtained in this step may include one or more simple words. When the simple word set includes only one simple word, this simple word is also the word to be represented itself.

[0074] In some embodiments, when there is a vector representation corresponding to the root of the word to be represented in the vector model, the vector representation corresponding to the root of the word to be represented is obtained; and further based on the vector representation corresponding to the root, the word vector representation of the word to be represented is determined. Exemplarily, the word to be represented is refrigerating, and its root refriger can be obtained by root extraction. The vector representation of the word to be represented is determined by searching for the vector of refriger in the trained vector model, that is, V("refrigerating") = V("refriger"), at which point the process of this method ends.

[0075] Step S303, obtaining a vector representation of each simple word in the simple word set according to the vector model.

[0076] In an embodiment of the present application, when the word representation device obtains the vector representation of each simple word in the simple word set according to the vector model, the vector representation of the simple word in the vector model can be obtained through the simple word, or the vector representation of each simple word can be obtained through the root of the simple word, or the vector representation of each simple word can be obtained through the simple word and the root of the simple word.

[0077] In an embodiment of the present application, when there are multiple simple words included in the simple word set, the word vector representation of each simple word is obtained; when the obtained simple word set includes only one simple word, the word to be represented can be converted into a synonym or a synonym of the word to be represented, and then the synonym of the word to be represented can be decomposed to obtain multiple simple words, thereby obtaining vector representations of multiple simple words.

[0078] Step S304, determining the word vector representation of the word to be represented based on the vector representation of each simple word.

[0079] In the embodiment of the present application, when implementing step S304, the vector representations of each simple word may be accumulated to determine the word vector representation of the word to be represented.

[0080] Example: word vector representation of robotcleaner:

[0081] V("robotcleaner")=V("robot")+V("cleaner").

[0082] The word vector representation of scaldabiberon is:

[0083] V("scaldabiberon")=V("bottle")+V("warmer")

[0084] Through the above word representation method, when the root of the word to be represented (such as a rare word) is not in the trained vector model, the word to be represented is decomposed into a simple word set, and the vector representation of each simple word in the simple word set is obtained in the vector model to determine the word to be represented. Because a simple word has only one meaning, the vector representation of the word to be represented is obtained through the vector representation corresponding to the simple word, thereby achieving a more accurate expression of the semantics of the word vector of the word to be represented.

[0085] In some embodiments, before step S302, the method further includes: obtaining a trained vector model.

[0086] In an embodiment of the present application, when determining the representation of a word to be represented, it is necessary to use the root of the word to be represented, so it is necessary to obtain a vector representation of each root of the word to be represented. Multiple words usually share a root, and sometimes the root is a new word. For example, the root of the words "produced" and "producing" is "produc", but "produc" is not a word in the corpus dictionary. Therefore, in an embodiment of the present application, the trained vector model can be an improved CBOW model, which can train word vectors and root vectors at the same time.

[0087] Figure 4 The schematic diagram of the improved CBOW model structure provided in the embodiment of the present application is as follows: Figure 4As shown, the model includes: an input layer 401, a mapping layer 402 and an output layer 403, wherein the input in the input layer 401 is the context of the word to be represented wt and the root of the context words, wherein the context includes: "w(t-2), w(t-1), w(t+1), w(t+2)"; the root of the context includes: "stem(w(t-2)), stem(w(t-1)), stem(w(t+1)), stem(w(t+2))". The input of the mapping layer 402 is the cumulative sum of the word vectors of the context and the cumulative sum of the root vectors of the context words. The output layer 403 outputs the word vector of the word to be represented wt. In each stochastic gradient ascent iterative solution process, the word vector and the root vector of the context are updated simultaneously, so as to finally obtain a converged improved CBOW model (in some embodiments, the improved CBOW model may also be referred to as a word and stem embedding model (WSE). By using this improved CBOW model, not only the word embedding of each word in the corpus vocabulary can be obtained, but also the vector of each word root (stem embedding) can be obtained.

[0088] In some embodiments, step S302: when there is no vector representation corresponding to the root of the word to be represented in the trained vector model, the word to be represented is decomposed with simple words as the decomposition granularity to obtain a simple word set, which can be achieved by the following steps:

[0089] Step S3021: The word representation device decomposes the word to be represented with simple words as the decomposition granularity to obtain a first decomposition result.

[0090] In the embodiment of the present application, when a word to be represented can be decomposed into multiple simple words, that is, the word to be represented is composed of two or more simple words, then the word to be represented is a compound word, that is, when the compound word is decomposed with simple words as the decomposition granularity, at least two simple words are obtained. For example, robotcleaner can be decomposed into robot and cleaner, and the obtained simple word set includes robot and cleaner.

[0091] In some embodiments, when the word to be represented is decomposed, the word to be represented itself is obtained, that is, the word to be represented itself is a simple word. For example, scaldabiberon is still obtained after the word form is decomposed, so the simple word set only includes the word to be represented itself.

[0092] Step S3022: The word representation device determines whether the simple word in the first decomposition result is the word to be represented.

[0093] In an embodiment of the present application, when the word representation device determines that the simple word in the first decomposition result is the word to be represented, it means that the word to be represented cannot be decomposed, and step S3023 is executed; when the word representation device determines that the simple word in the first decomposition result is not the word to be represented, it means that the word to be represented can be decomposed, and step S3026 is executed.

[0094] Step S3023: the word representation device converts the word to be represented into a second language to obtain the word to be represented in the second language.

[0095] In the embodiment of the present application, the first language is different from the second language. The conversion of the word to be represented into the second language may be to translate the word to be represented. Exemplarily, the first language may be English, and the second language may be Chinese, that is, the English word to be represented is translated into the Chinese word to be represented. For example, the word to be represented is scaldabiberon, and the word to be represented is translated into Chinese to obtain a milk heater, so the milk heater is the word to be represented in the second language.

[0096] In the embodiment of the present application, when implementing step S3023, the translation can be performed by calling the application programming interface (API) of the translation website to obtain the word to be represented in the second language.

[0097] Step S3024: the word representation device converts the word to be represented in the second language into the first language to obtain a target word.

[0098] In the embodiment of the present application, the target word and the word to be represented are in the first language. Converting the word to be represented in the second language into the first language may be translating the word to be represented in the second language into the target word in the first language. Continuing with the above example, the milk warmer is translated into English to obtain bottle warmer, and bottle warmer is used as the target word. In other words, the English word to be represented is translated into Chinese, and then the Chinese is reversely translated to obtain the English target word. In the embodiment of the present application, the target word can be considered as a synonym or a synonymous phrase of the word to be represented.

[0099] Step S3025, the word representation device decomposes the target word to obtain a second decomposition result, and obtains a simple word set based on the simple words in the second decomposition result, and this step process ends.

[0100] In the embodiment of the present application, the target word is decomposed by using simple words as the decomposition granularity. For example, the target word bottle warmer is decomposed into bottle and warmer, and the simple word set obtained at this time includes: bottle and warmer.

[0101] Step S3026: the word representation device obtains the simple words in the first decomposition result.

[0102] Exemplarily, after robotcleaner is decomposed into robot and cleaner, the first decomposition result is robot and cleaner.

[0103] Step S3027: the word representation device obtains a simple word set based on the simple words in the first decomposition result.

[0104] Exemplarily, the simple word set is determined to include robot and cleaner.

[0105] In an embodiment of the present application, when there is no vector representation corresponding to the root of the word to be represented in the trained vector model, the word to be represented is decomposed with simple words as the decomposition granularity, thereby determining a simple word set; when the word to be represented cannot be decomposed, a synonym or synonym phrase of the word to be represented is obtained, and the synonym or synonym phrase is decomposed to determine a simple word set.

[0106] Figure 5 This is a schematic diagram of a process for obtaining a vector representation of each simple word in a simple word set provided in an embodiment of the present application, such as Figure 5 As shown, the step S303: obtaining the vector representation of each simple word in the simple word set according to the vector model can be implemented by the following steps:

[0107] Step S3031: the word representation device obtains the i-th simple word in the simple word set.

[0108] In the embodiment of the present application, i=1, 2, ..., N, and there are N simple words in the simple word set.

[0109] In the embodiment of the present application, the simple word set may be determined based on the first decomposition result or the second decomposition result.

[0110] Step S3032: the word representation device determines whether there is a vector representation corresponding to the i-th simple word in the vector model.

[0111] In an embodiment of the present application, when there is a vector representation corresponding to the i-th simple word in the vector model, step S3033 is executed; when there is no vector representation of the i-th simple word in the vector model, step S3034 is executed.

[0112] Step S3033: the word representation device obtains the vector representation of the i-th simple word.

[0113] In the embodiment of the present application, by sequentially acquiring the vector representation of each simple word in the simple word set, the vector representation corresponding to all the simple words in the simple word set can be obtained. When the vector representation corresponding to all the simple words in the simple word set is obtained, the step process ends.

[0114] Step S3034: the word representation device extracts the root of the i-th simple word.

[0115] In the embodiment of the present application, the method of extracting the root of the i-th word is the same as the above-mentioned method of extracting the root of the word to be represented, which will not be repeated here.

[0116] Step S3035: the word representation device determines whether there is a vector representation corresponding to the root of the i-th simple word in the vector model.

[0117] In the embodiment of the present application, when there is a vector representation corresponding to the root of the i-th simple word in the vector model, step S3036 is executed. When there is no vector representation corresponding to the root of the i-th simple word in the vector model, step S3037 is executed.

[0118] Step S3036: The word representation device determines the vector representation corresponding to the root as the vector representation of the i-th simple word.

[0119] Step S3037: The word representation device determines the vector representation of the simple word as a preset vector.

[0120] In the embodiment of the present application, there may be a situation where the root of a simple word is not in the vector model. The vector representation of the simple word can be determined as a 0 vector.

[0121] In the embodiment of the present application, the vector representation corresponding to the simple word is obtained through the simple word or the root of the simple word, and then the vector representation of each simple word in the simple word set is obtained. This avoids the problem in the prior art that the vector representation of the word to be represented cannot be obtained due to the single acquisition method. Since a simple word has only one meaning, the vector representation of the word to be represented is obtained through the vector representation corresponding to the simple word, so that the semantic information of the vector representation of the word to be represented is more accurate.

[0122] This embodiment of the application further provides a method for representing a word, taking a rare word as an example: Figure 6 Schematic diagram of the word representation method provided in the embodiment of the present application, such as Figure 6 As shown, the method includes:

[0123] Step S601, the word representation device initializes the vector V of the rare word to a 0 vector.

[0124] Step S602: the word representation device extracts the root of the rare word w.

[0125] In the embodiment of the present application, the root of w is represented as ws.

[0126] Step S603: the word representation device determines whether there is a root vector of ws in the WSE model.

[0127] In the embodiment of the present application, if there is a root vector of ws in the WSE model, step S618 is executed; if there is no root vector of ws in the WSE model, step S604 is executed.

[0128] Step S604: the word representation device performs word form decomposition on the rare word w to obtain a set of simple words W after decomposition.

[0129] Step S605, the word representation device determines whether the number of words (ie, simple words) in the set W is equal to 1.

[0130] In the embodiment of the present application, if the number of words in the W set is equal to 1, it means that the rare word w itself is a simple word and cannot be decomposed into word forms. At this time, step S606 is executed. If the number of words in the W set is not equal to 1, step S609 is executed.

[0131] Step S606: the word representation device translates the rare word w from language L1 to language L2, and the translation result is S1.

[0132] In the embodiment of the present application, before step S606, the method further includes: determining that the language category of the rare word w is L1, and then determining that the intermediate language category is L2, wherein L2≠L1, for example, L1 is English and L2 is Chinese.

[0133] In the embodiment of the present application, the word representation device can translate by calling the API of the translation website.

[0134] Step S607: the word representation device reversely translates the translation result S1 from language L2 to language L1 to obtain the translation result S2.

[0135] Step S608: the word representation device performs word form decomposition on the translation result S2.

[0136] Step S609: the word representation device obtains a simple word set W.

[0137] In step S610, the word representation device sequentially selects simple words w0 from the simple word set W to obtain a vector representation M[w0] of w0 in the WSE model.

[0138] Step S611, the word representation device determines whether there is a vector representation M[w0] of w0 in the WSE model.

[0139] In the embodiment of the present application, if M[w0] exists in the WSE model, step S612 is executed; if M[w0] does not exist in the WSE model, step S613 is executed.

[0140] Step S612, the word representation device adds M[w0] to the vector V=V+M[w0].

[0141] In the embodiment of the present application, V may be a vector representation obtained by accumulating the vector V initialized to 0. That is, in the embodiment of the present application, the vector V changes dynamically, and when a vector representation of a simple word is obtained, the vector representation of the simple word is superimposed on the vector V. In the embodiment of the present application, step S617 is executed after step S612 is completed.

[0142] Step S613, the word representation device extracts the root ws0 of w0 to obtain the root vector M[ws0].

[0143] Step S614, the word representation device determines whether the vector representation M[ws0] of ws0 exists in the WSE model.

[0144] In the embodiment of the present application, if M[ws0] does not exist in the WSE model, step S615 is executed. If M[ws0] exists in the WSE model, step S616 is executed.

[0145] Step S615, the word representation device records M[ws0] as a 0 vector representation.

[0146] Step S616, the word representation device adds M[ws0] to V=V+M[ws0].

[0147] Step S617, the word representation device determines whether the simple words in W have been selected.

[0148] In the embodiment of the present application, if the selection is not completed, step S610 is executed; if the selection is completed, step S619 is executed.

[0149] Step S618, the word representation device determines V=M[ws].

[0150] In step S619, the word representation device returns the vector V as the distributed representation of the rare word w, and the process ends at this time.

[0151] For example, the word vector of the rare word "robotcleaner" can be represented by word form decomposition as the sum of two simple word vectors:

[0152] V("robotcleaner")=V("robot")+V("cleaner").

[0153] The rare word "refrigerating" can be represented by its root word "refriger" in the form of a vector in the WSE model through root extraction:

[0154] V("refrigerating") = V("refriger");

[0155] After root extraction and morphological decomposition, the rare word "scaldabiberon" remains itself. Its vector description needs to be obtained through bidirectional translation. Assuming the intermediate language is Chinese, first translate "scaldabiberon" into Chinese, and the translation result is "thermal milk dispenser". Then translate "thermal milk dispenser" back into English to get "bottlewarmer". After morphological decomposition of "bottle warmer", two simple words "bottle" and "warmer" are obtained. Finally, the distributed representation of "scaldabiberon" is obtained by superimposing the word vectors of these two simple words:

[0156] V("scaldabiberon") = V("bottle") + V("warmer")

[0157] A method for representing words proposed in the embodiments of the present application comprehensively utilizes morphological decomposition, root extraction, and bidirectional translation, and can realize the expression of synonym vectors for rare words. The pre-trained model used is the WSE model. Since the WSE model is no longer a simple word vector model but a model for training word and root vectors based on a text corpus, the WSE model can not only obtain the word vectors of each word in the vocabulary of the text corpus but also obtain the vectors of the roots of each word. In the embodiments of the present application, the smallest semantic element for expressing a rare word is a complete word or root. Compared with the method based on the sub-word vector model represented by FastText, each component unit has more explicit semantic information and can express the meaning of the original rare word more completely. Compared with the method based on the knowledge base, the method for representing words provided in the embodiments of the present application is not limited by the coverage of the knowledge base, has a better rare word coverage rate, and can provide vector inference representations for a wider range of words. In addition, the method for representing words provided in the embodiments of the present application can better express the semantic information of rare words such as domain-specific words, and is also applicable to different domains, having better universality.

[0158] Based on the foregoing embodiments, the embodiments of the present application provide a word representation device, and the modules included in the device, as well as the units included in each module, can be implemented by a processor in a computer device; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0159] Figure 7 A schematic diagram of the composition structure of a word representation device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the word representation device 700 includes a first acquisition module 701, a decomposition module 702, a second acquisition module 703 and a first determination module 704. Among them:

[0160] The first acquisition module 701 is used to acquire the root of the word to be represented.

[0161] The decomposition module 702 is used to decompose the word to be represented with simple words as the decomposition granularity to obtain a simple word set when there is no vector representation corresponding to the root of the word to be represented in the trained vector model.

[0162] The second acquisition module 703 is used to acquire the vector representation of each simple word in the simple word set according to the vector model.

[0163] The first determination module 704 is used to determine the word vector representation of the word to be represented according to the vector representation of each simple word.

[0164] In some embodiments, the decomposition module 702 includes: a first decomposition unit, a first conversion unit, a second conversion unit, and a second decomposition unit.

[0165] The first decomposition unit is used to decompose the word to be represented with simple words as the decomposition granularity to obtain a first decomposition result.

[0166] The first conversion unit is used for converting the word to be represented into a second language to obtain the word to be represented in the second language when it is determined that the simple word in the first decomposition result is the word to be represented.

[0167] The second conversion unit is used to convert the word to be represented in the second language into the first language to obtain a target word; wherein the target word and the word to be represented are in the first language.

[0168] The second decomposition unit is used to decompose the target word to obtain a second decomposition result, and obtain a simple word set according to the simple words in the second decomposition result.

[0169] In some embodiments, the apparatus 700 further includes: a third acquisition module and a second determination module.

[0170] in:

[0171] The third acquisition module is used to acquire the simple word in the first decomposition result when it is determined that the simple word in the first decomposition result is not the word to be represented.

[0172] The second determination module is used to obtain a simple word set according to the simple words in the first decomposition result.

[0173] In some embodiments, the second acquisition module 703 further includes: a first acquisition unit, a first judgment unit, and a second acquisition unit.

[0174] The first acquisition unit is used to acquire the i-th simple word in the simple word set, i=1, 2, ..., N, and there are N simple words in the simple word set.

[0175] The first judgment unit is used to judge whether there is a vector representation corresponding to the i-th simple word in the vector model.

[0176] The second acquisition unit is used to acquire the vector representation of the i-th simple word when there is a vector representation corresponding to the i-th simple word in the vector model.

[0177] In some embodiments, the second acquisition module 703 further includes: a first extraction unit, a second judgment unit and a first determination unit.

[0178] The first extraction unit is used to extract the root of the i-th simple word when the vector representation of the i-th simple word does not exist in the vector model.

[0179] The second judgment unit is used to judge whether there is a vector representation corresponding to the root of the i-th simple word in the vector model.

[0180] The first determining unit is used to determine the vector representation corresponding to the root as the vector representation of the i-th simple word when there is a vector representation corresponding to the root of the i-th simple word in the vector model.

[0181] In some embodiments, the second acquisition module 703 further includes:

[0182] The second determining unit is used to determine the vector representation of the simple word as a preset vector when there is no vector representation corresponding to the root of the i-th simple word in the vector model.

[0183] In some embodiments, the apparatus 700 further includes: a fourth acquisition module and a third determination module.

[0184] in:

[0185] The fourth acquisition module is used to acquire the vector representation corresponding to the root of the word to be represented when the vector representation corresponding to the root of the word to be represented exists in the vector model.

[0186] The third determination module is used to determine the word vector representation of the word to be represented according to the vector representation corresponding to the root word.

[0187] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0188] It should be noted that in the embodiment of the present application, if the above-mentioned word representation method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, ReadOnly Memory), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0189] Accordingly, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps in the word representation method provided in the above embodiment are implemented.

[0190] The embodiment of the present application provides a word representation device, Figure 8 A schematic diagram of the composition structure of a word representation device provided in an embodiment of the present application, such as Figure 8As shown, the word representation device 800 includes: a processor 801, at least one communication bus 802, a user interface 803, at least one external communication interface 804 and a memory 805. The communication bus 802 is configured to realize the connection and communication between these components. The user interface 803 may include a display screen, and the external communication interface 804 may include a standard wired interface and a wireless interface. The processor 801 is configured to execute the word representation program stored in the memory to implement the steps in the word representation method provided in the above embodiment.

[0191] The description of the above word representation device and storage medium embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the word representation device and storage medium embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0192] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0193] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0194] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0195] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0196] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0197] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0198] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read Only Memory), disks or optical disks, etc. Various media that can store program codes.

[0199] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for an AC to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0200] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for expressing a word, characterized in that: The method comprises: Get the root of the word to be represented; When there is no vector representation corresponding to the root of the word to be represented in the trained vector model, the word to be represented is decomposed with simple words as the decomposition granularity to obtain a simple word set; Obtaining a vector representation of each simple word in the simple word set according to the vector model; Determine the word vector representation of the word to be represented according to the vector representation of each simple word; Wherein, obtaining the vector representation of each simple word in the simple word set according to the vector model includes: Obtaining the i-th simple word in the simple word set, i=1, 2, ..., N, and there are N simple words in the simple word set; Determine whether there is a vector representation corresponding to the i-th simple word in the vector model; When there is a vector representation corresponding to the i-th simple word in the vector model, obtaining the vector representation of the i-th simple word; When there is no vector representation of the i-th simple word in the vector model, extracting the root of the i-th simple word; Determine whether there is a vector representation corresponding to the root of the i-th simple word in the vector model; When there is a vector representation corresponding to the root of the i-th simple word in the vector model, the vector representation corresponding to the root is determined as the vector representation of the i-th simple word.

2. The method according to claim 1, characterized in that The word to be represented is decomposed with simple words as the decomposition granularity to obtain a simple word set, including: Decomposing the word to be represented with simple words as decomposition granularity to obtain a first decomposition result; When it is determined that the simple word in the first decomposition result is the word to be represented, converting the word to be represented into a second language to obtain the word to be represented in the second language; Convert the word to be represented in the second language into the first language to obtain a target word; wherein the target word and the word to be represented are in the first language; The target word is decomposed to obtain a second decomposition result, and a simple word set is obtained according to the simple words in the second decomposition result.

3. The method according to claim 2, characterized in that The method further comprises: When it is determined that the simple word in the first decomposition result is not a word to be represented, obtaining the simple word in the first decomposition result; A simple word set is obtained according to the simple words in the first decomposition result.

4. The method according to claim 1, characterized in that The method further comprises: When there is no vector representation corresponding to the root of the i-th simple word in the vector model, the vector representation of the simple word is determined as a preset vector.

5. The method according to claim 1, characterized in that The method further comprises: When there is a vector representation corresponding to the root of the word to be represented in the vector model, obtaining the vector representation corresponding to the root of the word to be represented; According to the vector representation corresponding to the root word, a word vector representation of the word to be represented is determined.

6. A word representation device, characterized in that: The device comprises: A first acquisition module is used to acquire the root of the word to be represented; A decomposition module, used for decomposing the word to be represented with simple words as the decomposition granularity to obtain a simple word set when there is no vector representation corresponding to the root of the word to be represented in the trained vector model; The second acquisition module is used to acquire the i-th simple word in the simple word set, i=1, 2, ..., N, and there are N simple words in the simple word set; determine whether there is a vector representation corresponding to the i-th simple word in the vector model; when there is a vector representation corresponding to the i-th simple word in the vector model, acquire the vector representation of the i-th simple word; when there is no vector representation of the i-th simple word in the vector model, extract the root of the i-th simple word; determine whether there is a vector representation corresponding to the root of the i-th simple word in the vector model; when there is a vector representation corresponding to the root of the i-th simple word in the vector model, determine the vector representation corresponding to the root as the vector representation of the i-th simple word; A determination module is used to determine the word vector representation of the word to be represented based on the vector representation of each simple word.

7. A word representation device, characterized in that: include: processor; as well as a memory for storing a computer program executable on the processor; Wherein, when the computer program is executed by a processor, the steps of the word representation method described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are configured to execute the steps of the word representation method described in any one of claims 1 to 5.

Images