Synonym mining method, device, equipment and storage medium

Abstract

The present invention relates to the field of artificial intelligence technology, and discloses a synonym mining method, device, equipment and storage medium for improving the accuracy of synonym prediction. The method comprises: obtaining a target word in a question sentence input by a user; preprocessing the target word with one-hot encoding to obtain a one-hot code corresponding to the target word; inputting the one-hot code corresponding to the target word into a preset continuous bag of words model for vector representation and intermediate word prediction to obtain a corresponding intermediate word; vector representation of the intermediate word with a word embedding model to obtain a corresponding intermediate word vector; calculating the similarity between the intermediate word vector and the corresponding word vector of each word in a preset target domain vocabulary; sorting each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity, thereby obtaining a corresponding synonym dictionary.

Description

Technical Field

[0001] The present invention relates to the field of semantic parsing technology of artificial intelligence, and in particular to a synonym mining method, device, equipment and storage medium. Background Art

[0002] The existing word2vec algorithm adds up the word vectors corresponding to the context words to predict the intermediate word vector, but does not take into account the effect of the order of the context words on the prediction of the intermediate word vector. Therefore, the existing word2vec algorithm does not take into account the effect of the synonym part of speech when performing synonym mining. Summary of the invention

[0003] The main purpose of the present invention is to solve the technical problem that the existing word2vec algorithm does not take into account the role of synonym parts of speech when mining synonyms, resulting in insufficient prediction accuracy.

[0004] To achieve the above object, the present invention provides a synonym mining method in a first aspect, comprising:

[0005] Get the target word in the question sentence input by the user;

[0006] Using one-hot encoding to perform data preprocessing on the target word to obtain a one-hot code corresponding to the target word;

[0007] Input the unique hot code corresponding to the target word into a preset continuous bag-of-words model for vector representation and middle word prediction to obtain the corresponding middle word;

[0008] Using a word embedding model to perform vector representation on the intermediate word to obtain a corresponding intermediate word vector;

[0009] Calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, wherein the preset target domain vocabulary is a dictionary of the target domain search language;

[0010] Each word in the preset target domain vocabulary is sorted in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary.

[0011] Optionally, in another implementation of the first aspect of the present invention, before obtaining the target word in the question sentence input by the user, the method includes:

[0012] Obtaining questions to be trained in a question library of a target domain, and performing word segmentation processing on each question to be trained to obtain corresponding context words, wherein the questions to be trained contain words of different parts of speech;

[0013] Using one-hot encoding to perform data preprocessing on the context word block to obtain a one-hot code corresponding to the context word;

[0014] Multiply the one-hot code corresponding to each of the context words by the input weight matrix to obtain the corresponding word vector;

[0015] Perform vector splicing on the word vectors corresponding to each of the context words to obtain a splicing vector corresponding to the context word;

[0016] Input the concatenated vector corresponding to the context word into the continuous bag-of-words model, and output the one-hot encoded intermediate word;

[0017] Multiply the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector;

[0018] Using an activation function to process the intermediate word vector to obtain a corresponding probability distribution, and obtaining a word corresponding to a value with the maximum probability as the predicted intermediate word;

[0019] The preset loss function is used to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until the corresponding minimum function value is obtained, thereby obtaining the corresponding preset continuous bag-of-words model.

[0020] Optionally, in another implementation of the first aspect of the present invention, after using a preset loss function to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until a corresponding minimum function value is obtained and a corresponding preset continuous bag of words model is obtained, the method further includes:

[0021] According to the function value calculated by the preset loss function, the input weight matrix and the output weight matrix are updated by using a gradient descent algorithm to obtain an updated input weight matrix and an updated output weight matrix.

[0022] Optionally, in another implementation of the first aspect of the present invention, the preset loss function is:

[0023]

[0024] Among them, L(θ) represents the loss function value;

[0025] S represents the Sth sentence;

[0026] T j represents the number of target words in the jth sentence;

[0027] Represents the vector corresponding to the i-th non-leaf node in path k;

[0028] The word vector representing the concatenation of context words;

[0029] Represents the code corresponding to the i-th node in path k.

[0030] Optionally, in another implementation of the first aspect of the present invention, the acquiring the target word in the question sentence input by the user specifically includes:

[0031] Punctuation marks other than set punctuation marks in the question sentence input by the user are removed, and the question sentence input by the user is segmented to obtain corresponding target words, where the target words are words of the synonyms to be mined; wherein the set punctuation marks include at least one of punctuation marks used to express the tone of the question sentence input by the user and punctuation marks at the end of the sentence.

[0032] Optionally, in another implementation manner of the first aspect of the present invention, the similarity is at least one of cosine distance, Manhattan distance, correlation coefficient and Mahalanobis distance.

[0033] A second aspect of the present invention provides a synonym mining device, comprising:

[0034] A target word acquisition module is used to acquire the target word in the question sentence input by the user;

[0035] A one-hot code acquisition module, used for performing data preprocessing on the target word by using one-hot encoding to obtain a one-hot code corresponding to the target word;

[0036] An intermediate word acquisition module is used to input the one-hot code corresponding to the target word into a preset continuous bag-of-words model for vector representation and intermediate word prediction to obtain the corresponding intermediate word;

[0037] An intermediate word vector acquisition module is used to perform vector representation on the intermediate word using a word embedding model to obtain a corresponding intermediate word vector;

[0038] A similarity calculation module, used to calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, wherein the preset target domain vocabulary is a dictionary of the target domain search language;

[0039] The synonym dictionary acquisition module is used to sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary.

[0040] Optionally, in another implementation of the second aspect of the present invention, the synonym mining device further includes:

[0041] A context word acquisition module is used to acquire questions to be trained in the target domain question library, and to perform word segmentation processing on each question to be trained to acquire corresponding context words, wherein the questions to be trained contain words of different parts of speech;

[0042] A module for obtaining a one-hot code corresponding to a context word, used to perform data preprocessing on the context word block by using one-hot encoding to obtain a one-hot code corresponding to the context word;

[0043] A first word vector acquisition module is used to multiply the one-hot code corresponding to each of the context words by the input weight matrix to obtain a corresponding word vector;

[0044] A splicing vector acquisition module, used for performing vector splicing on the word vectors corresponding to each of the context words to obtain the splicing vectors corresponding to the context words;

[0045] A one-hot encoding intermediate word acquisition module is used to input the concatenated vector corresponding to the context word into a continuous bag-of-words model, and output a one-hot encoded intermediate word;

[0046] A second word vector acquisition module is used to multiply the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector;

[0047] A predicted intermediate word acquisition module, used for processing the intermediate word vector using an activation function to obtain a corresponding probability distribution, and obtaining a word corresponding to a value with the maximum probability as the predicted intermediate word;

[0048] The preset continuous bag-of-words model acquisition module is used to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word using a preset loss function until the corresponding minimum function value is obtained, thereby obtaining the corresponding preset continuous bag-of-words model.

[0049] Optionally, in another implementation of the second aspect of the present invention, the model training module further includes:

[0050] The matrix updating unit is used to update the input weight matrix and the output weight matrix using a gradient descent algorithm according to the function value calculated by the preset loss function to obtain an updated input weight matrix and an updated output weight matrix.

[0051] Optionally, in another implementation of the second aspect of the present invention, in the synonym mining device, the preset loss function is:

[0052]

[0053] Among them, L(θ) represents the loss function value;

[0054] S represents the Sth sentence;

[0055] T j represents the number of target words in the jth sentence;

[0056] Represents the vector corresponding to the i-th non-leaf node in path k;

[0057] The word vector representing the concatenation of context words;

[0058] Represents the code corresponding to the i-th node in path k.

[0059] Optionally, in another implementation of the second aspect of the present invention, the target word acquisition module is specifically used to:

[0060] Punctuation marks other than set punctuation marks in the question sentence input by the user are removed, and the question sentence input by the user is segmented to obtain corresponding target words, where the target words are words of the synonyms to be mined; wherein the set punctuation marks include at least one of punctuation marks used to express the tone of the question sentence input by the user and punctuation marks at the end of the sentence.

[0061] Optionally, in another implementation of the second aspect of the present invention, in the synonym mining device, the similarity is at least one of cosine distance, Manhattan distance, correlation coefficient and Mahalanobis distance.

[0062] The third aspect of the present invention provides a synonym mining device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the instructions in the memory so that the synonym mining device executes the method described in the first aspect above.

[0063] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect.

[0064] In the technical solution provided by the present invention, a target word in a question sentence input by a user is obtained; data preprocessing of the target word is performed using one-hot encoding to obtain a one-hot code corresponding to the target word; the one-hot code corresponding to the target word is input into a preset continuous bag of words model for vector representation and intermediate word prediction to obtain a corresponding intermediate word; a word embedding model is used to perform vector representation on the intermediate word to obtain a corresponding intermediate word vector; the similarity between the intermediate word vector and the corresponding word vector of each word in a preset target domain vocabulary is calculated, and the preset target domain vocabulary is a dictionary of a target domain retrieval language; each word in the preset target domain vocabulary is sorted in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary. The embodiment of the present invention is based on the problem that the existing word2vec algorithm does not take into account the role of synonym parts of speech when mining synonyms. The context word vectors are spliced, and the spliced ​​vectors are input into a continuous bag-of-words model to predict the probability of intermediate words, so as to train an improved continuous bag-of-words model; and the target word in the user input question sentence is input into the trained continuous bag-of-words model for vector representation and intermediate word prediction, and the obtained intermediate word vector is calculated for similarity with the word vector of the word in the vocabulary, and the words in the vocabulary are reordered according to the similarity calculation result, so as to obtain a synonym dictionary corresponding to the target word in the user input question sentence. The present invention improves the accuracy of synonym prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of an embodiment of a synonym mining method in an embodiment of the present invention;

[0066] Figure 2 A schematic diagram of another embodiment of the synonym mining method in an embodiment of the present invention;

[0067] Figure 3 A schematic diagram of an embodiment of a synonym mining device in an embodiment of the present invention;

[0068] Figure 4 A schematic diagram of another embodiment of the synonym mining device in an embodiment of the present invention;

[0069] Figure 5 Schematic diagram of an embodiment of a synonym mining device in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The embodiments of the present invention provide a synonym mining method, device, equipment and storage medium for improving the accuracy of synonym prediction.

[0071] In order to enable those skilled in the art to better understand the solutions of the present invention, the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0072] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0073] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the synonym mining method in the embodiment of the present invention includes:

[0074] 101. Obtain the target word in the question sentence input by the user.

[0075] Specifically, the server obtains the target word in the question sentence input by the user. Further, the server can remove the punctuation marks other than the set punctuation marks in the question sentence input by the user, and perform word segmentation processing on the question sentence input by the user to obtain the corresponding target word, which is the word of the synonym to be mined. Among them, the set punctuation marks include at least one of the punctuation marks used to express the tone of the question sentence input by the user and the punctuation marks at the end of the sentence. In specific implementation, the question input by the user can be an insurance question, so its target field is the insurance field. For example, if the question sentence input by the user is "Is Ben Federwin a legend in the insurance industry?", the punctuation mark "·" is removed, and the sentence is subjected to word segmentation processing to obtain the corresponding word as the target word. This step is the data preprocessing of the question sentence input by the user. By removing unnecessary punctuation marks and leaving punctuation marks that can represent a certain tone, the consideration of the synonym part of speech in the next step of using the preset continuous bag of words model to test the data can be improved. And the word segmentation processing of the question sentence input by the user can obtain words with part of speech. The algorithm or model used for word segmentation processing is not limited in the present invention.

[0076] 102. Use one-hot encoding to preprocess the target word data to obtain the one-hot code corresponding to the target word.

[0077] Specifically, the server also uses one-hot encoding to preprocess the target word data and obtain the one-hot code corresponding to the target word. That is, the target word is reprocessed using one-hot encoding, and the target word is represented by the one-hot code result, which is convenient for the next step of inputting the preset continuous bag of words model for prediction to prepare the data format, and also plays a role in expanding the characteristics of the target word to a certain extent.

[0078] 103. Input the unique hot code corresponding to the target word into the preset continuous bag-of-words model for vector representation and intermediate word prediction to obtain the corresponding intermediate word.

[0079] Specifically, the server inputs the obtained one-hot code into a preset continuous bag-of-words model for vector representation and intermediate word prediction to obtain the corresponding intermediate word. The present invention inputs the concatenated one-hot code into the preset continuous bag-of-words model for intermediate word prediction.

[0080] 104. Use the word embedding model to perform vector representation on the intermediate words and obtain the corresponding intermediate word vectors.

[0081] Specifically, the server also uses a word embedding model to perform vector representation on the intermediate words to obtain corresponding intermediate word vectors; since the output of the continuous bag of words model is still the predicted intermediate words of one-hot encoding, the present invention needs to further convert the predicted intermediate words of one-hot encoding into their corresponding intermediate word vectors through word embedding technology.

[0082] 105. Calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, where the preset target domain vocabulary is a dictionary of the target domain retrieval language.

[0083] Specifically, the server calculates the similarity between the intermediate word vector and the corresponding word vector of each word in the preset target domain vocabulary by performing similarity calculation. In specific implementation, the preset target domain vocabulary is a dictionary of the target domain retrieval language, such as the insurance vocabulary used in the insurance field. Further, the similarity is at least one of the cosine distance, Manhattan distance, correlation coefficient and Mahalanobis distance. For example, the similarity takes the cosine distance as an example, and calculates the cosine distance between the intermediate word vector and the corresponding word vector of each word in the preset target domain vocabulary. After the predicted intermediate word is vectorized using the word embedding model, it is necessary to further calculate the similarity between the intermediate word vector and the corresponding word vector of each word in the preset target domain vocabulary, so as to reorder the words in the preset target domain vocabulary according to the calculated similarity, thereby obtaining a synonym dictionary corresponding to the input target word.

[0084] 106. Sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary.

[0085] Specifically, the server will also sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain the corresponding synonym dictionary. For example, taking the cosine distance as an example of similarity, after calculating the cosine distance between the intermediate word vector and the corresponding word vector of each word in the preset target domain vocabulary, each word in the preset target domain vocabulary is sorted in descending order according to the corresponding cosine distance, and the words in the preset target domain vocabulary are reordered to obtain the synonym dictionary corresponding to the input target word, thereby completing the synonym mining for the target word.

[0086] As can be seen from the above, the method embodiment of the present invention performs vector representation and intermediate word prediction by inputting the target word in the user input question sentence into the trained preset continuous bag of words model, and performs similarity calculation between the obtained intermediate word vector and the word vector of the word in the vocabulary, and reorders the words in the vocabulary according to the similarity calculation result, thereby obtaining a synonym dictionary corresponding to the target word in the user input question sentence. When training and testing data, the preset continuous bag of words model uses the concatenated word vector to be transformed into the same dimension as the initialized input word vector through a layer of forward neural network, and then uses the word vector to predict the probability of the intermediate word appearing.

[0087] Through the synonym mining method of the present invention, the part of speech of the mined synonyms is made consistent with the part of speech of the target word in the question sentence input by the user, thereby improving the semantic matching degree between the question sentence input by the user and the relevant standard questions in the question library, thereby improving the accuracy of the answer to the target question sentence input by the user.

[0088] It can be seen that the present invention realizes fully automatic mining of synonyms in user input question sentences, so there is no need to manually judge whether the words are similar. And if there are 300,000 words in the target domain vocabulary used by the present invention, and the target words obtained by word segmentation of the user input question sentence are 5 words, that is, n is 5 in the top n hot word statistics, the present invention can be used to mine synonyms to obtain a synonym dictionary of 1.5 million words. It can be seen that the scale of the synonym dictionary obtained by the present invention through the synonym mining can be freely defined, and the present invention also realizes the scalability of synonym mining.

[0089] For further information, see Figure 2 In another embodiment of the synonym mining method in the embodiment of the present invention, the method includes:

[0090] 201. Obtain the questions to be trained in the target domain question library, and perform word segmentation on each question to be trained to obtain corresponding context words. The questions to be trained contain words of different parts of speech.

[0091] Specifically, some standard questions are extracted from the question bank or question-answer bank of the target field where the user inputs the question statement as materials for model training. In specific implementation, the user input question may be an insurance question, and thus the target field is the insurance field.

[0092] 202. Use one-hot encoding to preprocess the context word blocks to obtain the one-hot codes corresponding to the context words.

[0093] 203. Multiply the one-hot code corresponding to each context word by the input weight matrix to obtain a corresponding word vector. Specifically, the corresponding word vector can be obtained by multiplying the one-hot code corresponding to the context word by the shared input weight matrix.

[0094] 204. Concatenate the word vectors corresponding to each context word to obtain a concatenated vector corresponding to the context word.

[0095] 205. Input the concatenated vector corresponding to the context word into the continuous bag-of-words model, and output the one-hot encoded intermediate word.

[0096] The above steps 202, 204, and 205 are similar to steps 101 to 103 and are not described in detail here.

[0097] 206. Multiply the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector.

[0098] Specifically, the server also multiplies the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector.

[0099] In specific implementation, the word vectors corresponding to the context words with parts of speech are obtained by segmenting and encoding the training problems as mentioned above, and the obtained word vectors are further concatenated, and the concatenated vectors are input into the continuous bag-of-words model for intermediate word prediction; since the output of the continuous bag-of-words model is still the one-hot encoded intermediate word, the server also converts the one-hot encoded intermediate word into its corresponding intermediate word vector by multiplying the output weight matrix.

[0100] 207. Use the activation function to process the intermediate word vector to obtain the corresponding probability distribution, and obtain the word corresponding to the maximum probability value as the predicted intermediate word.

[0101] Specifically, the server uses an activation function to process the obtained intermediate word vector to obtain the corresponding probability distribution, and obtains the word corresponding to the maximum probability value as the predicted intermediate word. In this way, the probability of occurrence of the intermediate word is predicted by concatenating the context word vectors, and the word corresponding to the maximum probability value is selected as the predicted intermediate word. The activation function used here is not specifically limited.

[0102] 208. Use a preset loss function to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until the corresponding minimum function value is obtained, and obtain the corresponding preset continuous bag of words model.

[0103] Specifically, the server also uses a preset loss function to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until the corresponding minimum function value is obtained, and the corresponding preset continuous bag of words model is obtained. In specific implementation, since the intermediate word corresponding to the predicted maximum probability value and the one-hot encoded intermediate word obtained by the model output have an error, it is necessary to compare it with the one-hot code corresponding to the intermediate word, so that the error is as small as possible. The present invention uses a preset loss function to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word. The smaller the error, the better. Therefore, the training process of the continuous bag of words model is stopped until the corresponding minimum function value is obtained. The obtained continuous bag of words model is the optimal training model, that is, the preset continuous bag of words model required by the present invention. The present invention can perform vector representation and synonym prediction for each word in the user input question through the preset continuous bag of words model.

[0104] Furthermore, the preset loss function is:

[0105]

[0106] Among them, L(θ) represents the loss function value;

[0107] S represents the Sth sentence;

[0108] T j represents the number of target words in the jth sentence;

[0109] Represents the vector corresponding to the i-th non-leaf node in path k;

[0110] The word vector representing the concatenation of context words;

[0111] Represents the code corresponding to the i-th node in path k.

[0112] In specific implementation, when predicting the probability of the middle word, the word vectors corresponding to the context words are spliced, and then the spliced ​​word vectors are used to predict the probability of the middle word. Therefore, the biggest feature of the preset loss function of the present invention is that the word vector representing the context is used in this formula for splicing. The spliced ​​word vector is then transformed into the same dimension as the initialized word vector through a layer of forward neural network, and then the word vector is used to predict the probability of the middle word. Since the spliced ​​word vector takes into account the part-of-speech information contained in the order of appearance of the context words, that is, the grammar of the language is taken into account, the part-of-speech information is also taken into account when predicting the middle word.

[0113] Furthermore, the server also uses a gradient descent algorithm to update the input weight matrix and the output weight matrix according to the function value calculated by the preset loss function, and obtains an updated input weight matrix and an updated output weight matrix. That is, after the model training is completed, the input weight matrix and the output weight matrix required for word embedding in the input layer and output layer of the preset continuous word bag model need to be updated, that is, the weight matrix needs to be corrected in the gradient direction. The present invention uses a gradient descent algorithm to probabilistically derive the loss function for all function values ​​calculated by the preset loss function to update the weight matrix, and obtains an updated input weight matrix and an updated output weight matrix by stochastic gradient descent as the new weight matrix in the preset continuous word bag model.

[0114] 209. Obtain the target word in the question sentence input by the user.

[0115] 210. Use one-hot encoding to preprocess the target word data to obtain the one-hot code corresponding to the target word.

[0116] 211. Input the unique hot code corresponding to the target word into the preset continuous bag-of-words model for vector representation and intermediate word prediction to obtain the corresponding intermediate word.

[0117] 212. Use the word embedding model to vectorize the intermediate words and obtain the corresponding intermediate word vectors.

[0118] 213. Calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, where the preset target domain vocabulary is a dictionary of the target domain retrieval language.

[0119] 214. Sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary.

[0120] Specifically, the specific implementation description of steps 209-214 refers to steps 101-106, which will not be repeated here.

[0121] The above describes the synonym mining method in the embodiment of the present invention. The following describes the synonym mining device in the embodiment of the present invention. Figure 3 , an embodiment of the synonym mining device in the embodiment of the present invention includes:

[0122] The target word acquisition module 301 is used to acquire the target word in the question sentence input by the user.

[0123] The one-hot code acquisition module 302 is used to perform data preprocessing on the target word by using one-hot encoding to obtain the one-hot code corresponding to the target word.

[0124] The intermediate word acquisition module 303 is used to input the one-hot code corresponding to the target word into a preset continuous bag-of-words model for vector representation and intermediate word prediction to obtain the corresponding intermediate word.

[0125] The intermediate word vector acquisition module 304 is used to use the word embedding model to perform vector representation on the intermediate words to obtain the corresponding intermediate word vectors.

[0126] The similarity calculation module 305 is used to calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, where the preset target domain vocabulary is a dictionary of the target domain search language.

[0127] The synonym dictionary acquisition module 306 is used to sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary.

[0128] Optionally, in another implementation of the synonym mining device of the present invention, as Figure 4 As shown, the synonym mining device includes:

[0129] The target word acquisition module 301 is used to acquire the target word in the question sentence input by the user.

[0130] The one-hot code acquisition module 302 is used to perform data preprocessing on the target word by using one-hot encoding to obtain the one-hot code corresponding to the target word.

[0131] The intermediate word acquisition module 303 is used to input the one-hot code corresponding to the target word into a preset continuous bag-of-words model for vector representation and intermediate word prediction to obtain the corresponding intermediate word.

[0132] The intermediate word vector acquisition module 304 is used to use the word embedding model to perform vector representation on the intermediate words to obtain the corresponding intermediate word vectors.

[0133] The similarity calculation module 305 is used to calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, where the preset target domain vocabulary is a dictionary of the target domain search language.

[0134] The synonym dictionary acquisition module 306 is used to sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary.

[0135] The context word acquisition module 307 is used to acquire the questions to be trained in the target domain question library, and perform word segmentation processing on each question to be trained to acquire the corresponding context words. The questions to be trained contain words of different parts of speech.

[0136] The module 308 for obtaining the one-hot code corresponding to the context word is used to perform data preprocessing on the context word block by using one-hot encoding to obtain the one-hot code corresponding to the context word.

[0137] The first word vector acquisition module 309 is used to multiply the one-hot code corresponding to each context word by the input weight matrix to obtain the corresponding word vector.

[0138] The concatenated vector acquisition module 310 is used to concatenate the word vectors corresponding to each context word to obtain the concatenated vector corresponding to the context word.

[0139] The one-hot encoded intermediate word acquisition module 311 is used to input the concatenated vector corresponding to the context word into the continuous bag-of-words model, and output the one-hot encoded intermediate word.

[0140] The second word vector acquisition module 312 is used to multiply the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector.

[0141] The predicted intermediate word acquisition module 313 is used to process the intermediate word vector using an activation function to obtain a corresponding probability distribution, and obtain the word corresponding to the maximum probability value as the predicted intermediate word.

[0142] The preset continuous bag-of-words model acquisition module 314 is used to use a preset loss function to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until the corresponding minimum function value is obtained, thereby obtaining the corresponding preset continuous bag-of-words model.

[0143] Optionally, in another implementation of the synonym mining device of the present invention, the model training module further includes:

[0144] The matrix update unit is used to update the input weight matrix and the output weight matrix using a gradient descent algorithm according to the function value calculated by the preset loss function to obtain an updated input weight matrix and an updated output weight matrix.

[0145] Optionally, in another implementation of the synonym mining device of the present invention, in the synonym mining device, the preset loss function is:

[0146]

[0147] Among them, L(θ) represents the loss function value;

[0148] S represents the Sth sentence;

[0149] T j represents the number of target words in the jth sentence;

[0150] Represents the vector corresponding to the i-th non-leaf node in path k;

[0151] The word vector representing the concatenation of context words;

[0152] Represents the code corresponding to the i-th node in path k.

[0153] Optionally, in another implementation of the synonym mining device of the present invention, the target word acquisition module is specifically used for:

[0154] Punctuation marks other than set punctuation marks are removed from the question sentence input by the user, and the question sentence input by the user is segmented to obtain the corresponding target word, which is the word of the synonym to be mined. The set punctuation marks include at least one of punctuation marks used to express the tone of the question sentence input by the user and punctuation marks at the end of the sentence.

[0155] Optionally, in another implementation of the synonym mining device of the present invention, in the synonym mining device, the similarity of word vectors is at least one of cosine distance, Manhattan distance, correlation coefficient and Mahalanobis distance.

[0156] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For device or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0157] above Figure 3 and Figure 4 The synonym mining device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The synonym mining device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0158] Figure 51 is a schematic diagram of the structure of a synonym mining device provided by an embodiment of the present invention. The synonym mining device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 501 (for example, one or more processors) and a memory 509, and one or more storage media 508 (for example, one or more mass storage devices) storing application programs 507 or data 506. Among them, the memory 509 and the storage medium 508 can be short-term storage or permanent storage. The program stored in the storage medium 508 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the Boolean variable storage of the graph calculation. Furthermore, the processor 501 can be configured to communicate with the storage medium 508 to execute a series of instruction operations in the storage medium 508 on the synonym mining device 500.

[0159] The synonym mining device 500 may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input and output interfaces 504, and / or one or more operating systems 505, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the synonym mining device shown in the figure does not constitute a limitation of the synonym mining device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0160] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0161] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0164] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0165] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A synonym mining method, characterized in that: include: Get the target word in the question sentence input by the user; Using one-hot encoding to perform data preprocessing on the target word to obtain a one-hot code corresponding to the target word; Input the unique hot code corresponding to the target word into a preset continuous bag-of-words model for vector representation and middle word prediction to obtain the corresponding middle word; Using a word embedding model to perform vector representation on the intermediate word to obtain a corresponding intermediate word vector; Calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, wherein the preset target domain vocabulary is a dictionary of the target domain search language; Sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary; Before obtaining the target word in the question sentence input by the user, the following steps are included: Obtaining questions to be trained in a question library of a target domain, and performing word segmentation processing on each question to be trained to obtain corresponding context words, wherein the questions to be trained contain words of different parts of speech; Perform data preprocessing on the context words by using one-hot encoding to obtain a one-hot code corresponding to the context words; Multiply the one-hot code corresponding to each of the context words by the input weight matrix to obtain the corresponding word vector; Perform vector splicing on the word vectors corresponding to each of the context words to obtain a splicing vector corresponding to the context word; Input the concatenated vector corresponding to the context word into a continuous bag-of-words model, and output a one-hot encoded intermediate word; Multiply the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector; Using an activation function to process the intermediate word vector to obtain a corresponding probability distribution, and obtaining a word corresponding to a value with the maximum probability as the predicted intermediate word; The preset loss function is used to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until the corresponding minimum function value is obtained, thereby obtaining the corresponding preset continuous bag-of-words model.

2. The synonym mining method according to claim 1, characterized in that: After using the preset loss function to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until a corresponding minimum function value is obtained and a corresponding preset continuous bag-of-words model is obtained, the method further includes: According to the function value calculated by the preset loss function, the input weight matrix and the output weight matrix are updated by using a gradient descent algorithm to obtain an updated input weight matrix and an updated output weight matrix.

3. The synonym mining method according to claim 1, characterized in that: The preset loss function is: ； in, Represents the loss function value; s represents the sth sentence; represents the number of target words in the jth sentence; Represents the vector corresponding to the i-th non-leaf node in path k; Word vectors representing the concatenation of context words; Represents the code corresponding to the i-th node in path k.

4. The synonym mining method according to claim 1, characterized in that: The similarity is at least one of cosine distance, Manhattan distance, correlation coefficient and Mahalanobis distance.

5. The synonym mining method according to any one of claims 1 to 4, characterized in that: The step of obtaining the target word in the question sentence input by the user specifically includes: Punctuation marks other than set punctuation marks in the user input question sentence are removed, and the user input question sentence is segmented to obtain corresponding target words, where the target words are words for which synonyms are to be mined, and the set punctuation marks include at least one of punctuation marks used to express the tone of the user input question sentence and punctuation marks at the end of the sentence.

6. A synonym mining device, characterized in that: include: A target word acquisition module is used to acquire the target word in the question sentence input by the user; A one-hot code acquisition module, used for performing data preprocessing on the target word by using one-hot encoding to obtain a one-hot code corresponding to the target word; An intermediate word acquisition module is used to input the one-hot code corresponding to the target word into a preset continuous bag-of-words model for vector representation and intermediate word prediction to obtain the corresponding intermediate word; An intermediate word vector acquisition module is used to perform vector representation on the intermediate word using a word embedding model to obtain a corresponding intermediate word vector; A similarity calculation module, used to calculate the similarity between the intermediate word vector and the word vector corresponding to each word in the preset target domain vocabulary, wherein the preset target domain vocabulary is a dictionary of the target domain search language; A synonym dictionary acquisition module is used to sort each word in the preset target domain vocabulary in descending order according to the calculated corresponding similarity to obtain a corresponding synonym dictionary; Before obtaining the target word in the question sentence input by the user, the following steps are included: Obtaining questions to be trained in a question library of a target domain, and performing word segmentation processing on each question to be trained to obtain corresponding context words, wherein the questions to be trained contain words of different parts of speech; Perform data preprocessing on the context words by using one-hot encoding to obtain a one-hot code corresponding to the context words; Multiply the one-hot code corresponding to each of the context words by the input weight matrix to obtain the corresponding word vector; Perform vector splicing on the word vectors corresponding to each of the context words to obtain a splicing vector corresponding to the context word; Input the concatenated vector corresponding to the context word into a continuous bag-of-words model, and output a one-hot encoded intermediate word; Multiply the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector; Using an activation function to process the intermediate word vector to obtain a corresponding probability distribution, and obtaining a word corresponding to a value with the maximum probability as the predicted intermediate word; The preset loss function is used to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word until the corresponding minimum function value is obtained, thereby obtaining the corresponding preset continuous bag-of-words model.

7. The synonym mining device according to claim 6, characterized in that: Also includes: A context word acquisition module is used to acquire questions to be trained in the target domain question library, and to perform word segmentation processing on each question to be trained to acquire corresponding context words, wherein the questions to be trained contain words of different parts of speech; A module for obtaining a one-hot code corresponding to a context word, used for performing data preprocessing on the context word by using one-hot encoding to obtain a one-hot code corresponding to the context word; A first word vector acquisition module is used to multiply the one-hot code corresponding to each of the context words by the input weight matrix to obtain a corresponding word vector; A splicing vector acquisition module, used for performing vector splicing on the word vectors corresponding to each of the context words to obtain the splicing vectors corresponding to the context words; A one-hot encoding intermediate word acquisition module is used to input the concatenated vector corresponding to the context word into a continuous bag-of-words model and output a one-hot encoded intermediate word; A second word vector acquisition module is used to multiply the one-hot code corresponding to the middle word by the output weight matrix to obtain the corresponding middle word vector; A predicted intermediate word acquisition module, used for processing the intermediate word vector using an activation function to obtain a corresponding probability distribution, and obtaining a word corresponding to a value with the maximum probability as the predicted intermediate word; The preset continuous bag-of-words model acquisition module is used to calculate the error between the predicted intermediate word and the one-hot encoded intermediate word using a preset loss function until the corresponding minimum function value is obtained, thereby obtaining the corresponding preset continuous bag-of-words model.

8. A synonym mining device, characterized in that: The synonym mining device comprises: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the synonym mining device to execute the method according to any one of claims 1-5.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images