Word recommendation method and device, electronic equipment and storage medium

Abstract

This invention relates to the field of natural language processing technology, providing a word recommendation method, apparatus, electronic device, and storage medium. The method first obtains the definition to be queried; then, based on a reverse dictionary model, selects target words corresponding to the definition from a candidate word list; finally, based on the target words, determines the recommendation result. This method utilizes a reverse dictionary model obtained through multi-task learning to predict words and their parts of speech, considering the parts of speech rather than solely relying on the quality of definitions in the dictionary and the quality of the user-input definition. This ensures the accuracy and quality of the recommendation results, thereby improving the user experience. The introduction of parts of speech helps reduce the prediction space, constrains the recommendation results, makes the results more reliable, reduces interference from easily confused words, and avoids situations where the recommended results differ significantly from the user-input definition.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a word recommendation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, recommending words to users based on their provided definitions has become a new task in the field of natural language processing. For example, when a user experiences the tip-of-the-tongue phenomenon (also known as being unable to say what they want to say), that is, temporarily forgetting a word, it is necessary to look up the corresponding word using the known definition.

[0003] In existing technologies, the following two methods are typically used when recommending words to users:

[0004] 1) Definition-based matching methods: These methods compare the input query definition with stored word definitions one by one and return the word corresponding to the definition most similar to the input query definition. These methods primarily use handcrafted features and information retrieval techniques to search for target words, or train deep learning models to calculate definition similarity and obtain recommendation results.

[0005] 2) Definition and word matching methods: These methods typically learn word embeddings directly. For the input definition to be queried, they calculate the similarity between the definition and the word embeddings to obtain the corresponding recommendation results.

[0006] Method 1), which calculates based solely on the similarity between definitions, relies entirely on the quality of the definitions in the lexicon and the quality of the user-inputted definition, thus its effectiveness is generally limited. Method 2, while more direct in its calculation than Method 1 and not requiring a mapping between definitions and words, still fails to accurately produce recommendation results using its learned word embeddings. Summary of the Invention

[0007] This invention provides a word recommendation method, apparatus, electronic device, and storage medium to address the deficiencies in the prior art.

[0008] This invention provides a word recommendation method, comprising:

[0009] Get the definition of the query;

[0010] Based on the reverse dictionary model, the first matching result of each word in the candidate word list with the definition to be queried and the second matching result of the part of speech of each word with the definition to be queried are determined. Based on the first matching result and the second matching result, the target word corresponding to the definition to be queried is selected from the candidate word list.

[0011] Based on the target words, the recommendation results are determined;

[0012] The reverse dictionary model is obtained through multi-task learning based on each word, the part of speech of each word, and the definition samples of each word.

[0013] According to the word recommendation method provided by the present invention, the reverse dictionary model is trained based on the following steps:

[0014] For any word in the candidate word list, the first token set of the word is concatenated with the second token set of the definition sample of the word.

[0015] The first token set in the concatenated result is replaced with the mask identifier, and the replacement result is input into the initial model. The initial model encodes the identifier bits of the replacement result and the concatenated result respectively to obtain the first feature corresponding to the replacement result and the second feature corresponding to the identifier bit.

[0016] Based on the first feature, a third matching result between the mask identifier and each word is determined. Based on the third matching result and each word, a first loss function is calculated. Based on the second feature, a fourth matching result between the identifier and the part-of-speech of each word is determined. Based on the fourth matching result and the part-of-speech of each word, a second loss function is calculated.

[0017] Based on the first loss function and the second loss function, a target loss function is calculated, and based on the target loss function, the model parameters of the initial model are iteratively updated to obtain the inverse dictionary model.

[0018] According to a word recommendation method provided by the present invention, the step of concatenating a first token set of any word with a second token set of the definition samples of any word further includes:

[0019] The definition sample of any word is segmented into sentences to obtain several sentences, and the third token set of each sentence is determined.

[0020] Concatenate the first token set of any word with the third token set of each statement.

[0021] According to a word recommendation method provided by the present invention, the step of selecting the target word corresponding to the query definition from the candidate word table based on the first matching result and the second matching result specifically includes:

[0022] Based on the part-of-speech index matrix of the candidate word list, the first matching result, and the second matching result, the target word is selected from the candidate word list;

[0023] The part-of-speech index matrix is ​​constructed based on the part-of-speech category of each word, the number of words in the candidate word list, and the number of part-of-speech categories in the candidate word list.

[0024] According to a word recommendation method provided by the present invention, the step of selecting the target word from the candidate word list based on the part-of-speech index matrix, the first matching result, and the second matching result specifically includes:

[0025] Calculate the dot product of the part-of-speech index matrix and the second matching result;

[0026] Based on the dot product result and the first matching result, the comprehensive matching index of each word is calculated, and based on the comprehensive matching index of each word, several words with high comprehensive matching index are selected from the candidate word list as the target words.

[0027] According to a word recommendation method provided by the present invention, the target words include multiple terms, and the step of determining the recommendation result based on the target words specifically includes:

[0028] Determine the target meaning of each target word;

[0029] Calculate the similarity between each target definition and the definition to be queried, and determine a number of designated words with high similarity among each target word;

[0030] The frequency of each specified word in the corpus is determined, and the specified words are ranked based on the word frequency and the comprehensive matching index of each specified word. The ranking result is then used as the recommendation result.

[0031] According to a word recommendation method provided by the present invention, determining the recommendation result based on the target word specifically includes:

[0032] Determine whether the target word exists in the definition to be queried;

[0033] If they exist, the target words that exist in the definition to be queried are removed, and the recommendation result is determined based on the removal result.

[0034] The present invention also provides a word recommendation device, comprising:

[0035] The definition retrieval module is used to retrieve the definition to be queried;

[0036] The word selection module is used to determine the first matching result of each word in the candidate word list with the definition to be queried and the second matching result of the part of speech of each word with the definition to be queried based on the reverse dictionary model, and select the target word corresponding to the definition to be queried from the candidate word list based on the first matching result and the second matching result;

[0037] The recommendation module is used to determine recommendation results based on the target words;

[0038] The reverse dictionary model is obtained through multi-task learning based on each word, the part of speech of each word, and the definition samples of each word.

[0039] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the word recommendation method as described above.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the word recommendation method as described above.

[0041] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the word recommendation method as described above.

[0042] This invention provides a word recommendation method, apparatus, electronic device, and storage medium. The method first obtains the definition to be queried; then, based on a reverse dictionary model, it determines the first matching result between each word in the candidate word list and the definition to be queried, as well as the second matching result between the part-of-speech tag of each word and the definition to be queried. Based on the first and second matching results, it selects the target word corresponding to the definition to be queried from the candidate word list; finally, based on the target word, it determines the recommendation result. This method utilizes a reverse dictionary model obtained through multi-task learning to predict words and their parts of speech, considering the part of speech of words and not solely relying on the quality of definitions in the dictionary and the quality of the user-input definition to be queried. This ensures the accuracy and quality of the recommendation results, thereby improving the user experience. The introduction of part-of-speech tags helps reduce the prediction space, constrains the recommendation results, makes the recommendations more reliable, reduces interference from easily confused words, and avoids situations where the recommended results differ significantly from the user-input definition to be queried. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on the drawings described below without creative effort.

[0044] Figure 1 This is one of the flowcharts illustrating the word recommendation method provided by the present invention;

[0045] Figure 2 This is a schematic diagram of the structure of the reverse dictionary model in the word recommendation method provided by this invention;

[0046] Figure 3 This is the second flowchart illustrating the word recommendation method provided by this invention;

[0047] Figure 4 This is a schematic diagram of the word recommendation device provided by the present invention;

[0048] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] Reverse dictionaries are a new task in the field of natural language processing. While a regular dictionary provides a definition given a word, a reverse dictionary provides a list of words that match the given definition. Reverse dictionaries can help address tip-of-the-tongue phenomena in users; they can also help language learners learn and consolidate vocabulary; and they can improve the quality of life for people with word choice-based aphasia. Therefore, improving the capabilities of reverse dictionaries is of great significance.

[0051] In existing technologies, word recommendation to users typically employs either definition-based matching or definition-word-based matching. Definition-based matching relies on character-level and word-level features and information retrieval techniques to search for target words, which often results in poor timeliness for massive dictionary databases. Furthermore, methods that calculate based solely on the similarity between definitions depend on the quality of the definitions in the dictionary and the quality of the user-input definition, thus their effectiveness is generally limited. While definition-word-based matching is more direct in its computational form than definition-based matching, as it doesn't require a mapping between definitions and words, the learned word embeddings still cannot accurately yield recommendation results.

[0052] Based on this, this embodiment of the invention provides a word recommendation method to solve the problems existing in the current word recommendation method, and thus make accurate recommendations to users.

[0053] Figure 1 This is a flowchart illustrating a word recommendation method provided in an embodiment of the present invention, such as... Figure 1 As shown, the method includes:

[0054] S1, retrieve the definition to be queried;

[0055] S2, based on the reverse dictionary model, determine the first matching result of each word in the candidate word list with the definition to be queried and the second matching result of the part of speech of each word with the definition to be queried, and select the target word corresponding to the definition to be queried from the candidate word list based on the first matching result and the second matching result;

[0056] S3, Based on the target words, determine the recommendation results;

[0057] The reverse dictionary model is obtained through multi-task learning based on each word, the part of speech of each word, and the definition samples of each word.

[0058] Specifically, the word recommendation method provided in this embodiment of the invention is executed by a word recommendation device, which can be configured in a computer. The computer can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0059] First, step S1 is executed to obtain the definition to be queried. This definition can be input by the user, and can be obtained either through the voice module or directly through an external interface. That is, the definition can be in either voice or text form; no specific limitation is made here. It is understood that this embodiment of the invention performs text matching; therefore, if the definition to be queried is in voice form, it needs to be converted to text form. Conventional conversion methods can be used for this conversion.

[0060] Then, step S2 is executed, using a reverse dictionary model to determine the first matching result between each word in the candidate word list and the query definition, as well as the second matching result between the part of speech of each word and the query definition. The candidate word list can be constructed based on words in an existing dictionary, and the language of the existing dictionary is not specifically limited here. For example, the existing dictionary can be the Xinhua Dictionary and / or the Zhonghua Dictionary, in which case the word recommendation method is suitable for Chinese word recommendation; the existing dictionary can also be a foreign language dictionary, in which case the word recommendation method is suitable for foreign language word recommendation.

[0061] The inverse dictionary model can be a neural network model, such as a convolutional neural network model, a BERT model, or a RoBERTa model, etc., without being specifically limited here.

[0062] This inverse dictionary model has two functions: one is to determine the first matching result between each word in the candidate word list and the query definition, i.e., word prediction; the other is to determine the second matching result between the part-of-speech of each word in the candidate word list and the query definition, i.e., part-of-speech prediction. In other words, this inverse dictionary model can have one input, the query definition, two intermediate results, the first matching result and the second matching result, and one output, the target words, which can be one or more.

[0063] The first matching result can be the matching score between each word in the candidate word list predicted by the inverse dictionary model and the query definition. The higher the matching score, the greater the likelihood that the corresponding word corresponds to the query definition. Here, a word corresponding to the query definition means that the query definition can be used to explain the word.

[0064] The second matching result can be the matching probability of the part of speech of each word in the candidate word list predicted by the reverse dictionary model with the definition to be queried. The higher the matching probability, the closer the part of speech of the corresponding word is to the part of speech of the word that the definition to be queried can explain.

[0065] like Figure 2As shown, the first token set of each word in the candidate word list can be concatenated with the target token set of the definition to be queried to obtain the first concatenation result. The first token set and the target token set are separated by a separator character, which can be represented as SEP. Moreover, the starting position of the first concatenation result corresponding to each word is indicated by a flag bit, which can be represented as CLS.

[0066] For any word in the candidate word list, each character of that word corresponds to a token. The tokens corresponding to all characters in a word are arranged in order to form the first token set for that word. Similarly, each character in the definition to be queried also corresponds to a token. The tokens corresponding to all characters in the definition to be queried are arranged in order to form the target token set. If any word contains n characters, then the first token set contains n tokens, denoted as Tok1, ..., Tok... n If the definition to be queried contains m characters, then the target token set contains m tokens, denoted as Tok'1, ..., Tok'. m .

[0067] Subsequently, the first set of tokens in the first concatenation result can be replaced with a mask identifier to obtain the first replacement result. This mask identifier can be a MASK. Each token in the first set of tokens can be replaced with a MASK. If any word contains n characters, then the first replacement result contains a total of n MASKs.

[0068] Subsequently, the first replacement result can be input into the inverse dictionary model, which encodes the first replacement result and its identifier to obtain the first target features E1, ..., E1 corresponding to the first replacement result. n E [SEP] E'1, ..., E' m and the second target feature E corresponding to the identifier CLS [CLS] .

[0069] If the first replacement result is X = [x1, ..., x N ], where N is the number of positions in the first replacement result, i.e., the number of tokens. After passing through the inverse dictionary model (here, the inverse dictionary model is the RoBERTa model as an example), the first target feature corresponding to the i-th (1≤i≤N) token and the second target feature corresponding to the identifier CLS in the first replacement result can both be represented as:

[0070] E i =RoBERTa(x i )

[0071] Subsequently, the inverse dictionary model can obtain the matching results sc1, ..., sc1 at each position in the first replacement result through a fully connected layer based on the first target feature. n sc [SEP] ,sc'1, ...,sc' m Then, the matching result at the position corresponding to the mask identifier in the first replacement result is taken as the first matching result, and the second matching result sc can be directly determined through a fully connected layer and a softmax function based on the second target features. [CLS] In other words, the matching results at each position in the first replacement result obtained by the inverse dictionary model can be represented as:

[0072] sc = W c RoBERTa(X)+b c

[0073] Where sc represents the matching result at each position in the first replacement result. All of these are model parameters corresponding to word prediction in the inverse dictionary model, n c This represents the number of words in the candidate word list.

[0074] The score of the first matching result corresponding to the i-th word in the candidate word list w,i It can be represented as:

[0075]

[0076] Where, [x] i Let x represent the i-th element, and index(p) return the index of the p-th token in the candidate word list. Let be the set of target tokens corresponding to the i-th word.

[0077] The second matching result p corresponding to the i-th word in the candidate word list w,i It can be represented as:

[0078] p w,i =softmax(W pos RoBERTa(X) CLS +b pos )

[0079] in, All of these are model parameters corresponding to part-of-speech prediction in the inverse dictionary model, n pos This represents the number of part-of-speech categories in the candidate word list.

[0080] This inverse dictionary model can be trained using each word in the candidate vocabulary, its part of speech, and its definition samples as training samples, through multi-task learning on the initial model. The model structure of this initial model can be the same as that of convolutional neural network models, BERT models, and RoBERTa models.

[0081] The definition samples for each word can be obtained by searching existing dictionaries, which can be Xinhua Dictionary and / or Chinese Dictionary, or foreign language dictionaries. There is no specific limitation on the language of the existing dictionaries here.

[0082] When performing multi-task learning on the initial model, the multi-task can include a word prediction task that predicts words based on definition samples and a part-of-speech prediction task that predicts the part of speech of words based on definition samples. In the word prediction task, definition samples can be used as input, and the corresponding words in the definition samples can be used as labels. Similarly, in the part-of-speech prediction task, definition samples can be used as input, and the part of speech of the corresponding words in the definition samples can be used as labels.

[0083] During the multi-task learning process of the initial model, the loss function used for each task can be selected as needed. For example, a conventional loss function, a cross-entropy loss function, or other loss functions can be selected; no specific limitation is made here. The inverse dictionary model is obtained when each task reaches a preset number of rounds or the used loss function converges.

[0084] Subsequently, based on the first and second matching results, target words corresponding to the query definition can be selected from the candidate word list. For example, one can first select several first-category words with the highest first-matching results from the candidate word list based on the first-matching results, and then select several second-category words with the highest second-matching results from the first-category words as target words based on the second-matching results. Alternatively, one can first determine the comprehensive matching index of each word in the candidate word list by weighted summation based on the first and second matching results, and then select several words with high comprehensive matching indices from the candidate word list as target words based on the comprehensive matching indices of each word in the candidate word list.

[0085] Here, the target words can be one or more. If there are multiple target words, the number of target words can be set as needed, such as 100, 150, 200, etc.

[0086] Finally, step S3 is executed to determine the recommendation results based on the target words. Here, the target words can be directly used as the recommendation results, and recommended to the user in descending order of similarity between the target definition and the query definition corresponding to each target word; alternatively, the target words can be further filtered based on other indicators to select several words that meet the filtering criteria as the recommendation results. In particular, if there is only one target word, that target word can be directly used as the recommendation result.

[0087] The word recommendation method provided in this embodiment of the invention first obtains the definition to be queried; then, based on an inverse dictionary model, it determines the first matching result between each word in the candidate word list and the definition to be queried, as well as the second matching result between the part-of-speech tag of each word and the definition to be queried. Based on the first and second matching results, it selects the target word corresponding to the definition to be queried from the candidate word list; finally, it determines the recommendation result based on the target word. This method uses an inverse dictionary model obtained through multi-task learning to predict words and their parts of speech. It considers the part of speech of words and does not completely rely on the quality of the definitions in the dictionary or the quality of the definition to be queried input by the user. This can ensure the accuracy of the recommendation results, improve the quality of the recommendation results, and thus improve the user experience. The introduction of part-of-speech tagging can help reduce the prediction space, constrain the recommendation results, make the recommendation results more reliable, reduce the interference of easily confused words, and avoid situations where the recommendation results differ greatly from the definition to be queried input by the user.

[0088] Existing definition-based matching methods rely solely on character-level and word-level features and information retrieval techniques to search for target words, which often results in poor timeliness for massive word databases. Therefore, based on the above embodiments, the word recommendation method provided in this invention uses an inverse dictionary model trained according to the following steps:

[0089] For any word in the candidate word list, the first token set of the word is concatenated with the second token set of the definition sample of the word.

[0090] The first token set in the concatenated result is replaced with the mask identifier, and the replacement result is input into the initial model. The initial model encodes the identifier bits of the replacement result and the concatenated result respectively to obtain the first feature corresponding to the replacement result and the second feature corresponding to the identifier bit.

[0091] Based on the first feature, a third matching result between the mask identifier and each word is determined. Based on the third matching result and each word, a first loss function is calculated. Based on the second feature, a fourth matching result between the identifier and the part-of-speech of each word is determined. Based on the fourth matching result and the part-of-speech of each word, a second loss function is calculated.

[0092] Based on the first loss function and the second loss function, a target loss function is calculated, and based on the target loss function, the model parameters of the initial model are iteratively updated to obtain the inverse dictionary model.

[0093] Specifically, in this embodiment of the invention, when performing multi-task learning on the initial model to obtain the inverse dictionary model, the following steps are performed for each word in the candidate word list:

[0094] Taking any word in the candidate word list as an example, for any word, each character in the definition sample of any word also corresponds to a token. The tokens corresponding to all characters in the definition sample of any word can form a second token set when arranged in order.

[0095] Here, the first token set and the second token set can be concatenated. In the concatenated result (i.e., the second concatenation result), the first token set and the second token set are separated by the separator character SEP. Moreover, the starting position of the concatenation result corresponding to any word is indicated by the identifier CLS.

[0096] Subsequently, the first token set in the concatenated result can be replaced based on the mask identifier, and the replaced result (i.e., the second replaced result) is input into the initial model. The initial model encodes the identifier CLS of both the replaced result and the concatenated result to obtain the first feature corresponding to the replaced result and the second feature corresponding to the identifier. Here, the first feature may include the first feature corresponding to each position in the replaced result.

[0097] Subsequently, based on the first feature corresponding to any word, the third matching result between the mask identifier and any word is determined. In this process, the first feature corresponding to each position in the replacement result can be input into the fully connected layer of the initial model to obtain the fifth matching result of the mask identifier output by the fully connected layer and each position. Then, based on the fifth matching result corresponding to each position, the third matching result of the mask identifier and any word can be determined.

[0098] The third match result between the mask identifier and all words in the candidate vocabulary can be represented as: All words in the candidate word list can be represented as wtrue Then the first loss function It can be represented as:

[0099]

[0100] Where cross-entropy is the cross-entropy function.

[0101] Based on the second feature of any word, determine the fourth matching result p between the identifier CLS and any word. w,i Therefore, the fourth matching result of the identifier CLS with the part-of-speech of all words in the candidate word list can be represented as p w The part of speech of all words in the candidate word list can be represented as w. pos Then the second loss function It can be represented as:

[0102]

[0103] Subsequently, combined with the first loss function and the second loss function Calculate the target loss function The target loss function It can be done through the first loss function With the second loss function Adding them together, we get:

[0104]

[0105] Furthermore, based on this objective loss function The model parameters of the initial model are iteratively updated to obtain the inverse dictionary model.

[0106] In this embodiment of the invention, the first loss function is calculated by matching the third matching result with each word in the candidate word list, and the second loss function is calculated by matching the fourth matching result with the part-of-speech of each word in the candidate word list, thereby obtaining the target loss function. This differs from the traditional BERT prediction of the mask position by the word itself, and instead directly establishes the loss calculation on the candidate word list, which can greatly reduce the candidate space for model training and make model learning simpler.

[0107] Based on the above embodiments, the word recommendation method provided in this embodiment of the invention further includes, after concatenating the first token set of any word with the second token set of the definition samples of any word:

[0108] The definition sample of any word is segmented into sentences to obtain several sentences, and the third token set of each sentence is determined.

[0109] Concatenate the first token set of any word with the third token set of each statement.

[0110] Specifically, in this embodiment of the invention, since the definitions of some words are very long, excessively long text information is not conducive to model learning. Furthermore, for the same definition, there are cases where the meanings of the preceding and following descriptions differ significantly. This may be because polysemous information was compiled into the same definition during dictionary compilation, which is detrimental to the model's learning of word meanings. To address these issues and enhance the data, a toolkit can be used to segment the definition samples into sentences, resulting in several statements, and a third token set for each statement can be determined. The third token set for each statement is composed of the tokens of each character in that statement.

[0111] Then, the first token set of any word can be concatenated with the third token set of each sentence obtained by segmenting the definition sample of any word to obtain the third concatenation result. This third concatenation result is then combined with the second concatenation result and used together to train the initial model for multiple tasks.

[0112] Based on the above embodiments, the word recommendation method provided in this embodiment of the invention, wherein selecting the target word corresponding to the query definition from the candidate word list based on the first matching result and the second matching result, specifically includes:

[0113] Based on the part-of-speech index matrix of the candidate word list, the first matching result, and the second matching result, the target word is selected from the candidate word list;

[0114] The part-of-speech index matrix is ​​constructed based on the part-of-speech category of each word, the number of words in the candidate word list, and the number of part-of-speech categories in the candidate word list.

[0115] Specifically, in this embodiment of the invention, a part-of-speech index matrix can be constructed in advance based on the part-of-speech category of each word, the number of words in the candidate word list, and the number of part-of-speech categories in the candidate word list. For the i-th word in the candidate word list, if the part-of-speech of the i-th word is of the j-th category in the part-of-speech library, then let the part-of-speech index I... i,j If it is 1, otherwise I i,j The value is 0. The part-of-speech indexes corresponding to all words in the candidate word list constitute a part-of-speech index matrix. n w n represents the number of words in the candidate vocabulary. p This indicates the number of part-of-speech categories in the candidate word list.

[0116] Furthermore, when selecting target words corresponding to the query definition from the candidate word list, this can be achieved by combining the part-of-speech index matrix, the first matching result, and the second matching result. Specifically, the comprehensive matching index of each word in the candidate word list can be calculated first based on the part-of-speech index matrix, the first matching result, and the second matching result. Then, based on the comprehensive matching index of each word in the candidate word list, the words are ranked, and the words with the highest ranking are selected as target words. When calculating the comprehensive matching index of each word in the candidate word list, the part-of-speech matching index of each word in the candidate word list can be determined first based on the part-of-speech index matrix and the second matching result. Then, the comprehensive matching index of each word is determined by combining the part-of-speech matching index with the first matching result of each word. Afterward, several words can be selected as target words from these words based on their comprehensive matching index.

[0117] In this embodiment of the invention, by constructing a part-of-speech index matrix, the selection of target words can be assisted, making it easier to better combine parts of speech with words.

[0118] Based on the above embodiments, the word recommendation method provided in this embodiment of the invention, wherein selecting the target word from the candidate word list based on the part-of-speech index matrix, the first matching result, and the second matching result, specifically includes:

[0119] Calculate the dot product of the part-of-speech index matrix and the second matching result;

[0120] Based on the dot product result and the first matching result, the comprehensive matching index of each word is calculated, and based on the comprehensive matching index of each word, several words with high comprehensive matching index are selected from the candidate word list as the target words.

[0121] Specifically, in this embodiment of the invention, when selecting target words from the candidate word list using the part-of-speech index matrix, the first matching result, and the second matching result, the dot product of the part-of-speech index matrix and the second matching result can be calculated first. This dot product is the part-of-speech matching index. Subsequently, based on the dot product and the first matching result, the comprehensive matching index of each word in the candidate word list can be calculated. This comprehensive matching index can be expressed by the following formula:

[0122] Res w,i =[p w,i ·I T ]⊙score w,i

[0123] Among them, Res w,i It is the comprehensive matching index of the i-th word in the candidate word list, p w,i The score is the second matching result of the i-th word in the candidate word list.w,i It is the first matching result of the i-th word in the candidate word list.

[0124] Finally, based on the comprehensive matching index Res of each word in the candidate word list w,i Select several words with high comprehensive matching scores from the candidate word list as target words. For example, the words in the candidate word list can be sorted in descending order of their corresponding comprehensive matching scores, and the top 150 words in the sorting results can be selected as target words.

[0125] In this embodiment of the invention, the comprehensive matching index of each word in the candidate word list is calculated by using the part-of-speech index matrix, the first matching result, and the second matching result, and then the target word is selected, which can make the selection result of the target word more accurate.

[0126] Because existing matching methods based on definition and words mostly do not consider factors such as word frequency and part of speech, they may result in the appearance of low-frequency target words. This means the target words obtained may be obscure low-frequency words or words with significant part-of-speech deviations, leading to considerable confusion for users. Therefore, based on the above embodiments, the word recommendation method provided in this invention includes multiple target words, and the step of determining the recommendation result based on the target words specifically includes:

[0127] Determine the target meaning of each target word;

[0128] Calculate the similarity between each target definition and the definition to be queried, and determine a number of designated words with high similarity among each target word;

[0129] The frequency of each specified word in the corpus is determined, and the specified words are sorted based on the word frequency, the first matching result and the second matching result corresponding to each specified word, and the sorted result is used as the recommendation result.

[0130] Specifically, in this embodiment of the invention, when determining the recommendation results, the target definition corresponding to each target word can be determined first. The target definition can be obtained by querying a commonly used dictionary.

[0131] Then, the similarity between each target definition and the query definition is calculated, which can be represented by cosine distance or edit distance. Next, several target words with high similarity can be selected as designated words from the target words. At this point, the target words can be sorted according to their similarity to the query definition from high to low, and the top-ranked target words can be selected as designated words. The number of designated words can be set as needed; for example, if there are 150 target words, the top 100 can be selected as designated words from the ranking results.

[0132] Finally, the word frequency of each specified word in the corpus is determined. The word frequency of each specified word in the corpus is the number of times the specified word appears in the corpus. The corpus contains a large amount of commonly used data, which can be obtained by crawling from the Internet. For example, it can include encyclopedia data or a general corpus. No specific limitation is made here.

[0133] Based on the word frequency of each specified word in the corpus, the first matching result of each specified word, and the second matching result of each specified word, the specified words can be ranked. The resulting ranking is the recommendation result, in which the words are in a specific order. For example, we can first determine the comprehensive matching index for each specified word based on the first and second matching results. Then, we can weight and sum the word frequency and the comprehensive matching index to obtain the ranking score for each specified word. Finally, we can rank the specified words from highest to lowest according to their ranking scores to obtain the recommendation result.

[0134] In this embodiment of the invention, by combining the similarity between each target definition and the definition to be queried, word frequency, and the first and second matching results obtained from the reverse dictionary model, multi-dimensional re-ranking is achieved, which can improve the ranking accuracy of the recommendation results, make the words at the top of the recommendation results more reasonable and common, and thus improve the recommendation effect.

[0135] Based on the above embodiments, the word recommendation method provided in this embodiment of the invention includes multiple target words, and the step of determining the recommendation result based on the target words specifically includes:

[0136] Determine the target meaning of each target word;

[0137] Calculate the similarity between each target definition and the definition to be queried, and determine a number of designated words with high similarity among each target word;

[0138] The frequency of each specified word in the corpus is determined, and the specified words are ranked based on the word frequency and the comprehensive matching index of each specified word. The ranking result is then used as the recommendation result.

[0139] Specifically, in this embodiment of the invention, after determining the word frequency of each specified word in the corpus, the word frequency can first be scaled using a log function to reduce the impact of the word frequency differences of each specified word on the ranking result. Then, the scaling result is weighted and summed with the comprehensive matching index to obtain the ranking score corresponding to each specified word. The ranking score corresponding to the i-th specified word can be calculated using the following formula:

[0140] result i =λ1·Res w,i +λ2log (wordcount i )

[0141] Among them, result i The ranking score is the score corresponding to the i-th specified word, where λ1 and λ2 are constants, and wordcount is the ranking score. i For the frequency of the i-th specified word in the corpus, log(wordcount) i The result is obtained after scaling. In this embodiment of the invention, the values ​​of λ1 and λ2 can be λ1 = 0.8 and λ2 = 0.2, respectively.

[0142] In this embodiment of the invention, by combining the similarity and word frequency of each target definition and the definition to be queried, and by scaling the word frequency, the difficulty of ranking the recommendation results can be reduced, the ranking accuracy of the recommendation results can be further improved, and the recommendation effect can be enhanced.

[0143] Based on the above embodiments, the word recommendation method provided in this embodiment of the invention, wherein determining the recommendation result based on the target word, specifically includes:

[0144] Determine whether the target word exists in the definition to be queried;

[0145] If they exist, the target words that exist in the definition to be queried are removed, and the recommendation result is determined based on the removal result.

[0146] Specifically, in this embodiment of the invention, when determining the recommendation result, that is, after obtaining the target word, it can first be determined whether the target word exists in the definition to be queried. If it exists, it means that the user is already aware of the target word. If the target word is still included in the recommendation result, it will reduce the user experience. Therefore, it is necessary to remove the target word that exists in the definition to be queried and determine the recommendation result based on the result of the removal.

[0147] When determining the recommendation results based on the elimination results, we can first determine the similarity between the target definition and the query definition corresponding to each target word in the elimination results, and determine a number of designated words with high similarity among the target words in the elimination results; combined with the word frequency of each designated word in the corpus, based on the word frequency and the comprehensive matching index of each designated word, we can rank each designated word, and use the ranking result as the recommendation result.

[0148] In this embodiment of the invention, by removing target words that exist in the definition to be queried, it is possible to avoid recommending words that the user already knows to the user, thereby improving the recommendation effect and enhancing the user experience.

[0149] In summary, such as Figure 3 As shown in the figure, this embodiment of the invention provides a word recommendation method based on multi-task learning and multi-dimensional re-ranking, the method comprising:

[0150] The first step is to obtain the definition to be queried;

[0151] The second step involves using a reverse dictionary model to determine the first matching result between each word in the candidate word list and the query definition, as well as the second matching result between the part-of-speech of each word and the query definition. Based on the first and second matching results, a comprehensive matching index for each word is determined. Combining the comprehensive matching index of each word, the target word corresponding to the query definition is selected from the candidate word list. This reverse dictionary model is obtained by multi-task learning of the initial model based on each word in the candidate word list, the part-of-speech of each word, and the definition samples of each word.

[0152] When performing multi-task learning on the initial model, the definition samples of each word can be segmented into sentences to obtain several sentences, and the third token set of each sentence can be determined. The first token set of each word is concatenated with the third token set of each sentence of the definition samples of each word, and the concatenation result of the first token set of each word and the second token set of the definition samples of each word is used as training samples to learn the word prediction task of the initial model.

[0153] The third step is to determine whether the target word exists in the definition to be queried. If it exists, the target word that exists in the definition to be queried is removed, and the fourth step is executed; if it does not exist, the fourth step is executed directly.

[0154] The fourth step is to calculate the similarity between the target definition and the query definition corresponding to the target words processed in the third step, and to determine the word frequency of several specified words with high similarity in the corpus for each target word. Based on the word frequency of each specified word and the comprehensive matching index of each specified word, the specified words are ranked, and the ranking result is used as the recommendation result.

[0155] The word recommendation method based on multi-task learning and multi-dimensional re-ranking provided in this embodiment of the invention first segments the definition samples of each word in the candidate word list into sentences, i.e., data augmentation of the training samples. Then, a multi-task inverse dictionary model is trained for the words themselves and their parts of speech. Finally, multi-dimensional re-ranking is performed by combining word frequency information, word definitions, and the comprehensive matching index of each word obtained from the inverse dictionary model to obtain the final recommendation result. This method alleviates the problems of existing methods in handling polysemous words and large candidate spaces that are difficult to learn through data augmentation and multi-task learning. Multi-dimensional re-ranking is based on the target words output by the inverse dictionary model, combined with word frequency, word definitions, and other information to re-rank them, which greatly alleviates the problem of low-frequency target words and makes the final recommendation result more reasonable and common. In addition, the recommendation result obtained by re-ranking is further calculated based on the target words output by the inverse dictionary model. Unlike existing definition-based matching methods, its candidate space is very small, so it is faster and improves recommendation efficiency.

[0156] like Figure 4 As shown, based on the above embodiments, this embodiment of the invention provides a word recommendation device, including:

[0157] Definition retrieval module 41 is used to retrieve the definition to be queried;

[0158] The word selection module 42 is used to determine the first matching result of each word in the candidate word list with the definition to be queried and the second matching result of the part of speech of each word with the definition to be queried based on the reverse dictionary model, and select the target word corresponding to the definition to be queried from the candidate word list based on the first matching result and the second matching result;

[0159] Recommendation module 43 is used to determine recommendation results based on the target words;

[0160] The reverse dictionary model is obtained through multi-task learning based on each word, the part of speech of each word, and the definition samples of each word.

[0161] Based on the above embodiments, the word recommendation device provided in this embodiment of the invention further includes a training module, used for:

[0162] For any word in the candidate word list, the first token set of the word is concatenated with the second token set of the definition sample of the word.

[0163] The first token set in the concatenated result is replaced with the mask identifier, and the replacement result is input into the initial model. The initial model encodes the identifier bits of the replacement result and the concatenated result respectively to obtain the first feature corresponding to the replacement result and the second feature corresponding to the identifier bit.

[0164] Based on the first feature, a third matching result between the mask identifier and each word is determined. Based on the third matching result and each word, a first loss function is calculated. Based on the second feature, a fourth matching result between the identifier and the part-of-speech of each word is determined. Based on the fourth matching result and the part-of-speech of each word, a second loss function is calculated.

[0165] Based on the first loss function and the second loss function, a target loss function is calculated, and based on the target loss function, the model parameters of the initial model are iteratively updated to obtain the inverse dictionary model.

[0166] Based on the above embodiments, the word recommendation device provided in this embodiment of the invention further includes a training module for:

[0167] The definition sample of any word is segmented into sentences to obtain several sentences, and the third token set of each sentence is determined.

[0168] Concatenate the first token set of any word with the third token set of each statement.

[0169] Based on the above embodiments, the word recommendation device provided in this embodiment of the invention, wherein the word selection module is specifically used for:

[0170] Based on the part-of-speech index matrix of the candidate word list, the first matching result, and the second matching result, the target word is selected from the candidate word list;

[0171] The part-of-speech index matrix is ​​constructed based on the part-of-speech category of each word, the number of words in the candidate word list, and the number of part-of-speech categories in the candidate word list.

[0172] Based on the above embodiments, the word recommendation device provided in this embodiment of the invention, wherein the word selection module is specifically used for:

[0173] Calculate the dot product of the part-of-speech index matrix and the second matching result;

[0174] Based on the dot product result and the first matching result, the comprehensive matching index of each word is calculated, and based on the comprehensive matching index of each word, several words with high comprehensive matching index are selected from the candidate word list as the target words.

[0175] Based on the above embodiments, the word recommendation device provided in this embodiment of the invention includes multiple target words, and the recommendation module is specifically used for:

[0176] Determine the target meaning of each target word;

[0177] Calculate the similarity between each target definition and the definition to be queried, and determine a number of designated words with high similarity among each target word;

[0178] The frequency of each specified word in the corpus is determined, and the specified words are ranked based on the word frequency and the comprehensive matching index of each specified word. The ranking result is then used as the recommendation result.

[0179] Based on the above embodiments, the word recommendation device provided in this embodiment of the invention, wherein the recommendation module is specifically used for:

[0180] Determine whether the target word exists in the definition to be queried;

[0181] If they exist, the target words that exist in the definition to be queried are removed, and the recommendation result is determined based on the removal result.

[0182] Specifically, the functions of each module in the word recommendation device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0183] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute the word recommendation method provided in the above embodiments. The method includes: obtaining the definition to be queried; determining, based on a reverse dictionary model, a first matching result between each word in the candidate word list and the definition to be queried, and a second matching result between the part-of-speech tag of each word and the definition to be queried, and selecting a target word corresponding to the definition to be queried from the candidate word list based on the first matching result and the second matching result; determining a recommendation result based on the target word; wherein the reverse dictionary model is obtained by multi-task learning based on each word, the part-of-speech tag of each word, and the definition samples of each word.

[0184] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0185] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the word recommendation method provided in the above embodiments. The method includes: obtaining a definition to be queried; determining, based on a reverse dictionary model, a first matching result between each word in the candidate word list and the definition to be queried, and a second matching result between the part-of-speech tag of each word and the definition to be queried, and selecting a target word corresponding to the definition to be queried from the candidate word list based on the first matching result and the second matching result; and determining a recommendation result based on the target word; wherein the reverse dictionary model is obtained by multi-task learning based on each word, the part-of-speech tag of each word, and the definition samples of each word.

[0186] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the word recommendation method provided in the above embodiments. The method includes: obtaining a definition to be queried; determining, based on a reverse dictionary model, a first matching result between each word in a candidate word list and the definition to be queried, and a second matching result between the part-of-speech tag of each word and the definition to be queried, and selecting a target word corresponding to the definition to be queried from the candidate word list based on the first matching result and the second matching result; and determining a recommendation result based on the target word; wherein the reverse dictionary model is obtained by multi-task learning based on each word, the part-of-speech tag of each word, and the definition samples of each word.

[0187] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0188] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A word recommendation method, characterized in that, include: Get the definition of the query; Based on the reverse dictionary model, the first matching result of each word in the candidate word list with the definition to be queried and the second matching result of the part of speech of each word with the definition to be queried are determined. Based on the first matching result and the second matching result, the target word corresponding to the definition to be queried is selected from the candidate word list. Based on the target words, the recommendation results are determined; The reverse dictionary model is obtained through multi-task learning based on each word, the part of speech of each word, and the definition samples of each word; The reverse dictionary model is trained based on the following steps: For any word in the candidate word list, the first token set of the word is concatenated with the second token set of the definition sample of the word. The first token set in the concatenated result is replaced with the mask identifier, and the replacement result is input into the initial model. The initial model encodes the identifier bits of the replacement result and the concatenated result respectively to obtain the first feature corresponding to the replacement result and the second feature corresponding to the identifier bit. Based on the first feature, a third matching result between the mask identifier and each word is determined. Based on the third matching result and each word, a first loss function is calculated. Based on the second feature, a fourth matching result between the identifier and the part-of-speech of each word is determined. Based on the fourth matching result and the part-of-speech of each word, a second loss function is calculated. Based on the first loss function and the second loss function, a target loss function is calculated, and based on the target loss function, the model parameters of the initial model are iteratively updated to obtain the inverse dictionary model.

2. The word recommendation method according to claim 1, characterized in that, The step of concatenating the first token set of any word with the second token set of the definition samples of any word further includes: The definition sample of any word is segmented into sentences to obtain several sentences, and the third token set of each sentence is determined. Concatenate the first token set of any word with the third token set of each statement.

3. The word recommendation method according to any one of claims 1-2, characterized in that, The step of selecting the target word corresponding to the query definition from the candidate word list based on the first matching result and the second matching result specifically includes: Based on the part-of-speech index matrix of the candidate word list, the first matching result, and the second matching result, the target word is selected from the candidate word list; The part-of-speech index matrix is ​​constructed based on the part-of-speech category of each word, the number of words in the candidate word list, and the number of part-of-speech categories in the candidate word list.

4. The word recommendation method according to claim 3, characterized in that, The step of selecting the target word from the candidate word list based on the part-of-speech index matrix, the first matching result, and the second matching result specifically includes: Calculate the dot product of the part-of-speech index matrix and the second matching result; Based on the dot product result and the first matching result, the comprehensive matching index of each word is calculated, and based on the comprehensive matching index of each word, several words with high comprehensive matching index are selected from the candidate word list as the target words.

5. The word recommendation method according to claim 4, characterized in that, The target words include multiple terms, and the step of determining the recommendation results based on the target words specifically includes: Determine the target meaning of each target word; Calculate the similarity between each target definition and the definition to be queried, and determine a number of designated words with high similarity among each target word; The frequency of each specified word in the corpus is determined, and the specified words are ranked based on the word frequency and the comprehensive matching index of each specified word. The ranking result is then used as the recommendation result.

6. The word recommendation method according to any one of claims 1-2, characterized in that, The process of determining the recommendation results based on the target words specifically includes: Determine whether the target word exists in the definition to be queried; If they exist, the target words that exist in the definition to be queried are removed, and the recommendation result is determined based on the removal result.

7. A word recommendation device, characterized in that, include: The definition retrieval module is used to retrieve the definition to be queried; The word selection module is used to determine the first matching result of each word in the candidate word list with the definition to be queried and the second matching result of the part of speech of each word with the definition to be queried based on the reverse dictionary model, and select the target word corresponding to the definition to be queried from the candidate word list based on the first matching result and the second matching result; The recommendation module is used to determine recommendation results based on the target words; The reverse dictionary model is obtained through multi-task learning based on each word, the part of speech of each word, and the definition samples of each word; It also includes a training module for: For any word in the candidate word list, the first token set of the word is concatenated with the second token set of the definition sample of the word. The first token set in the concatenated result is replaced with the mask identifier, and the replacement result is input into the initial model. The initial model encodes the identifier bits of the replacement result and the concatenated result respectively to obtain the first feature corresponding to the replacement result and the second feature corresponding to the identifier bit. Based on the first feature, a third matching result between the mask identifier and each word is determined. Based on the third matching result and each word, a first loss function is calculated. Based on the second feature, a fourth matching result between the identifier and the part-of-speech of each word is determined. Based on the fourth matching result and the part-of-speech of each word, a second loss function is calculated. Based on the first loss function and the second loss function, a target loss function is calculated, and based on the target loss function, the model parameters of the initial model are iteratively updated to obtain the inverse dictionary model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the word recommendation method as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the word recommendation method as described in any one of claims 1-6.

Images