A similarity discrimination method and system for new word sense origin recommendation

Abstract

The application discloses a similarity discrimination method and system for new word sense primitive recommendation, and comprises the following steps: in the HowNet word set, similar words to a new word are selected by a similarity discrimination model to form a similar word set; a local "word-semantic item-semantic primitive" relationship network is constructed according to all the words in the similar word set, the corresponding concept semantic items and semantic primitives of the words; semantic primitive nodes are selected based on a network node importance sorting method, a recommendation index of the semantic primitive nodes is generated according to the standardization degree centrality and the intermediate centrality of the semantic primitive nodes, the importance of the semantic primitive nodes is evaluated, the correlation between an unregistered word and a semantic primitive is established by taking the similar word set as a bridge, the sorting and selection of the candidate semantic primitives of the unregistered word are completed through the recommendation index, and the HowNet is expanded through the new word; the application effectively solves the similarity discrimination problem between the unregistered word and the word table word, and effectively solves the selection problem of the candidate semantic primitives.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of new word sense primitive recommendation, and in particular to a similarity discrimination method and system for new word sense primitive recommendation. BACKGROUND

[0002] Since the appearance of HowNet, it has attracted extensive attention in the field of natural language processing. Domestic and foreign scholars have explored the important application value of HowNet in vocabulary semantic disambiguation, similarity calculation, text classification and other aspects. In recent years, relevant researches show that HowNet directly depicts semantic information through a unified sense primitive annotation system, and each sense primitive meaning is clear and fixed, which can be directly used as a semantic label to be integrated into a machine learning model, so that the natural language processing deep learning model has better robustness and explainability. However, like other knowledge base systems constructed by artificial means, HowNet has the problems of limited scale and high cost of updating and expansion. Personnel without HowNet background knowledge and without special training are difficult to complete the sense primitive prediction task well. This leads to the contradiction between the potential great application value of HowNet and its own limited scale, sparse semantic resources and difficulty in expansion. A feasible way to solve this contradiction is to carry out research on the automatic or semi-automatic construction technology of HowNet, and one of the core problems is to select appropriate sense primitives for the description of new concepts.

[0003] With the development of the times, new words and new concepts of old words emerge in an endless stream. The new word sense primitive recommendation (or prediction) task is to select a suitable set of sense primitives from the existing set of sense primitives in HowNet to construct the Def description of the concept. For example, in the existing HowNet knowledge base, the set of sense primitives contained in the Def description of the concept of "small rice" is {material | material, edible | food, crop | crop}. However, in the current actual language environment, "small rice" can not only have the concept of a crop, but also can be described as a company name or a brand of electronic products. Therefore, its set of sense primitives should also contain sense primitives such as {InstitutePlace | place} or {SpeBrand | specific brand}.

[0004] Generally, words or concepts with similar semantics will share the same sense primitive, so to solve the problem of new word concept sense primitive recommendation, the main idea of collaborative filtering (CF) can be used: that is, the set of sense primitives most likely to be used in the Def description of the current new word concept is predicted by using the set of sense primitives in the Def description of the existing concept in HowNet, and the key lies in measuring the semantic similarity between new and old words or concepts.

[0005] With the popularity of Internet applications, a large number of new words constantly appear in text big data, and the meanings of existing words are extended and expanded, so it is necessary to continuously modify and expand the semantic knowledge base based on semantic primitives.

[0006] In the aspect of automatic semantic primitive annotation of vocabulary, XIE et al. proposed the task of semantic primitive prediction, and based on word embeddings and semantic primitive embeddings, they proposed a variety of semantic primitive prediction models. With the help of collaborative filtering and matrix decomposition methods, the general relationship between vocabulary and semantic primitives is learned from existing manually annotated data sets, so as to automatically construct the semantic primitives of new words. The semantic representation method relying on the external context information of words cannot effectively process low-frequency words or out-of-vocabulary words. JIN et al. proposed a semantic primitive prediction framework based on internal word information and external context information, and by fusing internal and external models, the effect of low-frequency word semantic primitive prediction was improved. In order to overcome the problem of poor quality of low-frequency word embedding and low accuracy of semantic primitive prediction, LI et al. proposed a label distributed seq2seq model based on word and multi-label distribution sequence, which uses the definition and description information of words to predict semantic primitives. ZHANG Lei et al. based on the multi-label classification model architecture, by taking the words in the sentence as the input of the model, reduced the ambiguity of taking the word as the minimum unit. DU Jiaju et al. proposed a semantic primitive related pooling model, which uses local semantic correlation to predict semantic primitives, and this method depends on the quality of the definition text. In the case of low resources, the key to solving the problem of semantic primitive recommendation lies in calculating the semantic similarity between new words and HowNet word list words.

[0007] Word similarity calculation methods can be divided into corpus-based methods and knowledge base-based methods. At present, a large number of mainstream word semantic similarity calculation methods can be classified as corpus-based methods. These methods are based on distribution assumption, and learn the association between individual words from a large text collection. If the context environment of two words is more similar, or they appear together more frequently, it is assumed that their semantic similarity is higher. Generally, the distributed representation of words is obtained through the co-occurrence information of words in the corpus, and one of the most famous expression methods based on word adjacency relationship is word2vec, each word is mapped to a unique dense vector. Compared with the word embedding method represented by word2vec, BERT uses the encoder in the Transformer as a feature extractor. This method has good utilization of context and can dynamically model the polysemy phenomenon. The above methods are all based on the distributed representation of words or characters, ignoring the inconsistency between the word vector-based and HowNet-based semantic similarity relationship, and are not suitable for the semantic primitive recommendation task based on the collaborative framework. Therefore, there is an urgent need for a similarity discrimination method and system for new word semantic primitive recommendation, which can make the calculation result approach the similarity calculation result based on the HowNet semantic primitive, that is, achieve "similarity isomorphism". SUMMARY

[0008] In order to solve the problems in the prior art, the purpose of the present application is to provide a similarity discrimination method and system for new word semantic primitive recommendation. In the process of discriminating the semantic similarity between new words and HowNet word table words, the attention distribution of the known concept description semantic primitive sequence of the word table word is used to optimize the similarity discrimination model based on BERT+CNN, so as to provide a similar concept set for the semantic primitive recommendation task.

[0009] In order to achieve the above technical purpose, the present application provides a similarity discrimination method for new word semantic primitive recommendation, characterized in that it comprises the following steps:

[0010] In the word table word set of HowNet, the similar words to the new word are selected by the similarity discrimination model to form a similar word set, wherein the new word represents an out-of-vocabulary word in HowNet;

[0011] According to all word phrases in the similar word set, the concept semantic items corresponding to the word phrases and the semantic primitives, a local "word phrase-semantic item-semantic primitive" relationship network is constructed;

[0012] The selection of the semantic primitive node is based on the network node importance ranking method, the recommendation index of the semantic primitive node is generated according to the standardized degree centrality and the intermediate centrality of the semantic primitive node, the importance of the semantic primitive node is evaluated, the association between the unknown word and the semantic primitive is established by taking the similar word set as a bridge, and the sorting and selection of the candidate semantic primitive of the unknown word are completed through the recommendation index, so as to expand the HowNet through the new word.

[0013] Preferably, in the process of acquiring the HowNet, the HowNet is used to represent an unlimited concept set by using limited semantic primitives, wherein the semantic primitives are divided into Entity|entity, Event|event, Attribute|attribute, Value|value and SecondaryFeature|secondary feature.

[0014] Preferably, in the process of using the similarity discrimination model, the similarity discrimination model is a BERT-based similarity discrimination model, the attention distribution of the semantic primitive sequence is described by using the word list word known concept, the BERT-based similarity discrimination model is optimized by taking the similarity isomorphism as the target, and the similar word set is selected by using the optimized similarity discrimination model.

[0015] Preferably, in the process of constructing the similar word set, the word sequence corresponding to the new word and the word list word is respectively input into two SaBERT models to obtain semantic representation vectors fused with semantic primitive sequence information; the similarity discrimination between the unknown word and the word list word is completed by using the CNN classification model, and the similar word set is constructed.

[0016] Preferably, in the process of acquiring the semantic representation vector, the semantic primitive tree corresponding to the concept description item Def of the word list word is pre-ordered by using the SaBERT model, and the semantic primitive Chinese word sequence SL is obtained;

[0017] The semantic primitive sequence SL is obtained by using S-BERT to obtain its vector representation ESL;

[0018] ESL and the hidden layer vector hci of each character in the word after BERT are spliced, and then a full connection layer with an activation function tanh is added, wherein the parameter matrix w1 is R 2d×n , and the weight vector a of each character vector is obtained by using softmax.

[0019] The final vector representation e of the word is obtained by vector weighting.

[0020] Preferably, in the process of optimizing the BERT-based similarity discrimination model, the similarity discrimination problem is regarded as a binary classification problem with a classification threshold of 0.9, the continuous similarity value is converted into a similarity degree discrimination mark by using the discretization processing, and the model training is completed by using the cross-entropy loss function.

[0021] Preferably, in the process of evaluating the importance of the sense node, the measurement formula of the standardization degree centrality is:

[0022] C d (v i )=Σ j x ij / max(C d (v j )),i≠j∈N

[0023] Wherein, x ij =1 indicates that there is a direct contact between node i and node j, otherwise, x ij =0; N is the set of all nodes in the network.

[0024] Preferably, in the process of evaluating the importance of the sense node, the measurement formula of the intermediate centrality is:

[0025]

[0026] Wherein, σ st represents the number of shortest paths from node s to t, σ st (vi) represents the number of shortest paths from node s to t and passing through v i .

[0027] Preferably, in the process of obtaining the recommendation index, the recommendation index is represented as:

[0028] R(v i )=Cd ( v i )×log(C b (v i )+1)。

[0029] The application discloses a similarity discrimination system for new word sense origin recommendation, and the similarity discrimination system is used for expanding HowNet by obtaining a new word, and comprises the following steps:

[0030] A data retrieval module is used for selecting similar words to the new word from the word set of HowNet by a similarity discrimination model to form a similar word set, wherein the new word represents an out-of-vocabulary word in HowNet.

[0031] A data processing module is used for constructing a local "word-concept-sense origin" relationship network according to all words in the similar word set, the corresponding concept sense of the words and the sense origin.

[0032] The extended module is used to select semantic primitive nodes based on the network node importance ranking method. It generates a recommendation index for semantic primitive nodes based on the normalization centrality and mesocentric centrality of the semantic primitive nodes, evaluates the importance of the semantic primitive nodes, establishes the association between out-of-vocabulary words and semantic primitives by using a set of similar words as a bridge, and completes the ranking and selection of candidate semantic primitives for out-of-vocabulary words through the recommendation index.

[0033] The present invention discloses the following technical effects:

[0034] This invention effectively solves the problem of similarity determination between out-of-vocabulary words and words in the vocabulary list;

[0035] Experiments conducted on a semantic primitive recommendation task based on a collaborative filtering framework show that the degree of similarity isomorphism is positively correlated with the recommendation performance of semantic primitives, meaning that the present invention can effectively solve the problem of candidate semantic primitive selection. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is an example of the relationship between words, meanings, and semantic origins described in this invention;

[0038] Figure 2 This is the semantic primitive collaborative recommendation framework described in this invention;

[0039] Figure 3 The Sabert described in this invention 2 +CNN model;

[0040] Figure 4 This refers to the SaBERT model of semantic primitive attention described in this invention. Detailed Implementation

[0041] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0042] As shown in Figures 1-4 The present application provides a similarity discrimination method for new word sense primitive recommendation, characterized by comprising the following steps:

[0043] In the HowNet word set, similar words to the new word are selected by a similarity discrimination model to form a similar word set, wherein the new word represents an out-of-vocabulary word in the HowNet;

[0044] According to all the words in the similar word set, the concept meaning items corresponding to the words and the primitives, a local "word-meaning primitive" relationship network is constructed;

[0045] Based on a network node importance ranking method, a primitive node is selected, a recommendation index of the primitive node is generated according to the standardized degree centrality and the intermediate centrality of the primitive node, the importance of the primitive node is evaluated, the association between the out-of-vocabulary word and the primitive is established through the similar word set as a bridge, the sorting and selection of the candidate primitives of the out-of-vocabulary word are completed through the recommendation index, and the HowNet is expanded through the new word.

[0046] Further preferably, in the process of obtaining the HowNet, the HowNet mentioned in the present application is used to represent the description of an infinite concept set by limited primitives, wherein the primitives are divided into Entity|entity, Event|event, Attribute|attribute, Value|value and SecondaryFeature|secondary feature.

[0047] Further preferably, in the process of using the similarity discrimination model, the similarity discrimination model mentioned in the present application is a BERT-based similarity discrimination model, the attention distribution of the known concept description primitive sequence of the word table word is obtained, the BERT-based similarity discrimination model is optimized by taking similarity isomorphism as the target, and the similar word set is selected by the optimized similarity discrimination model.

[0048] Further preferably, in the process of constructing the similar word set, the sequence of characters corresponding to the new word and the word table word is respectively input into two SaBERT models to obtain semantic representation vectors integrating the sequence information of the semantic primitives; the similarity discrimination between the new word and the word table word is completed through the CNN classification model, and the similar word set is constructed.

[0049] Further preferably, in the process of obtaining the semantic representation vector, the present application performs pre-order traversal on the semantic primitive tree corresponding to the concept description semantic item Def of the word table word through the SaBERT model to obtain a sequence of semantic primitive Chinese words SL;

[0050] The sequence of semantic primitives SL is subjected to S-BERT to obtain its vector representation ESL;

[0051] ESL and the hidden layer vector hci obtained by BERT after each character in the word are spliced, and a fully connected layer with an activation function tanh is added, wherein the parameter matrix w1 is R 2d×n , and the weight vector a of each character vector is obtained through softmax.

[0052] The final vector representation e of the word is obtained through vector weighting and summing.

[0053] Further preferably, in the process of optimizing the similarity discrimination model based on BERT, the present application regards the similarity discrimination problem as a binary classification problem with a classification threshold of 0.9, converts the continuous similarity value into a similarity degree discrimination label through discretization processing, and completes model training by using a cross-entropy loss function.

[0054] Further preferably, in the process of evaluating the importance of the semantic primitive node, the measurement formula of the standardization degree centrality mentioned by the present application is:

[0055] C d (v i )=Σ j x ij / max(C d (v j )), i≠j∈N

[0056] Wherein, x ij =1 indicates that there is a direct connection between node i and node j, otherwise, x ij =0; N is the set of all nodes in the network.

[0057] Further preferably, in the process of evaluating the importance of the semantic primitive node, the measurement formula of the intermediate centrality is:

[0058]

[0059] Wherein, σ stLet sigma (s, t) denote the number of shortest paths from node s to t st (v i ) denotes the number of shortest paths from node s to t and passing through v i .

[0060] Preferably, in the process of obtaining the recommendation index, the recommendation index mentioned in the application is expressed as:

[0061] R(v i )=C d (v i )×log(C b (v i )+1)。

[0062] The application discloses a similarity discrimination system for new word sense origin recommendation, and the similarity discrimination system is used for expanding HowNet by obtaining a new word, and comprises the following steps:

[0063] A data retrieval module is used for selecting similar words to the new word from a word table word set of HowNet by a similarity discrimination model to form a similar word set, wherein the new word represents an out-of-vocabulary word in HowNet.

[0064] A data processing module is used for constructing a local "word-concept-semantic origin" relationship network according to all words of the similar word set, concept semantic items corresponding to the words and semantic origins of the words.

[0065] An expansion module is used for selecting semantic origin nodes based on a network node importance sorting method, generating a recommendation index of the semantic origin nodes according to a standardization degree centrality and an intermediate centrality of the semantic origin nodes, evaluating the importance of the semantic origin nodes, establishing an association relationship between the out-of-vocabulary word and the semantic origin by taking the similar word set as a bridge, and completing sorting and selection of candidate semantic origins of the out-of-vocabulary word through the recommendation index.

[0066] HowNet mentioned in the application is used for describing an infinite concept set by using limited semantic origins.

[0067] With the development of deep learning, neural network models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), graph neural networks (GNNs) and attention mechanisms are widely used to solve natural language processing (NLP) tasks. Compared with non-neural network model methods that rely heavily on discrete hand-crafted features, neural network methods usually use low-dimensional dense vectors to implicitly represent the syntactic or semantic features of language. In recent years, a large number of studies have shown that pre-training models (PTMs) trained using large-scale text corpora can learn approximate general language representations, and after fine-tuning on small data sets for specific tasks, they can significantly reduce the difficulty of individual natural language processing tasks while improving system performance.

[0068] In terms of word similarity calculation, distributed representation methods can capture the association between words by representing words as low-dimensional dense real number vectors. Therefore, this method can efficiently calculate the semantic association between words in a low-dimensional space and effectively solve the data sparsity problem. Although word2vec, as a representative of word representation learning models, has achieved good results in word similarity calculation tasks. However, each word in word2vec is mapped to a unique dense vector, which cannot handle the polysemy problem. In addition, existing representation learning models that learn word representation vectors based on the context distribution of words cannot effectively perform representation learning for words with low occurrence frequency or out-of-vocabulary (OOV), which is a major obstacle to the role of word distributed representation methods in similarity calculation tasks.

[0069] Compared with the distributed representation method represented by word2vec, one of the prominent advantages of BERT is the dynamic nature of word representation, which can model polysemy. In the task of recommending the original meaning of new word concepts, the need for distributed representation of out-of-vocabulary (OOV) words is widespread. In order to alleviate the impact of OOV, a common method is to expand the size of the dictionary to improve the coverage of words in the model training process, but this method cannot fundamentally solve the OOV problem.

[0070] With a word-based pre-trained language model, for example, the vector of the current word is obtained by using the hidden layer vector of the topmost word of the BERT encoder. However, the effect of the method of simply using the output of [CLS] as the word vector is not ideal. In addition, in HowNet, the key to measuring the similarity of concepts is to calculate the similarity between the sequences of primitives corresponding to the concepts. In terms of sequence semantic similarity (such as sentence similarity) calculation, the sentence embedding method Sentence-Transformers (S-BERT) using the twin BERT network structure has shown good performance. This method inputs a pair of sentences into two BERT models that share parameters, and obtains the sentence vector representation of each sentence by averaging and pooling all word vectors of the BERT output sentence. Therefore, the present application attempts to use the attention distribution of the known concept description primitives of the word table to optimize the BERT-based similarity discrimination model with the goal of similarity isomorphism.

[0071] In HowNet, concepts are constructed from primitives, and words are defined from concepts. The relationships of word, semantic item, and primitive are as shown in Figure 1 Here, taking the word "small rice" as an example, the description of the concept semantic item composed of primitives such as "material | material", "edible | food", and "crop | crops" and their relationships is: DEF = {material | material: MaterialOf = {edible | food}, material = {crop | crops} (senseID: 177381), and the word "small rice (millet)" is defined by this concept semantic item.

[0072] As described above, in HowNet, each word corresponds to the definition (Def) of one or more concept semantic items, and each Def contains one or more primitives. The primitives in the description of any concept are connected by embodying the mutual relationship between the concept and the concept and the attribute and the attribute. Usually, a primitive may exist in the definition of multiple concept semantic items. When modeling the semantic relationship of multiple words, a "word-semantic item-primitive" relationship network is formed.

[0073] The present application expects to use the structural characteristics of the "word-semantic item-primitive" relationship network and the collaborative filtering mechanism to discover new concepts of words and recommend related primitives.

[0074] Collaborative filtering mechanism is one of the most important techniques used in recommender systems. The basic principle is that if two users have similar purchase behavior, they will be interested in the same kind of goods, and the current user will be interested in the goods that the similar user likes. In the aspect of semantic similarity measurement, it can be considered that the concepts with similar semantics should have similar sets of semantic primitives according to the description of the concept in HowNet. Therefore, in the task of semantic primitive recommendation, it is a feasible way to use the collaborative filtering mechanism. The basic principle is to recommend the semantic primitives that have not been marked but are likely to express some aspect of semantic information in the actual language environment according to the semantic primitives contained in the similar concept.

[0075] The set of semantic primitives to be recommended is obtained by using the similar words corresponding to the concept. As shown in Figure 2 , first, in the HowNet word set, the similar words to the new word (unknown word) are selected by the similarity discrimination model to form a similar word set; second, based on the above all words, the concept semantic primitives corresponding to the words and the semantic primitives, a local "word-semantic primitive-semantic primitive" relationship network is constructed; finally, the semantic primitive node selection is based on the network node importance ranking method.

[0076] Here, two centrality measurement methods are used to evaluate the importance of semantic primitive nodes, and the importance is used as the basis for the selection and ranking of semantic primitives. Degree centrality is the most direct measurement index of node centrality in network analysis. The greater the node degree of a node, the higher the degree centrality of the node, and the more important the node in the network. The standardization degree centrality measurement formula is:

[0077] C d (v i )=Σ j x ij / max(C d (v j )),i≠j∈N

[0078] Where x ij =1 indicates that there is a direct connection between node i and node j, otherwise x ij= 0; N is the set of all nodes in the network. Since the calculation of the degree of the node does not consider the importance of the adjacent nodes in the graph, it cannot well reflect the sharing characteristics of the primitives between the words. Therefore, it is necessary to measure the centrality of the node from the dimension of the path. Here, the centrality measurement method based on betweenness centrality is introduced. All shortest paths between any two nodes in the network are calculated. If many of these shortest paths pass through a certain node, it is considered that the betweenness centrality of the node is high. The betweenness centrality measurement formula is:

[0079]

[0080] where σ st represents the number of shortest paths from node s to t, σ st (vi) represents the number of shortest paths from node s to t, and passing through v i The betweenness centrality of a node is an index for describing the importance of a node in terms of the number of shortest paths passing through the node. A primitive node with a high betweenness centrality indicates that it is more likely to be shared by similar words.

[0081] The recommendation index of the primitive node is calculated by combining the standardized degree centrality and the betweenness centrality of the primitive node:

[0082] R(v i )=C d (v i )×log(C b (v i )+1)

[0083] In the framework of collaborative recommendation, the association between the unknown word and the primitive is established based on the similar word set. The ranking and selection of the candidate primitives of the unknown word are completed based on the above-mentioned recommendation index of the primitive node.

[0084] The goal of the new word concept primitive recommendation task is to recommend suitable primitives for the unknown concepts in HowNet. Therefore, a similarity calculation model between the unknown concept and the known concept is needed. Since the concept is expressed in the form of words, the above problem can be converted into a similarity calculation problem between the unknown word and the word in the word list.

[0085] Unlike general word similarity calculation, on the one hand, based on the similarity isomorphism assumption, the similarity discrimination is based on the word similarity calculation results based on HowNet; on the other hand, the words participating in the similarity discrimination are composed of the unknown word (OOV) and the HowNet word list word (IV). For IV, the set of primitives constituting the concept description Def is known and can be used to improve the convergence of the model.

[0086] The overall framework of the similarity discrimination model is as shown in Figure 3 The right part. The model inputs the input new word and the word list word corresponding to the character sequence into two SaBERT models respectively, obtains the semantic representation vector integrating the semantic primitive sequence information, and then completes the similarity discrimination of the out-of-vocabulary word and the word list word through the CNN classification model.

[0087] The structure of the semantic primitive attention model SaBERT is as shown in Figure 4 The SaBERT first performs pre-order traversal on the semantic primitive tree corresponding to the concept description Def of the word list word, obtains the semantic primitive Chinese word sequence S L ; then the semantic primitive sequence S L is input into S-BERT to obtain its vector representation E SL ; the E SL is spliced with the hidden layer vector h ci obtained by BERT for each character in the word, and then a fully connected layer with an activation function tanh is connected, wherein the parameter matrix w1∈R 2d×n , and the weight vector α of each character vector is obtained by softmax; finally, the final vector representation e of the word is obtained by vector weighting and summing.

[0088] a=softmax(tanh(w1[E SL ;h ci ]))

[0089]

[0090] In the process of model training, in order to make the prediction result of the model approach the similarity calculation result based on the HowNet semantic primitive, that is, to achieve the purpose of "similarity isomorphism", this paper uses the classical HowNet semantic similarity calculation model as the standard model of similarity discrimination. The classical HowNet semantic similarity calculation result is a continuous real number in [0, 1], and the analysis of the calculation result shows that the proportion of the result value of the similarity calculation between similar words in the intervals [0.9, 1.0], [0.8, 0.9), [0.7, 0.8) and [0.0, 0.7) is about 6:2:1:1. Since the purpose of this paper is to select similar words for the construction of the candidate semantic primitive set, in order to improve the quality of similar word selection, this invention regards the similarity discrimination problem as a two-classification problem with 0.9 as the classification threshold, converts the continuous similarity value into a similarity degree discrimination label through discretization processing, and adopts the cross-entropy loss function to complete the model training.

[0091] The application proposes a heterogeneous problem of semantic similarity relationship, assumes that a word similarity discrimination model based on semantic similarity measurement of HowNet based on semantic primitives can more effectively recommend semantic primitives for new words. Based on the assumption, similarity isomorphism is taken as a target, and a pre-training language model is optimized by fusing semantic primitive attention. The application can effectively solve the similarity discrimination problem of out-of-vocabulary words and words in a word table. Experiments in a semantic primitive recommendation task based on a collaborative filtering framework show that the degree of similarity isomorphism is positively correlated with the semantic primitive recommendation effect, indicating the rationality of taking similarity isomorphism as the optimization target of the model.

[0092] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be in the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) having computer-usable program code embodied therein.

[0093] While the preferred embodiments of the application have been described, additional alternatives, modifications, and variations can become apparent to those skilled in the art once given the benefit of the foregoing description. Accordingly, the attached claims are intended to embrace all such alternatives, modifications, and variations that fall within the scope of the application.

Claims

1. A similarity determination method for new word sense origin recommendation, characterized by, The method comprises the following steps: In the vocabulary word set of HowNet, similar words to the new word are selected by a similarity discrimination model to form a similar word set, wherein the new word represents an out-of-vocabulary word in HowNet; A local "word-semantic item-semantic primitive" relationship network is constructed according to all word items in the similar word set, word item corresponding concepts and semantic primitives thereof; Based on a network node importance ranking method, a semantic primitive node is selected, a recommendation index of the semantic primitive node is generated according to the standardization degree centrality and the intermediate centrality of the semantic primitive node, the importance of the semantic primitive node is evaluated, and an association between the out-of-vocabulary word and the semantic primitive is established through the similar word set as a bridge, and the recommendation index is used to rank and select candidate semantic primitives of the out-of-vocabulary word, which is used to expand HowNet through the new word; In the process of using the similarity discrimination model, the similarity discrimination model is a BERT-based similarity discrimination model, the attention distribution of the concept description semantic primitive sequence of the vocabulary word is obtained, the BERT-based similarity discrimination model is optimized by taking similarity isomorphism as the target, and the similar word set is formed by selecting similar words to the new word through the optimized similarity discrimination model; In the process of forming the similar word set, the word sequence corresponding to the new word and the vocabulary word is respectively input into two SaBERT models to obtain semantic representation vectors fused with semantic primitive sequence information; the similarity between the out-of-vocabulary word and the vocabulary word is discriminated through a CNN classification model, and the similar word set is constructed; In the process of obtaining the semantic representation vector, the SaBERT model is used to perform a pre-order traversal on the semantic primitive tree corresponding to the concept description semantic item Def of the vocabulary word to obtain a semantic primitive Chinese word sequence SL; The semantic primitive sequence SL is input into the S-BERT to obtain a vector representation ESL thereof; The ESL and the hidden layer vector hci obtained by inputting each character in the word into the BERT are spliced, and then a fully connected layer with an activation function tanh is added, wherein, Parameter matrix w1∈R 2d×n and get the weight vector a of each word vector through softmax. The final vector representation e of the word is obtained through vector weighting.

2. The similarity discrimination method for new word semantic primitive recommendation according to claim 1, wherein: In the process of obtaining HowNet, HowNet is used to represent an infinite concept set described by limited semantic primitives, wherein the semantic primitives are divided into Entity|entity, Event|event, Attribute|attribute, Value|value and SecondaryFeature|second feature.

3. The similarity discrimination method for new word semantic primitive recommendation according to claim 2, wherein: In the process of optimizing the BERT-based similarity discrimination model, the similarity discrimination problem is regarded as a two-classification problem with a classification threshold of 0.9, a continuous similarity value is converted into a similarity degree discrimination label through discretization processing, and a cross-entropy loss function is used to complete model training.

4. The similarity discrimination method for new word semantic primitive recommendation according to claim 3, wherein: In the process of evaluating the importance of the semantic primitive node, the measurement formula of the standardization degree centrality is: ； where x ij = 1 indicates that there is a direct connection between node i and node j, otherwise, x ij = 0; N is the set of all nodes in the network.

5. The similarity discrimination method for new word sense primitive recommendation according to claim 4, characterized in that: In the process of evaluating the importance of the sense primitive node, the formula for measuring the intermediate centrality is: ； where σ st denotes the number of shortest paths from node s to t, σ st (v i ) denotes the number of shortest paths from node s to t that pass through v i .

6. The similarity discrimination method for new word sense primitive recommendation according to claim 5, characterized in that: In the process of obtaining the recommendation index, the recommendation index is expressed as: R(v i ) = C d (v i ) x log(C b (v i ) + 1).

7. A similarity discrimination system for new word sense origin recommendation, characterized by, The similarity discrimination system is used to expand HowNet by obtaining new words, comprising: A data retrieval module is configured to select similar words to the new word from the word set of HowNet by using the similarity discrimination model to form a similar word set, wherein the new word represents an out-of-vocabulary word in HowNet; A data processing module is configured to construct a local "word-concept-sense primitive" relationship network according to all words in the similar word set, the concept sense items corresponding to the words, and the sense primitives; An expansion module is configured to select sense primitive nodes based on a network node importance ranking method, generate a recommendation index of the sense primitive nodes according to the standardized degree centrality and the intermediate centrality of the sense primitive nodes, and evaluate the importance of the sense primitive nodes, so as to establish an association between the out-of-vocabulary word and the sense primitive through the similar word set, and complete the ranking and selection of the candidate sense primitive of the out-of-vocabulary word through the recommendation index; In the process of using the similarity discrimination model, the similarity discrimination model is a BERT-based similarity discrimination model, which uses the attention distribution of the concept description sense primitive sequence of the word set word as the target, optimizes the BERT-based similarity discrimination model, and selects similar words to the new word to form a similar word set through the optimized similarity discrimination model; In the process of forming the similar word set, the word sequence corresponding to the new word and the word set word is respectively input into two SaBERT models to obtain semantic representation vectors fused with the sense primitive sequence information; the similarity between the out-of-vocabulary word and the word set word is discriminated through a CNN classification model, and the similar word set is constructed; In the process of obtaining the semantic representation vector, the SaBERT model is used to perform a pre-order traversal on the sense primitive tree corresponding to the concept description sense item Def of the word set word to obtain a sense primitive Chinese word sequence SL; The sense primitive sequence SL is input into S-BERT to obtain its vector representation ESL; ESL and the hidden layer vector hci of each character in the word after BERT are spliced, and then a fully connected layer with an activation function tanh is added, wherein, Parameter matrix w1∈R 2d×n and get the weight vector a of each word vector by softmax. The final vector representation e of the word is obtained by vector weighting.

Images