A method and computer device for predicting word semantics based on semantic origin

Abstract

The present application relates to natural language processing technology in the field of artificial intelligence, and discloses a prediction method and computer device for word semantics based on semantics. The method, based on HowNet, constructs a semantic space of words based on semantics, and represents known words in HowNet in the semantic space. The vector representation of words in the semantic space consists of two parts, one part of which is determined by the category semantics, and the other part of which is determined by the extended description and the relationship between the extended description and the category semantics. Since the meaning of the semantics is clear, the vector used in this application to represent words is interpretable; in the face of newly added words, the correlation between the newly added words and the known words can be calculated, and the vector representation of the newly added words can be derived from the vector representation of the known words in the semantic space. The newly added words predicted by this method have strong representation capabilities. The method of this application can be combined with a recommendation system to solve the problem of understanding and representing new words in the recommendation system and improve the quality of recommendations.

Description

Technical Field

[0001] The present application relates to natural language processing technology in the field of artificial intelligence, and in particular to a method and computer device for predicting word semantics based on semantic primitives. Background Art

[0002] In recent years, recommendation systems have been widely used in mobile phones, TVs, smart speakers, smart headphones and other products. For example, in the application market of mobile products, recommendation systems can help users match more suitable APPs and improve the APP distribution efficiency of the application market itself; in information flow products, recommendation systems can help users match information that is more in line with their interests, saving users' time and increasing the overall reading volume of news. However, the rapid development of society has brought about the emergence of new words and concepts, especially on the Internet, where new words are created every day, such as: Lanshou Xianggu, Shidong Ranju, and Haoziweizhi. The rapid and accurate understanding and representation of the semantics of these words is a major challenge facing recommendation systems.

[0003] At present, the most commonly used method for representing word semantics is word vectorization, which is mainly divided into two types: A. One is based on the one-hot encoding of words, such as Figure 1 As shown in , each word is represented as an n-dimensional 0-1 vector, where n is the size of the vocabulary, usually between hundreds of thousands and tens of millions (the vector length is generally the length of the corresponding dictionary). Each dimension in the vector represents a word in the vocabulary, and in the n-dimensional 0-1 vector, only the value of the position where the current word is located is set to 1, and the rest of the positions are 0; B. The other is based on the dense embedding representation of words, such as Figure 2 As shown, each word can be represented as an n-dimensional real number vector, where n is usually between several hundred and one thousand. Each dimension of the vector is used to represent some characteristics / attributes of the word (such as gender, verb, plural, etc.). It does not have a specific meaning, and the closeness between the semantics of the words is represented by calculations between vectors (such as distance).

[0004] The above method A is equivalent to assigning a number to each word, but the relationship between words cannot be fully reflected, the representation ability is weak, the vector is very sparse, and the storage overhead is large; although the above method B can represent the semantic relationship between words through calculation, the meaning of each dimension in the vector is not clear, and the interpretability is relatively poor. Specifically in the recommendation system, this method of calculation is difficult to give reasons for recommendation. Summary of the invention

[0005] The embodiment of the present application provides a prediction method and computer device for the semantics of words based on semantics. The method constructs a semantic representation space of words (i.e., semantic space) based on semantics, and represents the defined words (i.e., the first words) in the semantic space. The vector representation of the words in the semantic space consists of two parts, one of which is determined by the category semantics of the words, and the other is determined by the extended description of the words and the association between the extended description and the category semantics (the specific value of this part is obtained through model learning). Since the meaning of the semantics is clear, the vector used to represent the words in the present application is interpretable; and, in the face of newly added words (i.e., the second word), the closeness relationship (i.e., correlation) between the newly added words and the known words can be calculated, and the vector representation of the newly added words in the semantic space is derived from the vector representation of the known words in the semantic space. The newly added words predicted by this method have strong representation capabilities. The method of the present application can be combined with a recommendation system to solve the problem of understanding and representing new words in the recommendation system and improve the quality of recommendation.

[0006] Based on this, the embodiments of the present application provide the following technical solutions:

[0007] In the first aspect, the embodiment of the present application first provides a semantic prediction method for words based on semantics, which can be used in the field of natural language processing. The method includes: first, constructing a semantic space based on category semantics, extended description, and the relationship between category semantics and extended description (i.e., association relationship). After that, the defined words (i.e., known words, also referred to as first words) are represented in the constructed semantic space, each known word can be represented as a vector (referred to as a first vector) in the semantic space, the first vector is composed of two parts, one part can be referred to as a first sub-vector, and the other part can be referred to as a second sub-vector, wherein the first sub-vector is determined by the category semantics corresponding to the known word (referred to as the first category semantics), the second sub-vector is determined by the extended description related to the known word (referred to as the first extended description), and the association relationship between the first extended description and the first category semantics (referred to as the first association relationship), the values ​​of each element in the second sub-vector are specifically obtained through model learning, and the obtained first vector can be used to represent the semantics of the first word, and each defined word can be represented as a vector in the semantic space. After that, the relevance between the second word and each first word is calculated, and the second word is an undefined word. After obtaining the relevance between the second word and each first word, the vector representation of the second word in the semantic space (which can be called the second vector) can be obtained according to the first vector and relevance of each first word in the semantic space, and the second vector is used to represent the semantics of the second word.

[0008] In the above-mentioned implementation mode of the present application, a semantic representation space (i.e., semantic space) of words is constructed based on semantic originals, and the defined words (i.e., the first words) are represented in the semantic original space. The vector representation of words in the semantic original space consists of two parts, one of which is determined by the category semantic original of the word, and the other is determined by the extended description of the word and the association between the extended description and the category semantic original (the specific value of this part is obtained through model learning). Since the meaning of the semantic original is clear, the vector used to represent words in the present application is interpretable; and, in the face of newly added words (i.e., the second word), the closeness relationship (i.e., correlation) between the newly added words and the known words can be calculated, and the vector representation of the newly added words in the semantic original space is derived from the vector representation of the known words in the semantic original space. The newly added words predicted by this method have strong representation ability. The method of the present application can be combined with a recommendation system to solve the problem of understanding and representing new words in the recommendation system and improve the quality of recommendation.

[0009] In a possible implementation of the first aspect, for a known word, when the known word has n (n≥2) different semantics, the vector representation (which may be called the target vector) of each semantic in the semantic space can be obtained respectively, that is, n target vectors of n word pairs in the semantic space can be obtained, and one target vector is used to represent one semantic of the known word. Finally, the n target vectors are fused, and the fused result is the vector representation of the word in the semantic space, that is, the first vector.

[0010] In the above-mentioned implementation of the present application, it is explained that when a known word has multiple semantics, a target vector can be obtained for each semantics, and then the target vectors are fused to obtain the final first vector, which is feasible.

[0011] In a possible implementation of the first aspect, since the representation of a word in the semantic space can actually be a fusion of the vector representations of the word's various word pairs in the semantic space, the vector representations of the various semantics of the same word in the semantic space (e.g., n target vectors) can be added and averaged to obtain the vector representation of the word in the semantic space (i.e., the first vector).

[0012] In the above-mentioned implementation of the present application, a method for fusing multiple target vectors is provided, and the fusing method is simple and easy to operate.

[0013] In a possible implementation of the first aspect, assuming that there are m first words, since each first word corresponds to a first vector, each first word and the second word can calculate a correlation, so there are also m first vectors, m correlations, one first word corresponds to a first vector, and one first vector corresponds to a correlation. Therefore, an implementation method of obtaining the second vector of the second word in the semantic original space according to the first vector and the correlation can be: first, according to the value sorting of the m correlations, the dynamic weight of each first word is calculated (one dynamic weight corresponds to one first vector), and the value of the dynamic weight is positively correlated with the value of the correlation, that is, the larger the correlation value, the higher the ranking, indicating that the corresponding first word is more closely associated with the second word, and the dynamic weight of the first word is greater; then, each first vector, the correlation corresponding to each first vector, and the dynamic weight corresponding to each first vector are multiplied to obtain m product results; finally, these m product results are added to obtain the second vector of the second word in the semantic original space.

[0014] In the above-mentioned embodiment of the present application, a method for obtaining the second vector of the second word in the semantic original space according to the first vector and the relevance is specifically described, which has wide applicability.

[0015] In a possible implementation of the first aspect, a method for calculating the relevance between the second word and each first word may be: first, project each first word and the second word into a first point (multiple first words are each projected into a corresponding first point) and a second point in a high-dimensional word vector space through a language model (such as word2vec, bert, etc.), that is, each projected word can be represented as a point in the word vector space (which can be called a projection point), and then calculate the projection point (that is, the second point) corresponding to the second word and the projection point corresponding to each first word. The distance between points (i.e., each first point). For example, assuming there are m first words, these m words can all be projected into a point in the word vector space (i.e., the projection point). Then, the distance between the projection point of the second word and the projection points of the m first words can be calculated, and m distances can be obtained. Each distance is used to characterize the value of the correlation between the second word and the corresponding first word, and the value of the distance is inversely correlated with the value of the correlation, that is, the smaller the distance value, the closer the association, and the larger the correlation value; the larger the distance value, the looser the association, and the smaller the correlation value.

[0016] In the above-mentioned implementation mode of the present application, a method for calculating the relevance between the second word and each first word is specifically described, and the similarity comparison problem is converted into a distance comparison problem of projection points in the word vector space, which is feasible.

[0017] In a possible implementation of the first aspect, another method for calculating the relevance between the second word and each first word can also be: first, through a language model (such as word2vec, bert, etc.), each first word and the second word are projected into a first space vector in a high-dimensional word vector space (multiple first words are each projected into a corresponding first space vector) and a second space vector, that is, each projected word can be represented as a space vector of the word vector space passing through the origin of the space, and then, the angle between the second space vector and each first space vector is calculated. For example, assuming that there are m first words, these m words can all be projected into a point in the word vector space (that is, the projection point), and each point is It can be represented as a space vector passing through the origin of space, one word corresponds to one space vector, and then the angle between the space vector of the second word and the space vectors of the m first words can be calculated (for example, the cosine similarity between the two space vectors can be calculated), and m angles (for example, m cosine similarities) can be obtained. Each angle (for example, cosine similarity) is used to characterize the value of the correlation between the second word and the corresponding first word, and the value of the angle (for example, cosine similarity) is inversely correlated with the value of the correlation, that is, the smaller the value of the angle (for example, cosine similarity), the closer the association, and the larger the value of the correlation; the larger the value of the angle (for example, cosine similarity), the looser the association, and the smaller the value of the correlation.

[0018] In the above-mentioned embodiment of the present application, another method for calculating the relevance between the second word and each first word is specifically described, which converts the similarity comparison problem into the angle comparison problem between vectors in the word vector space, and has flexibility.

[0019] In a possible implementation of the first aspect, the dimension corresponding to the category semantics in the constructed semantic space can be expressed as a one-hot encoding (also denoted as 0-1), that is, the first sub-vector is a one-hot encoding representation; the dimension corresponding to the combination of "extended description + association relationship" can be expressed as a dense embedding (dense embedding) form, that is, the second sub-vector is a dense embedding representation.

[0020] In the above implementation of the present application, a form of the first vector is specifically described, that is, one part is in a one-hot encoding form and the other part is in a dense embedding form. This form takes into account both the storage problem and the interpretability problem of the vector.

[0021] In a possible implementation of the first aspect, after obtaining the second vector of the second word in the semantic space, the method may also include: constructing a user portrait based on the obtained second vector and user behavior information (such as click status), in the embodiment of the present application, the click status is the situation that the target user account clicks on the article containing the second word through the terminal device; then, for the candidate article (such as news, information, etc.), extract the keywords of the candidate article (how to extract keywords has a lot of existing work, such as TFIDF method, TextRank method, etc., which are not discussed in the present invention), and obtain the vector representation of these keywords in the semantic space (which can be called the third vector), which is used to characterize the characteristics of the candidate article. Next, calculate the correlation between the vector representation of the existing words in the semantic space in the user portrait (that is, the second vector described above) and the third vector of the keywords in the semantic space in the candidate article, and determine the matching degree between the candidate article and the user portrait based on the obtained correlation, that is, obtain the matching degree between the current user and the candidate article, which can be used for recommendation sorting. Finally, when the matching degree reaches a preset threshold, the candidate article is recommended to the device currently logged in by the target user account. As an example, the matching degree can be ranked in order. Assuming that the matching degree is ranked in the top three, and the set threshold is to recommend the top three candidate articles with matching degree to the user, then it means that the matching degree reaches the preset threshold. As another example, the matching degree can be a specific value. In this case, the preset threshold is also a specific value. Assuming that the matching degree value is 0.8 (the total value is assumed to be 1), and the preset threshold is 0.6, then it means that the matching degree reaches the preset threshold.

[0022] In the above-mentioned embodiments of the present application, it is specifically explained that the word semantics prediction method based on semantic primitives provided in the above-mentioned embodiments of the present application can be further combined with the recommendation system to solve the new word understanding and expression problems of the recommendation system, so as to improve the recommendation quality.

[0023] In a possible implementation of the first aspect, the click situation may be the total number of clicks within a preset time period, from which the user's interest in the article including the new word can be inferred, so that the recommendation system can recommend articles that meet the user's interests; the click situation may also be a click time period in which the click frequency reaches a preset threshold, from which it can be inferred that the user has a greater interest in reading in what time period, so that the recommendation system can push at the right time. It should be noted that in addition to the above two categories, the click situation may also be other situations. As long as it can provide additional information to the recommendation system to improve the recommendation effect, it belongs to the click situation described in this application, and this application does not limit it specifically.

[0024] In the above-mentioned implementation mode of the present application, the expression form of the click situation is explained, which is optional and the user can make a choice based on his or her own needs, thereby improving the user experience.

[0025] A second aspect of the embodiment of the present application provides a computer device, which has the function of implementing the method of the first aspect or any possible implementation of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0026] A third aspect of an embodiment of the present application provides a computer device, which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to call the programs stored in the memory to execute the method of the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0027] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer can execute the method of the first aspect or any possible implementation of the first aspect.

[0028] A fifth aspect of an embodiment of the present application provides a computer program or a computer program product. When the computer program or the computer program product runs on a computer, the computer executes the method of the first aspect or any possible implementation of the first aspect.

[0029] The sixth aspect of the embodiment of the present application provides a chip, which includes at least one processor and at least one interface circuit, the interface circuit is coupled to the processor, the at least one interface circuit is used to perform a transceiver function, and send instructions to at least one processor, the at least one processor is used to run a computer program or instruction, which has the function of implementing the method of the first aspect or any possible implementation of the first aspect, the function can be implemented by hardware, can also be implemented by software, can also be implemented by a combination of hardware and software, the hardware or software includes one or more modules corresponding to the above functions. In addition, the interface circuit is used to communicate with other modules outside the chip. For example, the interface circuit can encapsulate the second vector of the second word obtained by the processor on the chip in the semantic space and send it to other computer devices, such as, sending it to other terminal devices (such as mobile phones, personal computers, smart watches, etc.) or cloud-side devices (such as cloud servers, clusters, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of word-based one-hot encoding representation;

[0031] Figure 2 A schematic diagram of the dense embedding representation of words;

[0032] Figure 3 A flowchart of a method for predicting word semantics based on semantic primitives provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of a flow chart of a method for predicting words based on semantic elements provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of the relationship between the word "kid" and related semantic primitives in HowNet provided in the embodiment of the present application;

[0035] Figure 6 A schematic diagram of a vector representation of the word pair "child / kid" in the semantic space provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of the vector representation of the word pair "song and dance troupe / chorus" in the semantic space provided in the embodiment of the present application;

[0037] Figure 8 A schematic diagram of a vector representation of a word in a semantic space provided by an embodiment of the present application;

[0038] Fig. 9 An overall flow chart of the application of the vector representation of the newly added words in the semantic space in the recommendation system provided by the embodiment of the present application;

[0039] Fig.10 A schematic diagram of a user portrait provided in an embodiment of the present application for a recommendation system;

[0040] Fig.11 An overall flow chart of the application of the vector representation of events in the semantic primitive space provided by the embodiment of the present application in event mining and alarming;

[0041] Fig.12 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0042] Fig.13 Another structural schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The embodiment of the present application provides a prediction method and computer device for the semantics of words based on semantics. The method, based on HowNet (a linguistic resource), constructs a semantic representation space (i.e., semantic space) of words based on semantics, and represents known words (i.e., the first word) in HowNet in the semantic space. The vector representation of words in the semantic space consists of two parts, one of which is determined by the category semantics of the word, and the other is determined by the extended description of the word and the association between the extended description and the category semantics (the specific value of this part is obtained through model learning). Since the meaning of the semantics is clear, the vector used to represent words in the present application is interpretable; and, in the face of newly added words (i.e., the second word), the closeness relationship (i.e., correlation) between the newly added words and the known words can be calculated, and the vector representation of the newly added words in the semantic space is derived from the vector representation of the known words in the semantic space. The newly added words predicted by this method have strong representation capabilities. The method of the present application can be combined with a recommendation system to solve the problem of understanding and representing new words in the recommendation system and improve the quality of recommendation.

[0044] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0045] The embodiments of this application involve a lot of relevant knowledge about HowNet, semantics, etc. In order to better understand the solutions of the embodiments of this application, the following introduces the relevant terms and concepts that may be involved in the embodiments of this application. It should be understood that the interpretation of related concepts may be limited due to the specific circumstances of the embodiments of this application, but it does not mean that this application is limited to this specific situation. The specific circumstances in different embodiments may also be different, which is not specifically limited here.

[0046] HowNet: A bilingual common sense database in Chinese and English, which describes concepts represented by Chinese and English vocabulary, and describes the concepts of words by semantic primitives, aiming to reveal the relationship between concepts and the attributes of concepts. The semantic primitives in HowNet are organized into a tree structure according to subordinate relationships. According to different functions, they can be further divided into entity semantic primitive trees, event semantic primitive trees, attribute semantic primitive trees, attribute value semantic primitive trees and secondary feature primitive trees, with a total of more than 2,000 semantic primitives. Based on these more than 2,000 semantic primitives, more than 35,000 concepts and more than 229,000 Chinese and English word pairs (English is word pair, also called entries) have been defined so far. In the embodiment of the present application, a word pair is a pair of words corresponding to Chinese and English, such as "child / kid" is a word pair.

[0047] Sememe: In linguistics, it refers to the smallest indivisible and unambiguous semantic unit, which can be further divided into two categories: categorical sememe and extended description.

[0048] Categorical sememe: Categorical sememe is the basic category to which a word belongs. It is the basis and important component of constructing the knowledge of word meaning.

[0049] Extended elaboration: Extended elaboration is a further description and supplementary explanation of the category semantics. The association between two semantics (e.g., category semantic a and extended description b) can usually be expressed in the form of a triple: T = {(h, r, t)|h, t∈S; r∈R}, where h and t represent semantics (it is not limited to what semantics they are, they can be category semantics or extended descriptions), and r represents the association between the two semantics h and t.

[0050] Sememe space: In the embodiment of the present application, it is a high-dimensional vector space constructed based on category sememes, extended descriptions, and the association relationship between category sememes and extended descriptions.

[0051] Word embedding space: also known as word embedding space, is a high-dimensional real number vector space. Each word is represented as a point in the space. It can be trained on large-scale corpora by language models such as word2vec, glove, and BERT. After training, these language models can project each word into a point in the word embedding space.

[0052] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0053] In order to facilitate understanding of this solution, first, the basic principle of the semantic prediction method based on semantic primitives provided in the embodiment of the present application is introduced. Figure 3 , Figure 3 The process framework diagram of the prediction method of the semantics of words based on the semantics of the semantics provided in the embodiment of the present application can be specifically divided into two parts: the learning process and the prediction process. In the learning process, first, a semantic space is defined based on the relationship between the category semantics, the extended description, and the category semantics and the extended description, and then, according to the semantic features (i.e., word pairs) of the known words (i.e., the first words), the vector representation of these semantic features in the semantic space is further designed, and the specific values ​​of the elements in the vector representation are obtained through model learning, and the vectorized representation results of the words of all known semantics in the semantic space can be obtained accordingly; in the prediction process, in the face of the newly added words (i.e., the second words), the closeness relationship between the newly added words and the known words is first calculated, and the closeness relationship can be expressed by the correlation value, and the vectorized representation result of the newly added words in the semantic space is derived according to the vectorized representation results of the known words in the semantic space and the obtained closeness relationship.

[0054] Next, based on the basic principles described above, the word prediction method based on semantic origin provided in the embodiment of the present application is introduced. Figure 4 , Figure 4 A flowchart of a method for predicting words based on semantic elements provided in an embodiment of the present application is provided. The method may include the following steps:

[0055] 401. Obtain a first vector of a first word in a semantic space, where the first word is a defined word. The semantic space is constructed according to a category semantic, an extended description, and an association relationship, where the association relationship is used to represent a relationship between the category semantic and the extended description, and the first vector is used to represent the semantics of the first word.

[0056] First, a semantic primitive space is constructed based on the category semantic primitive, the extended description, and the relationship between the category semantic primitive and the extended description (i.e., the association relationship). After that, the words defined in HowNet (i.e., known words, also called first words) are represented in the constructed semantic primitive space. Each known word can be represented as a vector in the semantic primitive space (which can be called the first vector). The first vector consists of two parts, one part can be called the first subvector, and the other part can be called the second subvector. The first subvector is determined by the category semantic primitive related to the known word (which can be called the first category semantic primitive), and the second subvector is determined by the extended description related to the known word (which can be called the first extended description) and the association relationship between the first extended description and the first category semantic primitive (which can be called the first association relationship). The values ​​of each element in the second subvector are specifically obtained through model learning. The obtained first vector can be used to represent the semantics of the first word. Each word defined in HowNet can be represented as a vector in the semantic primitive space.

[0057] In an embodiment of the present application, the representation of the constructed semantic space can be in the form of a dense embedded vector, each category semantic primitive in the semantic primitive set of HowNet corresponds to a dimension of the vector, and each extended description and the association relationship related to the extended description as a whole corresponds to a dimension of the vector. For ease of understanding, an example is given here: assuming that a total of 2000 semantic primitives are defined in HowNet, of which 800 are category semantic primitives and the other 1200 are extended descriptions, then each of the 800 category semantic primitives corresponds to a dimension in the semantic primitive space, that is, there are 800 dimensions in the semantic primitive space that correspond one-to-one to the category semantic primitive; and among these 1200 extended descriptions, some extended descriptions may have an association relationship with only one category semantic primitive, and some extended descriptions may have an association relationship with multiple category semantic primitives (the association relationships are different), then the combination of "extended description + association relationship" each corresponds to a dimension in the semantic primitive space. For example, in addition to the association relationship "belong" between the extended description "family" and the category semantic primitive "human", the extended description "family" also has association relationships a and b with category semantic primitives A and B respectively (for illustration only). Then "family,belong" corresponds to one dimension of the vector, and "family,a" and "family,b" correspond to the other two dimensions of the vector. In other words, the same extended description has as many different association relationships with the category semantic primitives as there are, and they correspond to several dimensions of the vector. Assuming that there are 4200 combinations of "extended description + association relationship" corresponding to these 1200 extended descriptions, the constructed semantic primitive space includes 5000 dimensions, of which 800 dimensions correspond to category semantic primitives one-to-one, and 4200 dimensions correspond to combinations of "extended description + association relationship" one-to-one.

[0058] After defining the construction method of the sememe space, each word can be projected as a point in the sememe space according to its semantics in HowNet. For example, assuming that the constructed sememe space includes 5000 dimensions, each word can be represented as a vector including 5000 elements.

[0059] It should be noted that in some embodiments of the present application, the dimensions corresponding to the category sememes in the constructed sememe space can be represented in the form of one-hot encoding (also denoted as 0-1), that is, the first sub-vector is represented by one-hot encoding; the dimensions corresponding to the combination of "extended description + association relationship" can be represented in the form of dense embedding, that is, the second sub-vector is represented by dense embedding. In some other embodiments of the present application, each dimension of the constructed sememe space can be represented in the form of dense embedding, that is, the first vector (including the first sub-vector and the second sub-vector) is represented by dense embedding. Specifically, the present application does not limit the representation method of the vector. For the convenience of description, in the following embodiments of the present application, the form in which the first sub-vector is represented by one-hot encoding and the second sub-vector is represented by dense embedding is used for description.

[0060] To facilitate the understanding of the above steps, the following takes the known word "kid" as an example to introduce in detail how to represent the known word "kid" into the constructed sememe space.

[0061] As Figure 5 shown, Figure 5 is a schematic diagram of the relationship between the word "kid" and related sememes in HowNet. It can be seen from Figure 5 that the word "kid" has 3 semantics in HowNet, which can be represented by 3 Chinese-English word pairs "孩子 / kid", "逗 / kid", and "羔羊 / kid" respectively. Each word pair accurately represents the semantics of "kid" in a certain aspect. And the word pair is represented by specific sememes. Taking "孩子 / kid" as an example, it is represented as the category sememe "人 / human". Further, the category sememe "人 / human" has 3 extended descriptions, namely "家庭 / family", "小辈 / junior", and "直系 / lineal". The category sememe "人 / human" and these 3 extended descriptions are each associated through 3 association relationships (i.e., the edges in Figure 5 ). Therefore, the category sememe and the extended description can be represented in the form of a triple. For example, ("人 / human", "belong", "家庭 / family") is a triple.

[0062] Since the class sememes and the combination of "extended description + association relationship" respectively correspond to a dimension of the sememe space, for a known word in HowNet, when the known word has n (n≥2) different semantic meanings, the Chinese-English word pairs of each semantic meaning of the known word can be obtained first. One semantic meaning corresponds to one word pair. Then, the vector representation (which can be called the target vector) of each word pair in the sememe space can be obtained respectively. Thus, n target vectors of n word pairs in the sememe space can be obtained. One word pair corresponds to one target vector, and one target vector is used to represent one semantic meaning of the known word. Finally, the result of merging the vector representations of all word pairs in the sememe space (that is, fusing n target vectors) is the vector representation of the word in the sememe space, which is the so-called first vector.

[0063] Still taking Figure 5 as an example, for the word pair "孩子 / kid", its vector representation in the sememe space can be as Figure 6 shown. This vector consists of two parts. One part corresponds to the class sememe and is represented as a 0-1 vector. For the class sememe "人 / human", only the position corresponding to this class sememe "人 / human" in the 0-1 vector is 1, and the rest of the positions are 0, indicating that the current word pair belongs to this class sememe. The other part of this vector corresponds to the combination of "extended description + association relationship" and is represented as a dense embedding vector. The values of the elements in this dense embedding vector are real numbers. On the one hand, it indicates that the word pair has an association relationship with the current extended description. On the other hand, the values of the elements in this dense embedding vector indicate the strength of this association relationship. The specific values of the elements are determined by the extended description and the relationship between the extended description and the class sememe (such as in the above example, the extended description is "家庭 / family", and the relationship between "家庭 / family" and the class sememe "人 / human" is "belong"). The calculation method of the specific values of the elements will be given below.

[0064] Since the word "kid" has 3 Chinese-English word pairs, Figure 5 the corresponding embodiment shows the vector representation of one of the word pairs "孩子 / kid" in the sememe space. Similarly, the vector representations of the other 2 Chinese-English word pairs of the word "kid" in the sememe space are also obtained in a similar way, which will not be elaborated here specifically. Finally, by fusing the vector representations of the 3 Chinese-English word pairs in the sememe space respectively, the vector representation of the word "kid" in the sememe space can be obtained.

[0065] It should be noted that in the embodiments of this application, if the known word has only one semantic meaning, that is, n = 1, it means that the known word has only one Chinese-English word pair, and that word pair will also be processed according to the above Figure 5The method described in the corresponding embodiment is expressed as a vector representation of the semantic primitive space. Since the known word has only one semantic meaning, the vector representation of the word pair in the semantic primitive space is the vector representation of the known word in the semantic primitive space.

[0066] The following mathematically describes how to calculate the vector representation of a word in the semantic space. As mentioned above, a word is represented by a Chinese-English word pair to represent different semantics. Therefore, we can first give a vector representation method for a word pair in the semantic space, as shown in the following formula (1):

[0067]

[0068] The first term in the formula is the word pair p i The 0-1 vector corresponding to the category semantic primitive; the second term in the formula is all entities with the category semantic primitive as the head (i.e., h j ) triple, the extended description (ie, t j ) and the relationship (i.e. r j ) is the sum of the vectors corresponding to the extended description and the association relationship. The specific value of the vector is determined by the extended description and the association relationship. The specific determination method is described below. The third term in the formula is similar to the second term. It is based on the class semantic original as the tail entity (i.e., t k ) triple, the extended description (i.e. h k ) and the relationship (i.e. r k ) The fourth term of the formula is the bias term, which corresponds one-to-one to the word pairs representing the corresponding semantics.

[0069] As an example, Figure 5 The known word "kid" in includes a word pair "child / kid". The vector of the word pair "child / kid" in the semantic space can be expressed as shown in the following formula (2):

[0070]

[0071] As another example, Figure 7 As shown in the figure, the word pair "chorus" contains both the category semantics "group / community" and the extended descriptions "art / entertainment", "performance / perform", etc., and also contains the associations "domain" and "agent" between the category semantics and the extended descriptions. The vector of the word pair "chorus" in the semantic space can be expressed as shown in the following formula (3):

[0072]

[0073] Since some words are composed of one word pair, and some words are composed of multiple word pairs, such as the above Figure 5 The word "kid" mentioned above, it is known that the word "kid" has not only the semantics represented by the word pair "child / kid", but also the semantics represented by the word pairs "dou / kid" and "lamb / kid". These two word pairs can be expressed as two vectors in the semantic original space according to formula (1), which will not be repeated here. Therefore, the representation of a word in the semantic original space can actually be a fusion of the vector representations of each word pair (i.e., each different semantics) of the word in the semantic original space. Specifically, in some embodiments of the present application, the vector representations of each word pair of the same word in the semantic original space (e.g., n target vectors) can be added and averaged to obtain the vector representation of the word in the semantic original space (i.e., the first vector). The vector representation of the word obtained can be shown in the following formula (4):

[0074]

[0075] Taking the word "kid" as an example, "kid" contains three different semantics, which are represented by the word pairs "child / kid", "funny / kid" and "lamb / kid" respectively. The vector representation of the word "kid" in the semantic space is the average of the vector representations of these three word pairs in the semantic space.

[0076] In summary, based on the constructed semantic space and formula (1), each word defined in HowNet can be represented as a vector in the semantic space. The vector corresponding to each word can be composed of two parts, one part is the first sub-vector, represented as a 0-1 vector; the other part is the second sub-vector, represented as a dense embedding vector. Since the value of the element in the dense embedding vector is a real number, on the one hand, it indicates that the word has an association with the current extended description, and on the other hand, the value of the element in the dense embedding vector indicates the strength of this association. The specific value of the element is determined by the extended description and the relationship between the extended description and the category semantics, and the specific value of each element can be learned through the model. The following is a detailed introduction on how to learn the specific value of each vector of the word.

[0077] First, it can be seen from the above that the word pair of a known word and the vector representation of the known word in the semantic space can be expressed by formula (1) and formula (4) respectively. In the embodiment of the present application, the second to fourth items in formula (1) can be regarded as parameters to be learned (i.e., the values ​​of the elements are not determined). Since formula (4) is the addition and averaging of the representation of formula (1), each item in formula (4) The second to fourth items of each element can be regarded as parameters to be learned (i.e., the values ​​of the elements are not determined). The following is an introduction to the learning process of the vector in steps:

[0078] Step 1: Randomly initialize the parameters to be learned.

[0079] Since the words in HowNet are all defined, it is possible to obtain in advance which semantic primitives are related to each known word. For details, please refer to Figure 8 , Figure 8 A schematic diagram of the vector representation of words in the semantic space provided in the embodiment of the present application, assuming that the vector of a known word C in the semantic space in HowNet can be represented as Vc. Since the 0-1 vector part corresponds to the category semantics of the word C, based on the definition of HowNet, the specific value of each element in the 0-1 vector can be known, that is, which elements in the 0-1 vector are 1 and which are 0 can be obtained. In the embodiment of the present application, since the 0-1 vector is real and reliable, it can be used as the true value data for model learning; and the part of the dense embedded vector corresponds to the combination of "extended description + association relationship" of the word C, the values ​​of this part of the elements are real numbers, which are obtained based on learning.

[0080] It should be noted here that in some embodiments of the present application, each element in the dense embedded vector can be learned as a parameter to be learned to obtain the specific value of each element, but generally speaking, the number of dimensions corresponding to the combination of "extended description + association relationship" is very large (reaching several thousand). If each element is learned as a parameter to be learned, not only the amount of calculation will be large but also the learning accuracy will be reduced.

[0081] Therefore, in order to reduce the amount of parameters to be learned, in other embodiments of the present application, the element values ​​on the dimensions corresponding to the combination of "extended description + associated relationship" that are irrelevant to the word can be excluded in advance based on the definition of HowNet. That is, based on the definition of HowNet, which dimensions in the dense embedding vector are irrelevant to the word C can be excluded in advance. As an example, assuming that the constructed semantic primitive space includes 5000 dimensions, of which 800 dimensions correspond to the category semantic primitives one-to-one, 4200 dimensions correspond to the combination of "extended description + associated relationship" one-to-one, and the combination of "extended description + associated relationship" of word C corresponds to 100 dimensions in the dense embedding vector, then the elements of these 100 dimensions in the dense embedding vector can be regarded as the parameters x, y, z... to be learned, and the elements of the dimensions other than these 100 dimensions in the dense embedding vector are set to 0, and the elements on these 100 dimensions are used as parameters to be learned (i.e., unknown parameters).

[0082] After determining the vector representation of each known word in the semantic space and the parameters to be learned, the unknown parameters of each known word can be randomly initialized, such as Perform random initialization.

[0083] Step 2: Model learning process.

[0084] 1) First, all known words in HowNet are used as training data, and these known words are represented as Figure 8 The vector form shown.

[0085] 2) The dense embedding vector of each known word is used as the input data of the model (that is, the parameters to be learned as described above), so that the model predicts the category semantics to which the word belongs based on the input data. In the embodiment of the present application, since the number of category semantics is limited, the above prediction problem can be converted into a classification problem in machine learning, that is, a classifier in machine learning can be selected as a predictor, such as a fully connected neural network as a classifier. Specifically, it can be a recurrent neural network (RNN), a convolutional neural network (CNN), a transformer, etc., and examples are not continued here.

[0086] 3) Based on 2), the predicted result of the category semantics of the known word is compared with the true category semantics of the known word (that is, the 0-1 vector of the training word is taken as the true value), and the value of the loss function is calculated.

[0087] 4) According to the calculated value of the loss function, back propagation updates the network parameter values ​​of the model and the parameters to be learned in the known words, that is, in turn, the parameters of the model and the parameters to be learned of the known words are learned and corrected. It should be noted that in the embodiment of the present application, the dense embedded vectors in the known words are both the input data of the model and the part that needs to be trained as the model.

[0088] 5) Repeat 1) to 4) to train all known words in HowNet, thereby obtaining the specific values ​​of the elements in the dense embedding vector of each known word in HowNet. At the same time, combined with the category semantic information of the known word, the complete representation of the known word in the semantic space is obtained, that is, the elements in the vector representation of each known word have specific values.

[0089] It should be noted that in some embodiments of the present application, some known words in HowNet (e.g., typical and representative words, such as only one of multiple words with similar semantics) may be selected for training to obtain a complete representation of the known words in the semantic space. The advantage of doing so is that the amount of calculation can be reduced.

[0090] After the above steps 1 and 2, all known words in HowNet, or some representative known words, can be obtained with vector representations with clear values ​​in the semantic space. In the embodiment of the present application, for the sake of convenience, the following is an example of obtaining vector representations with clear values ​​for all known words in HowNet.

[0091] 402. Calculate the relevance between a second word and the first word, where the second word is an undefined word.

[0092] In step 401, the vector representation of the words defined in HowNet (i.e., the first words) in the semantic space is described, and the specific value of each first vector is obtained through model learning. Next, step 402 further describes how to obtain the vector representation of these words in the semantic space when encountering words other than HowNet (i.e., words not defined by HowNet, which can be called second words or newly added words).

[0093] First, the correlation between the second word and each first word is calculated, and the second word is a word that is not defined in HowNet. The correlation can be calculated in different ways, including but not limited to: projecting the second word and each first word into a high-dimensional word vector space through a language model (such as word2vec, bert, etc.), and each projected word can be represented as a point in the word vector space (or a space vector passing through the origin of the space). 1) If the second word and each first word are projected as points in the word vector space, then the distance between the projection point corresponding to the second word and the projection point corresponding to each first word can be calculated. For example, assuming that there are m first words, these m words can all be projected as a point in the word vector space (i.e., projection point), then the distance between each projection point of the second word and the projection points of the m first words can be calculated, and m distances can be obtained. Each distance is used to characterize the value of the correlation between the second word and the corresponding first word, and the value of the distance is inversely correlated with the value of the correlation, that is, the smaller the distance value, the closer the association, and the larger the correlation value; the larger the distance value, the looser the association, and the smaller the correlation value. 2) If the second word and each first word are projected as vectors passing through the origin of the space in the word vector space, then the angle between the space vector corresponding to the second word and the space vector corresponding to each first word can be calculated. For example, assuming there are m first words, these m words can be projected as a point in the word vector space (i.e., projection point), and each point can be represented as a space vector passing through the origin of the space. One word corresponds to one space vector, and then the angle between the space vector of the second word and the space vector of the m first words can be calculated. The angle between the two space vectors (e.g., the cosine similarity between the two space vectors can be calculated), and m angles (e.g., m cosine similarities) can be obtained. Each angle (e.g., cosine similarity) is used to characterize the value of the correlation between the second word and the corresponding first word, and the value of the angle (e.g., cosine similarity) is inversely correlated with the value of the correlation, that is, the smaller the value of the angle (e.g., cosine similarity), the closer the association, and the larger the correlation value; the larger the value of the angle (e.g., cosine similarity), the looser the association, and the smaller the correlation value.

[0094] 403. Obtain a second vector of the second word in the semantic space according to the first vector and the correlation, where the second vector is used to represent the semantics of the second word.

[0095] After obtaining the correlation between the second word and each first word, the vector representation of the second word in the semantic space (which may be called the second vector) can be obtained based on the first vector and correlation of each first word in the semantic space. The second vector is used to represent the semantics of the second word.

[0096] Specifically, assuming that there are m first words, since each first word corresponds to a first vector, each first word and the second word can calculate a correlation, so there are also m first vectors, and there are also m correlations. One first word corresponds to one first vector, and one first vector corresponds to one correlation. Therefore, in some embodiments of the present application, the implementation method of obtaining the second vector of the second word in the semantic original space according to the first vector and the correlation can be: first, according to the value sorting of the m correlations, the dynamic weight of each first word is calculated (one dynamic weight corresponds to one first vector), and the value of the dynamic weight is positively correlated with the value of the correlation, that is, the larger the correlation value, the higher the ranking, indicating that the corresponding first word is more closely associated with the second word, and the larger the dynamic weight of the first word; then, each first vector, the correlation corresponding to each first vector, and the dynamic weight corresponding to each first vector are multiplied to obtain m product results; finally, these m product results are added to obtain the second vector of the second word in the semantic original space. As an example, the calculation of the second vector of the second word in the semantic space can be obtained based on the following formula (5):

[0097]

[0098] in, For the second word and the first word The cosine similarity of is used to characterize two words and The correlation between them can also be replaced by other metrics according to actual conditions, such as various distance metrics. rj is a dynamic weight, c is a hyperparameter, and its specific value can be set according to the needs. For example, the value range can be between [0,1], r j Indicates the correlation between words (here is ) in descending order, i.e. the first word The second word The rth j More closely related words, is the vector representation of all known words in the semantic space, which is learned in step 2 above. It should be noted that in some embodiments of the present application, the dynamic weight It can also be directly set to 1.

[0099] In summary, the method provided in the embodiment of the present application is based on HowNet (a linguistic resource), and constructs a semantic representation space (i.e., a semantic original space) of words based on semantic originals, and represents the known words in HowNet (i.e., the first word) in the semantic original space. The vector representation of the word in the semantic original space consists of two parts, one of which is determined by the category semantic original of the word, and the other is determined by the extended description of the word and the association between the extended description and the category semantic original (the specific value of this part is obtained through model learning). Since the meaning of the semantic original is clear, the vector used to represent the word in this application is interpretable; and, in the face of a new word (i.e., the second word), the closeness relationship (i.e., correlation) between the new word and the known word can be calculated, and the vector representation of the new word in the semantic original space is derived from the vector representation of the known word in the semantic original space. The new word predicted by this method has a strong representation ability. The method of this application can be combined with a recommendation system to solve the problem of understanding and representing new words in the recommendation system and improve the quality of recommendation.

[0100] It should be noted that the above embodiments of the present application predict the vector representation of the newly added words in the semantic primitive space based on the vector representation of the known words in the semantic primitive space in HowNet. In addition to the entity and attribute semantic primitive trees that describe words and concepts in HowNet, there are also time semantic primitive trees that describe events. Therefore, in other embodiments of the present application, the semantic primitive resources in the above embodiments are replaced with event semantic primitives, and the events can be represented in the semantic primitive space. In other words, the event semantic primitives can be used in combination with the above-described method to calculate the vector representation of the event in the semantic primitive space. The process is similar to the above-mentioned calculation of the vector representation of the word in the semantic primitive space, which will not be repeated here.

[0101] The word semantics prediction method based on semantic primitives provided in the above embodiments of the present application can be further combined with a recommendation system to solve the problem of understanding and expressing new words in the recommendation system, so as to improve the quality of recommendations.

[0102] Next, we will discuss the application of the vector representation of the new words in the semantic space in the recommendation system. For details, please refer to Fig. 9 , Fig. 9 The overall flow chart of the application of the vector representation of the newly added words in the semantic space in the embodiment of the present application in the recommendation system, first, in the offline system, the relevant new word mining process (not in Fig. 9(displayed in the text) to mine new words on the Internet in real time. These new words, through the device corresponding to the method described in the above embodiment of the present application, obtain the vector representation of the new words in the semantic space (i.e., the second vector described above), and the vector representation can characterize the semantics of the new word. After that, a user portrait is constructed based on the obtained vector representation and user behavior information (such as click situation). For example, the click situation refers to the situation in which the target user account clicks on the article containing the new word through the terminal device. For example, the click situation can be the total number of clicks within a preset time length, from which the user's interest in the article including the new word can be inferred, so that the recommendation system can recommend articles that meet the user's interests; the click situation can also be a click time period in which the click frequency reaches a preset threshold, from which it can be inferred that the user is more interested in reading in what time period, so that the recommendation system can push at the right time. It should be noted that in addition to the above two categories, the click situation can also be other situations. As long as it can provide additional information to the recommendation system for improving the recommendation effect, it belongs to the click situation described in this application, and the specific application is not limited. Finally, the user portrait and new word information are sent to the recommendation algorithm and used as the characteristics of the algorithm. The recommendation algorithm uses these features to ultimately generate recommendation results, and pushes the recommended content to the user's client (such as a mobile phone, personal computer, smart watch, etc.) based on the recommendation results.

[0103] It should be noted that in the embodiments of the present application, there are many methods for forming a user portrait by combining the vector representation of a word in the semantic space and the user behavior information. As an example, the user portrait can be formed by the method described in the following formula (6):

[0104]

[0105] in, The word w j The vector representation in the sememe space is, Is user U i The total number of clicks on articles containing word w within the preset time period is finally calculated. As user U i The user portrait is stored on the user portrait server.

[0106] Next, we will introduce the process of using user portraits. Fig.10 , Fig.10A schematic diagram of a user portrait provided for a recommendation system in an embodiment of the present application, first, obtain a user portrait from a user portrait library, which is a vector representation of a word in a semantic original space in the present application. Then, for candidate articles (such as news, information, etc.), extract the keywords of the candidate articles (there are many existing works on how to extract keywords, such as TFIDF method, TextRank method, etc., which are not within the scope of discussion of the present invention), and obtain the vector representation of these keywords in the semantic original space (which can be called the third vector), which is used to characterize the characteristics of the candidate articles. Next, calculate the correlation between the vector representation of the existing words in the semantic original space in the user portrait (i.e., the second vector described above) and the third vector of the keywords in the semantic original space in the candidate article, and determine the matching degree between the candidate article and the user portrait based on the obtained correlation, that is, obtain the matching degree between the current user and the candidate article, which can be used for recommendation sorting. Finally, when the matching degree reaches a preset threshold, the candidate article is recommended to the device currently logged in by the target user account. As an example, the matching degree can be ranked in order. Assuming that the matching degree is ranked in the top three, and the set threshold is to recommend the top three candidate articles with matching degree to the user, then it means that the matching degree reaches the preset threshold. As another example, the matching degree can be a specific value. In this case, the preset threshold is also a specific value. Assuming that the matching degree value is 0.8 (the total value is assumed to be 1), and the preset threshold is 0.6, then it means that the matching degree reaches the preset threshold.

[0107] It should be noted that the above recommendation system may also adopt a more complex recommendation system, such as a deep neural network, which may use a deeper neural network to calculate the relationship between users and items (eg, candidate articles).

[0108] In the above implementation of the present application, the semantic prediction method based on semantic primitives provided in the embodiment of the present application can be combined with the recommendation system to enhance the recommendation system's ability to understand new words, thereby enhancing the performance of the online recommendation system. The online recommendation system then pushes high-quality recommended content to the user's client to provide information services to the user, which can improve the quality of the recommendation system and user experience.

[0109] In addition, since we can also use the event semantics and combine the above method to calculate the vector representation of the event in the semantic space, we will now describe how to apply the vector representation of the event in the semantic space to event mining. Fig.11 , Fig.11The overall flow chart of the vector representation of the event in the semantic original space provided by the embodiment of the present application in the application of event mining and alarm, first, in the offline mining process, the event-related keywords are extracted for the input text (such as network articles, internal system logs, etc.), and the extraction method adopted can be a vocabulary matching method, a TFIDF method, a textrank method, etc. The keywords of the event are represented as vector representations in the event semantic original space, and then the vector representation of the input text in the event semantic original space is further obtained to represent the event represented by the text and stored in the event library. In the online detection process, the vector representation in the event semantic original space is obtained by the same method for the online input text, and then the event library is searched and matched, and finally appropriate actions are taken for the matching results. For example, for dangerous events in the internal system log, an alarm operation is performed; for known events in news articles, event merging operations are performed, etc.

[0110] In the above-mentioned implementation manner of the present application, the method described in the embodiment of the present application may be based on an event mining and detection method, which can mine and detect events in the text to achieve the function of early warning.

[0111] In addition to the business scenarios described above, the methods provided in the above embodiments of the present application can be applied to multiple other business scenarios, such as user portrait modules, recall modules, sorting modules in information flow recommendation systems, alarm event mining in network service logs, event monitoring in Huawei user forums, etc. Specific examples are not given here.

[0112] exist Figure 4 On the basis of the corresponding embodiments, in order to better implement the above solutions of the embodiments of the present application, the following also provides related devices for implementing the above solutions. Fig.12 , Fig.12A schematic diagram of a computer device provided in an embodiment of the present application, the computer device 1200 may specifically include: an acquisition module 1201, a first calculation module 1202 and a second calculation module 1203, wherein the acquisition module 1201 is used to obtain a first vector of a first word in a semantic primitive space, the first word is a defined word, the first vector is used to characterize the semantics of the first word, the semantic primitive space is constructed according to the category semantic primitive, the extended description and the association relationship, the association relationship is used to characterize the relationship between the category semantic primitive and the extended description, the first vector includes a first sub-vector and a second sub-vector, the first sub-vector is determined by the first category semantic primitive, The second sub-vector is determined by the first extended description and the first association relationship, and the second sub-vector is obtained through model learning. The first category semantic primitive is the category semantic primitive corresponding to the first word, the first extended description is the extended description of the first category semantic primitive, and the first association relationship is the association relationship between the first category semantic primitive and the first extended description; the first calculation module 1202 is used to calculate the correlation between the second word and the first word, and the second word is a word that is not defined by the HowNet; the second calculation module 1203 is used to obtain the second vector of the second word in the semantic primitive space according to the first vector and the correlation, and the second vector is used to represent the semantics of the second word.

[0113] In the above-mentioned embodiment of the present application, the computer device 1200 constructs a semantic representation space (i.e., a semantic original space) of words based on semantic originals, and represents known words (i.e., the first words) in the semantic original space. The vector representation of a word in the semantic original space consists of two parts, the value of one part is determined by the category semantic original of the word, and the value of the other part is determined by the extended description of the word and the association between the extended description and the category semantic original (the specific value of this part is obtained through model learning). Since the meaning of the semantic original is clear, the vector used to represent the words in the present application is interpretable; and, in the face of a newly added word (i.e., the second word), the closeness relationship (i.e., the correlation) between the newly added word and the known word can be calculated, and the vector representation of the newly added word in the semantic original space is derived from the vector representation of the known word in the semantic original space. The newly added words predicted by this method have strong representation capabilities.

[0114] In a possible design, the acquisition module 1201 is specifically used to: when a first word has n different semantics, obtain n target vectors corresponding to the n semantics in the semantic space, and one target vector is used to represent one semantic of the first word; and fuse the n target vectors to obtain the first vector.

[0115] In the above implementation of the present application, it is explained that when a known word has multiple semantics, the acquisition module 1201 can obtain a target vector for each semantics, and then fuse the target vectors to obtain the final first vector, which is feasible.

[0116] In a possible design, the acquisition module 1201 is further specifically used to: add the n target vectors and average them to obtain the first vector.

[0117] In the above-mentioned implementation of the present application, a method for fusing multiple target vectors is provided, and the fusing method is simple and easy to operate.

[0118] In a possible design, it is assumed that there are m first words. Since each first word corresponds to a first vector, each first word and the second word can calculate a correlation, so there are also m first vectors and m correlations. One first word corresponds to one first vector, and one first vector corresponds to one correlation. Therefore, the second calculation module 1203 is specifically used to: calculate the dynamic weight of each first word according to the value sorting of the m correlations, the dynamic weight is positively correlated with the correlation, one dynamic weight corresponds to one first vector, and multiply each first vector, the correlation corresponding to each first vector, and the dynamic weight corresponding to each first vector to obtain m product results, and finally add the m product results to obtain the second vector of the second word in the semantic original space.

[0119] In the above-mentioned embodiment of the present application, a method for implementing the second calculation module 1203 to obtain the second vector of the second word in the semantic original space according to the first vector and the correlation is specifically described, which has wide applicability.

[0120] In a possible design, the first calculation module 1202 is specifically used to: project each first word and the second word into a first point and a second point in a high-dimensional space vector space through a language model (such as word2vec, BERT, etc.), and each projected word can be represented as a point (i.e., a projection point) in the word vector space. Then, calculate the distance between the projection point (i.e., the second point) corresponding to the second word and the projection point (i.e., each first point) corresponding to each first word. For example, assuming that there are m first words, these m words can be projected into a point (i.e., a projection point) in the word vector space, then calculate the distance between each projection point of the second word and the projection points of the m first words, and obtain m distances. Each distance is used to characterize the value of the correlation between the second word and the corresponding first word, and the value of the distance is inversely correlated with the value of the correlation, that is, the smaller the distance value, the closer the association, and the larger the correlation value; the larger the distance value, the looser the association, and the smaller the correlation value.

[0121] In the above-mentioned embodiment of the present application, a method for the first calculation module 1202 to calculate the relevance between the second word and each first word is specifically described, and the similarity comparison problem is converted into a distance comparison problem of projection points in the word vector space, which is feasible.

[0122] In a possible design, the first computing module 1202 can also be specifically used to: project each first word and the second word into a first space vector and a second space vector in a high-dimensional word vector space through a language model (such as word2vec, bert, etc.), that is, each projected word can be represented as a space vector of the word vector space passing through the origin of the space, and then calculate the angle between the second space vector corresponding to the second word and the first space vector corresponding to each first word. For example, assuming there are m first words, these m words can all be projected into a point in the word vector space (i.e., a projection point), and each point can be represented as a space vector passing through the origin of the space. A space vector is a space vector, one word corresponds to one space vector, and then the angle between the space vector of the second word and the space vectors of the m first words can be calculated (for example, the cosine similarity between the two space vectors can be calculated), and m angles (for example, m cosine similarities) can be obtained. Each angle (for example, cosine similarity) is used to characterize the value of the correlation between the second word and the corresponding first word, and the value of the angle (for example, cosine similarity) is inversely correlated with the value of the correlation, that is, the smaller the value of the angle (for example, cosine similarity), the closer the association, and the larger the value of the correlation; the larger the value of the angle (for example, cosine similarity), the looser the association, and the smaller the value of the correlation.

[0123] In the above-mentioned embodiment of the present application, another method for the first calculation module 1202 to calculate the relevance between the second word and each first word is specifically described, which converts the similarity comparison problem into the angle comparison problem between vectors in the word vector space, and has flexibility.

[0124] In a possible design, the computer device may also include an application module 1204, which is used to: construct a user portrait based on the obtained second vector and user behavior information (such as click status). In the embodiment of the present application, the click status is the situation in which the target user account clicks on the article containing the second word through the terminal device; then, for the candidate articles (such as news, information, etc.), extract the keywords of the candidate articles, and obtain the vector representation of these keywords in the semantic space (which can be called the third vector), which is used to characterize the characteristics of the candidate articles. Next, calculate the correlation between the vector representation of the existing words in the user portrait in the semantic space (that is, the second vector mentioned above) and the third vector of the keywords in the candidate article in the semantic space, and determine the matching degree between the candidate article and the user portrait based on the obtained correlation, that is, obtain the matching degree between the current user and the candidate article, which can be used for recommendation sorting. Finally, when the matching degree reaches a preset threshold, the candidate article is recommended to the device currently logged in by the target user account. As an example, the matching degree can be ranked in order. Assuming that the matching degree is ranked in the top three, and the set threshold is to recommend the top three candidate articles with matching degree to the user, then it means that the matching degree reaches the preset threshold. As another example, the matching degree can be a specific value. In this case, the preset threshold is also a specific value. Assuming that the matching degree value is 0.8 (the total value is assumed to be 1), and the preset threshold is 0.6, then it means that the matching degree reaches the preset threshold.

[0125] In the above-mentioned embodiments of the present application, it is specifically explained that the word semantics prediction method based on semantic primitives provided in the above-mentioned embodiments of the present application can be further combined with the recommendation system to solve the new word understanding and expression problems of the recommendation system, so as to improve the recommendation quality.

[0126] In one possible design, the click situation can be the total number of clicks within a preset time period, from which the user's interest in the article containing the new word can be inferred, so that the recommendation system can recommend articles that meet the user's interests; the click situation can also be a click time period in which the click frequency reaches a preset threshold, from which it can be inferred that the user has a greater interest in reading in what time period, so that the recommendation system can push at the right time. It should be noted that in addition to the above two categories, the click situation can also be other situations. As long as it can provide additional information to the recommendation system to improve the recommendation effect, it belongs to the click situation described in this application, and this application does not limit it specifically.

[0127] In the above-mentioned implementation mode of the present application, the expression form of the click situation is explained, which is optional and the user can make a choice based on his or her own needs, thereby improving the user experience.

[0128] It should be noted that Fig.12The information interaction, execution process, etc. between the modules / units in the computer device 1200 described in the corresponding embodiment are the same as those in the present application. Figure 4 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0129] Next, a computer device provided by an embodiment of the present application is introduced. Fig.13 , Fig.13 A schematic diagram of a computer device provided in an embodiment of the present application, wherein the computer device 1300 may be deployed with Fig.12 The modules described in the corresponding embodiments are used to implement Fig.12 Functions of the computer device in the corresponding embodiment. The computer device 1300 is implemented by one or more servers. The computer device 1300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1322 (for example, one or more central processing units) and memory 1332, one or more storage media 1330 (for example, one or more mass storage devices) storing application programs 1342 or data 1344. Among them, the memory 1332 and the storage medium 1330 can be short-term storage or permanent storage. The program stored in the storage medium 1330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device 1300. Furthermore, the central processing unit 1322 can be configured to communicate with the storage medium 1330 to execute a series of instruction operations in the storage medium 1330 on the computer device 1300.

[0130] The computer device 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input and output interfaces 1358, and / or one or more operating systems 1341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0131] In the embodiment of the present application, the computer device 1300 can be used to execute Figure 4In the corresponding steps of the embodiment, for example, the central processor 1322 can be used to: construct a semantic primitive space based on the category semantic primitive, the extended description, and the relationship between the category semantic primitive and the extended description (i.e., the association relationship). Afterwards, the defined words (i.e., known words, also referred to as first words) are represented in the constructed semantic primitive space, and each known word can be represented as a vector (referred to as a first vector) in the semantic primitive space, and the first vector is composed of two parts, one part can be referred to as a first sub-vector, and the other part can be referred to as a second sub-vector, wherein the first sub-vector is determined by the category semantic primitive (referred to as the first category semantic primitive) related to the known word, the second sub-vector is related to the extended description (referred to as the first extended description) related to the known word, and the association relationship (referred to as the first association relationship) between the first extended description and the first category semantic primitive, and the values ​​of each element in the second sub-vector are specifically obtained through model learning, and the obtained first vector can be used to represent the semantics of the first word, and each defined word can be represented as a vector in the semantic primitive space. Afterwards, the correlation between the second word and each first word is calculated, and the second word is an undefined word. After obtaining the correlation between the second word and each first word, the vector representation of the second word in the semantic space (which may be called the second vector) can be obtained based on the first vector and correlation of each first word in the semantic space. The second vector is used to represent the semantics of the second word.

[0132] The central processing unit 1322 is used to execute Figure 4 Corresponding to any step in the embodiment. For specific content, please refer to the description in the method embodiment shown in the previous application, which will not be repeated here.

[0133] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program for performing signal processing. When the program is run on a computer, the computer executes the steps executed by the computer device in the description of the embodiment shown above.

[0134] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0135] Through the description of the above implementation mode, the technicians in the relevant field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0136] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0137] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, training equipment or data center to another website site, computer, training equipment or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a high-density digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid-state drive (solid statedisk, SSD)), etc.

Claims

1. A method for predicting word semantics based on semantic origin, characterized in that: include: Obtain a first vector of a first word in a semantic primitive space, wherein the first word is a defined word, the first vector is used to characterize the semantics of the first word, the semantic primitive space is constructed according to the category semantic primitive, the extended description and the associated relationship, each of the category semantic primitives corresponds to a dimension of the semantic primitive space, each of the extended description and the associated relationship related to the extended description as a whole corresponds to a dimension of the semantic primitive space, the associated relationship is used to characterize the relationship between the category semantic primitive and the extended description, the first vector includes a first subvector and a second subvector, the first subvector is a one-hot encoding one-hot representation, the second subvector is a dense embedding denseembedding representation, the first subvector is determined by the first category semantic primitive, the second subvector is determined by the first extended description and the first associated relationship, the second subvector is obtained through model learning, the first category semantic primitive is the category semantic primitive corresponding to the first word, the first extended description is the extended description of the first category semantic primitive, and the first associated relationship is the associated relationship between the first category semantic primitive and the first extended description; Calculating a correlation between a second word and the first word, where the second word is an undefined word; A second vector of the second word in the semantic primitive space is obtained according to the first vector and the correlation, and the second vector is used to represent the semantics of the second word.

2. The method according to claim 1, characterized in that The step of obtaining a first vector of the first word in the semantic space includes: In the case where the first word has n different semantics, obtaining n target vectors corresponding to the n different semantics in the semantic original space, where one target vector is used to represent one semantic of the first word; The n target vectors are fused to obtain the first vector.

3. The method according to claim 2, characterized in that The fusing the n target vectors to obtain the first vector comprises: The n target vectors are added and averaged to obtain the first vector.

4. The method according to any one of claims 1 to 3, characterized in that The number of the first words is m, the number of the first vectors is m, the number of the relevances is m, one first word corresponds to one first vector, one first vector corresponds to one relevance, m≥2, and obtaining the second vector of the second word in the semantic primitive space according to the first vector and the relevance includes: Calculate the dynamic weight of each first word according to the order of the values ​​of the m relevances, wherein the dynamic weight is positively correlated with the relevance; Multiply each first vector, the correlation corresponding to each first vector, and the dynamic weight of each first vector to obtain m product results; The m product results are added together to obtain a second vector of the second word in the semantic space.

5. The method according to any one of claims 1 to 4, characterized in that The calculating the relevance between the second word and the first word comprises: Projecting the first word and the second word into a first point and a second point in a word vector space through a language model; The distance between the first point and the second point is calculated, and the distance is used to represent the relevance between the second word and the first word, and the distance is inversely correlated with the relevance.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Projecting the first word and the second word into a first space vector and a second space vector passing through a space origin in a word vector space through a language model; An angle between the first space vector and the second space vector is calculated, where the angle is used to characterize the relevance between the second word and the first word, and the angle is inversely correlated with the relevance.

7. The method according to any one of claims 1 to 6, characterized in that After obtaining the second vector of the second word in the semantic space, the method further includes: Building a user portrait according to the second vector and the click situation, where the click situation is a situation in which the target user account clicks on the article containing the second word through a terminal device; Obtaining the third vector of the keyword of the candidate article in the sememe space; Calculating the correlation between the second vector and the third vector, and determining the matching degree between the candidate article and the user portrait according to the correlation; When the matching degree reaches a first preset threshold, the candidate article is recommended to the terminal device currently logged in by the target user account.

8. The method according to claim 7, characterized in that The click conditions include: The total number of clicks within a preset time period, or the click time period in which the click frequency reaches a second preset threshold.

9. A computer device, characterized in that: include: An acquisition module is used to acquire a first vector of a first word in a semantic primitive space, wherein the first word is a defined word, the first vector is used to characterize the semantics of the first word, the semantic primitive space is constructed according to a category semantic primitive, an extended description, and an associated relationship, each of the category semantic primitives corresponds to a dimension of the semantic primitive space, each of the extended descriptions and the associated relationship related to the extended descriptions as a whole corresponds to a dimension of the semantic primitive space, the associated relationship is used to characterize the relationship between the category semantic primitive and the extended description, the first vector includes a first subvector and a second subvector, the first subvector is represented by a one-hot encoding, the second subvector is represented by a dense embedding, the first subvector is determined by the first category semantic primitive, the second subvector is determined by the first extended description and the first associated relationship, the second subvector is obtained through model learning, the first category semantic primitive is the category semantic primitive corresponding to the first word, the first extended description is the extended description of the first category semantic primitive, and the first associated relationship is the associated relationship between the first category semantic primitive and the first extended description; A first calculation module, used for calculating the relevance between a second word and the first word, wherein the second word is an undefined word; The second calculation module is used to obtain a second vector of the second word in the semantic primitive space according to the first vector and the correlation, wherein the second vector is used to represent the semantics of the second word.

10. The device according to claim 9, characterized in that The acquisition module is specifically used for: In the case where the first word has n different semantics, obtaining n target vectors corresponding to the n different semantics in the semantic original space, wherein one target vector is used to represent one semantic of the first word; The n target vectors are fused to obtain the first vector.

11. The device according to claim 10, characterized in that The acquisition module is further specifically used for: The n target vectors are added and averaged to obtain the first vector.

12. The device according to any one of claims 9 to 11, characterized in that There are m first words, m first vectors, and m relevances, one first word corresponds to one first vector, one first vector corresponds to one relevance, m≥2, and the second calculation module is specifically used for: Calculate the dynamic weight of each first word according to the order of the values ​​of the m relevances, wherein the dynamic weight is positively correlated with the relevance; Multiply each first vector, the correlation corresponding to each first vector, and the dynamic weight corresponding to each first vector to obtain m product results; The m product results are added together to obtain a second vector of the second word in the semantic space.

13. The device according to any one of claims 9 to 12, characterized in that The first calculation module is specifically used for: Projecting the first word and the second word into a first point and a second point in a word vector space through a language model; The distance between the first point and the second point is calculated, and the distance is used to represent the relevance between the second word and the first word, and the distance is inversely correlated with the relevance.

14. The device according to any one of claims 9 to 12, characterized in that The first calculation module is further specifically used for: Projecting the first word and the second word into a first space vector and a second space vector passing through a space origin in a word vector space through a language model; An angle between the first space vector and the second space vector is calculated, where the angle is used to characterize the relevance between the second word and the first word, and the angle is inversely correlated with the relevance.

15. The device according to any one of claims 9 to 14, characterized in that The device further comprises an application module, wherein the application module is configured to: Building a user portrait according to the second vector and the click situation, where the click situation is a situation in which the target user account clicks on the article containing the second word through a terminal device; Obtaining the third vector of the keyword of the candidate article in the sememe space; Calculating the correlation between the second vector and the third vector, and determining the matching degree between the candidate article and the user portrait according to the correlation; When the matching degree reaches a first preset threshold, the candidate article is recommended to the terminal device currently logged in by the target user account.

16. The device according to claim 15, characterized in that The click conditions include: The total number of clicks within a preset time period, or the click time period in which the click frequency reaches a second preset threshold.

17. A computer device, comprising a memory and one or more processors, wherein the one or more processors are coupled to the memory, and the computer device is used as a source device, characterized in that: The memory is used to store programs; The one or more processors are configured to execute the program in the memory so that the computer device executes the method according to any one of claims 1 to 8.

18. A computer-readable storage medium comprising computer-readable instructions, characterized in that: When the computer-readable instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.

19. A computer program product comprising computer readable instructions, characterized in that: When the computer-readable instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.

20. A chip, characterized in that: The chip includes a memory and one or more processors, and the chip is used to read a computer program stored in the memory so that the one or more processors execute the method according to any one of claims 1 to 8.

Images