A neural zero-shot fine-grained entity classification method based on type attention

Abstract

The present invention discloses a neural zero-shot fine-grained entity classification method based on type attention, which is specifically as follows: based on the word vectors corresponding to each word in the target entity text, a representation vector of the target entity text is calculated; based on the context word vectors corresponding to each word on both sides of the target entity text in the sentence, a basic context vector of the target entity text is obtained; based on the types corresponding to each entity in the target entity text, an entity type vector is constructed; based on the target entity type vector and the basic context vector, relevant attention values ​​corresponding to the words on both sides are calculated; based on the relevant attention value and the basic context vector, a relevant context vector of the target entity text is calculated; the entity representation vector and the relevant context vector of the target entity text are merged to obtain the representation vector of the entire sentence, and the score of the target entity text belonging to each given category is obtained by using the created entity type classifier model, and the effect of fine-grained entity classification is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to a neural zero-shot fine-grained entity classification method based on type attention. Background Art

[0002] The role of entity classification is to identify the semantic type of entity text, such as person name, place name, organization name, etc. This role helps to accurately locate entities in the text and is of great significance to many other tasks in natural language processing.

[0003] Entity recognition is a key basic task in the field of information extraction. The output of entity recognition can be used as input features for many downstream natural language processing tasks to improve the performance of downstream tasks, such as relation extraction, question answering, topic modeling, etc.

[0004] Currently, researchers have found that finer-grained entity types will have a greater improvement on downstream tasks. Therefore, researchers have refined the granularity of entity types, expanded the number of entity types, and developed fine-grained entity classification tasks (Fine-Grained Entity Typing).

[0005] Early methods for coarse-grained entity recognition and fine-grained entity classification tasks mainly rely on manual features. Since manual features are time-consuming and labor-intensive, and have a limited scope of application, researchers have used deep learning to replace traditional manual feature methods in recent years. On the other hand, in many cases, training models requires a large amount of labeled data, and the cost of manually labeled data is very high. Entity classification and recognition with few or even zero samples has also become a topic that needs to be studied. In addition, since there are many types of fine-grained entities, and one entity mention may have multiple types of fine-grained entities, the sequence annotation model used for coarse-grained entity recognition is not suitable for end-to-end fine-grained entity recognition, which is also a problem that needs to be solved. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects in the prior art and provide a neural zero-shot fine-grained entity classification method based on type attention, so as to effectively improve the stability of the classification model and effectively improve the effect of fine-grained entity classification.

[0007] The specific technical solutions adopted by the present invention are as follows:

[0008] A neural zero-shot fine-grained entity classification method based on type attention, comprising the following steps:

[0009] S1: Calculate the entity representation vector of the target entity text based on the word vectors corresponding to each word in the target entity text;

[0010] S2: Based on the context word vectors corresponding to the words on both sides of the target entity text in the sentence where the target entity text is located, obtain the basic context vector of the target entity text;

[0011] S3: construct a target entity type vector based on the types corresponding to each entity in the target entity text;

[0012] S4: Based on the target entity type vector and the basic context vector, the attention values ​​corresponding to the words on both sides of the target entity text are calculated, and then the attention values ​​are normalized to obtain normalized attention values;

[0013] S5: Calculate the relevant context vector of the target entity text based on the normalized attention value and the basic context vector;

[0014] S6: Merge the entity representation vector and related context vector of the target entity text to obtain the representation vector of the entire sentence; based on the representation vector of the sentence, use the created entity type classifier model to obtain the score of the target entity text belonging to each given category.

[0015] Preferably, S1 is specifically as follows: obtaining an average value of word vectors corresponding to each word in the target entity text, and using the average value as the entity representation vector of the target entity text.

[0016] Preferably, it is characterized in that said S2 is specifically as follows:

[0017] The front word vectors corresponding to the front words of the target entity text in the sentence where the target entity text is located and the rear word vectors corresponding to the rear words are respectively input into the bidirectional recurrent neural network to obtain the forward basic context vectors corresponding to the front words and the backward basic context vectors corresponding to the rear words.

[0018] Furthermore, it is characterized in that the bidirectional recurrent neural network is a bidirectional LSTM recurrent neural network.

[0019] Preferably, the specific steps of S3 are as follows:

[0020] For each entity type in the target entity text, select entity mention as the prototype of the entity type; use the average of the sum of the pre-trained word embedding vectors of the prototypes in the prototype set as the embedded representation of the initial entity type; then use the embedded representation of the initial entity type to form a matrix, combined with the hierarchical information of the entity type, to obtain the embedded representation of the entity type as the target entity type vector.

[0021] Preferably, the specific steps of S4 are as follows:

[0022] Based on the forward basic context vector and the target entity type vector, the forward knowledge base related attention values ​​corresponding to each of the front words are calculated, as follows:

[0023]

[0024] In the formula, Represents the forward basic context vector corresponding to the i-th word adjacent to the front side of the target entity text in the sentence where the target entity text is located; Indicates the type t corresponding to the i-th word in front of the target entity text in the sentence where the target entity text is located k The attention value; is a bilinear matrix, with entity type t k The more relevant the word, the higher its weight will be;

[0025] Then the attention value Perform normalization:

[0026]

[0027] In the formula, exp(x) represents the exponential operation of x. Indicates the type t corresponding to the i-th word in front of the target entity text in the sentence where the target entity text is located k The normalized attention value, C represents a fixed value of the number of forward basis context vectors;

[0028] Based on the backward basic context vector and the target entity type vector, the backward knowledge base related attention values ​​corresponding to each word on the rear side are calculated using a given non-negative function formula, and the specific method is the same as the forward knowledge base related attention values.

[0029] Furthermore, the specific steps of S5 are as follows:

[0030] The forward knowledge base related attention value and the backward basic context vector are used as corresponding weights, the forward basic context vector and the backward basic context vector are weighted and averaged to calculate the relevant context vector of the target entity text.

[0031] Preferably, the specific steps of creating the entity type classifier model in S6 are as follows:

[0032] First, the entity representation vector r of the target entity text is merged according to the following formula: m and the relevant context vector r c,k , get the representation vector c of the entire sentence,

[0033]

[0034] Then, based on the representation vector c of the entire sentence, an initial classifier model is established according to the nonlinear layer function and the ranking function; the pairwise weighted approximation loss of all dimensions of the classifier model is used as the optimization target, the initial classifier model is trained using given training samples, and the optimization target is optimized by the gradient descent method to obtain the entity type classifier model.

[0035] Furthermore, the steps of creating the entity type classifier model are as follows:

[0036] S61: Map the representation vector c of the sentence where the target entity text is located to a vector space of a certain dimension

[0037] r=B·c

[0038] In the formula, r represents the sentence representation vector after mapping, and D is the vector space dimension; B represents the mapping matrix, which is used to map the sentence representation vector to a vector space of a certain dimension;

[0039] S62: Map the target entity type vector to the vector space

[0040]

[0041] Wherein, t represents the target entity type vector after mapping; A represents the mapping matrix, which is used to map the target entity type vector to a vector space of a certain dimension;

[0042] S63: Calculate the dot product of the mapped sentence representation vector r and the mapped target entity type vector t to represent the similarity between the entity mention and context representation and the entity type representation.

[0043] s=r·t

[0044] In the formula, s represents the similarity between the entity mention and the context representation and the entity type representation. The higher the similarity, the higher the match between the entity mention and the entity type.

[0045] S64: Taking the pairwise weighted approximate loss of the classifier model as the optimization target, using given training samples to train the classifier model, optimizing the optimization target by gradient descent, and obtaining the entity type classifier model, as follows:

[0046]

[0047]

[0048]

[0049] Where J(θ) represents the optimization objective, t is the set of positive categories for a given entity mention m, is the set of negative categories for a given entity mention m; Y and are the positive category set and the negative category set respectively; max(a,b) represents the larger value of a and b; k is the intermediate variable.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention adopts a neural zero-shot fine-grained entity classification model based on an entity type attention mechanism. The model uses an entity type-based attention mechanism to enable the model to focus on contextual information related to the required entity type. At the same time, a learning to rank method is used to replace the traditional classification model. Combined with the embedded representation of entity type, zero-shot entity classification is performed. The model can simultaneously predict fine-grained entity types that have been seen and not seen in the training data, and has good robustness to data noise, effectively improving the stability of the model, improving the effect of entity fine classification, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of calculating the attention values ​​corresponding to the words on both sides of the target entity text in an embodiment of the present invention;

[0053] Figure 2 The present invention is a flowchart for creating an entity type classifier model in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described and illustrated below in conjunction with the accompanying drawings and specific embodiments. The technical features of each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0055] As an embodiment of the present invention, this embodiment provides a neural zero-shot fine-grained entity classification method based on type attention, comprising the following steps:

[0056] S1: Based on the word vectors corresponding to each word in the target entity text, calculate the entity representation vector of the target entity text.

[0057] The details are as follows: the average value of the word vectors corresponding to each word in the target entity text is obtained and used as the entity representation vector of the target entity text.

[0058] S2: Based on the context word vectors corresponding to the words on both sides of the target entity text in the sentence where the target entity text is located, obtain the basic context vector of the target entity text.

[0059] Specifically, the front word vectors corresponding to the front words of the target entity text in the sentence where the target entity text is located and the back word vectors corresponding to the back words are respectively input into the bidirectional recurrent neural network to obtain the forward basic context vectors corresponding to the front words and the backward basic context vectors corresponding to the back words. The bidirectional recurrent neural network is a bidirectional LSTM recurrent neural network.

[0060] S3: Based on the types corresponding to each entity in the target entity text, construct the target entity type vector. The specific steps are as follows:

[0061] For each entity type in the target entity text, select entity mention as the prototype of the entity type; use the average of the sum of the pre-trained word embedding vectors of the prototypes in the prototype set as the embedded representation of the initial entity type; then use the embedded representation of the initial entity type to form a matrix, combined with the hierarchical information of the entity type, to obtain the embedded representation of the entity type as the target entity type vector.

[0062] S4: Based on the target entity type vector and the basic context vector, the attention values ​​corresponding to the words on both sides of the target entity text are calculated, and then the attention values ​​are normalized to obtain normalized attention values.

[0063] The specific steps are as follows:

[0064] Based on the forward basic context vector and the target entity type vector, the forward knowledge base related attention values ​​corresponding to each of the front words are calculated, as follows:

[0065]

[0066] In the formula, Represents the forward basic context vector corresponding to the i-th word adjacent to the front side of the target entity text in the sentence where the target entity text is located; Indicates the type t corresponding to the i-th word in front of the target entity text in the sentence where the target entity text is located k The attention value; is a bilinear matrix, with entity type t k The more relevant the word, the higher its weight will be;

[0067] Then the attention value Perform normalization:

[0068]

[0069] In the formula, exp(x) represents the exponential operation of x. Indicates the type t corresponding to the i-th word in front of the target entity text in the sentence where the target entity text is located k The normalized attention value, C represents a fixed value of the number of forward basis context vectors;

[0070] Based on the backward basic context vector and the target entity type vector, the backward knowledge base related attention values ​​corresponding to each word on the rear side are calculated using a given non-negative function formula, and the specific method is the same as the forward knowledge base related attention values.

[0071] S5: Based on the normalized attention value and the basic context vector, calculate the relevant context vector of the target entity text.

[0072] The specific steps are as follows:

[0073] The forward knowledge base related attention value and the backward basic context vector are used as corresponding weights, the forward basic context vector and the backward basic context vector are weighted and averaged to calculate the relevant context vector of the target entity text.

[0074] S6: Merge the entity representation vector and related context vector of the target entity text to obtain the representation vector of the entire sentence; based on the representation vector of the sentence, use the created entity type classifier model to obtain the score of the target entity text belonging to each given category.

[0075] The specific steps to create an entity type classifier model in S6 are as follows:

[0076] First, the entity representation vector r of the target entity text is merged according to the following formula: m and the relevant context vector r c,k , get the representation vector c of the entire sentence,

[0077]

[0078] Then, based on the representation vector c of the entire sentence, an initial classifier model is established according to the nonlinear layer function and the ranking function; the pairwise weighted approximate loss of all dimensions of the classifier model is used as the optimization target, the initial classifier model is trained using a given training sample, and the optimization target is optimized by the gradient descent method to obtain the entity type classifier model, specifically:

[0079] S61: Map the representation vector c of the sentence where the target entity text is located to a vector space of a certain dimension

[0080] r=B·c

[0081] In the formula, r represents the sentence representation vector after mapping, and D is the vector space dimension; B represents the mapping matrix, which is used to map the sentence representation vector to a vector space of a certain dimension;

[0082] S62: Map the target entity type vector to the vector space

[0083]

[0084] Wherein, t represents the target entity type vector after mapping; A represents the mapping matrix, which is used to map the target entity type vector to a vector space of a certain dimension;

[0085] S63: Calculate the dot product of the mapped sentence representation vector r and the mapped target entity type vector t to represent the similarity between the entity mention and context representation and the entity type representation.

[0086] s=r·t

[0087] In the formula, s represents the similarity between the entity mention and the context representation and the entity type representation. The higher the similarity, the higher the match between the entity mention and the entity type.

[0088] S64: Taking the pairwise weighted approximate loss of the classifier model as the optimization target, using given training samples to train the classifier model, optimizing the optimization target by gradient descent, and obtaining the entity type classifier model, as follows:

[0089]

[0090]

[0091]

[0092] Where J(θ) represents the optimization objective, t is the set of positive categories for a given entity mention m, is the set of negative categories for a given entity mention m; Y and are the positive category set and the negative category set respectively; max(a,b) represents the larger value of a and b; k is the intermediate variable.

[0093] The following is a detailed description of each step:

[0094] Step S1 can be understood as: for the target entity to be classified, it corresponds to the corresponding target entity text, which is usually composed of one or more words. In order to convert natural language into a language that can be recognized by a computer, the words in the entity library are usually represented as a vector form, such as a one-hot Representation word vector, a distributed Representation word vector, etc., to form a word vector library.

[0095] In actual processing, the word vector library is queried according to each word contained in the target entity text, the word vector corresponding to each constituent word is obtained, and the representation vector corresponding to the target entity text is calculated according to each word vector, that is, the entity representation vector.

[0096] The step S1 specifically includes: obtaining the average value of the word vectors corresponding to each word in the target entity text, and using it as the entity representation vector of the target entity text.

[0097] It can be understood as first looking up the table according to each word corresponding to the target entity text to obtain the pre-trained word vectors corresponding to each component word. For each group of word vectors, the vector average is calculated, that is, the average value of the word vectors corresponding to each word is calculated as the entity representation vector according to the following formula:

[0098]

[0099] In the formula, is the entity representation vector, r represents the entity representation vector corresponding to the target entity, and t k Indicates the position of the word corresponding to the target entity text in the text, v i represents the word vector corresponding to the i-th word in the target entity text, and n represents the total number of words contained in the target entity text.

[0100] Step S2 can be understood as that, for a single entity text, it may usually represent different meanings and therefore be classified into different categories. Therefore, the specific meaning and classification of the target entity text need to be determined in combination with the specific context of the sentence in which the target entity text is located. For the sentence in which the target entity text is located, the words that appear before the target entity text represent the past state, and the words that appear after the target entity text will represent the future state.

[0101] In this step, the word vectors corresponding to the words before and after the target entity text in the sentence where the target entity text is located are determined by table lookup or other methods, namely, the context word vectors. Then, the context word vectors on both sides of the target entity text are used to output the vector representations of the context corresponding to the context words in the two directions respectively through a bidirectional recurrent neural network, namely, the basic context vectors.

[0102] The step S2 includes: inputting the front word vectors corresponding to the front words of the target entity text in the sentence where the target entity text is located and the rear word vectors corresponding to the rear words into the bidirectional recurrent neural network, respectively, to obtain the forward basic context vectors corresponding to the front words and the backward basic context vectors corresponding to the rear words.

[0103] It can be understood that first, the context words in the sentence except the target entity text are looked up in the table to obtain the pre-trained word vectors corresponding to each context word. The context words include the words in front of the target entity and the words in the back of the target entity, and the pre-trained word vectors include the forward word vectors corresponding to the front words and the backward word vectors corresponding to the back words.

[0104] In one embodiment, the bidirectional recurrent neural network is further specifically a bidirectional LSTM recurrent neural network. It can be understood that in the calculation of the basic context vector in the above embodiment, a long short-term memory bidirectional recurrent neural network, i.e., a bidirectional LSTM recurrent neural network, is specifically used. The LSTM network can memorize long-term information, which is of great significance to the stability of the network.

[0105] Step S3 can be understood as, for each entity type, selecting some entity mentions as the prototype of this entity type, and using the average of the sum of the pre-trained word embedding vectors of the prototypes in its prototype set as the embedded representation of the initial entity type. The matrix composed of the acquired entity type embedded representations is combined with the hierarchical information of the entity type to obtain the embedded representation of the entity type.

[0106] S3 specifically includes:

[0107] S31, for each entity type, select some entity mentions as the prototype of this entity type, and do not need the sentence where the entity mentions are located. For entity types in the training set, manually sample some entity mentions of this type in the training set as the prototype of this entity type. For entity types that do not exist in the training set, according to common sense, manually write out 20 entity mentions of this type as the prototype of this entity type.

[0108] S32, uses the average of the sum of the pre-trained word embedding vectors of the prototypes in the prototype set of the entity type as the embedded representation of the initial entity type.

[0109]

[0110] In the formula, v i represents the i-th word in the entity type prototype set, Embedded representation of entity type, tk Indicates the position of the entity type, and n indicates the number of entity mentions.

[0111] S33, using the matrix consisting of the acquired initial entity type embedded representations and combining it with the hierarchical information of the entity type, to acquire the embedded representation of the entity type.

[0112] It can be understood as mapping each initial entity type to a row of the matrix S. If the entity type corresponding to the element in the row exists in the hierarchical path of the entity type represented by this row, the element is 1, otherwise it is 0. For example, the row of entity type / person is [1,0,…], the row of / person / artist type is [1,1,0,…], and the row of / person / artist / actor is [1,1,1,0,…]. Using the matrix Let d be the dimension of the embedded representation of the entity, K is the number of entity types, and each row of the matrix is ​​the embedded representation of an entity. is a binary sparse matrix used to encode the hierarchical information of entity types. The matrix W represents the embedded representation of entity types with added hierarchical information. The concatenated matrix, W, is calculated as follows:

[0113] W T =V T S T

[0114] Step S4 can be understood as, after obtaining the basic context vector of the sentence where the target entity text is located according to the above steps, considering that the embedded representation of the entity type has a certain attention response, when processing the basic context vector, the embedded representation of the entity type is used as the query vector of the attention mechanism. Specifically, according to the above forward basic context vector and backward basic context vector in the sentence, and the knowledge base representation vector of the target entity text, the relevant attention values ​​of the embedded representation of the entity type corresponding to each word in the two directions are calculated respectively.

[0115] The further processing steps of S4 refer to Figure 1 , is a flowchart of calculating the attention value related to the embedded representation of the entity type corresponding to each word on both sides according to an embodiment of the present invention, specifically including:

[0116] S41, based on the forward basic context vector and the entity type embedded representation vector, calculate the forward entity type related attention values ​​corresponding to the preceding words respectively.

[0117] It can be understood that, according to the above embodiment, the entity type-related attention values ​​corresponding to each word in the two directions are calculated respectively. This step specifically calculates the entity type-related attention value of the front side of the target entity text.

[0118] For the forward basic context vector in the above embodiment Assume that the representation vector of the entity type is The entity type is t k , l indicates that the vector is a forward basic context vector, and the forward entity type related attention value is calculated using the following formula:

[0119]

[0120] In the formula, represents the forward basis vector corresponding to the i-th word adjacent to the front side of the target entity in the sentence, Indicates the type t corresponding to the i-th word adjacent to the front side of the target entity text in the sentence k The attention value, is a bilinear matrix, with entity type t k The more relevant the word, the higher its weight will be. Then the attention value is normalized:

[0121]

[0122] Where exp(x) represents the exponential operation on x. Indicates the type t of the i-th word in front of the target entity k The normalized attention value, C represents a fixed value of the number of forward basis context vectors.

[0123] S42, based on the backward basic context vector and the entity type embedded representation vector, using the bilinear matrix, calculate the backward entity type related attention values ​​corresponding to the subsequent words respectively.

[0124] It can be understood that, similar to the above step, this step specifically calculates the backward entity type-related attention value at the back of the target entity text. Assume that the representation vector of the entity type is The entity type is t k , r indicates that the vector is changed to the backward basic context vector, and the backward entity type related attention value is calculated using the following formula:

[0125]

[0126] In the formula, Represents the backward basis vector corresponding to the i-th word adjacent to the target entity in the sentence, Indicates the type t corresponding to the i-th word adjacent to the target entity text in the sentence k The attention value, is a bilinear matrix, with entity type t k The more relevant the word, the higher its weight will be. Then the attention value is normalized:

[0127]

[0128] Where exp(x) represents the exponential operation on x. Indicates the type t of the i-th word behind the target entity k The normalized attention value, C represents a fixed value of the number of forward basis context vectors.

[0129] Step S5 can be understood as, when calculating the embedded representation vector of the target entity text with respect to a given entity type according to the above embodiment, weighted summation of the forward basic context vectors corresponding to each word on the front side and the backward basic context vectors corresponding to each word on the back side is performed to calculate the relevant context vector of the target entity text, and the calculation is specifically performed using the following formula:

[0130]

[0131] In the formula, r c,k A context vector representing the target entity.

[0132] Step S6 can be understood as that, for a given target entity, its classification is affected by the corresponding target entity text and the contextual environment of the sentence in which it is located. Therefore, the entity representation vector of the target entity text and the knowledge base-related context vector of the context words in the sentence in which it is located are merged, and the composed vector is used as the sentence representation vector of the sentence. That is, the entity representation vector corresponding to the target entity obtained in the above step and the relevant context vector of the target entity text are merged to form the following sentence:

[0133]

[0134] In the formula, c represents the sentence representation vector of the sentence where the target entity text is located, r m is the entity representation vector, r c , k is the relevant context vector of the target entity text.

[0135] Then, the sentence representation vector is used as the input vector of the classifier model, and the classifier model is used to calculate the score of the output entity text belonging to a given category.

[0136] An optional step for establishing the target classifier model in step S6 is as follows: Figure 2, is a flow chart of establishing a target classifier model according to an embodiment of the present invention, including:

[0137] S61, based on the sentence representation vector, mapping the sentence representation vector where the target entity text is located to a vector space of a certain dimension Where D is the dimension of the vector space:

[0138] r=B·c

[0139] In the formula, r represents the mapped sentence representation vector, and B represents the mapping matrix, which maps the sentence representation vector to a vector space of a certain dimension.

[0140] S62, based on the embedded representation vector of the entity type, mapping the embedded representation vector of the entity type to a vector space of a certain dimension Where D is the dimension of the vector space:

[0141]

[0142] In the formula, t represents the embedded representation vector of the mapped entity type, and A represents the mapping matrix, which maps the embedded representation vector of the entity type to a vector space of a certain dimension.

[0143] S63, calculate the dot product of the mapped sentence representation vector and the embedded representation vector representing the mapped entity type to represent the similarity between the entity mention and context representation and the entity type representation:

[0144] s=r·t

[0145] Where s represents the similarity between the entity mention and the context representation and the entity type representation. The higher the similarity, the more likely the entity mention belongs to the entity type.

[0146] S64, taking the pairwise weighted approximate loss of the classifier model as the optimization target, using given training samples to train the classifier model, optimizing the optimization target by gradient descent, and obtaining the target classifier model.

[0147] It can be understood that after obtaining the expression of the classifier model according to the above steps, the classifier model is trained using the given training samples. Specifically, the gradient descent method is used to optimize the parameters of the model, and the optimization target is the pairwise weighted approximation loss of the classifier model, which is expressed as follows:

[0148]

[0149]

[0150]

[0151] Where J(θ) represents the optimization objective, t is the set of positive categories for a given entity mention m, is the set of negative categories for a given entity mention m, where Y, are the positive category set and the negative category set respectively, max(a,b) represents the larger one of the values ​​a and b, s is the similarity function mentioned above; k is the intermediate variable.

[0152] An embodiment of the present invention provides a neural zero-shot fine-grained entity classification method based on type attention, which adopts a neural zero-shot fine-grained entity classification model based on type attention, while taking into account the effectiveness in the zero-shot case and the consistency of training and testing, which can effectively improve the stability of the model and improve the effect of fine-grained entity classification and has strong practicality.

[0153] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A neural zero-shot fine-grained entity classification method based on type attention, characterized in that The steps include: S1: Calculate the entity representation vector of the target entity text based on the word vectors corresponding to each word in the target entity text; S2: Based on the context word vectors corresponding to the words on both sides of the target entity text in the sentence where the target entity text is located, obtain the basic context vector of the target entity text; S3: construct a target entity type vector based on the types corresponding to each entity in the target entity text; S4: Based on the target entity type vector and the basic context vector, the attention values ​​corresponding to the words on both sides of the target entity text are calculated, and then the attention values ​​are normalized to obtain normalized attention values; S5: Calculate the relevant context vector of the target entity text based on the normalized attention value and the basic context vector; S6: merging the entity representation vector and the relevant context vector of the target entity text to obtain the representation vector of the entire sentence; based on the representation vector of the sentence, using the created entity type classifier model, obtaining the score of the target entity text belonging to each given category; The specific steps of creating the entity type classifier model in S6 are as follows: First, the entity representation vector r of the target entity text is merged according to the following formula: m and the relevant context vector r c,k , get the representation vector c′ of the entire sentence, Then, based on the representation vector c′ of the entire sentence, an initial classifier model is established according to a nonlinear layer function and a ranking function; the pairwise weighted approximation loss of all dimensions of the classifier model is used as an optimization target, the initial classifier model is trained using a given training sample, and the optimization target is optimized by a gradient descent method to obtain the entity type classifier model; The steps for creating the entity type classifier model are as follows: S61: Map the representation vector c′ of the sentence where the target entity text is located to a vector space of a certain dimension r=B·c′ In the formula, r represents the sentence representation vector after mapping, and D is the vector space dimension; B represents the mapping matrix, which is used to map the sentence representation vector to a vector space of a certain dimension; S62: Map the target entity type vector to the vector space Wherein, t_v represents the target entity type vector after mapping; A represents the mapping matrix, which is used to map the target entity type vector to a vector space of a certain dimension; S63: Calculate the dot product of the mapped sentence representation vector r and the mapped target entity type vector t_v to represent the similarity between the entity mention and context representation and the entity type representation. s=r·t_v In the formula, s represents the similarity between the entity mention and the context representation and the entity type representation. The higher the similarity, the higher the match between the entity mention and the entity type. S64: Taking the pairwise weighted approximate loss of the classifier model as the optimization target, using given training samples to train the classifier model, optimizing the optimization target by gradient descent, and obtaining the entity type classifier model, as follows: Where J(θ) represents the optimization objective, t is the set of positive categories for a given entity mention m, is the set of negative categories for a given entity mention m; Y and are the positive category set and the negative category set respectively; max(a,b) represents the larger value of a and b; x is any intermediate variable.

2. The neural zero-shot fine-grained entity classification method according to claim 1, characterized in that: The S1 is specifically as follows: obtaining the average value of the word vectors corresponding to each word in the target entity text, and using it as the entity representation vector of the target entity text.

3. The neural zero-shot fine-grained entity classification method according to claim 1, characterized in that: The S2 is specifically as follows: The front word vectors corresponding to the front words of the target entity text in the sentence where the target entity text is located and the rear word vectors corresponding to the rear words are respectively input into the bidirectional recurrent neural network to obtain the forward basic context vectors corresponding to the front words and the backward basic context vectors corresponding to the rear words.

4. The neural zero-shot fine-grained entity classification method according to claim 3, characterized in that: The bidirectional recurrent neural network is a bidirectional LSTM recurrent neural network.

5. The neural zero-shot fine-grained entity classification method according to claim 1, characterized in that: The specific steps of S3 are as follows: For each entity type in the target entity text, select entity mention as the prototype of the entity type; use the average of the sum of the pre-trained word embedding vectors of the prototypes in the prototype set as the embedded representation of the initial entity type; then use the embedded representation of the initial entity type to form a matrix, combined with the hierarchical information of the entity type, to obtain the embedded representation of the entity type as the target entity type vector.

6. The neural zero-shot fine-grained entity classification method according to claim 3, characterized in that: The specific steps of S4 are as follows: Based on the forward basic context vector and the target entity type vector, the forward knowledge base related attention values ​​corresponding to each of the front words are calculated, as follows: In the formula, Represents the forward basic context vector corresponding to the i-th word adjacent to the front side of the target entity text in the sentence where the target entity text is located; Indicates the type t corresponding to the i-th word in front of the target entity text in the sentence where the target entity text is located k The attention value; is a bilinear matrix, with entity type t k The more relevant the word, the higher its weight will be; Then the attention value Perform normalization: In the formula, exp(x) represents the exponential operation of x. Indicates the type t corresponding to the i-th word in front of the target entity text in the sentence where the target entity text is located k The normalized attention value, C represents a fixed value of the number of forward basis context vectors; Based on the backward basic context vector and the target entity type vector, the backward knowledge base related attention values ​​corresponding to each word on the rear side are calculated using a given non-negative function formula, and the specific method is the same as the forward knowledge base related attention values.

7. The neural zero-shot fine-grained entity classification method according to claim 6, characterized in that: The specific steps of S5 are as follows: The forward knowledge base related attention value and the backward knowledge base related attention value are used as corresponding weights, the forward basic context vector and the backward basic context vector are weighted and averaged to calculate the relevant context vector of the target entity text.

Images