A method for associating entities in text to a knowledge base using context

Abstract

The application relates to a method for associating entities in text to a knowledge base by using context, and belongs to the technical field of natural language processing. The method comprises the following steps: establishing a knowledge base; establishing a knowledge base index; preprocessing characters of entity references in input text, obtaining a candidate entity set corresponding to the entity references; establishing a pair of <reference, entity>; selecting multiple candidate entity description sentences with high context relevance to the entity references as selection basis; and judging entity nouns in the knowledge base corresponding to the entity references or judging that there is no entity noun in the knowledge base corresponding to the entity references according to the selection basis. The method provided by the application can expand the matching range and effectively improve the accuracy of association; the MLP multi-layer perception model which cannot link the classification tags is added at the prediction result mark to judge the entity nouns corresponding to the entity references, so that the problem that, in the prior art, when the entity is not in the knowledge base, only the item with the maximum association degree is found to cause an error association is solved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method for associating entities in text with a knowledge base using context. Background Technology

[0002] Natural language text, as the primary form of data carrier on the internet today, including manually written news reports, explanatory analyses, etc., exists in greater quantity and contains more information than structured data. Entities, often as words corresponding to real-world objects and concepts, are the core of querying and analysis. Therefore, processing entity nouns is an important method in modern data processing. Recommending words semantically related to entity nouns can effectively provide users with data navigation capabilities. However, entity nouns suffer from ambiguity and polysemy, and cannot be directly linked to specific entries in the knowledge base.

[0003] Therefore, the existing technology has the following problems: abbreviations appearing in the text are difficult to find in the knowledge base by traditional retrieval; there are generally not many parts that are related to the reference in the context, and the general alignment does not refine the entity description sentences in the knowledge base, which reduces the relevance between the correct entity and the reference; traditional entities are judged by thresholding, which can only consider whether they are related separately; entities may not necessarily appear in the knowledge base, and simply finding the item with the highest relevance can lead to incorrect associations. Summary of the Invention

[0004] The present invention aims to provide a method for associating entities in text with a knowledge base using context, in order to overcome the shortcomings of the prior art. The technical problem to be solved by the present invention is achieved through the following technical solution.

[0005] This invention provides a method for associating entities in text with a knowledge base using context, the method comprising:

[0006] Acquire internet knowledge encyclopedia data and build a knowledge base that includes a dictionary collection of entity nouns of various categories;

[0007] Establish a knowledge base index corresponding to each entity noun in the knowledge base;

[0008] The characters representing entities in the input text are preprocessed and decomposed into multiple query characters. The query characters are then input into the knowledge base for searching, and a set of candidate entities containing K candidate entities in the knowledge base corresponding to the entity reference is obtained.

[0009] Establish K <label, entity> pairs consisting of entity labels and K candidate entities from the corresponding candidate entity set;

[0010] Calculate the feature vector of each candidate entity description and the feature vector of the entity reference context in each of the K <reference, entity> pairs, and select multiple candidate entity description sentences with high relevance to the entity reference context as the selection criteria.

[0011] The selection criteria determine whether the entity reference is a noun in the knowledge base or whether there is no entity reference in the knowledge base.

[0012] In the above scheme, the preprocessing of the characters referring to entities in the input text includes:

[0013] Remove punctuation marks;

[0014] Convert uppercase letters to lowercase letters;

[0015] Convert numeric characters into Arabic numerals.

[0016] In the above scheme, the entity reference characters are decomposed into multiple query characters using the 2-gram algorithm.

[0017] In the above scheme, the feature vectors of the candidate entity descriptions and the feature vectors of the entity reference context are calculated for each of the K <reference, entity> pairs. Several candidate entity description sentences with high relevance to the entity reference context are selected as the selection criteria, including:

[0018] The Doc2vec model is used to calculate the feature vectors of the candidate entity descriptions and the feature vectors of the entity reference context for each of the K <referential, entity> pairs.

[0019] Calculate the cosine similarity between feature vectors, find multiple candidate entity description sentences that are similar to the context of the entity reference based on the cosine similarity, and use the multiple candidate entity description sentences that are similar to the context of the entity reference as the selection criterion.

[0020] In the above scheme, the sum of the sentence lengths of multiple candidate entity description sentences that are close to the context of the entity reference is less than 512 characters.

[0021] In the above scheme, determining whether an entity reference exists in the knowledge base or whether there is no corresponding entity reference in the knowledge base based on the selection criteria includes:

[0022] The BERT pre-trained model is used to directly concatenate the selection criteria and the context of the entity reference corresponding to the selection criteria into an input string.

[0023] The above input string is used as the token for the BERT pre-trained model and then labeled.

[0024] The first-last-avg feature is used to average all vectors from the first and last layers of the BERT pre-trained model, which serves as the selection criterion and the relation representation vector of the candidate entity corresponding to the selection criterion.

[0025] All the relation representation vectors are concatenated sequentially to form a long vector;

[0026] The long vector is input into the MLP multilayer perceptron, which adds the unlinkable classification label at the prediction result mark to obtain the entity noun in the knowledge base corresponding to the entity reference in the selection criterion, or to determine that there is no entity noun in the knowledge base corresponding to the entity reference.

[0027] In the above scheme, establishing the knowledge base index corresponding to each entity noun in the knowledge base includes:

[0028] The entity nouns of each category in the knowledge base are split to obtain multiple n-gram words. The obtained n-gram words are then analyzed, expanded, and the obtained n-gram words and expansion results are stored in the corresponding category documents in the knowledge base.

[0029] In the above scheme, establishing the knowledge base index corresponding to each entity noun in the knowledge base further includes:

[0030] Calculate the IDF value of each n-gram and remove n-grams in each document whose IDF value is greater than the preset IDF threshold;

[0031] Based on the remaining n-grams corresponding to each entity noun, a knowledge base index is built for each entity noun in the knowledge base.

[0032] In the above scheme, the entity nouns of each category in the knowledge base are split and processed separately using the constructed word segmentation library.

[0033] In the above scheme, the IDF value of each n-gram is calculated using the IDF calculation formula, where the IDF calculation formula is:

[0034] Among them, IDF i Let |D| be the IDF value of the i-th n-gram, and |D| be the total number of documents in the knowledge base. This indicates that the knowledge base contains n-gram words t. i The total number of documents, d j Let j be the j-th document in the knowledge base.

[0035] The embodiments of the present invention have the following advantages:

[0036] The method for associating entities in text with a knowledge base using context provided in this invention expands the matching range by splitting entity nouns of various categories in the knowledge base. By initially screening candidate entity description sentences, multiple candidate entity description sentences with context similar to the entity reference are obtained, which can effectively improve the accuracy of association. By adding an MLP multilayer perceptron model that cannot link classification labels to the prediction result mark to judge the entity nouns in the knowledge base corresponding to the entity reference, it can be determined whether the reference does not have a corresponding knowledge base entity. This solves the problem in the prior art that entities may not necessarily appear in the knowledge base, and simply finding the item with the highest relevance will lead to incorrect association. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the steps of an embodiment of the method for associating entities in text with a knowledge base using context, according to the present invention.

[0038] Figure 2 This is a flowchart illustrating the steps involved in establishing a knowledge base index according to the present invention.

[0039] Figure 3 This is a flowchart of the steps for obtaining the selection criteria in this invention.

[0040] Figure 4 This is a flowchart of the steps for determining the entity noun in the knowledge base corresponding to the entity reference in this invention.

[0041] Figure 5 This is a schematic diagram of the BERT pre-trained model of the present invention.

[0042] Figure 6 This is a schematic diagram illustrating the working process of the BERT pre-trained model of this invention. Detailed Implementation

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] like Figure 1 As shown, this invention provides a method for associating entities in text with a knowledge base using context, the method comprising:

[0045] Step S1: Obtain internet knowledge encyclopedia data and build a knowledge base that includes a dictionary set of entity nouns of various categories.

[0046] Step S2: Establish a knowledge base index corresponding to each entity noun in the knowledge base.

[0047] In this embodiment, the purpose of establishing a knowledge base index corresponding to each entity noun in the knowledge base is to remove the parts of the term description that are not related to the entity and to remove sentences in the description text whose subject is not the entity from the content index, thereby ensuring that no errors occur when searching and calculating similarity.

[0048] like Figure 2 As shown, establishing a knowledge base index for each entity noun in the knowledge base includes:

[0049] Step S21: Split the entity nouns of each category in the knowledge base to obtain multiple n-gram words, analyze the obtained n-gram words, expand the n-gram words, and store the obtained n-gram words and expansion results in the corresponding category documents in the knowledge base;

[0050] Step S22: Calculate the IDF value of each n-gram and remove n-grams in each document whose IDF value is greater than the preset IDF threshold;

[0051] Step S23: Based on the remaining n-grams corresponding to each entity noun, establish a knowledge base index corresponding to each entity noun in the knowledge base.

[0052] In this embodiment, an entity is each term in the knowledge base, which typically contains a full name in a very standard format and related descriptive text.

[0053] In this embodiment, the entity nouns of each category in the knowledge base are split and processed separately using the constructed word segmentation library.

[0054] In this embodiment, the IDF value of each n-gram is calculated using the IDF calculation formula, whereby:

[0055] Among them, IDF i Let |D| be the IDF value of the i-th n-gram, and |D| be the total number of documents in the knowledge base. This indicates that the knowledge base contains n-gram words t. i The total number of documents, d j Let j be the j-th document in the knowledge base.

[0056] In one embodiment of the present invention, the IDF values ​​of the top 30 words for country names are shown in the table below:

[0057] Table 1. IDF Values ​​for Country Names

[0058] token IDF country 0.504717 and 0.410377 common 0.377358 Heguo 0.377358 Republic 0.377358 Republic 0.377358 Asia 0.216981 Si 0.127358 Ni 0.117925 you 0.0943396 profit 0.0849057 pull 0.0801887 -bar 0.0754717 West 0.0707547 horse 0.0707547 gram 0.0660377 add 0.0613208 Asian Communist Party 0.0613208 king 0.0613208 kingdom 0.0613208 Alliance 0.0566038 orchid 0.0566038 Ah 0.0518868 Nia 0.0518868 Compare 0.0518868 sluice 0.0518868 civil 0.0471698 state 0.0471698 federal 0.0471698 elder brother 0.0471698 host 0.0424528 Luo 0.0424528 inside 0.0424528 dimension 0.0424528 many 0.0424528 Tan 0.0424528 democracy 0.0377358 West Asia 0.0377358 Stan 0.0377358 watt 0.0330189 Na 0.0330189 Da 0.0330189 cloth 0.0330189 big 0.0283019 Nisi 0.0283019 Bia 0.0283019 main communist 0.0283019 special 0.0283019 Inside 0.0283019 island 0.0235849 Lia 0.0235849 Mo 0.0235849 Lan Gong 0.0235849

[0059] In another embodiment of the invention, the names of weapons and equipment are often composed of multiple parts, such as model-subtype-classification. The J-20 attack aircraft is an example, and its model number is a key identifier. This invention employs sequence labeling to specifically identify the root words of entities in the weapon and equipment category. In particular, the root word is necessarily part of the entity name. Since entity labeling datasets are relatively abundant, this invention uses a large amount of entity recognition data annotation information to assist in the root word identification of weapons and equipment.

[0060] For the labeled and identified weapons, the IDF calculation formula is used to calculate statistical root words, and the top 40 root words with the highest IDF values ​​are shown in the table below:

[0061] Table 2. Weapon Root Words Ranking Top 40 by IDF Value

[0062] Quantity sorting root 1 - 2 "(English) 3 "(Chinese) 4 Number 5 type 6 (space) 7 missile 8 class 9 engine 10 / 11 Mode 12 helicopter 13 2 14 1 15 tank 16 — 17 fighter 18 system 18 F 20 ship 21 destroyer 22 Frigate 23 airplane 24 millimeters 25 3 26 \xa0 27 （ 28 ） 29 submarine 30 " 31 radar 32 － 33 A 34 nuclear submarine 35 air defense 36 drones 37 series 38 5 39 ballistic missiles 40 4

[0063] Step S3: Preprocess the characters that refer to entities in the input text, decompose the characters that refer to entities into multiple query characters, input the query characters into the knowledge base for searching, and obtain the candidate entity set containing K candidate entities in the knowledge base corresponding to the entity reference.

[0064] In this embodiment, "referencing" generally refers to entity nouns appearing in the input text, namely, names of people, organizations, places, and all other entities identified by names. In this invention, "referencing" refers to the textual names of entity items such as names of people and organizations appearing in the processed text object.

[0065] In this embodiment, the preprocessing of the characters referring to entities in the input text includes: removing punctuation marks; converting uppercase letters to lowercase letters; and converting numeric characters to Arabic numerals.

[0066] In this embodiment, the entity reference characters are decomposed into multiple query characters using the 2-gram algorithm.

[0067] Step S4: Establish K <label, entity> pairs consisting of entity labels and K candidate entities from the corresponding candidate entity set.

[0068] Step S5: Calculate the feature vector of each candidate entity description and the feature vector of the entity reference context in each of the K <reference, entity> pairs, and select multiple candidate entity description sentences with high relevance to the entity reference context as the selection criteria.

[0069] In this embodiment, the multiple characters to the left and right of the reference in the input text constitute the context. For encoding convenience, 64 or 128 characters are selected for the top and bottom. To preserve the integrity of the sentence, the input text contains a clause with the reference.

[0070] like Figure 3 As shown, step S5 includes:

[0071] Step S51: Calculate the feature vectors of the candidate entity descriptions and the feature vectors of the entity reference context for each of the K <referential, entity> pairs using the Doc2vec model;

[0072] Step S52: Calculate the cosine similarity between feature vectors, find multiple candidate entity description sentences that are similar to the context of the entity reference based on the cosine similarity, and use the multiple candidate entity description sentences that are similar to the context of the entity reference as the selection criterion.

[0073] In this embodiment, the entity description is a descriptive fragment of the entity, which is used as the entity's features for feature extraction.

[0074] In this embodiment, the sum of the sentence lengths of multiple candidate entity description sentences that are close to the context of the entity reference is less than 512 characters, in order to meet the input constraints of the subsequent model.

[0075] Step S6: Determine whether the entity reference is in the knowledge base or whether there is no entity reference in the knowledge base based on the selection criteria.

[0076] like Figure 4 As shown, step S6 includes:

[0077] Step S61: Use the BERT pre-trained model to directly concatenate the strings of the selection criteria and the context of the entity reference corresponding to the selection criteria to form the input string;

[0078] Step S62: Use the above input string as the token for the BERT pre-trained model and mark it;

[0079] Step S63: Using first-last-avg features, the average of all vectors from the first and last layers of the BERT pre-trained model is used as the selection criterion and the relation representation vector of the candidate entity corresponding to the selection criterion.

[0080] Step S64: Concatenate all relation representation vectors sequentially to form a long vector;

[0081] Step S65: Input the long vector into the MLP multilayer perceptron model that adds the unlinkable classification label at the prediction result mark to obtain the entity noun in the knowledge base corresponding to the entity reference in the selection criterion, or determine that there is no entity noun in the knowledge base corresponding to the entity reference.

[0082] In this embodiment, step S61 uses a BERT pre-trained model to directly concatenate the selection criteria and the context of the entity reference corresponding to the selection criteria into an input string:

[0083] [CLS]description[SPE]mention[SEP]context[SEP];

[0084] In step S62, the above string is used as the token for inputting BERT, and the segment tags [CLS]description[SPE] and mention[SEP]context[SEP] are used to mark the segments.

[0085] like Figure 5 As shown, the BERT pre-trained model first concatenates the entity description sentence and entity referential context corresponding to each candidate entity to form an input string as the model input. The concatenated input string is then transformed into token_ids_i and segment_ids_i, where i=0,1,2,3. Next, each pair of token_ids_i and segment_ids_i is transformed into model_1 through a Functional transformation. The four transformed results are then concatenated through a concatenate layer. The output of the concatenate layer is then concatenated together and fed into a Dence layer, where it undergoes a Dense linear transformation. Finally, the model passes through a Softmax layer, and the entity with the highest score is selected as the final candidate entity.

[0086] like Figure 6 As shown, the BERT pre-trained model converts each candidate entity's corresponding entity description sentence and each character in the entity's referential context into a one-dimensional vector by querying the character vector table, which serves as the model input. The model output is the vector representation of each input character after fusing full-text semantic information. In addition to character vectors, the model input also includes:

[0087] Text vector Token_ids: The values ​​of this vector are automatically learned during model training and are used to characterize the global semantic information of the text and are fused with the semantic information of individual characters or words;

[0088] Position vector segment_ids: Due to the differences in semantic information carried by characters or words appearing in different positions in the text.

[0089] Furthermore, after Dropout, the model input undergoes a linear transformation through a fully connected layer called Dense. Then, the feature vectors of multiple candidate entities are merged into one using the Concat operation. After another Dense linear transformation, softmax is used to calculate the similarity probability sampling, thereby obtaining the most likely classification result.

[0090] It should be noted that the above detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0091] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0092] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0093] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0094] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways, such as rotated 90 degrees or in other orientations, and the spatial relative descriptions used herein will be interpreted accordingly.

[0095] In the detailed description above, reference has been made to the accompanying drawings, which form part of this document. In the drawings, similar symbols typically identify similar parts unless the context otherwise indicates otherwise. The illustrated embodiments described in the detailed specification, drawings, and claims are not intended to be limiting. Other embodiments may be used and other changes may be made without departing from the spirit or scope of the subject matter presented herein.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for associating entities in text with a knowledge base using context, characterized in that, The method includes: Acquire internet knowledge encyclopedia data and build a knowledge base that includes a dictionary collection of entity nouns of various categories; Establish a knowledge base index corresponding to each entity noun in the knowledge base; The characters representing entities in the input text are preprocessed and decomposed into multiple query characters. The query characters are then input into the knowledge base for searching, and a set of candidate entities containing K candidate entities in the knowledge base corresponding to the entity reference is obtained. Establish K <label, entity> pairs consisting of entity labels and K candidate entities from the corresponding candidate entity set; Calculate the feature vector of each candidate entity description and the feature vector of the entity reference context in each of the K <reference, entity> pairs, and select multiple candidate entity description sentences with high relevance to the entity reference context as the selection criteria. The selection criteria determine whether the entity reference is a corresponding entity name in the knowledge base or whether there is no corresponding entity name in the knowledge base. The specific steps include: The BERT pre-trained model is used to directly concatenate the selection criteria and the descriptions of the candidate entities corresponding to the selection criteria into a string to form the input string. The above input string is used as the token for the BERT pre-trained model and then labeled. The first-last-avg feature is used to average all vectors from the first and last layers of the BERT pre-trained model, which serves as the selection criterion and the relation representation vector of the candidate entity corresponding to the selection criterion. All the relation representation vectors are concatenated sequentially to form a long vector; The long vector is input into the MLP multilayer perceptron, which adds the unlinkable classification label at the prediction result mark to obtain the entity noun in the knowledge base corresponding to the entity reference in the selection criterion, or to determine that there is no entity noun in the knowledge base corresponding to the entity reference.

2. The method for associating entities in text with a knowledge base using context according to claim 1, characterized in that, The preprocessing of the characters representing entities in the input text includes: Remove punctuation marks; Convert uppercase letters to lowercase letters; Convert numeric characters into Arabic numerals.

3. The method for associating entities in text with a knowledge base using context according to claim 1, characterized in that, The entity reference characters are decomposed into multiple query characters using the 2-gram algorithm.

4. The method for associating entities in text with a knowledge base using context according to claim 1, characterized in that, Calculate the feature vectors of the candidate entity descriptions and the feature vectors of the entity's reference context for each of the K <reference, entity> pairs. Then, select several candidate entity description sentences with high relevance to the entity's reference context as the selection criteria, including: The Doc2vec model is used to calculate the feature vectors of the candidate entity descriptions and the feature vectors of the entity reference context for each of the K <referential, entity> pairs. Calculate the cosine similarity between feature vectors, find multiple candidate entity description sentences that are similar to the context of the entity reference based on the cosine similarity, and use the multiple candidate entity description sentences that are similar to the context of the entity reference as the selection criterion.

5. The method for associating entities in text with a knowledge base using context according to claim 4, characterized in that, The sum of the sentence lengths of multiple candidate entity descriptions that are close to the context of the entity reference is less than 512 characters.

6. The method for associating entities in text with a knowledge base using context according to claim 1, characterized in that, The knowledge base index corresponding to each entity noun in the knowledge base includes: The entity nouns of each category in the knowledge base are split to obtain multiple n-gram words. The obtained n-gram words are then analyzed, expanded, and the obtained n-gram words and expansion results are stored in the corresponding category documents in the knowledge base.

7. The method for associating entities in text with a knowledge base using context according to claim 6, characterized in that, The process of establishing a knowledge base index corresponding to each entity noun in the knowledge base also includes: Calculate the IDF value of each n-gram and remove n-grams in each document whose IDF value is greater than the preset IDF threshold; Based on the remaining n-grams corresponding to each entity noun, a knowledge base index is built for each entity noun in the knowledge base.

8. The method for associating entities in text with a knowledge base using context according to claim 7, characterized in that, The entity nouns of each category in the knowledge base are split and processed separately using the constructed word segmentation library.

9. The method for associating entities in text with a knowledge base using context according to claim 8, characterized in that, The IDF value of each n-gram is calculated using the IDF calculation formula, where the IDF calculation formula is: Among them, IDF i Let i be the IDF value of the i-th n-gram. It is the total number of documents in the knowledge base. This indicates that the knowledge base contains n-gram words t. i The total number of documents, d j Let j be the j-th document in the knowledge base.

Images