Entity linking method and related device

Abstract

The application provides an entity linking method and related equipment, the method comprising: obtaining a text comprising an entity referring item; encoding and recognizing the text by a pre-trained entity recognition model to obtain the entity referring item; performing dictionary matching on the entity referring item by the entity recognition model to obtain candidate entities having a matching relationship with the entity referring item, a candidate entity set being a subset of candidate entities obtained by mapping the entity referring item; scoring the candidate entity set by a pre-trained disambiguation model according to the correlation between the entity referring item and the candidate entities to obtain a scoring result; and selecting a candidate entity with the highest score in the scoring result as an entity linking result.

Description

Technical Field

[0001] This application relates to the field of knowledge graph technology, and in particular to an entity linking method and related equipment. Background Technology

[0002] In the era of big data, the internet generates massive amounts of data daily, a large portion of which is stored as text in natural language form, such as news headlines and microblog posts, leading to an information explosion. Moreover, much of this text lacks annotation and explanation, and the inherent complexity and ambiguity of natural language pose significant challenges to computers' accurate text localization and understanding. With the continuous development of information extraction technology, structured knowledge graphs are constantly being enriched and constructed. Researchers are attempting to link text and knowledge graphs, and entity linking is one of the necessary steps in this process. Entity linking first identifies the entity references mentioned in the text and then links them to the corresponding entities in a given knowledge graph. Utilizing the implicit knowledge in knowledge graphs can assist in mining natural language texts containing massive amounts of information; simultaneously, the acquired knowledge can supplement the knowledge graph, further expanding the knowledge base.

[0003] Typically, entity linking methods first utilize external data sources for offline entity referential mining. For a given text, a candidate entity set is generated based on entity recognition methods. Finally, candidate entity disambiguation is modeled as a binary classification problem, scoring and ranking the candidate entities, and selecting the candidate entity with the highest score as the disambiguation result. However, the entity recognition model first identifies entity referentials in the text, and then the disambiguation model performs independent disambiguation. This causes errors from the entity recognition model to accumulate in the disambiguation process, resulting in a significant decrease in the overall performance of entity linking. Furthermore, existing disambiguation models lack fine-grained interaction between the context of entity referentials and the context of candidate entities at the lexical and grammatical levels, leading to poor disambiguation capabilities. Moreover, in knowledge graph-oriented entity linking scenarios, there is a lack of effective extraction of relational semantics from candidate entities. Thus, when the ambiguity of the candidate entities to be disambiguated is high and the number is large, the disambiguation effect of the disambiguation model decreases. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a physical linking method and related equipment to solve or partially solve the above-mentioned technical problems.

[0005] For the purposes described above, the first aspect of this application provides an entity linking method, comprising:

[0006] Retrieve the text including entity references;

[0007] The entity reference is obtained by encoding and recognizing the text using a pre-trained entity recognition model;

[0008] The entity recognition model is used to perform dictionary matching on the entity reference to obtain candidate entities that have a matching relationship with the entity reference. The candidate entity set is a subset of the candidate entities obtained by mapping the entity reference.

[0009] Based on the correlation between the entity reference and the entity context corresponding to the candidate entity and the historical entity information corresponding to the correlation, a pre-trained disambiguation model is used to score the candidate entity set to obtain the scoring result.

[0010] The candidate entity with the highest score in the scoring results is selected as the entity link result.

[0011] A second aspect of this application provides a physical linking device, comprising:

[0012] The retrieval module is configured to retrieve text including entity references;

[0013] The recognition module is configured to encode and recognize the text using a pre-trained entity recognition model to obtain the entity reference.

[0014] The matching module is configured to use the entity recognition model to perform dictionary matching on the entity reference to obtain candidate entities that have a matching relationship with the entity reference, and the candidate entity set includes the entity reference and the candidate entities;

[0015] The scoring module is configured to score the candidate entity set using a pre-trained disambiguation model based on the correlation between the entity reference and the entity context corresponding to the candidate entity and the historical entity information corresponding to the correlation, and to obtain the scoring result.

[0016] The selection module is configured to select the candidate entity with the highest score from the scoring results as the entity link result.

[0017] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0018] As can be seen from the above, the entity linking method and related equipment provided in this application, through the character-level encoding capability and the data-driven capability of the neural network in the entity recognition model, realize the adjustment of the boundary of the entity reference item, and enable the candidate entity to adapt to the subsequent disambiguation process; the disambiguation model is used to generate candidate entity scores based on the semantic correlation between the entity reference item and the entity context corresponding to the candidate entity, which improves the disambiguation model's ability to extract key semantic information of the candidate entity, and the historical entity information in the disambiguation model is associated with the candidate entity scores, which effectively improves the accuracy of the entity linking results. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the entity linking method according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating step 104.

[0022] Figure 3 This is a schematic diagram of the structure of the physical linking device according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0025] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] As described in the background section, an entity recognition model refers to a text consisting of a set of identifiers, denoted as D, where an entity reference can include multiple identifiers w, i.e., D = {w1, w2, ..., w...}. N Output a set of M triples. s I e A list consisting of , t>, where each triple in the list represents an entity reference in s. s ∈[1,N] is the starting index of the entity reference; I e ∈[1,N] is the ending index of the entity reference; t refers to the type of the entity reference selected from the predefined categories. The disambiguation model refers to: the input is a sequence of N characters W={w1,…,w…} N The set of range sequences corresponding to M non-overlapping entity references in sequence W, where the entity references can be represented as the target to be disambiguated, M = {m1, ..., m}. M The output is the most likely candidate entity {e1,…,e} mentioned for each instance. M}∈E, where E is the set of entities in the knowledge graph.

[0027] like Figure 1 As shown, the method in this embodiment includes:

[0028] Step 101: Obtain the text including entity references.

[0029] In this step, entity references refer to entities with specific meanings, such as names of people, places, organizations, and proper nouns. This provides the data foundation for subsequent extraction of entity references from the text using an entity recognition model.

[0030] ​Step 102: Encode and recognize the text using a pre-trained entity recognition model to obtain the entity reference.

[0031] In this step, the entity recognition model refers to a model capable of extracting entity references from text and obtaining candidate entities corresponding to those entity references. The preferred entity recognition model in this embodiment not only extracts entity references from text and obtains candidate entities corresponding to those entity references, but also calculates entity recognition accuracy.

[0032] Specifically, the first step is dictionary construction. Based on a large-scale corpus, aliases are extracted from encyclopedia websites (such as titles, redirect pages, and disambiguation pages) to describe independent entities, or from existing knowledge graphs such as DBpedia (a semantic web application example) to construct an alias dictionary. Then, entity referential identification is performed, identifying the boundaries mentioned in the text and determining whether consecutive fields at positions i to j in the text constitute entity referentials. The BERT-CRF model (an unsupervised model) can be used to identify entity referentials in the text. The text can be labeled using BIO (begin-inside-outside). In the BERT-CRF model, BERT (an unsupervised model) learns the pattern of each word and symbol in the text corresponding to entity referential labels, while CRF (conditional random field) learns the transition rules between adjacent entity referential labels. The text is input into the BERT encoding layer. The embedding vector of each character is passed through the CRF layer and the fully connected layer and mapped to the label set. Then, the label of each character is predicted using the Softmax activation function (a normalized exponential function). The introduction of the CRF layer can effectively reduce the number of incorrect prediction sequences.

[0033] In addition, this embodiment introduces an embedding vector statistical method for knowledge base entity names to calculate the probability of entity references for subsequent model optimization.

[0034] Specifically, for the embedding vectors at the start and end positions of entity references in the CRF output layer, we have For learnable parameters, for the character vector at the middle position, we have For each possible entity referent, the recognition accuracy is obtained. The entity referential probability will be applied to the subsequent training and optimization process of the entity recognition model and disambiguation model.

[0035] This provides optimization parameters for the subsequent training and optimization of entity recognition and disambiguation models, enabling candidate entities to adapt to the subsequent disambiguation process.

[0036] Step 103: Use the entity recognition model to perform dictionary matching on the entity reference to obtain candidate entities that have a matching relationship with the entity reference. The candidate entity set is a subset of the candidate entities obtained by mapping the entity reference.

[0037] In this step, dictionary matching refers to matching entity references in the text with corresponding candidate entities in a dictionary. For each entity reference, a subset of candidate entities mapped through dictionary matching is used as the candidate entity set. This provides the disambiguation data foundation for the subsequent disambiguation model.

[0038] Step 104: Based on the correlation degree between the entity reference and the entity context corresponding to the candidate entity and the historical entity information corresponding to the correlation degree, a pre-trained disambiguation model is used to score the candidate entity set to obtain the scoring result.

[0039] In this step, the disambiguation model refers to the model that finds the candidate entity with the best feature match from the candidate entity set. In this embodiment, a preferred disambiguation model is one that scores the matching features in the candidate entity set.

[0040] In some embodiments, the disambiguation model includes a local disambiguation model and a global disambiguation model, such as... Figure 2 As shown, step 104 specifically includes:

[0041] Step 1041: Input the text corresponding to the entity reference item into the local disambiguation model using a predetermined format;

[0042] In some embodiments, the predetermined format is Equation (1):

[0043]

[0044] in, The text corresponding to the entity reference item. The string is concatenated between the predefined text and the string of the j-th candidate entity corresponding to the i-th entity designation. This is a string concatenation of the previous order relation sequence pair of the i-th entity reference item corresponding to the j-th candidate entity, where [CLS] is a flag indicating the first character of the string concatenation and [SEP] is a flag indicating the separation between the two string concatenations.

[0045] In the above scheme, the local disambiguation model can extract the semantic and relational correlation between entity referents and entity contexts based on a pre-trained language model. The basic idea of ​​the pre-trained language model in the local disambiguation model is to design tasks related to language feature learning, pre-train the model, and then generate deep bidirectional language representations. The pre-trained local disambiguation model only needs to be fine-tuned by adding an extra output layer to adapt to various downstream natural language-related tasks. Due to the characteristics of the pre-trained language model and the features of the disambiguation process, when using the pre-trained language model to solve entity linking problems, it is necessary to reasonably design the model's input and output, and add additional neural network layers to adapt to the specific requirements of the disambiguation task.

[0046] Specifically, in the pre-trained language model of the local disambiguation model, the input can be a type of sentence, which differs from the task format of the local disambiguation model. The disambiguation task of the local disambiguation model involves the semantic information processing between entity referents and candidate entities in the text. Therefore, the preferred predetermined format in this embodiment can be Equation (1):

[0047]

[0048] in, The text corresponding to the entity reference item. The string is concatenated between the predefined text and the string of the j-th candidate entity corresponding to the i-th entity designation. This is a string concatenation of the previous order relation sequence pair of the i-th entity reference item corresponding to the j-th candidate entity, where [CLS] is a flag indicating the first character of the string concatenation and [SEP] is a flag indicating the separation between the two string concatenations.

[0049] Furthermore, this embodiment can use the BERT bidirectional encoding model to model the semantic and relational features between candidate entity contexts and candidate entities, and utilize the BERT bidirectional encoding model's ability to predict the next sentence to capture the deep lexical, syntactic, and semantic connections between candidate entities and contexts.

[0050] In this way, the predefined format in the local disambiguation model includes the context of the candidate entity, enriches the information of the candidate entity by using the set of all attributes of the candidate entity in the knowledge base, and improves the disambiguation model's ability to extract key semantic information of the candidate entity.

[0051] Step 1042: The local disambiguation score corresponding to the candidate entity is calculated by the multilayer perceptron in the local disambiguation model based on the correlation between the entity reference and the entity context corresponding to the candidate entity.

[0052] In the above scheme, the specific structure of the local disambiguation model may include a BERT bidirectional coding layer, a multilayer perceptron layer, and a sigmoid activation function. The input to the local disambiguation model may include three parts: word embeddings, positional embeddings, and segment embeddings, and the text... For each corresponding word ID, the position embedding is the temporal information of the sentence in the input text, with the segment embedding set to 0 and the rest to 1. After processing the input by the BERT bidirectional encoding layer, the output vector of the last Transformer (a deep learning model based on self-attention mechanism) is used as the classification token (password) as the semantic representation of the corresponding candidate entities in the two sentences before and after the entity referent. This semantic representation is then input into a multilayer perceptron, and the sigmoid function is used as the activation function to obtain the local disambiguation policy score of the candidate entity.

[0053] This provides a score basis for the subsequent ranking of entity references.

[0054] Step 1043: Sort the entity references in descending order according to the local disambiguation scores to obtain the sorting result.

[0055] In some embodiments, the entity designations are sorted in descending order according to equation (2):

[0056]

[0057] in, Let n be the nth entity designation, where n is the number of entity designations in the candidate entity set, and Rank is a descending sorting function. Let K be the local disambiguation score corresponding to the j-th candidate entity, where 1 ≤ j ≤ K. Let K be the average local disambiguation score of all candidate entities corresponding to the i-th entity reference, 1≤i≤n, and K be the number of candidate entities corresponding to the entity reference.

[0058] In the above scheme, entity references need to be roughly ranked based on disambiguation confidence scores. A higher disambiguation confidence score indicates a greater certainty in the disambiguation model's ability to disambiguate an entity reference. In this embodiment, the preferred disambiguation confidence score can be the variance of the local disambiguation prediction probability of candidate entities.

[0059] In this way, starting disambiguation with the entity references that are easiest to disambiguate helps to reduce the introduction of noise information in subsequent disambiguation processes and reduce the erroneous impact of disambiguation results on entity linking results.

[0060] Step 1044: Input the sorting result into the gated recurrent neural network in the global disambiguation model to obtain historical entity information.

[0061] In some embodiments, the relationship between the sorting result and the historical entity information is calculated according to equations (3)-(6):

[0062]

[0063]

[0064]

[0065]

[0066] Where t is the t-th time point in the gated cyclic network. For the J-th state represented by the entity in the sorting result, W z W r W and W represent the first weight matrix, the second weight matrix, and the third weight matrix, respectively. This represents the J-th state of the hidden layer, where σ is the activation function, tanh is the hyperbolic tangent function, and ⊙ is the XNOR logical operation. It is the reset gate for the j-th candidate entity. It is the update gate for the j-th candidate entity. For the historical entity information, h t-1 This refers to the historical entity information corresponding to the (t-1)th time point.

[0067] In the above scheme, the basic goal of the global disambiguation model is to accumulate knowledge from the entity links that have completed disambiguation as a dynamic context to enhance subsequent disambiguation decisions. This knowledge comes not only from the inherent attributes of previous entity links (e.g., attributes, relationships) but also from entities closely related to the entity links, which endow the global disambiguation model with important associative capabilities. Considering that in real-world scenarios, some entities from previous entity links may be irrelevant to the current candidate entity, or some erroneous entity links may even introduce noisy context in future linking steps, this embodiment can use a gated recurrent neural network to model the global disambiguation process as a sequence problem. By dynamically and selectively introducing historical entity information from previously disambiguated entities, the co-occurrence relationship between the candidate entity and other entity references in the candidate reference context is calculated, strengthening the next disambiguation decision.

[0068] In this way, the gating mechanism in the gated recurrent neural network controls the combination and filtering of historical entity information, thereby capturing the interdependencies between entity references.

[0069] Step 1045: Calculate the global disambiguation score corresponding to the candidate entity using the multilayer perceptron in the global disambiguation model based on the historical entity information.

[0070] In some embodiments, the global disambiguation score is calculated according to equation (7):

[0071]

[0072] in, Let b1 be the global disambiguation score, and b2 be the global disambiguation parameter. Here, σ is the global disambiguation weight matrix, and σ is the activation function. This refers to the historical entity information.

[0073] In the above scheme, historical entity information is input into a global disambiguation model based on a multilayer perceptron classifier to obtain the probability that the candidate reference to be disambiguated is mapped to the current candidate entity. The global disambiguation score for candidate entities is calculated using the following formula: And select the information corresponding to the candidate entity with the highest score. As historical entity information for the next disambiguation. t ,Right now

[0074] In this way, by associating historical entity information in the disambiguation model with candidate entity scores, the accuracy of entity linking results is effectively improved.

[0075] Step 1046: Determine the scoring result based on the local disambiguation score and the global disambiguation score.

[0076] In some embodiments, the scoring result is calculated according to equation (8):

[0077]

[0078] Among them, y i For the scoring results, The global disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is given by [reference to the i-th entity designation]. Let α be the local disambiguation score of the j-th candidate entity corresponding to the i-th entity designation, and let α be the scoring parameter.

[0079] In the above scheme, the scoring result refers to the comprehensive score obtained based on the local disambiguation score and the global disambiguation score.

[0080] The above scheme provides a scoring basis for the selection of subsequent entity linking results.

[0081] Step 105: Select the candidate entity with the highest score from the scoring results as the entity link result.

[0082] In this step, entity linking refers to the task of associating entity references appearing in natural language text with corresponding knowledge graph entities. A preferred embodiment of this approach involves selecting the candidate entity most similar to and closest to the context of the entity reference from among multiple candidate entities associated with it.

[0083] In some embodiments, S1. The recognition accuracy of the entity designation is obtained according to equation (9):

[0084]

[0085] Where p([s,e]) is the recognition accuracy, σ is the activation function, and m start (s) is the starting character vector of the entity reference, m end (e) is the vector of the ending character of the entity reference item. This is the intermediate character vector for the entity reference.

[0086] S2. Obtain the first loss function during the training process of the entity recognition model according to equation (10):

[0087]

[0088] in, Let y be the first loss function. [s,e] Let p([s,e]) be the candidate entity, p([s,e]) be the recognition accuracy, s be the start number of the character vector corresponding to the entity reference, e be the end number of the character vector corresponding to the entity reference, i be the number of the entity reference, 1≤i≤n, and n be the number of the entity references in the candidate entity set.

[0089] S3. Obtain the second loss function during the training process of the disambiguation model according to equation (11):

[0090]

[0091] Among them, L MD For the second loss function, Let τ be the expected function, and τ be the time constant. The positive sample corresponding to the global disambiguation score of the candidate entity corresponding to the i-th entity designation is... The negative sample corresponding to the global disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is... The positive sample corresponding to the local disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is... The negative sample is the local disambiguation score corresponding to the j-th candidate entity corresponding to the i-th entity designation.

[0092] S4. Train the entity recognition model and the disambiguation model jointly based on the first loss function and the second loss function.

[0093] In the above scheme, to jointly train the entity recognition model and the disambiguation model, the training of the entity recognition model and the disambiguation model is completed by optimizing the sum of their loss functions. Furthermore, the recognition accuracy, reflecting the influence between the entity recognition model and the disambiguation model, is incorporated into the loss function required for training. This helps improve the overall performance of entity linking.

[0094] The above scheme utilizes the character-level encoding capability and the data-driven capability of neural networks in the entity recognition model to adjust the boundary of entity references and enable candidate entities to adapt to the subsequent disambiguation process. The disambiguation model generates candidate entity scores based on the correlation between entity references and the corresponding entity contexts, which improves the disambiguation model's ability to extract key semantic information of candidate entities. Furthermore, historical entity information corresponding to the correlation is added to the candidate entity scores, effectively improving the accuracy of entity linking results.

[0095] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0096] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0097] Based on the same inventive concept, and on the basis of the corresponding implementation schemes of the above-mentioned embodiments, the following specific implementations are possible.

[0098] In some embodiments, the present application may further include:

[0099] Step 201: First, a dictionary is constructed. Based on a large-scale corpus, an alias dictionary is constructed by extracting aliases (corresponding to entities in the embodiments of this application) from independent entities described in encyclopedia websites, such as titles, redirect pages, and disambiguation pages, or from existing knowledge graphs.

[0100] Step 202: For example, “Author A’s representative work is a certain novel” is used as the original short text and input into the entity recognition module (corresponding to the entity recognition model in the embodiment of this application). In the training set, the text has two entities, author A and a certain novel, as well as the sequence range of the entities in the text.

[0101] Step 203: Input the short text into the BERT (an unsupervised model) bidirectional encoding layer, then pass it through the CRF (conditional random field) layer and the fully connected layer respectively. The softmax activation function maps the entities to the corresponding label types, and the identified entities are extracted based on the annotations.

[0102] Step 204: Candidate entity generation. The identified designations (corresponding to entity designations in this embodiment) are matched using a dictionary to obtain the mapping relationship between the entity and the corresponding candidate entity set. Specifically, given a designation, its possible candidate entities in the dictionary are retrieved and presented in the form of key-value pairs (designation, candidate entity list). To reduce computational load, the maximum number of candidate entities is set to 64. If the candidate entity list is too large, a preliminary screening is performed based on the entity tag type to obtain a new candidate entity list.

[0103] Step 205: The entity disambiguation model (corresponding to the disambiguation model in this embodiment) generally adopts a binary classification approach to derive the semantic similarity and contextual matching degree between the referent and the candidate entity, and scores each candidate entity of the candidate referent one by one. First, the first-order relation pairs of the entities to be disambiguated in the knowledge graph are concatenated into a relation sequence string. The candidate entity and the relation sequence string are then concatenated together to form a set of various attributes of the candidate entity, including various related entity information. For example, in the question "When did Wu Cheng'en write 'Journey to the West'?", the triple information of a certain candidate entity of the referent "'Journey to the West'" in the knowledge graph is: (Journey to the West, Director, Yang Jie), which is concatenated together as the input of the model.

[0104] In step 206, the local disambiguation strategy first uses the concatenated string of the original text and candidate entities as input to the BERT model. The output vector of the first CLS position (a marker indicating the first character of the concatenated string) of the encoding layer is taken as the output. Then, it is concatenated with the vectors of the candidate entities at the start and end positions in the BERT model output vector. Finally, the local disambiguation probability score of each candidate entity is obtained through a multilayer perceptron and a sigmoid activation function.

[0105] Step 207, in the global disambiguation strategy, firstly, the variance of the local disambiguation probability score of each candidate entity is calculated, and the candidates are ranked to form the disambiguation order. Candidates with smaller variances have a higher certainty of disambiguation results and should be disambiguated first. The candidate entities are input into the BERT model in the same way as in step 6. The output vector of the first CLS position of the encoding layer is concatenated with the vectors of the candidate entity at the corresponding start and end positions in the BERT model output vector, and used as the input to the GRU (Gated Recurrent) unit. The output of the hidden layer is taken as the candidate entity representation incorporating historical entity knowledge and input into a sigmoid (a type of S-shaped function) classifier based on a multilayer perceptron layer to obtain the probability of the candidate candidate being mapped to the current candidate entity, which is used as the global disambiguation probability score of the candidate entity. The hidden layer output of the correctly linked candidate entity is then passed to the next GRU unit.

[0106] Step 208: For all candidate entities of each reference item, sort them according to the sum of the local disambiguation probability score (corresponding to the local disambiguation score in this embodiment) and the global disambiguation probability score (corresponding to the global disambiguation score in this embodiment), and select the candidate entity with the highest score as the link result of the current reference item, that is, the entity that is correctly linked.

[0107] The above embodiments are used to implement the corresponding entity linking methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0108] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a physical linking device.

[0109] refer to Figure 3 The physical linking device includes:

[0110] The acquisition module 201 is configured to acquire text including entity references;

[0111] The recognition module 202 is configured to encode and recognize the text using a pre-trained entity recognition model to obtain the entity reference.

[0112] The matching module 203 is configured to use the entity recognition model to perform dictionary matching on the entity reference to obtain candidate entities that have a matching relationship with the entity reference, and the candidate entity set is a subset of the candidate entities obtained by mapping the entity reference.

[0113] The scoring module 204 is configured to score the candidate entity set using a pre-trained disambiguation model based on the correlation degree between the entity reference and the entity context corresponding to the candidate entity and the historical entity information corresponding to the correlation degree, and obtain the scoring result.

[0114] Selection module 205 is configured to select the candidate entity with the highest score in the scoring results as the entity link result.

[0115] In some embodiments, the disambiguation model includes a local disambiguation model and a global disambiguation model; the scoring module 204 specifically includes:

[0116] The formatting unit is configured to input the text corresponding to the entity designation into the local disambiguation model using a predetermined format.

[0117] The local disambiguation unit is configured to calculate the local disambiguation score corresponding to the candidate entity based on the correlation between the entity reference and the entity context corresponding to the candidate entity through the multilayer perceptron in the local disambiguation model.

[0118] The sorting unit is configured to sort the entity references in descending order according to the local disambiguation score to obtain a sorting result.

[0119] The historical unit is configured to input the sorting result into the gated recurrent neural network in the global disambiguation model to obtain historical entity information;

[0120] The global disambiguation unit is configured to calculate the global disambiguation score corresponding to the candidate entity based on the historical entity information using a multilayer perceptron in the global disambiguation model.

[0121] The score unit is configured to determine the scoring result based on the local disambiguation score and the global disambiguation score.

[0122] In some embodiments, the predetermined format in the format unit is Equation (1):

[0123]

[0124] in, The text corresponding to the entity reference item. The string is concatenated between the predefined text and the string of the j-th candidate entity corresponding to the i-th entity designation. This is a string concatenation of the previous order relation sequence pair of the i-th entity reference item corresponding to the j-th candidate entity, where [CLS] is a flag indicating the first character of the string concatenation and [SEP] is a flag indicating the separation between the two string concatenations.

[0125] In some embodiments, the sorting unit is specifically configured to sort the entity designations in descending order according to equation (2):

[0126]

[0127] in, Let n be the nth entity designation, where n is the number of entity designations in the candidate entity set, and Rank is a descending sorting function. Let K be the local disambiguation score corresponding to the j-th candidate entity, where 1 ≤ j ≤ K. Let K be the average local disambiguation score of all candidate entities corresponding to the i-th entity reference, 1≤i≤n, and K be the number of candidate entities corresponding to the entity reference.

[0128] In some embodiments, the historical unit is specifically configured to calculate the relationship between the sorting result and the historical entity information according to equations (3)-(6):

[0129]

[0130]

[0131]

[0132]

[0133] Where t is the t-th time point in the gated cyclic network. For the J-th state represented by the entity in the sorting result, W z W r W and W represent the first weight matrix, the second weight matrix, and the third weight matrix, respectively. This represents the J-th state of the hidden layer, where σ is the activation function, tanh is the hyperbolic tangent function, and ⊙ is the XNOR logical operation. It is the reset gate for the j-th candidate entity. It is the update gate for the j-th candidate entity. For the historical entity information, h t-1 This refers to the historical entity information corresponding to the (t-1)th time point.

[0134] In some embodiments, the global disambiguation unit is configured to calculate the global disambiguation score according to equation (7):

[0135]

[0136] in, Let b1 be the global disambiguation score, and b2 be the global disambiguation parameter. Here, σ is the global disambiguation weight matrix, and σ is the activation function. This refers to the historical entity information.

[0137] In some embodiments, the score unit is configured to calculate the scoring result according to equation (8):

[0138]

[0139] Among them, y i For the scoring results, The global disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is given by [reference to the i-th entity designation]. Let α be the local disambiguation score of the j-th candidate entity corresponding to the i-th entity designation, and let α be the scoring parameter.

[0140] In some embodiments, the training process of the entity recognition model in the recognition module 202 and the disambiguation model in the scoring module 204 includes:

[0141] S1. Obtain the recognition accuracy of the entity reference item according to equation (9):

[0142]

[0143] Where p([s,e]) is the recognition accuracy, σ is the activation function, and m start (s) is the starting character vector of the entity reference, m end (e) is the vector of the ending character of the entity reference item. This is the intermediate character vector for the entity reference.

[0144] S2. Obtain the first loss function during the training process of the entity recognition model according to equation (10):

[0145]

[0146] in, Let y be the first loss function. [s,e] Let p([s,e]) be the candidate entity, p([s,e]) be the recognition accuracy, s be the start number of the character vector corresponding to the entity reference, e be the end number of the character vector corresponding to the entity reference, i be the number of the entity reference, 1≤i≤n, and n be the number of the entity references in the candidate entity set.

[0147] S3. Obtain the second loss function during the training process of the disambiguation model according to equation (11):

[0148]

[0149] Among them, L MD For the second loss function, Let τ be the expected function, and τ be the time constant. The positive sample corresponding to the global disambiguation score of the candidate entity corresponding to the i-th entity designation is... The negative sample corresponding to the global disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is... The positive sample corresponding to the local disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is... The negative sample is the local disambiguation score corresponding to the j-th candidate entity corresponding to the i-th entity designation.

[0150] S4. Train the entity recognition model and the disambiguation model jointly based on the first loss function and the second loss function.

[0151] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0152] The apparatus described above is used to implement the corresponding entity linking method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0153] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the entity linking method described in any of the above embodiments.

[0154] Figure 4 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0155] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0156] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0157] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0158] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0159] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0160] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0161] The electronic devices described above are used to implement the corresponding entity linking methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0162] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the entity linking method as described in any of the above embodiments.

[0163] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0164] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the entity linking method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0165] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0166] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0167] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0168] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. An entity linking method, characterized by, include: Retrieve the text including entity references; The entity reference is obtained by encoding and recognizing the text using a pre-trained entity recognition model; The entity recognition model is used to perform dictionary matching on the entity reference to obtain candidate entities that have a matching relationship with the entity reference. The candidate entity set is a subset of the candidate entities obtained by mapping the entity reference. Based on the correlation between the entity reference and the entity context corresponding to the candidate entity and the historical entity information corresponding to the correlation, a pre-trained disambiguation model is used to score the candidate entity set to obtain the scoring result. Select the candidate entity with the highest score from the scoring results as the entity link result; The disambiguation model includes a local disambiguation model and a global disambiguation model; Based on the correlation between the entity reference and the entity context corresponding to the candidate entity, and the historical entity information corresponding to the correlation, a pre-trained disambiguation model is used to score the candidate entity set, resulting in a scoring result, including: The text corresponding to the entity designation is input into the local disambiguation model using a predetermined format; Based on the correlation between the entity reference and the entity context corresponding to the candidate entity, the local disambiguation score of the candidate entity is calculated by the multilayer perceptron in the local disambiguation model. The entity references are sorted in descending order based on the local disambiguation scores to obtain the sorting result. The sorting result is input into the gated recurrent neural network in the global disambiguation model to obtain the historical entity information; The global disambiguation score corresponding to the candidate entity is calculated by the multilayer perceptron in the global disambiguation model based on the historical entity information. The scoring result is determined based on the local disambiguation score and the global disambiguation score; The training process of the entity recognition model and the training process of the disambiguation model include: S1. Obtain the recognition accuracy of the entity reference item according to equation (9): wherein, is the recognition accuracy, is an activation function, is a start character vector of the entity mention, is an end character vector of the entity mention, is an intermediate character vector of the entity mention; S2. Obtain the first loss function during the training process of the entity recognition model according to equation (10): wherein, is the first loss function, is the candidate entity, is the recognition accuracy, s is the start number of the character vector corresponding to the entity mention, e is the end number of the character vector corresponding to the entity mention, i is the number of the entity mention, 1≤i≤n, n is the number of entity mentions in the candidate entity set. S3. Obtain the second loss function during the training process of the disambiguation model according to equation (11): wherein, is the second loss function, is the expected function, is the time constant, is a positive sample corresponding to the global disambiguation score of the jth candidate entity corresponding to the ith entity mention, is a negative sample corresponding to the global disambiguation score of the jth candidate entity corresponding to the ith entity mention, is a positive sample corresponding to the local disambiguation score of the jth candidate entity corresponding to the ith entity mention, is a negative sample corresponding to the local disambiguation score of the jth candidate entity corresponding to the ith entity mention. S4. Train the entity recognition model and the disambiguation model jointly based on the first loss function and the second loss function.

2. The method according to claim 1, characterized in that, The predetermined format is Equation (1): in, The text corresponding to the entity reference item. For the predefined text and the i-th entity reference item corresponding to the first Concatenation of strings of candidate entities This is a string concatenation of the previous order relation sequence pair of the i-th entity reference item corresponding to the j-th candidate entity, where [CLS] is a flag indicating the first character of the string concatenation and [SEP] is a flag indicating the separation between the two string concatenations.

3. The method according to claim 1, characterized in that, The step of sorting the entity designations in descending order based on the local disambiguation scores to obtain the sorting result includes: Sort the entity references in descending order according to equation (2): in, Let n be the nth entity designation, where n is the number of entity designations in the candidate entity set, and Rank is a descending sorting function. Let K be the local disambiguation score corresponding to the j-th candidate entity, where 1 ≤ j ≤ K. Let K be the average local disambiguation score of all candidate entities corresponding to the i-th entity reference, 1≤i≤n, and K be the number of candidate entities corresponding to the entity reference.

4. The method according to claim 1, characterized in that, The step of inputting the sorting result into the gated recurrent neural network in the global disambiguation model to obtain historical entity information includes: The relationship between the sorting result and the historical entity information is calculated according to equations (3)-(6): Where t is the t-th time point in the gated cyclic network. Let J be the state representing the entity in the sorting result. , , These are the first weight matrix, the second weight matrix, and the third weight matrix, respectively. It is the Jth state of the hidden layer. Here, is the activation function, tanh is the hyperbolic tangent function, and ⊙ is the XOR logical operator. It is the reset gate for the j-th candidate entity. It is the update gate for the j-th candidate entity. For the historical entity information, This refers to the historical entity information corresponding to the (t-1)th time point.

5. The method according to claim 1, characterized in that, The step of calculating the global disambiguation score corresponding to the candidate entity link based on the historical entity information using the multilayer perceptron in the global disambiguation model includes: The global disambiguation score is calculated according to equation (7): in, The global disambiguation score is... These are global disambiguation parameters. This is the global disambiguation weight matrix. For activation function, This refers to the historical entity information.

6. The method according to claim 1, characterized in that, Determining the scoring result based on the local disambiguation score and the global disambiguation score includes: The scoring result is calculated according to formula (8): in, For the scoring results, The global disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is given by [reference to the i-th entity designation]. The local disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is... These are the scoring parameters.

7. A physical linking device, characterized in that, include: The retrieval module is configured to retrieve text including entity references; The recognition module is configured to encode and recognize the text using a pre-trained entity recognition model to obtain the entity reference. The matching module is configured to use the entity recognition model to perform dictionary matching on the entity reference to obtain candidate entities that have a matching relationship with the entity reference, and the candidate entity set is a subset of the candidate entities obtained by mapping the entity reference. The scoring module is configured to score the candidate entity set using a pre-trained disambiguation model based on the correlation between the entity reference and the entity context corresponding to the candidate entity and the historical entity information corresponding to the correlation, and to obtain the scoring result. The selection module is configured to select the candidate entity with the highest score from the scoring results as the entity link result. The disambiguation model includes a local disambiguation model and a global disambiguation model; the scoring module specifically includes: The formatting unit is configured to input the text corresponding to the entity designation into the local disambiguation model using a predetermined format. The local disambiguation unit is configured to calculate the local disambiguation score corresponding to the candidate entity based on the correlation between the entity reference and the entity context corresponding to the candidate entity through the multilayer perceptron in the local disambiguation model. The sorting unit is configured to sort the entity references in descending order according to the local disambiguation score to obtain a sorting result. The historical unit is configured to input the sorting result into the gated recurrent neural network in the global disambiguation model to obtain historical entity information; The global disambiguation unit is configured to calculate the global disambiguation score corresponding to the candidate entity based on the historical entity information using a multilayer perceptron in the global disambiguation model. The score unit is configured to determine the scoring result based on the local disambiguation score and the global disambiguation score; The training process for the entity recognition model in the recognition module and the disambiguation model in the scoring module includes: S1. Obtain the recognition accuracy of the entity reference item according to equation (9): in, The recognition accuracy is... For activation function, The starting character vector of the entity reference item. The vector of the ending characters of the entity reference item. The intermediate character vector of the entity reference item; S2. Obtain the first loss function during the training process of the entity recognition model according to equation (10): in, Let the first loss function be... For the candidate entity, The recognition accuracy is denoted as s, where s is the start number of the character vector corresponding to the entity reference item, e is the end number of the character vector corresponding to the entity reference item, i is the number of the entity reference item, 1≤i≤n, and n is the number of the entity reference items in the candidate entity set. S3. Obtain the second loss function during the training process of the disambiguation model according to equation (11): in, For the second loss function, Let be the expected function. It is a time constant. The positive sample corresponding to the global disambiguation score of the candidate entity corresponding to the i-th entity designation is... The negative sample corresponding to the global disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is... The positive sample corresponding to the local disambiguation score of the j-th candidate entity corresponding to the i-th entity designation is... The negative sample corresponding to the local disambiguation score of the j-th candidate entity corresponding to the i-th entity designation; S4. Train the entity recognition model and the disambiguation model jointly based on the first loss function and the second loss function.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 6.

Images