Entity linking method, device, equipment and storage medium

Abstract

The present application relates to the field of natural language processing technology, and discloses an entity linking method, apparatus, device, and storage medium, including: obtaining entity mentions, candidate entities, and entity descriptions corresponding to input text; constructing a first fusion sequence containing entity mentions and entity descriptions and a second fusion sequence containing candidate entities and input text; using a first model to calculate the similarity between the first fusion sequence and the second fusion sequence of entity mentions, and determining the linked entities of the entity mentions from the candidate entities based on the similarity; a third model is obtained by training a pre-trained model using a contrastive loss function using positive and negative samples of entity mention training samples of training texts through contrastive learning. The present application incorporates entity description information into entity linking, and at the same time trains the model through positive and negative sample contrastive learning to achieve entity disambiguation, improve the accuracy of entity linking, and improve the performance of entity linking on open knowledge graphs.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to an entity linking method, apparatus, device and storage medium. Background Art

[0002] Entity linking is the process of linking entities mentioned in a text to corresponding entities in a knowledge base. It is the first and crucial step for machines to understand natural language. The input of entity linking usually includes the entity's reference (mention) and context, as well as the knowledge base to be linked. The output of entity linking is the entity in the knowledge base corresponding to the reference. When there is a one-to-one correspondence between the reference and the entity, that is, when there is no ambiguity, the entity linking problem is very simple. However, in practical applications, it often has ambiguities. On the one hand, an entity can be expressed in multiple ways; on the other hand, the same name can refer to different entities.

[0003] Existing entity linking methods generally include three steps: named entity recognition (MD), candidate entity generation, and entity disambiguation. However, a drawback of this approach is that if an error occurs in the first mention detection step, subsequent candidate entity generation and disambiguation operations will produce cumulative errors, resulting in poor results.

[0004] Therefore, how to improve the accuracy of entity linking is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an entity linking method, apparatus, device, and storage medium that can improve the accuracy of entity linking and the performance of entity linking on open knowledge graphs. The specific solution is as follows:

[0006] A first aspect of the present application provides an entity linking method, comprising:

[0007] Obtain entity mentions corresponding to input text, candidate entities for the entity mentions, and entity descriptions for the candidate entities;

[0008] Constructing a first fused sequence comprising the entity mention and the entity description and a second fused sequence comprising the candidate entity and the input text;

[0009] A first model is used to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions, and a linked entity of the entity mention is determined from the candidate entities based on the similarity; wherein the first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples of training texts; positive samples are sequences consisting of the entity mention training samples and the entity description training samples of the correct entity, and the correct entity and the training text, respectively; negative samples are sequences consisting of the entity mention training samples and the entity description training samples of the incorrect entity, and the incorrect entity and the training text, respectively.

[0010] Optionally, obtaining entity mentions corresponding to the input text includes:

[0011] The second model is used to determine entity mention locations of the input text, and the entity mention corresponding to the input text is determined based on the entity mention locations.

[0012] Optionally, the second model includes a BERT neural network and a CRF neural network;

[0013] Accordingly, determining the entity mention location of the input text using the second model includes:

[0014] The word vector of the input text is processed by the BERT neural network and the CRF neural network in sequence to obtain a BIO tag representing the entity mention location.

[0015] Optionally, obtaining candidate entities mentioned by the entity and entity descriptions of the candidate entities includes:

[0016] Calculating the matching degree between the entity mention and the combined text of each category of alias in the knowledge base entity list using the third model, and determining the entity of the alias category corresponding to the matching degree greater than a first threshold as the candidate entity;

[0017] The entity description of the candidate entity is read from the entity list.

[0018] Optionally, the entity linking method further includes:

[0019] Obtaining the training text;

[0020] Performing entity extraction on the training text using the second model to obtain the entity mention training sample corresponding to the training text, and determining candidate entity training samples corresponding to the entity mention training sample using the third model;

[0021] Determining correct entities and incorrect entities and corresponding entity description training samples from the candidate entity training samples;

[0022] Determine a sequence consisting of the entity mention training sample and the entity description training sample of the correct entity as a first positive sample sequence, and determine a sequence consisting of the correct entity and the training text as a second positive sample sequence, and determine a sequence consisting of the entity mention training sample and the entity description training sample of the incorrect entity as a first negative sample sequence, and determine a sequence consisting of the incorrect entity and the training text as a second negative sample sequence;

[0023] The first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence.

[0024] Optionally, the step of training a pre-trained model using a contrastive loss function by contrastive learning using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence to obtain the first model includes:

[0025] The pre-trained model is used to respectively calculate the representation vectors of the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence, and the loss value of the NCE_LOSS function is calculated according to the similarity between the representation vectors, and then the relevant network parameters are adjusted so that the loss value is less than a second threshold.

[0026] Optionally, the entity linking method further includes:

[0027] integrating the second model for obtaining the entity mentions, the third model for obtaining the candidate entities, and the third model into one model to obtain a corresponding end-to-end integrated model;

[0028] When performing model training, the first model is trained by training the second model using a cross entropy loss function and using the output of the trained second model as the input of the first model;

[0029] When performing entity linking, the input text is input into the end-to-end integrated model to be processed by the second model, the third model and the first model in sequence, and then the corresponding similarity is output.

[0030] A second aspect of the present application provides a physical linking device, comprising:

[0031] An acquisition module, configured to acquire entity mentions corresponding to the input text, candidate entities of the entity mentions, and entity descriptions of the candidate entities;

[0032] A construction module, configured to construct a first fused sequence comprising the entity mention and the entity description and a second fused sequence comprising the candidate entity and the input text;

[0033] A calculation and determination module is used to use a first model to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions, and determine the linked entity of the entity mentions from the candidate entities based on the similarity; wherein, the first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples of training texts; positive samples are sequences consisting of the entity mention training samples and the entity description training samples of the correct entity, and the correct entity and the training text, respectively; negative samples are sequences consisting of the entity mention training samples and the entity description training samples of the incorrect entity, and the incorrect entity and the training text, respectively.

[0034] A third aspect of the present application provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned entity linking method.

[0035] A fourth aspect of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the aforementioned entity linking method is implemented.

[0036] In this application, the entity mentions corresponding to the input text, the candidate entities of the entity mentions, and the entity descriptions of the candidate entities are first obtained; then a first fusion sequence containing the entity mentions and the entity descriptions and a second fusion sequence containing the candidate entities and the input text are constructed; finally, a first model is used to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions, and the linked entities of the entity mentions are determined from the candidate entities based on the similarity; wherein, the first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples of the training text; the positive sample is a sequence composed of the entity mention training sample and the entity description training sample of the correct entity, and the correct entity and the training text, respectively; the negative sample is a sequence composed of the entity mention training sample and the entity description training sample of the incorrect entity, and the incorrect entity and the training text, respectively. It can be seen that the present application is applicable to the entity linking task of any open knowledge graph. When performing entity linking on the input text, on the basis of preliminarily extracting the entity mentions of the input text and determining the candidate entities, the entity description corresponding to the candidate entity is further obtained, and the entity description information is integrated into the entity link to obtain the corresponding fusion sequence. At the same time, the model is trained by positive and negative sample comparative learning to achieve entity disambiguation, and the trained model is used to calculate the similarity of the fusion sequence to screen out the correct entity among the candidate entities, that is, the linked entity, thereby improving the accuracy of entity linking and the performance of entity linking on the open knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A flow chart of an entity linking method provided in this application;

[0039] Figure 2 A specific flow chart of the first model training method provided in this application;

[0040] Figure 3 A flow chart of a specific entity linking method provided in this application;

[0041] Figure 4 A flow chart of a specific entity linking method provided in this application;

[0042] Figure 5A specific second model structure diagram provided for this application;

[0043] Figure 6 A specific entity link logic diagram provided for this application;

[0044] Figure 7 A schematic diagram of the structure of a physical link device provided in this application;

[0045] Figure 8 This is a structural diagram of an entity-linked electronic device provided in this application. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] Currently, the method of entity linking that sequentially performs named entity recognition (MD, mention detection), candidate entity generation, and entity disambiguation results in poor final entity linking results. That is, if an error occurs in the first step of mention detection, the subsequent candidate entity generation and disambiguation operations will produce a superposition of errors. In response to the above technical defects, the present application provides an entity linking solution that incorporates entity description information into entity linking and simultaneously trains a model through positive and negative sample comparative learning to achieve entity disambiguation, thereby improving the accuracy of entity linking and the performance of entity linking on open knowledge graphs.

[0048] Figure 1 This is a flow chart of an entity linking method provided in an embodiment of the present application. Figure 1 As shown, the entity linking method includes:

[0049] S11: Obtain entity mentions corresponding to the input text, candidate entities of the entity mentions, and entity descriptions of the candidate entities.

[0050] In this embodiment, for the input text to be entity linked, entity mentions corresponding to the input text are first obtained. These entity mentions are names of people, places, and other entities that are initially determined to be entities in the input text. Next, candidate entities are obtained for these entity mentions. These candidate entities are aliases or synonyms of these entity mentions that exist in the knowledge base. Based on this, entity descriptions of these candidate entities are obtained to incorporate the entity description information into the entity linking process. The entity description contains information such as what the entity is and what characteristics it has.

[0051] It's understandable that the candidate entities include both correct and incorrect entities. The ultimate goal of entity linking is to filter out the correct entity from the candidate entities. For example, when the text "In which year did Li Na win the Australian Open?" is input, the entity reference "Li Na" is first identified. Initial candidate entities obtained in the knowledge base include tennis player Li Na, singer Li Na, and gymnast Li Na. The ultimate goal is to use the context "...Australian Open champion" to link the name "Li Na" to the correct entity in the knowledge base, tennis player Li Na.

[0052] S12: Constructing a first fusion sequence including the entity mention and the entity description and a second fusion sequence including the candidate entity and the input text.

[0053] In this embodiment, after obtaining the entity mentions, the candidate entities and the entity descriptions of the input text, it is necessary to integrate the entity description information into the entity linking processing. Specifically, it is necessary to construct a first fusion sequence containing the entity mentions and the entity descriptions and a second fusion sequence containing the candidate entities and the input text. It can be understood that the first fusion sequence and the second fusion sequence are generally represented in the form of vectors, that is, the first fusion sequence is obtained by splicing the vector of the entity mentions and the vector of the entity descriptions, and the second fusion sequence is obtained by splicing the vector of the candidate entities and the vector of the input text.

[0054] S13: Calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions using a first model, and determine the linked entity of the entity mentions from the candidate entities based on the similarity; wherein, the first model is obtained by training a pre-training model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples of training texts; positive samples are sequences consisting of the entity mention training samples and the entity description training samples of the correct entity, and the correct entity and the training text, respectively; negative samples are sequences consisting of the entity mention training samples and the entity description training samples of the incorrect entity, and the incorrect entity and the training text, respectively.

[0055] In this embodiment, the first model is first used to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mention, and then the linked entity of the entity mention is determined from the candidate entity based on the similarity. The first model is also the entity disambiguation model, and the first model is mainly used to calculate the similarity between the first fusion sequence and the second fusion sequence. The first model is obtained by training a pre-trained model using a contrast loss function through contrast learning using positive and negative samples of entity mention training samples in training texts. The training process is as follows: Figure 2 The specific method is as follows:

[0056] S131: Obtain training text.

[0057] S132: Using the second model to perform entity extraction on the training text to obtain entity mention training samples corresponding to the training text, and using the third model to determine candidate entity training samples corresponding to the entity mention training samples.

[0058] S133: Determine correct entities and incorrect entities and corresponding entity description training samples from the candidate entity training samples.

[0059] S134: Determine a sequence consisting of the entity mention training sample and the entity description training sample of the correct entity as a first positive sample sequence and a sequence consisting of the correct entity and the training text as a second positive sample sequence, and determine a sequence consisting of the entity mention training sample and the entity description training sample of the incorrect entity as a first negative sample sequence and a sequence consisting of the incorrect entity and the training text as a second negative sample sequence.

[0060] In this embodiment, it is necessary to construct training samples, including positive samples and negative samples for comparative learning training. First, the training text is obtained, and then the second model is used to extract entities from the training text to obtain entity mention training samples corresponding to the training text, and the third model is used to determine the candidate entity training samples corresponding to the entity mention training samples. Then, the correct entities and incorrect entities and the corresponding entity description training samples are determined from the candidate entity training samples. The second model and the third model are the same as the models in the previous steps. Assume that the training text is S, and its vector is represented as sent = {x1, x2…x n}, the entity mention training sample is represented as EM = {em1,em2…em i}, i means there are i entity mentions in the training sample; the candidate entity training sample is represented as 30 means there are 30 candidate entities for each entity mention); the entity description training sample is represented as

[0061] On this basis, the sequence consisting of the entity mention training sample and the entity description training sample of the correct entity is determined as the first positive sample sequence, and the sequence consisting of the correct entity and the training text is determined as the second positive sample sequence, and the sequence consisting of the entity mention training sample and the entity description training sample of the incorrect entity is determined as the first negative sample sequence, and the sequence consisting of the incorrect entity and the training text is determined as the second negative sample sequence. Assume that for the entity mention em i , the correct entity in the candidate entity training sample is represented as The entity description training sample of the correct entity is represented as Em i and After splicing, the first positive sample sequence is expressed as Will After concatenating with sent, the second positive sample sequence is expressed as Similarly, for entity mentions em i , the incorrect entity in the candidate entity training sample is represented as The entity description training sample of incorrect entity is represented as Em i and After splicing, the first negative sample sequence is expressed as Will After concatenating with sent, the second negative sample sequence is expressed as Among them, p≠r.

[0062] S135: Using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence, a pre-trained model using a contrastive loss function is trained by contrastive learning to obtain a first model.

[0063] In this embodiment, the first model is obtained by training a pre-training model using a contrast loss function using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence through contrast learning. Specifically, the representation vectors of the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence are calculated respectively using the pre-training model, and the loss value of the NCE_LOSS function is calculated based on the similarity between the representation vectors, and the relevant network parameters are adjusted so that the loss value is less than the second threshold. That is, the obtained MD sequence and CS sequence are combined in pairs, input into the same pre-training model, and the output of the first position of the model is used as the representation vector of the sequence. The representation vector is expressed as Then calculate the similarity score by calculating the inner product between the two vectors. The formula is expressed as:

[0064] score(f(MD),f(CS))=exp(f(MD) T f(CS))

[0065] After calculating the similarity between two vectors using the above formula, we further calculate the contrast loss function L, which is as follows:

[0066]

[0067]

[0068] L=L1+L2

[0069] The loss value calculated by the above formula is used to adjust the network parameters of the pre-trained network until L is less than the second threshold or three training rounds are completed. In this embodiment, the second threshold can be set to 0.01.

[0070] In this embodiment, after the first model is trained, in the actual entity linking operation, the output similarity score is directly calculated, and the candidate entity with the highest score is used as the linked entity mentioned by the entity.

[0071] It can be seen that the embodiment of the present application first obtains the entity mentions corresponding to the input text, the candidate entities of the entity mentions, and the entity descriptions of the candidate entities; then constructs a first fusion sequence containing the entity mentions and the entity descriptions and a second fusion sequence containing the candidate entities and the input text; finally, the first model is used to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions, and the linked entities of the entity mentions are determined from the candidate entities based on the similarity; wherein, the first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples of the training text; the positive sample is a sequence composed of the entity mention training sample and the entity description training sample of the correct entity, the correct entity and the training text, respectively, and the negative sample is a sequence composed of the entity mention training sample and the entity description training sample of the incorrect entity, the incorrect entity and the training text, respectively. The embodiments of the present application are applicable to the entity linking task of any open knowledge graph. When performing entity linking on the input text, based on the preliminary extraction of entity mentions in the input text and the determination of candidate entities, the entity description corresponding to the candidate entity is further obtained, and the entity description information is integrated into the entity link to obtain the corresponding fusion sequence. At the same time, the model is trained by positive and negative sample comparative learning to achieve entity disambiguation. The trained model is used to calculate the similarity of the fusion sequence to screen out the correct entity among the candidate entities, that is, the link entity, thereby improving the accuracy of entity linking and the performance of entity linking on the open knowledge graph.

[0072] Figure 3 This is a flow chart of a specific entity linking method provided in the embodiment of this application. Figure 3 As shown, the entity linking method includes:

[0073] S21: Integrate the second model for obtaining entity mentions, the third model for obtaining candidate entities, and the third model into one model to obtain a corresponding end-to-end integrated model.

[0074] In this embodiment, entity extraction, candidate entity acquisition, and entity disambiguation are integrated into a single model for end-to-end entity linking. Specifically, the second model for acquiring entity mentions, the third model for acquiring candidate entities, and the third model are integrated into a single model to obtain the corresponding end-to-end integrated model.

[0075] S22: When performing entity linking, the input text is input into the end-to-end integrated model to output the corresponding similarity after being processed by the second model, the third model and the first model in sequence, and the linked entity mentioned by the entity is determined from the candidate entities based on the similarity.

[0076] In this embodiment, after obtaining the end-to-end integrated model, when linking entities, the input text is directly input into the end-to-end integrated model, so that the corresponding similarity is output after being processed by the second model, the third model and the first model in sequence, and the linked entity mentioned by the entity is determined from the candidate entities based on the similarity. The specific method is as follows Figure 4 As shown, the following steps are included:

[0077] S221: Determine entity mention locations of the input text using the second model, and determine entity mentions corresponding to the input text based on the entity mention locations.

[0078] In this embodiment, the entity mention process is to use the second model to determine the entity mention position of the input text, and determine the entity mention corresponding to the input text based on the entity mention position. Specifically, the second model includes a BERT neural network and a CRF neural network. On this basis, the word vector of the input text is processed by the BERT neural network and the CRF neural network in turn to obtain a BIO tag representing the entity mention position. The model is shown in FIG. Figure 5 As shown. After the input text that needs to be recognized as an entity is converted into a word vector, it is input into a pre-trained BERT neural network. The output of the BERT neural network is then passed through a CRF neural network, and finally the label probability of BIO is output. The entity mention position is obtained by the BIO label of each position. Of course, other models that can perform entity extraction can also be used in the entity linking method of this application, and this embodiment of the application is not limited to this.

[0079] S222: Calculate the matching degree between the entity mention and the combined text of each category of alias in the knowledge base entity list using the third model, and determine the entity of the alias category corresponding to the matching degree greater than the first threshold as a candidate entity.

[0080] S223: Read the entity description of the candidate entity from the entity list.

[0081] In this embodiment, candidate entities are primarily determined through a text similarity matching method, using a third model to calculate the degree of match between the entity mention and the combined text consisting of each category of aliases in the knowledge base entity list. Entities with alias types corresponding to the degree of match greater than a first threshold are then identified as candidate entities. The third model can be a BM25 algorithm model. Other algorithms for text similarity calculation can also achieve the same technical effect. The knowledge base entity list is a Wikipedia entity list, also known as the Falcon Candidates vocabulary, which expands each entity label in Wikipedia into many aliases. A match is calculated between each entity mention and each category of aliases in the Wikipedia entity list. Each entity mention serves as a query, and the aliases of each entity in the entity list form a document. The BM25 algorithm is used to calculate the match between the query and the document. Finally, the calculated match is sorted, and the top 30 entities with the highest match for each entity mention constitute the candidate entity set. The first paragraph of the Wikipedia explanation corresponding to the top 30 candidate entities for each entity mention is used as supplementary information, i.e., the entity description.

[0082] S224: Construct a first fusion sequence including the entity mention and the entity description and a second fusion sequence including the candidate entity and the input text.

[0083] In this embodiment, regarding the specific process of the above-mentioned step S224, reference may be made to the corresponding content disclosed in the above-mentioned embodiment, and no further details will be given here.

[0084] S225: Calculate the similarity between the first fusion sequence and the second fusion sequence mentioned by the entity using the first model.

[0085] In this embodiment, the specific process of the above steps S224 and S225 can refer to the corresponding content disclosed in the above embodiments, and will not be repeated here. It should be noted that the first model in this embodiment can be obtained by performing comparative learning training on the pre-trained RoBerta model, and can be regarded as a high-order model of the RoBerta model. The logical diagram of the above steps is as follows Figure 6 shown.

[0086] S23: When performing model training, the first model is trained by training the second model using a cross entropy loss function and using the output of the trained second model as the input of the first model.

[0087] In this embodiment, to maximize the accuracy of the entity introduction operation results, after obtaining the end-to-end integrated model, during model training, the second model is trained using a cross-entropy loss function, and the output of the trained second model is used as the input to train the first model. Compared to jointly training the two or individual subtasks described above, this embodiment combines the three tasks to obtain an integrated entity linking model, and uses a comparative learning method based on the fusion of entity description information to improve entity linking performance.

[0088] See also Figure 7 As shown, the embodiment of the present application also discloses a physical linking device, including:

[0089] An acquisition module 11 is configured to acquire entity mentions corresponding to the input text, candidate entities of the entity mentions, and entity descriptions of the candidate entities;

[0090] A construction module 12 is configured to construct a first fused sequence comprising the entity mention and the entity description and a second fused sequence comprising the candidate entity and the input text;

[0091] A calculation and determination module 13 is used to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions using a first model, and determine the linked entity of the entity mentions from the candidate entities based on the similarity; wherein, the first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples of training texts; positive samples are sequences consisting of the entity mention training samples and the entity description training samples of the correct entity, and the correct entity and the training text, respectively; negative samples are sequences consisting of the entity mention training samples and the entity description training samples of the incorrect entity, and the incorrect entity and the training text, respectively.

[0092] It can be seen that the embodiment of the present application first obtains the entity mentions corresponding to the input text, the candidate entities of the entity mentions, and the entity descriptions of the candidate entities; then constructs a first fusion sequence containing the entity mentions and the entity descriptions and a second fusion sequence containing the candidate entities and the input text; finally, the first model is used to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions, and the linked entities of the entity mentions are determined from the candidate entities based on the similarity; wherein, the first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples of the training text; the positive sample is a sequence composed of the entity mention training sample and the entity description training sample of the correct entity, the correct entity and the training text, respectively, and the negative sample is a sequence composed of the entity mention training sample and the entity description training sample of the incorrect entity, the incorrect entity and the training text, respectively. The embodiments of the present application are applicable to the entity linking task of any open knowledge graph. When performing entity linking on the input text, based on the preliminary extraction of entity mentions in the input text and the determination of candidate entities, the entity description corresponding to the candidate entity is further obtained, and the entity description information is integrated into the entity link to obtain the corresponding fusion sequence. At the same time, the model is trained by positive and negative sample comparative learning to achieve entity disambiguation. The trained model is used to calculate the similarity of the fusion sequence to screen out the correct entity among the candidate entities, that is, the link entity, thereby improving the accuracy of entity linking and the performance of entity linking on the open knowledge graph.

[0093] In some specific embodiments, the acquisition module 11 specifically includes:

[0094] an extraction unit, configured to determine entity mention locations of the input text using a second model, and determine the entity mention corresponding to the input text based on the entity mention locations;

[0095] a matching unit, configured to calculate, by using a third model, a degree of matching between the entity mention and a combined text consisting of each type of alias in the knowledge base entity list, and determine an entity of an alias type corresponding to the degree of matching greater than a first threshold as the candidate entity;

[0096] A reading unit is configured to read the entity description of the candidate entity from the entity list.

[0097] In some specific embodiments, the physical linking device further includes:

[0098] A sample acquisition module is configured to acquire the training text; perform entity extraction on the training text using the second model to obtain the entity mention training sample corresponding to the training text; and determine candidate entity training samples corresponding to the entity mention training sample using the third model;

[0099] A determination module, configured to determine correct entities and incorrect entities and corresponding entity description training samples from the candidate entity training samples;

[0100] a positive and negative sample sequence construction module, configured to determine a sequence consisting of the entity mention training sample and the entity description training sample of the correct entity as a first positive sample sequence and a sequence consisting of the correct entity and the training text as a second positive sample sequence, and to determine a sequence consisting of the entity mention training sample and the entity description training sample of the incorrect entity as a first negative sample sequence and a sequence consisting of the incorrect entity and the training text as a second negative sample sequence;

[0101] A model training module is used to train a pre-trained model using a contrastive loss function by contrastive learning using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence to obtain the first model.

[0102] In some specific embodiments, the model training module is specifically used to use a pre-trained model to respectively calculate the representation vectors of the first positive sample sequence, the second positive sample sequence, the first negative sample sequence and the second negative sample sequence, and adjust the relevant network parameters after calculating the loss value of the NCE_LOSS function based on the similarity between each representation vector so that the loss value is less than a second threshold.

[0103] In some specific embodiments, the physical linking device further includes:

[0104] a model integration module, configured to integrate the second model for obtaining the entity mentions, the third model for obtaining the candidate entities, and the third model into one model to obtain a corresponding end-to-end integrated model;

[0105] An integrated training module is configured to train the first model by training the second model using a cross entropy loss function and using the output of the trained second model as the input of the first model during model training;

[0106] The integration calculation module is used to input the input text into the end-to-end integration model when performing entity linking, so as to output the corresponding similarity after being processed by the second model, the third model and the first model in sequence.

[0107] Furthermore, an embodiment of the present application also provides an electronic device. Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0108] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the entity linking method disclosed in any of the aforementioned embodiments.

[0109] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0110] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon may include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0111] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, so as to enable the processor 21 to calculate and process the massive amount of data 223 in the memory 22. The operating system 221 can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the entity linking method disclosed in any of the aforementioned embodiments and executed by the electronic device 20, the computer program 222 can further include computer programs capable of performing other specific tasks. The data 223 can include data such as text information collected by the electronic device 20.

[0112] Furthermore, an embodiment of the present application also discloses a storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the entity linking method steps disclosed in any of the aforementioned embodiments are implemented.

[0113] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0114] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0115] The above describes in detail the entity linking method, apparatus, device, and storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. An entity linking method, characterized in that: include: Integrate the second model for obtaining entity mentions, the third model for obtaining candidate entities, and the first model into one model to obtain a corresponding end-to-end integrated model; When performing model training, the second model is trained by adopting a cross entropy loss function, and the first model is trained by using the output of the trained second model as the input of the first model; The process of inputting the input text into the end-to-end integrated model, and the end-to-end integrated model sequentially processing the text through the second model, the third model, and the first model and then outputting the corresponding similarity includes: Obtaining entity mentions corresponding to the input text through the second model, obtaining candidate entities of the entity mentions and entity descriptions of the candidate entities through the third model; constructing a first fusion sequence including the entity mentions and the entity descriptions and a second fusion sequence including the candidate entities and the input text; calculating the similarity between the first fusion sequence and the second fusion sequence of the entity mentions using the first model, and determining the linked entity of the entity mentions from the candidate entities based on the similarity; The first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using positive and negative samples of entity mention training samples in training texts: Obtaining the training text; Performing entity extraction on the training text using the second model to obtain the entity mention training sample corresponding to the training text, and determining candidate entity training samples corresponding to the entity mention training sample using the third model; Determining correct entities and incorrect entities and corresponding entity description training samples from the candidate entity training samples; Determine a sequence consisting of the entity mention training sample and the entity description training sample of the correct entity as a first positive sample sequence, and determine a sequence consisting of the correct entity and the training text as a second positive sample sequence, and determine a sequence consisting of the entity mention training sample and the entity description training sample of the incorrect entity as a first negative sample sequence, and determine a sequence consisting of the incorrect entity and the training text as a second negative sample sequence; The first model is obtained by training a pre-trained model using a contrastive loss function through contrastive learning using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence.

2. The entity linking method according to claim 1, characterized in that Obtaining entity mentions corresponding to the input text includes: The second model is used to determine entity mention locations of the input text, and the entity mention corresponding to the input text is determined based on the entity mention locations.

3. The entity linking method according to claim 2, characterized in that The second model includes a BERT neural network and a CRF neural network; Accordingly, determining the entity mention location of the input text using the second model includes: The word vector of the input text is processed by the BERT neural network and the CRF neural network in sequence to obtain a BIO tag representing the entity mention location.

4. The entity linking method according to claim 1, characterized in that Obtain candidate entities mentioned by the entity and entity descriptions of the candidate entities, including: Calculating the matching degree between the entity mention and the combined text of each category of alias in the knowledge base entity list using the third model, and determining the entity of the alias category corresponding to the matching degree greater than a first threshold as the candidate entity; The entity description of the candidate entity is read from the entity list.

5. The entity linking method according to claim 1, characterized in that The method of training a pre-trained model using a contrastive loss function by contrastive learning using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence to obtain the first model includes: The pre-trained model is used to respectively calculate the representation vectors of the first positive sample sequence, the second positive sample sequence, the first negative sample sequence, and the second negative sample sequence, and the loss value of the NCE_LOSS function is calculated according to the similarity between the representation vectors, and then the relevant network parameters are adjusted so that the loss value is less than a second threshold.

6. A physical link device, characterized in that: include: a model integration module, configured to integrate the second model for obtaining entity mentions, the third model for obtaining candidate entities, and the first model into one model to obtain a corresponding end-to-end integrated model; An integrated training module is used to train the first model by training the second model using a cross entropy loss function and using the output of the trained second model as the input of the first model during model training; an integration calculation module, configured to input the input text into the end-to-end integration model when performing entity linking, and the end-to-end integration model outputs corresponding similarity after being processed by the second model, the third model, and the first model in sequence; an acquisition module, configured to acquire entity mentions corresponding to the input text through the second model, and acquire candidate entities of the entity mentions and entity descriptions of the candidate entities through the third model; A construction module, configured to construct a first fused sequence comprising the entity mention and the entity description and a second fused sequence comprising the candidate entity and the input text; A calculation and determination module is used to calculate the similarity between the first fusion sequence and the second fusion sequence of the entity mentions using a first model, and determine the linked entity of the entity mention from the candidate entities based on the similarity; wherein, the first model is obtained by training a pre-training model using a contrast loss function by using positive samples and negative samples of the entity mention training samples of the training text through contrast learning: obtaining the training text; using the second model to perform entity extraction on the training text to obtain the entity mention training samples corresponding to the training text, and determining the candidate entity training samples corresponding to the entity mention training samples through a third model; determining the correct entity from the candidate entity training samples and incorrect entities and corresponding entity description training samples; determining a sequence consisting of the entity mention training samples and the entity description training samples of the correct entity as a first positive sample sequence and determining a sequence consisting of the correct entity and the training text as a second positive sample sequence, and determining a sequence consisting of the entity mention training samples and the entity description training samples of the incorrect entity as a first negative sample sequence and determining a sequence consisting of the incorrect entity and the training text as a second negative sample sequence; using the first positive sample sequence, the second positive sample sequence, the first negative sample sequence and the second negative sample sequence to train a pre-trained model using a contrastive loss function through contrastive learning to obtain the first model.

7. An electronic device, characterized in that: The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the entity linking method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that Used to store computer-executable instructions, which, when loaded and executed by a processor, implement the entity linking method according to any one of claims 1 to 5.

Images