A method of agricultural entity linking based on RoBERTa model

Abstract

The present invention discloses an agricultural entity linking method based on a RoBERTa model, which includes: identifying a target entity in a target text, extracting entity mentions expressing the target entity and the context of the entity mentions from the target text, and constructing a first text according to the entity mentions and the context; inputting the first text into a first RoBERTa model for text embedding to obtain a first embedding vector; inputting n second texts into a second RoBERTa model for text embedding to obtain n second embedding vectors; calculating the similarity between the first embedding vector and the n second embedding vectors, and determining the k second embedding vectors with the highest similarity to the first embedding vector, using the k entities corresponding to the k second embedding vectors as k candidate entities, and performing entity disambiguation based on the k candidate entities to obtain a target candidate entity; linking the entity mentions of the target entity to the target candidate entity in the knowledge graph. The present invention improves the accuracy of entity linking.

Description

Technical Field

[0001] The present invention relates to the field of entity linking technology, and in particular to an agricultural entity linking method based on a RoBERTa model. Background Art

[0002] An entity is an object or thing that exists objectively and can be distinguished from each other in the real world. A mention is a language fragment that expresses an entity in a natural text, that is, a natural language fragment that involves an entity. For example, for the entity "corn", "corn is an annual herbaceous plant of the Poaceae family and the genus Zea" is a mention of the entity.

[0003] Knowledge graph is a semantic network that aims to describe the conceptual entities of the objective world and the relationships between them. Entity is the basic unit of knowledge graph, so the mention of entity can find the corresponding linked entity in the knowledge graph. The process of finding entity for mention is called entity linking. Entity linking is the task of associating entities appearing in the text with corresponding knowledge graph entities, and is a key method for knowledge fusion in knowledge graph. Entity linking has applications in many aspects such as question answering, text analysis, information / retrieval, and agricultural knowledge base expansion.

[0004] The entity linking method mainly includes two steps: entity recognition and entity disambiguation. Among them, entity recognition is to use the entity recognition model to identify the mention of the expression entity in a given text; entity disambiguation mainly includes the initial screening and sorting steps of candidate entities. Among them, the initial screening of candidate entities is to screen the entities that the mentions may link, and the candidate entity sorting is to select the entity that the mention should actually link from the candidate entities; then, the mention can be linked to the entity with the highest ranking. Entity linking can be applied in the agricultural field, such as linking entities such as crop names and pest names in agricultural reports. In this way, agricultural data can be understood and analyzed more accurately.

[0005] With the development of deep learning, there is a practice of using neural network models to generate candidate entities. It mainly uses sequence labeling models to model agricultural texts and mark the scope of entities. The current mainstream solution is the Bi-LSTM-CRF labeling model method based on word embedding.

[0006] However, when the sequence labeling model performs word embedding, the same word will be represented as the same vector. However, some words are polysemous, or the same word has different semantics in different contexts. This will lead to agricultural entity linking errors. Summary of the invention

[0007] In order to solve the above problems existing in the prior art, the present invention provides an agricultural entity linking method based on the RoBERTa model.

[0008] The present invention discloses an agricultural entity linking method based on a RoBERTa model, which comprises:

[0009] Identify a target entity in a target text, extract entity mentions expressing the target entity and contexts of the entity mentions from the target text, and construct a first text based on the entity mentions and the contexts;

[0010] Input the first text into a first RoBERTa model for text embedding to obtain a first embedding vector; input n second texts into a second RoBERTa model for text embedding to obtain n second embedding vectors;

[0011] Calculate the similarity between the first embedding vector and the n second embedding vectors, and determine the k second embedding vectors with the highest similarity to the first embedding vector; the n second texts correspond one-to-one to all the n entities in the knowledge graph, and each second text is constructed according to the title and description of the corresponding entity in the knowledge graph;

[0012] The k entities corresponding to the k second embedding vectors are taken as k candidate entities, and entity disambiguation is performed based on the k candidate entities to obtain a target candidate entity; and the entity mention of the target entity is linked to the target candidate entity in the knowledge graph.

[0013] Furthermore, the first RoBERTa model is trained based on multiple first text samples, and the second RoBERTa model is trained based on multiple second text samples.

[0014] Furthermore, performing entity disambiguation based on the k candidate entities to obtain a target candidate entity includes:

[0015] Obtain k second texts corresponding to the k candidate entities, and construct a third text according to the first text and each second text in the k second texts, to obtain k third texts;

[0016] Inputting the k third texts into a third RoBERTa model for text embedding to obtain k third embedding vectors; the third RoBERTa model is trained based on multiple third text samples;

[0017] Calculate k scores corresponding to the k third embedding vectors as the scores of the k candidate entities; and select the candidate entity with the highest score as the target candidate entity.

[0018] Furthermore, the third RoBERTa model is obtained through training for multiple training cycles, and the training method in each training cycle is as follows:

[0019] From the plurality of third text samples, select a third text sample that has not participated in training and input it into a third RoBERTa model in training to obtain a third embedding vector;

[0020] The obtained third embedding vector is input into the Softmax function to calculate the probability value;

[0021] Calculating a second loss value according to the probability value and the label of the selected third text sample;

[0022] If the second loss value is not less than the preset threshold, adjust the network weight parameters of the current third RoBERTa model, and then enter the next training cycle;

[0023] If the second loss value is less than the threshold, the training is terminated to obtain a third RoBERTa model that has completed the training.

[0024] Furthermore, the label is equal to 0 or 1; when the label is equal to 0, it indicates that the entity corresponding to the context and mention in the selected third text sample is different from the entity corresponding to the title and description in the third text sample; when the label is equal to 1, it indicates that the entity corresponding to the context and mention in the selected third text sample is the same as the entity corresponding to the title and description in the third text sample.

[0025] Furthermore, the third text is constructed as follows:

[0026] x m,e =[CLS]ctxl l mention ctxl r [ENT]title[ENT]description[SEP];

[0027] Among them, mention is the entity mention contained in the first text, ctxl l is the above text in the context contained in the first text, ctxl r is the following text in the context contained in the first text; title is the title contained in the second text, description is the description contained in the second text; [CLS] is an inserted sentence start symbol, [SEP] is an inserted sentence end symbol, [ENT] is an inserted separator, x m,e Indicates the third text.

[0028] Further, k scores are calculated according to the k third embedding vectors, including:

[0029] Perform a linear transformation on the k third embedding vectors to obtain a linear transformation value corresponding to each third embedding vector as a score calculated corresponding to the third embedding vector.

[0030] Furthermore, the first RoBERTa and the second RoBERTa are obtained through synchronous training over multiple training cycles, and the training method in each training cycle is as follows:

[0031] From the multiple first text samples, select a first text sample that has not participated in the training and input it into the first RoBERTa model being trained, and input all the second text samples into the second RoBERTa model being trained, respectively, to obtain a first embedding vector output by the first RoBERTa model and multiple second embedding vectors output by the second RoBERTa model;

[0032] Calculate the similarity between the obtained first embedding vector and each obtained second embedding vector;

[0033] Calculate a first loss value according to the similarity calculation result;

[0034] If the first loss value is not less than a preset threshold, the network weight parameters of the current first RoBERTa model and the second RoBERTa model are adjusted synchronously, and then the next training cycle is entered;

[0035] If the first loss value is less than the threshold, the training is terminated to obtain the trained first RoBERTa model and the second RoBERTa model.

[0036] Furthermore, the loss function used to calculate the first loss value is as follows:

[0037]

[0038] Where loss represents the first loss value, a is an adjustable parameter to balance MSE_loss and Triplet_loss, a∈[0,1]; y m represents the first text sample input to the first RoBERTa model, Represents y m The second text sample with the highest similarity, for y m and similarity; Represents y m The second text sample with the lowest similarity, for y m and Similarity of label i Equal to 0 or 1, when labeli =1 indicates The corresponding entity and y m The corresponding entities are the same, when label i =0 indicates The corresponding entity and y m The corresponding entities are different; margin is another adjustable parameter used to increase and The gap between.

[0039] Furthermore, the first text is constructed in the following manner:

[0040] x m =[CLS]ctxl l mention ctxl r [SEP];

[0041] Among them, mention is the entity mentioned, ctxl l For the above context, ctxl r is the following text in the context, [CLS] is the inserted beginning symbol, [SEP] is the inserted end symbol, x m Indicates the first text;

[0042] The second text is constructed as follows:

[0043] x e =[CLS]title[ENT]description[SEP];

[0044] Wherein, title is the title, description is the description, [CLS] is the inserted sentence start symbol, [SEP] is the inserted sentence end symbol, [ENT] is the inserted separator, x e Indicates the second text.

[0045] In the agricultural entity linking method based on the RoBERTa model provided by the present invention, n second embedding vectors are constructed in advance according to all n entities in the knowledge graph using the second RoBERTa model, and the title and description of the entity in the knowledge graph are used in the process of constructing the second embedding vector; in the actual linking process, the mention of the target entity and the context of the mention are extracted from the target text, and the first text is constructed accordingly, which is input into the first RoBERTa model to obtain the first embedding vector. Therefore, the process of calculating the similarity between the first embedding vector and the n second embedding vectors is equivalent to retrieving the candidate entity that best matches the semantics of the target entity in the target text from the knowledge graph, so that even if the target entity is a polysemous word or has different semantics in different contexts, the candidate entity can be retrieved according to its true semantics in the target text, thereby performing entity disambiguation based on the candidate entity, and the mention of the target entity can be connected to its true corresponding entity, which is not easy to cause entity linking errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0047] Figure 1 It is a flow chart of an agricultural entity linking method based on a RoBERTa model provided by an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the structure of the existing RoBERTa model;

[0049] Figure 3 It is a flow chart of another agricultural entity linking method based on the RoBERTa model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described in conjunction with the accompanying drawings and embodiments, and the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present invention.

[0051] The embodiment of the present invention provides an agricultural entity linking method based on the RoBERTa model, which can make full use of entity mentions and their contextual information by using a deep learning model, thereby effectively reducing the error probability of entity linking.

[0052] See also Figure 1As shown, the method provided by the embodiment of the present invention includes the following steps:

[0053] S10: Identify a target entity in a target text, and extract entity mentions expressing the target entity and the context of the entity mentions from the target text.

[0054] Specifically, some existing entity recognition tools such as CoreNLP, NLTK, foolnltk, LTP, LAC, BosonNLP or HanLP can be used to identify the target entity in the target text. Then, according to the punctuation marks in the target text, the sentences containing the target entity are extracted from the target text as entity mentions, and the context of the entity mention is extracted accordingly.

[0055] S20: Construct a first text according to the entity mention and the context.

[0056] Specifically, the first text is constructed as follows:

[0057] x m =[CLS]ctxl l mention ctxl r [SEP];

[0058] Among them, mention is the entity mention, ctxl l The context above the entity mention, ctxl r is the following text in the context of the entity mention, [CLS] is the inserted sentence start symbol, [SEP] is the inserted sentence end symbol, x m Indicates the first text.

[0059] S30: Input the first text into the first RoBERTa model for text embedding to obtain a first embedding vector.

[0060] The first RoBERTa model has the same network structure as the existing RoBERTa (Robustly optimized BERT approach) model.

[0061] The RoBERTa model (proposed in the paper "RoBERTa: A Robustly Optimized BERT Pretraining Approach") is an enhanced version of BERT and a more finely tuned version than the BERT model. The BERT model is an improved architecture fine-tuning method based on the bidirectional encoding representation of Transformer (proposed by Google in the paper Attention Is All You Need published at the end of 2017). The RoBERTa model further optimizes this. Its model structure is as follows Figure 2 As shown: E1~E N They correspond to individual words in the text input into the model, Trm represents the Transformer Block of the Transformer model, T1~T N Both are hidden layers, used to output hidden layer vectors; among them, T1 is the embedding vector at the beginning of the sentence [CLS] tag generated after processing by the BERT model, which can be used to represent the entire sentence text input into the RoBERTa model.

[0062] The RoBERTa model can be fine-tuned through an additional output layer, making it suitable for building state-of-the-art models for a wide range of tasks, such as question-answering tasks and language reasoning, without making major architectural modifications for specific tasks.

[0063] In this embodiment of the present invention, the first embedding vector is the hidden layer vector output by T1, which can be expressed as:

[0064] y m =CLS(RoBERTa(x m ));

[0065] In this formula, x m is the first text, RoBERTa(x m ) indicates that the first RoBERTa model is used to train x m Text embedding, CLS(RoBERTa(x m )) means taking the hidden layer vector at the beginning [CLS] tag in the embedding result.

[0066] The first RoBERTa model can be obtained by training based on a plurality of first text samples. The first text samples have the same format as the first text and are training samples used when training the first RoBERTa model.

[0067] S40: Calculate the similarity between the first embedding vector and the preset n second embedding vectors, and determine the k second embedding vectors with the highest similarity to the first embedding vector, k<n; wherein the n second embedding vectors are obtained by inputting n second texts into the second RoBERTa model for text embedding; the n second texts correspond one-to-one to all the n entities in the knowledge graph, and each second text is constructed according to the title and description of the corresponding entity in the knowledge graph.

[0068] It is understandable that each entity in the knowledge graph has a corresponding title and description. For example, in the agricultural knowledge base, for the entity "corn", its title is crop plant, and its description is: corn is an annual herbaceous crop plant of the genus Zea mays of the family Poaceae. Therefore, the second text can be pre-constructed according to the knowledge graph, and the second embedding vector can be constructed accordingly.

[0069] Among them, the second text is constructed as follows:

[0070] x e =[CLS]title[ENT]description[SEP];

[0071] Among them, title is the title of the entity in the knowledge graph, description is the description of the entity in the knowledge graph, [CLS] is the inserted sentence start symbol, [SEP] is the inserted sentence end symbol, [ENT] is the inserted separator, x e Indicates the second text.

[0072] The embodiment of the present invention does not limit the type of knowledge graph, for example, it can be an agricultural knowledge base, etc.

[0073] The second RoBERTa model also has the same network structure as the existing RoBERTa model. The second embedding vector is the hidden layer vector output by T1 of the second RoBERTa model. The second embedding vector can be expressed as:

[0074] y e =CLS(RoBERTa(x e ));

[0075] In this formula, x e For the second text, RoBERTa(x e ) indicates that the second RoBERTa model is used to train x e For text embedding, CLS(RoBERTa(x e )) means taking the hidden layer vector at the beginning [CLS] tag in the embedding result.

[0076] The second RoBERTa model can be trained based on multiple second text samples. The second text sample has the same format as the second text and is a training sample used when training the second RoBERTa model. In practical applications, the second text sample used to train the second RoBERTa model can be directly used as the n second texts constructed when constructing the n second embedding vectors.

[0077] In this step, the similarity between the first embedding vector and each second embedding vector may be calculated by calculating the cosine similarity of the two vectors, or by calculating the Euclidean distance.

[0078] S50: taking the k entities corresponding to the k second embedding vectors as k candidate entities, and performing entity disambiguation based on the k candidate entities to obtain a target candidate entity.

[0079] For entity disambiguation based on k candidate entities, reference may be made to existing entity disambiguation methods, or an implementation method is proposed by the embodiment of the present invention, which specifically includes:

[0080] (1) Obtain k second texts corresponding to k candidate entities;

[0081] (2) constructing a third text based on the first text and each of the k second texts, thereby obtaining k third texts;

[0082] (3) inputting the k third texts into the third RoBERTa model for text embedding to obtain k third embedding vectors;

[0083] (4) Calculate k scores according to the k third embedding vectors as the scores of the k candidate entities;

[0084] (5) Select the candidate entity with the highest score as the target candidate entity.

[0085] It can be understood that when the knowledge graph is used to construct n second embedding vectors in advance, the second text has been pre-constructed for each entity in the knowledge graph. Therefore, in step (1), the k second texts corresponding to the k candidate entities that have been pre-constructed and stored can be directly obtained.

[0086] In step (2), the third text is constructed as follows:

[0087] x m，e =[CLS]ctxl l mentionctxl r [ENT]title[ENT]description[SEP];

[0088] Among them, mention is the entity mention contained in the first text, ctxl l The context contained in the first text, ctxl r is the following text in the context of the first text; title is the title contained in the second text, description is the description contained in the second text; [CLS] is the inserted beginning symbol, [SEP] is the inserted end symbol, [ENT] is the inserted separator, x m，e Indicates the third text.

[0089] The third RoBERTa model has the same network structure as the existing RoBERTa model. The third embedding vector is the hidden layer vector output by T1 of the third RoBERTa model, and its expression is:

[0090] y m，e =CLS(RoBERTa(x m，e ));

[0091] In this formula, x m，e For the third text, RoBERTa(x m，e ) indicates that the third RoBERTa model is used to train x m，e For text embedding, CLS(RoBERTa(x m，e )) means taking the hidden layer vector at the beginning [CLS] tag in the embedding result.

[0092] The third RoBERTa model can be trained based on multiple third text samples; the third text samples have the same format as the third text and are training samples used when training the third RoBERTa model. The third text samples include positive samples and negative samples; wherein, for the third text samples belonging to positive samples, the entities corresponding to the context and entity mentions contained therein are the same as the entities corresponding to the titles and descriptions contained therein; and for the third text samples belonging to negative samples, the entities corresponding to the context and entity mentions contained therein are different from the entities corresponding to the titles and descriptions contained therein.

[0093] In step (4), k scores are calculated according to the k third embedding vectors. Specifically, the k third embedding vectors are linearly transformed to obtain the linear transformation value corresponding to each third embedding vector as the score calculated according to the third embedding vector. This calculation process can be expressed as follows:

[0094] s(y m,e )=Wy m,e ;

[0095] Among them, y m,e represents the third embedding vector, W is a function that implements the linear transformation operation, which can be regarded as a linear layer, s(y m,e ) is the corresponding calculated score.

[0096] In practical applications, nn.Linear in Pytorch can be used to implement the functionality of function W. PyTorch is an open source Python machine learning library, and nn.Linear is a function in the library.

[0097] S60: Link the entity mentions of the target entity to the target candidate entity in the knowledge graph.

[0098] Thus, an entity linking task is completed.

[0099] In summary, in the agricultural entity linking method based on the RoBERTa model provided in the embodiment of the present invention, n second embedding vectors are constructed in advance according to all n entities in the knowledge graph using the second RoBERTa model, and the title and description of the entity in the knowledge graph are used in the process of constructing the second embedding vector; in the actual linking process, the entity mention expressing the target entity and the context of the entity mention are extracted from the target text, and the first text is constructed accordingly, which is input into the first RoBERTa model to obtain the first embedding vector. Therefore, the process of calculating the similarity between the first embedding vector and the n second embedding vectors is equivalent to retrieving the candidate entity that best matches the semantics of the target entity in the target text from the knowledge graph, so that even if the target entity is a polysemous word or has different semantics in different contexts, the candidate entity can be retrieved according to its true semantics in the target text, thereby performing entity disambiguation based on the candidate entity, and the entity mention of the target entity can be connected to its true corresponding entity, which is not easy to cause entity linking errors.

[0100] Optionally, in one implementation, the first RoBERTa and the second RoBERTa may be obtained through synchronous training of multiple training cycles, and the training method in each training cycle is specifically as follows:

[0101] (a) From a plurality of first text samples, a first text sample that has not participated in training is selected and input into a first RoBERTa model being trained, and all second text samples are respectively input into a second RoBERTa model being trained, to obtain a first embedding vector output by the first RoBERTa model and a plurality of second embedding vectors output by the second RoBERTa model;

[0102] (b) calculating the similarity between the obtained first embedding vector and each obtained second embedding vector;

[0103] (c) calculating a first loss value according to the similarity calculation result;

[0104] (d) if the first loss value is not less than a preset threshold, synchronously adjust the network weight parameters of the current first RoBERTa model and the second RoBERTa model, and then enter the next training cycle;

[0105] (e) If the first loss value is less than the threshold, the training is terminated to obtain the first RoBERTa model and the second RoBERTa model that have been trained.

[0106] The similarity calculated in step (b) may be calculated by calculating cosine similarity or Euclidean distance, etc.

[0107] The loss function used to calculate the first loss value in step (c) is as follows:

[0108]

[0109] in,

[0110] loss represents the first loss value, a is an adjustable parameter to balance MSE_loss and Triplet_loss, a∈[0,1]; y m represents the first text sample input to the first RoBERTa model, Represents y m The second text sample with the highest similarity, for y m and similarity; Represents y m The second text sample with the lowest similarity, for y m and Similarity of label i Equal to 0 or 1, when label i =1 indicates The corresponding entity and y m The corresponding entities are the same, when label i =0 indicates The corresponding entity and y m The corresponding entities are different; margin is another adjustable parameter used to widen the gap between the correct matching pairs and the incorrect matching pairs, that is, to increase and The gap between.

[0111] In the above loss function, each calculation of MSE_loss uses It is from y m The loss should be considered from the perspective of linking to the correct entity. However, this method of calculating the loss makes it difficult for the model to learn the differences in similarity samples. Each time riplet_loss is calculated, the triple It considers correctly linked entities to be a better match for the description than incorrectly linked entities.

[0112] riplet_loss is a triplet loss. The purpose of calculating the triplet loss is to make the distance between the feature expression as small as possible through learning, and there is a gap between the feature expression. It can learn the fine-grained distinction pattern of similar samples well, and can learn a better representation vector for two inputs with small differences, thereby achieving more accurate classification.

[0113] However, if the triplet loss is calculated independently to train the RoBERTa model, the sample triplets selected when calculating the triplet loss may not be evenly distributed, resulting in unstable performance during model training, slower convergence, and more prone to overfitting.

[0114] It is understandable that the model optimization goal is to make the positive example similarity as large as possible and the negative example similarity as small as possible. Therefore, in order to achieve this goal, the embodiment of the present invention adopts the joint synchronous training of the first RoBERTa model and the second RoBERTa model, and fuses the two losses to form an overall loss, thereby avoiding a series of problems such as poor similarity sample learning effect, slow convergence speed, and easy overfitting that may be caused by calculating MSE_loss or riplet_loss alone.

[0115] In addition, the synchronous adjustment in step (d) refers to using the same network weight optimization method to consistently adjust the network weight parameters of the current first RoBERTa model and the second RoBERTa model, wherein the network weight optimization method is such as the gradient descent method.

[0116] The training method of the third RoBERTa model is described below; the third RoBERTa model is obtained through training for multiple training cycles, and the training method in each training cycle is as follows:

[0117] (A) selecting a third text sample that has not participated in training from multiple third text samples and inputting it into a third RoBERTa model being trained to obtain a third embedding vector;

[0118] (B) Input the third embedding vector into the Softmax function to calculate a probability value;

[0119] (C) calculating a second loss value according to the probability value and the label of the selected third text sample;

[0120] (D) If the second loss value is not less than the preset threshold, adjust the network weight parameters of the current third RoBERTa model, and then enter the next training cycle;

[0121] (E) if the second loss value is less than the threshold, the training is terminated to obtain a third RoBERTa model that has completed the training;

[0122] Among them, the label of the third text sample is equal to 0 or 1; when the label of the third text sample is equal to 0, the third text sample is a negative sample, indicating that the entity corresponding to the context and entity mention in the selected third text sample is different from the entity corresponding to the title and description in the third text sample; when the label of the third text sample is equal to 1, the third text sample is a positive sample, indicating that the entity corresponding to the context and entity mention in the selected third text sample is the same as the entity corresponding to the title and description in the third text sample.

[0123] In steps (B) and (C), the probability value is calculated using the Softmax function according to the third embedding vector, and the second loss value is calculated based on the probability value and the label of the third text sample, so that the score of the correct entity and description pair can be maximized. Among them, what is specifically calculated in step (C) is the statistical average of the difference between the probability value and the label of the selected third text sample.

[0124] In a specific embodiment, the process of implementing entity linking based on the RoBERTa model can be as follows: Figure 3 As shown, the steps in the rounded rectangle box are completed in advance and are not included in the actual entity connection process. It can be seen that in this embodiment, a total of three RoBERTa models are used from the preparation stage to the actual use stage; among them, the second RoBERTa model is used in the preparation stage to encode the titles and descriptions of all entities in the knowledge graph to form n second embedding vectors. In the actual use stage, the first RoBERTa model is used to encode the entity mentions and context of the target entity in the target text to form a first embedding vector; then, by calculating the similarity of the first embedding vector and the n second embedding vectors prepared in advance, the k second embedding vectors with the highest similarity are selected, and their corresponding entities are used as candidate entities. Then, for each candidate entity, its x e x with the target entity m After splicing, the obtained x m,e The third RoBERTa model is used for encoding to form a third embedding vector. The third embedding vector of each candidate entity is then sent to the linear layer to calculate the score, and finally the entity mention of the target entity is linked to the candidate entity with the highest score.

[0125] Based on this embodiment, it can be seen that the use of the first RoBERTa model makes full use of the entity mentions and context information of the target entity to extract the semantic information of the target entity in the target text; the use of the second RoBERTa model also makes use of the semantic information contained in the description of the entity in the knowledge graph; thus, in the process of retrieving candidate entities, these two aspects of semantic information are effectively utilized, making the retrieval of candidate entities more accurate. In addition, the use of the third RoBERTa model is equivalent to the use of the entity mentions and context information from x m The semantic information in x e The synonymy of the semantic information in the sentence is scored, which solves the problem of entity linking errors caused by different semantic information of the same entity in different contexts, thereby making the final selected target candidate entity closer to the real target entity.

[0126] In summary, the agricultural entity linking method based on the RoBERTa model provided by the embodiment of the present invention can effectively reduce the error probability of agricultural entity linking and improve the accuracy of agricultural entity linking. The method is simple, effective, accurate and fast.

[0127] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for agricultural entity linking based on the RoBERTa model, characterized in that: include: Identify a target entity in a target text, extract entity mentions expressing the target entity and contexts of the entity mentions from the target text, and construct a first text based on the entity mentions and the contexts; Input the first text into a first RoBERTa model for text embedding to obtain a first embedding vector; input n second texts into a second RoBERTa model for text embedding to obtain n second embedding vectors; Calculate the similarity between the first embedding vector and the n second embedding vectors, and determine the k second embedding vectors with the highest similarity to the first embedding vector; the n second texts correspond one-to-one to all the n entities in the knowledge graph, and each second text is constructed according to the title and description of the corresponding entity in the knowledge graph; Taking the k entities corresponding to the k second embedding vectors as k candidate entities, and performing entity disambiguation based on the k candidate entities to obtain a target candidate entity; linking the entity mention of the target entity to the target candidate entity in the knowledge graph; The first RoBERTa model is trained based on a plurality of first text samples, and the second RoBERTa model is trained based on a plurality of second text samples; The performing entity disambiguation based on the k candidate entities to obtain a target candidate entity includes: Obtain k second texts corresponding to the k candidate entities, and construct a third text according to the first text and each second text in the k second texts, to obtain k third texts; Inputting the k third texts into a third RoBERTa model for text embedding to obtain k third embedding vectors; the third RoBERTa model is trained based on multiple third text samples; Calculating k scores corresponding to the k third embedding vectors as scores of the k candidate entities; selecting the candidate entity with the highest score as the target candidate entity; The third RoBERTa model is obtained through training for multiple training cycles, and the training method in each training cycle is as follows: From the plurality of third text samples, select a third text sample that has not participated in training and input it into a third RoBERTa model in training to obtain a third embedding vector; The obtained third embedding vector is input into the Softmax function to calculate the probability value; Calculating a second loss value according to the probability value and the label of the selected third text sample; If the second loss value is not less than the preset threshold, adjust the network weight parameters of the current third RoBERTa model, and then enter the next training cycle; If the second loss value is less than the threshold, the training is terminated to obtain a third RoBERTa model that has completed the training.

2. The method according to claim 1, characterized in that: The label is equal to 0 or 1; when the label is equal to 0, it indicates that the entity corresponding to the context and mention in the selected third text sample is different from the entity corresponding to the title and description in the third text sample; when the label is equal to 1, it indicates that the entity corresponding to the context and mention in the selected third text sample is the same as the entity corresponding to the title and description in the third text sample.

3. The method according to claim 1, characterized in that The third text is constructed as follows: x m,e =[CLS]ctxl l mention ctxl r [ENT]title[ENT]description[SEP]; Among them, mention is the entity mention contained in the first text, ctxl l is the above text in the context contained in the first text, ctxl r is the following text in the context contained in the first text; title is the title contained in the second text, description is the description contained in the second text; [CLS] is an inserted sentence start symbol, [SEP] is an inserted sentence end symbol, [ENT] is an inserted separator, x m,e Indicates the third text.

4. The method according to claim 1, characterized in that: Calculating k scores according to the k third embedding vectors, including: Perform a linear transformation on the k third embedding vectors to obtain a linear transformation value corresponding to each third embedding vector as a score calculated corresponding to the third embedding vector.

5. The method according to claim 1, characterized in that The first RoBERTa and the second RoBERTa are obtained through synchronous training over multiple training cycles, and the training method in each training cycle is as follows: From the multiple first text samples, select a first text sample that has not participated in the training and input it into the first RoBERTa model being trained, and input all the second text samples into the second RoBERTa model being trained, respectively, to obtain a first embedding vector output by the first RoBERTa model and multiple second embedding vectors output by the second RoBERTa model; Calculate the similarity between the obtained first embedding vector and each obtained second embedding vector; Calculate a first loss value according to the similarity calculation result; If the first loss value is not less than a preset threshold, the network weight parameters of the current first RoBERTa model and the second RoBERTa model are adjusted synchronously, and then the next training cycle is entered; If the first loss value is less than the threshold, the training is terminated to obtain the trained first RoBERTa model and the second RoBERTa model.

6. The method according to claim 1, characterized in that The loss function used to calculate the first loss value is as follows: Where loss represents the first loss value, a is an adjustable parameter to balance MSE_loss and Triplet_loss, a∈[0,1]; y m represents the first text sample input to the first RoBERTa model, Indicates that m The second text sample with the highest similarity, for y m and similarity; Indicates that m The second text sample with the lowest similarity, for y m and Similarity of label i Equal to 0 or 1, when label i =1 indicates The corresponding entity and y m The corresponding entities are the same, when label i =0 indicates The corresponding entity and y m The corresponding entities are different; margin is another adjustable parameter used to increase and The gap between.

7. The method according to claim 1, characterized in that The first text is constructed as follows: x m =[CLS]ctxl l mention ctxl r [SEP]; Among them, mention is the entity mentioned, ctxl l For the above context, ctxl r is the following text in the context, [CLS] is the inserted beginning symbol, [SEP] is the inserted end symbol, x m Indicates the first text; The second text is constructed as follows: x e =[CLS]title[ENT]description[SEP]; Wherein, title is the title, description is the description, [CLS] is the inserted sentence start symbol, [SEP] is the inserted sentence end symbol, [ENT] is the inserted separator, x e Indicates the second text.

Images