Low-resource cross-language entity linking method, device, electronic device and storage medium

Abstract

The present invention provides a low-resource cross-language entity linking method, device, electronic device and storage medium, which can train a pivot model using a high-resource language as a pivot language of a low-resource language, the pivot language is a character-level neural network, and is migrated to the low-resource language in a zero-transfer manner, so that after obtaining the low-resource entity referent and the candidate entity set in the text to be processed, the target English entity linked to the low-resource entity referent is determined from multiple candidate English entities in the candidate entity set based on the pivot model. Based on the present invention, low-resource cross-language entity connection can be achieved, solving the problem of difficulty in cross-language entity linking, especially for languages ​​lacking bilingual resources and Wikipedia resources.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and more specifically, to a low-resource cross-language entity linking method, device, electronic device and storage medium. Background Art

[0002] Entity linking refers to correctly linking entity references in the text to entities in the target library. First, it can improve the user's reading experience and eliminate ambiguity in the text. It can also be used for other downstream tasks to improve their accuracy and obtain accurate information about the entity.

[0003] In entity linking, cross-language entity linking is one of the difficult tasks. Cross-language entity linking is to map entity references in the source language to entity items in the target language knowledge base, that is, the text (source language) where the entity references are located and the knowledge base to be linked (target language) are two different languages. In this case, the entity references and the entity items in the knowledge base are not similar in most cases, so methods such as string matching and Wikipedia anchor text links in a single language are not applicable. Summary of the invention

[0004] In view of this, in order to solve the above problems, the present invention provides a low-resource cross-language entity linking method, device, electronic device and storage medium, and the technical solution is as follows:

[0005] A low-resource cross-language entity linking method, the method comprising:

[0006] Obtaining low-resource entity referents and a candidate entity set in the text to be processed, wherein the candidate entity set includes multiple candidate English entities;

[0007] Retrieving a pivot model that has been transferred to a low-resource language to which the low-resource entity referent belongs, wherein the pivot model is obtained based on bilingual vocabulary training between a high-resource language and English and transferred to the low-resource language in a zero-transfer manner, the pivot model is a character-level neural network, and the high-resource language is the pivot language of the low-resource language;

[0008] A target English entity to which the low-resource entity reference term is linked is determined from the multiple candidate English entities based on the pivot model.

[0009] Preferably, the process of obtaining the pivot model based on bilingual vocabulary training between a high-resource language and English includes:

[0010] Based on the bilingual vocabulary, respectively obtain a first data pair and a second data pair as positive samples, wherein the first data pair includes a high-resource entity and an English entity, and the second data pair includes a high-resource entity referent and an English entity;

[0011] Constructing a third data pair and a fourth data pair as negative samples, wherein the third data pair contains the same high-resource entity as the first data pair but different English entities, and the fourth data pair contains the same high-resource entity referent as the second data pair but different English entities;

[0012] Select sample data for this training from the positive samples and the negative samples, and input the sample data into the basic model to achieve:

[0013] The high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are randomly initialized by the character embedding layer in the basic model to obtain their respective corresponding embedded representations; the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are encoded by the graph convolutional network in the basic model based on the embedded representation to obtain their respective corresponding encoded representations; the similarities between the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are calculated by the similarity function in the basic model based on the encoded representation;

[0014] According to the similarity corresponding to the sample data, the loss function value of the basic model is calculated by negative sampling with maximum interval, the weight parameters of the character embedding layer and the graph convolutional network are reversely adjusted according to the loss function value, and the step of selecting the sample data for this training from the positive samples and the negative samples is returned to execute until the loss function value meets the end condition, and the adjusted basic model is used as the pivot model.

[0015] Preferably, determining the target English entity linked to the low-resource entity referent from the plurality of candidate English entities based on the pivot model comprises:

[0016] Determine the high-resource entity corresponding to each candidate English entity;

[0017] Inputting the low-resource entity designation and each candidate English entity into the pivot model to obtain a first similarity between the low-resource entity designation and each candidate English entity through the pivot model; and inputting the low-resource entity designation and the high-resource entity corresponding to each candidate English entity into the pivot model to obtain a second similarity between the low-resource entity designation and the high-resource entity corresponding to each candidate English entity through the pivot model;

[0018] Taking the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity;

[0019] The candidate English entity with the largest target similarity is selected as the target English entity linked to the low-resource entity referent.

[0020] Preferably, the method further comprises:

[0021] Determine at least one alias corresponding to each candidate English entity;

[0022] Inputting the low-resource entity designation and at least one alias corresponding to each candidate English entity into the pivot model respectively, so as to obtain a third similarity between the low-resource entity designation and each alias through the pivot model;

[0023] Taking the maximum value of each third similarity and the first similarity as the target first similarity between the low-resource entity referent and each candidate English entity;

[0024] Accordingly, taking the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity includes:

[0025] The maximum value between the target first similarity and the second similarity is used as the target similarity between the low-resource entity referent and each candidate English entity.

[0026] A low-resource cross-language entity linking device, the device comprising:

[0027] An acquisition module, used to acquire low-resource entity reference items and a candidate entity set in the text to be processed, wherein the candidate entity set includes multiple candidate English entities;

[0028] A linking module is used to retrieve a pivot model that has been migrated to a low-resource language to which the low-resource entity reference item belongs, wherein the pivot model is obtained based on bilingual vocabulary training between a high-resource language and English and migrated to the low-resource language in a zero-transfer manner, the pivot model is a character-level neural network, and the high-resource language is the pivot language of the low-resource language; based on the pivot model, a target English entity to which the low-resource entity reference item is linked is determined from the multiple candidate English entities.

[0029] Preferably, the process of obtaining the pivot model by training the link module based on a bilingual vocabulary between a high-resource language and English includes:

[0030] Based on the bilingual vocabulary, a first data pair and a second data pair are respectively obtained as positive samples, wherein the first data pair includes a high-resource entity and an English entity, and the second data pair includes a high-resource entity reference item and an English entity; a third data pair and a fourth data pair are constructed as negative samples, wherein the third data pair includes the same high-resource entity as the first data pair and a different English entity, and the fourth data pair includes the same high-resource entity reference item as the second data pair and a different English entity; sample data for this training is selected from the positive samples and the negative samples, and the sample data is input into a basic model to achieve:

[0031] The high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are randomly initialized by the character embedding layer in the basic model to obtain their respective corresponding embedded representations; the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are encoded by the graph convolutional network in the basic model based on the embedded representation to obtain their respective corresponding encoded representations; the similarities between the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are calculated by the similarity function in the basic model based on the encoded representation;

[0032] According to the similarity corresponding to the sample data, the loss function value of the basic model is calculated by negative sampling with maximum interval, the weight parameters of the character embedding layer and the graph convolutional network are reversely adjusted according to the loss function value, and the step of selecting the sample data for this training from the positive samples and the negative samples is returned to execute until the loss function value meets the end condition, and the adjusted basic model is used as the pivot model.

[0033] Preferably, the linking module determines the target English entity to which the low-resource entity referent is linked from the plurality of candidate English entities based on the pivot model, including:

[0034] Determine the high-resource entity corresponding to each candidate English entity; input the low-resource entity reference term and each candidate English entity into the pivot model to obtain a first similarity between the low-resource entity reference term and each candidate English entity through the pivot model; and input the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity into the pivot model to obtain a second similarity between the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity through the pivot model; use the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity reference term and each candidate English entity; and select the candidate English entity with the largest target similarity as the target English entity linked to the low-resource entity reference term.

[0035] Preferably, the link module is also used for:

[0036] Determine at least one alias corresponding to each candidate English entity; input the low-resource entity designation and at least one alias corresponding to each candidate English entity into the pivot model respectively, so as to obtain a third similarity between the low-resource entity designation and each alias through the pivot model; and use the maximum value of each third similarity and the first similarity as the target first similarity between the low-resource entity designation and each candidate English entity;

[0037] Accordingly, the linking module uses the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity, including:

[0038] The maximum value between the target first similarity and the second similarity is used as the target similarity between the low-resource entity referent and each candidate English entity.

[0039] An electronic device comprises: at least one memory and at least one processor; the memory stores an application program, the processor calls the application program stored in the memory, and the application program is used to implement the low-resource cross-language entity linking method.

[0040] A storage medium stores computer executable instructions for executing the low-resource cross-language entity linking method.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention provides a low-resource cross-language entity linking method, device, electronic device and storage medium, which can train a pivot model using a high-resource language as a pivot language of a low-resource language, the pivot language is a character-level neural network, and is migrated to the low-resource language in a zero-transfer manner, so that after obtaining the low-resource entity referent and the candidate entity set in the text to be processed, the target English entity linked to the low-resource entity referent is determined from multiple candidate English entities in the candidate entity set based on the pivot model. Based on the present invention, low-resource cross-language entity connection can be achieved, solving the problem of difficulty in cross-language entity linking, especially for languages ​​lacking bilingual resources and Wikipedia resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 A method flow chart of a low-resource cross-language entity linking method provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the structure of a basic model provided by an embodiment of the present invention;

[0046] Figure 3 A partial method flow chart of a low-resource cross-language entity linking method provided by an embodiment of the present invention;

[0047] Figure 4 A migration structure diagram of a pivot model provided by an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of the structure of a low-resource cross-language entity linking device provided by an embodiment of the present invention;

[0049] Figure 6 A hardware structure block diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] To facilitate understanding of the present invention, firstly, the relevant concepts appearing in the present invention are explained:

[0053] Low-resource languages ​​(LRL) refer to languages ​​with little or no Wikipedia data, that is, the language has only a few Wikipedia pages, for example, Tigrinya has only 168 Wikipedia pages. Correspondingly, low-resource entity referents are entity referents in low-resource languages.

[0054] High-resource languages ​​(HRL): refers to languages ​​with enough Wikipedia data, for example, English has more than 5 million Wikipedia pages. Accordingly, high-resource entities are entities in high-resource languages, high-resource entity referents are entity referents in high-resource languages, and English entities are entities in English.

[0055] Pivot language: It refers to low-resource languages. It is a high-resource language with similar character forms to low-resource languages ​​(most likely a cognate language). For example, if the low-resource language is Marathi (such as entity ), then its pivot language is Hindi language (such as entity ).

[0056] With the continuous advancement of technology and the continuous increase in Internet users, network data is showing an explosive growth trend, and text data is the main way for people to obtain information. However, natural language itself has problems such as ambiguity and diversity, which is particularly serious for named entities.

[0057] Entity ambiguity means that an entity reference in a text may refer to multiple entities, and entity diversity means that an entity has multiple expressions, such as abbreviations, aliases, standard names, etc. Due to the diversity and ambiguity of natural language expressions, people need to process a large amount of irrelevant information to obtain the target information. On the other hand, knowledge communities such as Wikipedia contain a large number of real-world entities and contain rich entity knowledge. Therefore, if the entity name of a single information in the text can be mapped to the entity in the knowledge base, then the ambiguity and diversity of natural language will be solved, which requires entity linking technology.

[0058] Entity linking refers to correctly linking entity references in the text to entities in the target library. First, it can improve the user's reading experience and eliminate ambiguity in the text. It can also be used for other downstream tasks to improve their accuracy and obtain accurate information about the entity.

[0059] In entity linking, cross-language entity linking is one of the difficult tasks. Cross-language entity linking means that the text (source language) where the entity referent is located and the knowledge base to be linked (target language) are in two different languages. In this case, the entity referent and the entity in the knowledge base are not similar in most cases, so methods such as string matching and Wikipedia anchor text links in a single language are not applicable. Cross-language entity linking is not only beneficial to international communication in non-English regions, but can also be used for a variety of downstream tasks, such as cross-language question-answering systems and information extraction in languages ​​without rich knowledge base resources.

[0060] There are also some methods for cross-lingual entity linking, which use bilingual resources to bridge the gap between different languages, such as cross-lingual links between dictionaries and Wikipedia. These methods have been successful in high-resource languages, but this cannot be extended well to low-resource languages ​​because these resources are scarce or even unavailable for low-resource languages. There are also some methods that directly extract "entity referent-entity" pairs from Wikipedia in low-resource languages, and some methods first translate the entity referents in low-resource languages ​​into English and then obtain the corresponding entity items. However, most languages ​​in the world are low-resource, with few or even no bilingual resources, and there are few entity items in these languages ​​in Wikipedia.

[0061] Recently, transfer learning methods have been shown to reduce the demand for resources in low-resource languages ​​(e.g., Wikipedia, bilingual interaction resources between low-resource languages ​​and applications) by leveraging resources in closely related pivot languages.

[0062] In this regard, the present invention adopts an entity linking method based on a pivot model to achieve low-resource cross-language entity linking. Although many languages ​​have very few resources, there are still high-resource languages ​​that are closely related to them, such as low-resource languages ​​Marathi and Lao, which come from the same language family as high-resource languages ​​Hindi and Thai, respectively. The present invention uses bilingual dictionaries and structural information in these high-resource languages ​​to improve entity linking in low-resource languages, which are pivot languages.

[0063] However, previous pivot-based entity linking methods also have some shortcomings, such as the mismatch between entity names in the knowledge base and entity referents in the text. The reason is that the training data consists of "entity-entity" pairs, and its variant information cannot be obtained. Therefore, the present invention adds "entity-entity referent" pairs to the training data to provide clear supervision, helping the model to better obtain the differences between entity referents and entities. In addition, it is noted that many variants of the source language and variants of entities in English are similar, so the present invention obtains aliases for English entities and allows the model to query these aliases when linking. In summary, the improved pivot-based entity linking method adopted by the present invention is superior to previous methods and has high application value.

[0064] See also Figure 1 , Figure 1 The low-resource cross-language entity linking method provided by the embodiment of the present invention comprises the following steps:

[0065] S10, obtaining low-resource entity referents and a candidate entity set in the text to be processed, where the candidate entity set includes multiple candidate English entities.

[0066] In the embodiment of the present invention, multiple candidate English entities in the candidate entity set may be pre-specified, which is not limited in the embodiment of the present invention.

[0067] S20, retrieve the pivot model that has been migrated to the low-resource language to which the low-resource entity referent belongs. The pivot model is trained based on the bilingual vocabulary between the high-resource language and English, and migrated to the low-resource language in a zero-time migration manner. The pivot model is a character-level neural network, and the high-resource language is the pivot language of the low-resource language.

[0068] In the embodiment of the present invention, for the low-resource language to which the low-resource entity referent belongs, it has a corresponding pivot model, which is trained based on a bilingual vocabulary (also referred to as a bilingual dictionary) between a high-resource language and English as the pivot language of the low-resource language. The pivot model is a character-level neural network that can connect the entity referent in the high-resource language to the English knowledge base. In addition, after the pivot model is trained on the high-resource language, it can be transferred to the low-resource language without any fine-tuning operation, that is, it can be transferred to the low-resource language in a zero-transfer manner. This is because the low-resource language is highly similar to the high-resource language as its pivot language, so the pivot model of the high-resource language can be used to process the low-resource language.

[0069] It should be noted that by using a high-resource language as the pivot language between low-resource languages ​​and English, entity references in the low-resource language (i.e., low-resource entity references) can not only be directly linked to English entities, but can also first be linked to entities in a closely related high-resource language (i.e., the pivot language), and then the bilingual vocabulary can be used to map the entities in the high-resource language to the English entities.

[0070] In order to solve the mismatch problem between entity names in the knowledge base and entity referents in the text, the present invention adds entity-entity referent pairs on the basis of entity-entity pairs when training the pivot model. In the specific implementation process, the process of obtaining the pivot model based on the bilingual vocabulary training between the high-resource language and English can include the following steps:

[0071] Based on the bilingual vocabulary, a first data pair and a second data pair are respectively obtained as positive samples, wherein the first data pair includes a high-resource entity and an English entity, and the second data pair includes a high-resource entity referent and an English entity;

[0072] Constructing a third data pair and a fourth data pair as negative samples, wherein the third data pair contains the same high-resource entity as the first data pair but different English entities, and the fourth data pair contains the same high-resource entity referent as the second data pair but different English entities;

[0073] Select sample data for this training from positive samples and negative samples, and input the sample data into the basic model to achieve:

[0074] The high-resource entities and English entities / high-resource entity referents and English entities in the sample data are randomly initialized through the character embedding layer in the basic model to obtain their corresponding embedded representations; the high-resource entities and English entities / high-resource entity referents and English entities in the sample data are encoded based on the embedded representations through the graph convolutional network in the basic model to obtain their corresponding encoded representations; the similarity between the high-resource entities and English entities / high-resource entity referents and English entities in the sample data is calculated based on the encoded representations through the similarity function in the basic model;

[0075] According to the similarity corresponding to the sample data, the loss function value of the basic model is calculated by negative sampling with the maximum interval, and the weight parameters of the character embedding layer and the graph convolutional network are reversely adjusted according to the loss function value. Then, the step of selecting sample data for this training from positive and negative samples is returned to execute until the loss function value meets the end condition, and the adjusted basic model is used as the pivot model.

[0076] In the embodiment of the present invention, the pivot model can be applied to multiple low-resource languages ​​at the same time. Accordingly, during training, it is necessary to prepare high-resource languages ​​closely related to each low-resource language, that is, the pivot language of each low-resource language, such as a high-resource language of the same family. For example, the pivot language of the low-resource language Marathi can be the high-resource language Hindi, where:

[0077] The correspondence between Hindi and English in the training set is as follows:

[0078] 2991300|||Cobie Smulders||| |||PER

[0079] The correspondence between Marathi and English in the test set is as follows:

[0080] 2991300|||Cobie Smulders|||

[0081] Thus, we can see that for the same entity, the surface forms of Hindi and Marathi are similar.

[0082] Furthermore, entity-entity pairs and entity-entity referent pairs in high-resource languages ​​(i.e., pivot languages) that are closely related to low-resource languages ​​can be used as training sets. Specifically:

[0083] First, based on the bilingual vocabulary, we can directly obtain entity-entity pairs between high-resource languages ​​and English, that is, first data pairs, which contain entities in high-resource languages ​​(i.e., high-resource entities, which are subsequently referred to as e HRL Indicates) and entities in English (i.e. English entities, which are subsequently represented by e EN The first data pair is a high-resource entity-English entity pair, represented by e HRL -e EN In order to solve the mismatch between the entity name in the knowledge base and the entity reference in the text, we continue to obtain the entity reference in the high-resource language (i.e., high-resource entity reference, which will be used later in m HRL Indicates) and entities in English (i.e. English entities, represented by e EN The second data pair is the high-resource entity reference item-English entity pair, represented by m HRL -e EN Specifically, we can first collect high-resource entity referent-high-resource entity pairs (i.e., m HRL -e HRL ), and then the high resource entity e HRL Redirect to the corresponding English Wikipedia, and finally get the high-resource entity referent-English entity pair m HRL -e EN . Let's continue with the high-resource language Hindi as an example. If the entity in Hindi can correspond to Cobie Smulders in English, then the two constitute a high-resource entity-English entity pair; (Smulders) can link to (Cobie Smulders), then and CobieSmulders constitute a high-resource entity referent-English entity pair.

[0084] Furthermore, in the present invention, the first data pair and the second data pair are used as positive samples, and the third data pair and the fourth data pair can be constructed as negative samples based on the first data pair and the second data pair. The third data pair corresponds to the first data pair, and the high-resource entities in the two are the same, but the English entities are different; the fourth data pair corresponds to the second data pair, and the high-resource entity referents in the two are the same, but the English entities are different. The English entities in the third data pair and the fourth data pair are subsequently used express.

[0085] See also Figure 2The structural diagram of the basic model shown in Figure 2. The basic model includes a character embedding layer (i.e., character embedding in the figure), a graph convolution network (i.e., GCN in the figure, full name Graph convolution Network) and a similarity function (i.e., Cosine Similarity in the figure). In Figure 2, the high-resource entity and the high-resource entity referent use the same graph convolution network (i.e., HRL-GCN), while the English entity uses another graph convolution network (i.e., English-GCN).

[0086] exist Figure 2 In the character-level model shown, each character of a word has an embedding representation, which is randomly initialized through the character embedding layer, and the embedding dimension of the word is 1024. Taking the English entity Cobie Smulders as an example, the embedding (i.e., embedding representation) of the character C is:

[0087] [1.3275, -1.5848, 0.7928, ..., -0.7260, -1.7137, -0.7433]

[0088] Similarly, the embeddings of other characters are:

[0089] [1.3866, -0.0981, 1.8502, ..., 0.8748, 0.1385, 0.3183]

[0090] [-0.9857, 0.2828, 2.1188, ..., 0.9051, -0.4908, -2.3782] ...

[0092] [-0.7647, -0.6149, -1.5059, ..., -1.4924, -0.9465, 2.5793]

[0093] [-0.7850, 2.9807, 2.4863, ..., -1.1880, -1.1957, 0.7120]

[0094] It should be noted that the embedding dimension of all words in the present invention is 1024.

[0095] Continue to see Figure 2 , taking the characters in the word as nodes, constructing a graph convolutional network, obtaining the correlation between characters, and thus obtaining the encoded character representation. After average pooling, a single representation is obtained, namely the encoded representation. HRL -e EN As an example, consider a high-resource entity eHRL and its corresponding English entity e EN , each entity is a sequence of characters: e HRL = <c1,c2,...,c M >, e EN = <k1,k2,...,k N >, each character has a fixed-size character embedding, which is used as the input of the graph convolutional network to obtain the encoded word vector (i.e., encoded representation):

[0096] v HRL =HRL-GCN( <c1,c2,...,c M >)

[0097] v EN =English-GCN( <k1,k2,...,k N >)

[0098] Continuing with the English entity Cobie Smulders as an example, the embedding representation is encoded through a graph convolutional network to obtain the corresponding embedding (encoded representation) of the entity:

[0099] [0.4441, 2.1919, 0.4573, ..., 0.6015, -0.5455, 0.4618]

[0100] Similarly, its corresponding high resource entity The embedding (encoding representation) is:

[0101] [0.1781, 2.3919, -2.0442, ..., -1.3068, -0.6113, -0.6740]

[0102] Continue to see Figure 2 , with high-resource entity-English entity pairs HRL -e EN For example, the obtained encoding representation can be used to represent the high-resource entity e HRL The encoding representation is the same as the English entity e EN The similarity between the two:

[0103] sim(e HRL , e EN )=cosine(v HRL , v EN )

[0104] Continuing with the English entity Cobie Smulders as an example, the encoding representation of Cobie Smulders obtained is The encoding representation of the two can be used to calculate the similarity between them. For example, the similarity between the two is 0.0289.

[0105] During the actual training process, sample data for this training can be selected from the positive samples and negative samples, and the sample data can be input into the basic model. The character embedding layer, graph convolutional network and similarity function in the basic model can refer to the above description to process the high-resource entities and English entities / high-resource entity referents and English entities in the sample data, so as to obtain the similarity between the high-resource entities and English entities / high-resource entity referents and English entities in the sample data.

[0106] In order to enable the model to effectively sort the knowledge base entities of a given entity referent, the present invention uses the maximum margin negative sampling to train the basic model based on existing work, and the loss function is as follows:

[0107]

[0108] Among them, λ is the preset interval, e EN is the English entity in the positive sample, is an English entity in the negative sample, and

[0109] According to the above loss function and the similarity corresponding to the sample data, the loss function value of this training can be calculated, such as 1.356. According to the loss function value, the loss is back-propagated. During the loss back-propagation process, the weight parameters of the character embedding layer and the graph convolution network are continuously updated, and the next training process is entered until the loss function value meets the end condition, such as the loss function value converges. The adjusted basic model is used as the pivot model, and finally a trained pivot model can be obtained.

[0110] In practical applications, the dataset prepared for training the pivot model may include 7 low-resource languages ​​and 5 high-resource languages ​​that serve as pivot languages ​​for the 7 low-resource languages. See the table below, which shows the correspondence between low-resource languages ​​and high-resource languages ​​as their pivot languages. The seven low-resource languages ​​(LRLs) include Tigrinya (ti for short; a Semitic language used in Eritrea and Ethiopia, written in Ethiopic script), Oromo (om for short; an Afro-Asiatic language used in the Horn of Africa, written in Roman script), Kinyarwanda (rw for short; a language of the Niger-Congo family, used in Rwanda, written in Roman script), Sinhala (si for short; an Indo-Aryan language used in Sri Lanka, written in its own script), Marathi (mr for short; an Indo-Aryan language used in western India, written in Sanskrit), Lao (lo for short; a Kra-Dai language written in Lao script), and Telugu (te for short; a Dravidian language used in southeastern India, written in Telugu script). There are five high-resource languages ​​(HRLs), namely Amharic (am), Indonesian (id), Tagalog (t1), Hindi (hi), and Thai (th).

[0111] In addition, the representation of the language is divided into phonetic representation (Phoneme) and grapheme representation (Grapheme). For phonetic representation, all input strings of LRL, HRL and English are converted to IPA; for grapheme representation, the string retains its original representation.

[0112]

[0113]

[0114] S30, determining a target English entity linked to the low-resource entity referent from multiple candidate English entities based on the pivot model.

[0115] In an embodiment of the present invention, the low-resource entity reference term and each candidate English entity can be input into the pivot model respectively. The character embedding layer in the pivot model can be used to randomly initialize the low-resource entity reference term and each candidate English entity to obtain their corresponding embedding representations; the graph convolution network in the pivot model can be used to encode the low-resource entity reference term and each candidate English entity based on their corresponding embedding representations to obtain their corresponding encoding representations; the similarity function in the pivot model can be used to calculate the similarity between the low-resource entity reference term and each candidate English entity based on their corresponding encoding representations.

[0116] The similarities of the candidate English entities are ranked, and the candidate English entity with the greatest similarity is selected as the target English entity linked to the low-resource entity referent.

[0117] The low-resource entity reference term m is For example, its encoding represents v m for:

[0118] [2.0770, -0.4010, 0.1718, ..., 0.2389, 1.4626, 0.9979]

[0119] Take candidate English entity e en Take Cobie Smulders as an example, its encoding represents v en for:

[0120] [0.2320, 0.8645, -0.1574, ..., -0.0936, -0.2306, -1.0863]

[0121] Therefore, you can use and Cobie Smulders's encoding representation, and calculate the similarity between the two, for example:

[0122] sim(m,e en )=cosine(v m , v en )=0.0561

[0123] Among them, v en =English-GCN(e en )∈K, K is the candidate English entity set.

[0124] In some embodiments, step S30 "determining the target English entity linked to the low-resource entity referent from multiple candidate English entities based on the pivot model" can adopt the following steps, and the method flow chart is as follows: Figure 3 As shown:

[0125] S301, determining the high resource entity corresponding to each candidate English entity.

[0126] In the embodiment of the present invention, the high-resource entity corresponding to each candidate English entity can be directly obtained based on the bilingual vocabulary, that is, for a candidate English entity, the candidate English entity and the corresponding high-resource entity point to the same entity.

[0127] S302, input the low-resource entity reference term and each candidate English entity into the pivot model to obtain a first similarity between the low-resource entity reference term and each candidate English entity through the pivot model; and, input the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity into the pivot model to obtain a second similarity between the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity through the pivot model.

[0128] See also Figure 4 The migration structure diagram of the pivot model shown in the figure. en , the high resource entity e corresponding to each candidate English entity HRL Input into the pivot model, and output the low-resource entity referent m and each candidate English entity e through the character embedding layer and graph convolutional network in the pivot model en , the high resource entity e corresponding to each candidate English entity HRL The corresponding encoding representations are then used to calculate the low-resource entity referent m and each candidate English entity e through the similarity function in the pivot model. en The similarity between them (i.e., the first similarity), the low-resource entity reference m, and the high-resource entity e corresponding to each candidate English entity HRL The similarity between (i.e. the second similarity)

[0129] Take the high resource entity eHRL corresponding to a candidate English entity as For example, its encoding represents v HRL for:

[0130] [-0.3504, -0.3207, -0.5349, ..., 0.7053, -0.3089, 1.7641]

[0131] Similarly, we also need to calculate the low-resource entity referent m and the high-resource entity e corresponding to the candidate English entity HRL The similarity between the two, for example:

[0132] sim(m,e HRL )=cosine(v m , v HRL)=0.0689

[0133] Among them, v m =HRL-GCN(m),v HRL =HRL-GCN(e HRL ), it should be noted that e HRL and e en Point to the same entity.

[0134] S303: Taking the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity.

[0135] In the embodiment of the present invention, for each candidate English entity, the maximum value of the first similarity and the second similarity corresponding to the candidate English entity can be used as the similarity between the low-resource entity referent and the candidate English entity (ie, the target similarity).

[0136] Based on the above scheme, we can get the low-resource entity reference item m and the candidate English entity e en The maximum value of the two similarities is used as the target similarity:

[0137] score(m,e en )=max(sim(m,e en ), sim(m, e HRL ))

[0138] It should be noted that not every candidate English entity in K has a corresponding high-resource language entity. In this case, consider sim(m, e HRL )=-∞.

[0139] In some embodiments, although the above-mentioned solution of the present invention has enabled the pivot model to be more capable of handling subtle differences between entity referents and entities, it is still difficult to match strings with significant differences. To alleviate this problem, the present invention can use alias resources in Wikidata, and obtain a ready-made alias table through the Wikipedia entry of the entity. For example, a record of the alias table is as follows:

[0140] Q200566|||2991300|||Cobie Smulders|||Jacoba Francisca Maria Smulders-Killam||Jacoba Francisca Maria Smulders

[0141] Therefore, the embodiment of the present invention may further include the following steps:

[0142] Determine at least one alias corresponding to each candidate English entity;

[0143] The low-resource entity reference term and at least one alias corresponding to each candidate English entity are respectively input into the pivot model to obtain the third similarity between the low-resource entity reference term and each alias through the pivot model; the maximum value of each third similarity and the first similarity is used as the target first similarity between the low-resource entity reference term and each candidate English entity.

[0144] In the embodiment of the present invention, for any alias of a candidate English entity, the low-resource entity reference item and the alias may be input into the pivot model, and the similarity between the low-resource entity reference item and the alias (i.e., the third similarity) may be obtained through the pivot model. For a candidate English entity, a maximum value may be selected from each third similarity and the first similarity as the similarity between the low-resource entity reference item and the candidate English entity, i.e., the target first similarity, specifically:

[0145]

[0146] Among them, A is a set of a candidate English entity and its alias, that is, a set of entity Wikipedia title and entity alias.

[0147] For example, the candidate English entity Cobie Smulders has aliases Jacoba Francisca Maria Smulders-Killam and Jacoba Francisca Maria Smulders. For each of these aliases, the pivot model should be used to encode them respectively to obtain their encoded representations:

[0148] [-0.9396, -0.6383, -1.0303, ..., 0.1209, -0.6716, -0.3704]

[0149] [0.6549, -1.3883, -0.4443, ..., -1.3121, 0.3756, 0.5742]

[0150] Then, the low-resource entity referent The encoding representation of is calculated with the encoding representation of each alias above, for example, (0.0748, 0.0534) are obtained respectively, then the low-resource entity reference item The target first similarity with the candidate English entity Cobie Smulders is:

[0151]

[0152] Accordingly, step S303 of "taking the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity" can adopt the following steps:

[0153] The maximum value between the target first similarity and the second similarity is taken as the target similarity between the low-resource entity referent and each candidate English entity.

[0154] Continuing with the above example:

[0155] score(m,e en )=max(sim(m,e en ), sim(m, e HRL )) =max (0.0748, 0.0689) = 0.0748

[0156] The target similarity is the low-resource entity reference item The final similarity with the candidate English entity Cobie Smulders. Then, the target similarity of each candidate English entity is sorted, and the candidate English entity with the largest target similarity is selected as the target English entity linked to the low-resource entity referent. For example:

[0157]

[0158] From the above, we can see that the candidate English entity Cobie Smulders has the highest score, so Cobie Smulders is the low-resource entity reference item. The entity to link to, i.e. the target entity.

[0159] The low-resource cross-language entity linking method provided by the embodiment of the present invention can realize low-resource cross-language entity connection and solve the problem that cross-language entity linking is difficult, especially for languages ​​that lack bilingual resources and Wikipedia resources.

[0160] Based on the low-resource cross-language entity linking method provided in the above embodiment, an embodiment of the present invention provides a device for executing the low-resource cross-language entity linking method. The structural diagram of the device is shown in FIG. Figure 5 As shown, including:

[0161] An acquisition module 10 is used to acquire low-resource entity reference items and a candidate entity set in the text to be processed, where the candidate entity set includes multiple candidate English entities;

[0162] The linking module 20 is used to retrieve the pivot model that has been migrated to the low-resource language to which the low-resource entity referent belongs. The pivot model is trained based on the bilingual vocabulary between the high-resource language and English and migrated to the low-resource language in a zero-transfer manner. The pivot model is a character-level neural network, and the high-resource language is the pivot language of the low-resource language; based on the pivot model, the target English entity to which the low-resource entity referent is linked is determined from multiple candidate English entities.

[0163] Optionally, the process of linking module 20 obtaining a pivot model based on bilingual vocabulary training between a high resource language and English includes:

[0164] Based on the bilingual vocabulary, the first data pair and the second data pair are respectively obtained as positive samples, wherein the first data pair includes high-resource entities and English entities, and the second data pair includes high-resource entity reference items and English entities; the third data pair and the fourth data pair are constructed as negative samples, wherein the third data pair includes the same high-resource entities as the first data pair but different English entities, and the fourth data pair includes the same high-resource entity reference items as the second data pair but different English entities; sample data for this training is selected from the positive samples and the negative samples, and the sample data is input into the basic model to achieve:

[0165] The high-resource entities and English entities / high-resource entity referents and English entities in the sample data are randomly initialized through the character embedding layer in the basic model to obtain their corresponding embedded representations; the high-resource entities and English entities / high-resource entity referents and English entities in the sample data are encoded based on the embedded representations through the graph convolutional network in the basic model to obtain their corresponding encoded representations; the similarity between the high-resource entities and English entities / high-resource entity referents and English entities in the sample data is calculated based on the encoded representations through the similarity function in the basic model;

[0166] According to the similarity corresponding to the sample data, the loss function value of the basic model is calculated by negative sampling with the maximum interval, and the weight parameters of the character embedding layer and the graph convolutional network are reversely adjusted according to the loss function value. Then, the step of selecting sample data for this training from positive and negative samples is returned to execute until the loss function value meets the end condition, and the adjusted basic model is used as the pivot model.

[0167] Optionally, the linking module 20 determines a target English entity to which the low-resource entity referent is linked from a plurality of candidate English entities based on a pivot model, including:

[0168] Determine the high-resource entity corresponding to each candidate English entity; input the low-resource entity reference term and each candidate English entity into the pivot model to obtain a first similarity between the low-resource entity reference term and each candidate English entity through the pivot model; and input the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity into the pivot model to obtain a second similarity between the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity through the pivot model; use the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity reference term and each candidate English entity; select the candidate English entity with the largest target similarity as the target English entity linked to the low-resource entity reference term.

[0169] Optionally, the link module 20 is further used for:

[0170] Determine at least one alias corresponding to each candidate English entity; input the low-resource entity designation and at least one alias corresponding to each candidate English entity into the pivot model respectively, so as to obtain the third similarity between the low-resource entity designation and each alias through the pivot model; take the maximum value of each third similarity and the first similarity as the target first similarity between the low-resource entity designation and each candidate English entity;

[0171] Accordingly, the linking module 20 uses the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity, including:

[0172] The maximum value between the target first similarity and the second similarity is taken as the target similarity between the low-resource entity referent and each candidate English entity.

[0173] It should be noted that the detailed functions of each module in the embodiment of the present invention can be found in the corresponding disclosed part of the above-mentioned low-resource cross-language entity linking method embodiment, and will not be repeated here.

[0174] Based on the low-resource cross-language entity linking method provided in the above embodiment, an embodiment of the present invention further provides an electronic device, see Figure 6 The hardware structure block diagram of the electronic device shown in the figure may include: a processor 11, a communication interface 12, a memory 13 and a communication bus 14;

[0175] In the embodiment of the present application, the number of the processor 11 , the communication interface 12 , the memory 13 , and the communication bus 14 is at least one, and the processor 11 , the communication interface 12 , and the memory 13 communicate with each other through the communication bus 14 .

[0176] The processor 11 may be a central processing unit (CPU), a graphics processing unit (GPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0177] The memory 13 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0178] The memory 13 stores the application and the data generated by the application, and the processor 11 executes the application to realize the following functions:

[0179] A low-resource entity reference term and a candidate entity set in a text to be processed are obtained, wherein the candidate entity set includes a plurality of candidate English entities; a pivot model that has been migrated to a low-resource language to which the low-resource entity reference term belongs is retrieved, wherein the pivot model is obtained based on bilingual vocabulary training between a high-resource language and English and migrated to the low-resource language in a zero-transfer manner, the pivot model is a character-level neural network, and the high-resource language is the pivot language of the low-resource language; and a target English entity to which the low-resource entity reference term is linked is determined from the plurality of candidate English entities based on the pivot model.

[0180] It should be noted that the refinement and expansion of the functions implemented by the processor executing the application program can refer to the description of the embodiment of the low-resource cross-language entity linking method above.

[0181] Based on the low-resource cross-language entity linking method provided in the above embodiment, an embodiment of the present invention further provides a storage medium, in which computer executable instructions are stored, and the computer executable instructions are used to execute the low-resource cross-language entity linking method.

[0182] The above is a detailed introduction to a low-resource cross-language entity linking method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0183] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0184] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device that includes a series of elements is inherent to the elements, or also includes elements inherent to these processes, methods, articles or devices. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0185] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A low-resource cross-language entity linking method, characterized in that: The method comprises: Obtaining low-resource entity reference items and a candidate entity set in the text to be processed, wherein the candidate entity set includes multiple candidate English entities; Retrieving a pivot model that has been transferred to a low-resource language to which the low-resource entity referent belongs, wherein the pivot model is obtained based on bilingual vocabulary training between a high-resource language and English and transferred to the low-resource language in a zero-transfer manner, the pivot model is a character-level neural network, and the high-resource language is the pivot language of the low-resource language; Determining a target English entity linked to the low-resource entity reference term from the plurality of candidate English entities based on the pivot model; The process of obtaining the pivot model based on bilingual vocabulary training between a high-resource language and English includes: Based on the bilingual vocabulary, respectively obtain a first data pair and a second data pair as positive samples, wherein the first data pair includes a high-resource entity and an English entity, and the second data pair includes a high-resource entity referent and an English entity; Constructing a third data pair and a fourth data pair as negative samples, wherein the third data pair contains the same high-resource entity as the first data pair but different English entities, and the fourth data pair contains the same high-resource entity referent as the second data pair but different English entities; Select sample data for this training from the positive samples and the negative samples, and input the sample data into the basic model to achieve: The high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are randomly initialized by the character embedding layer in the basic model to obtain their respective corresponding embedded representations; the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are encoded by the graph convolutional network in the basic model based on the embedded representation to obtain their respective corresponding encoded representations; the similarities between the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are calculated by the similarity function in the basic model based on the encoded representation; According to the similarity corresponding to the sample data, the loss function value of the basic model is calculated by means of maximum interval negative sampling, the weight parameters of the character embedding layer and the graph convolutional network are reversely adjusted according to the loss function value, and the step of selecting the sample data for this training from the positive samples and the negative samples is returned to be executed until the loss function value meets the end condition, and the adjusted basic model is used as the pivot model; The step of determining the target English entity linked to the low-resource entity reference from the plurality of candidate English entities based on the pivot model includes: Determine the high-resource entity corresponding to each candidate English entity; Inputting the low-resource entity designation and each candidate English entity into the pivot model to obtain a first similarity between the low-resource entity designation and each candidate English entity through the pivot model; and inputting the low-resource entity designation and the high-resource entity corresponding to each candidate English entity into the pivot model to obtain a second similarity between the low-resource entity designation and the high-resource entity corresponding to each candidate English entity through the pivot model; Taking the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity; The candidate English entity with the largest target similarity is selected as the target English entity linked to the low-resource entity referent.

2. The method according to claim 1, characterized in that The method further comprises: Determine at least one alias corresponding to each candidate English entity; Inputting the low-resource entity designation and at least one alias corresponding to each candidate English entity into the pivot model respectively, so as to obtain a third similarity between the low-resource entity designation and each alias through the pivot model; Taking the maximum value of each third similarity and the first similarity as the target first similarity between the low-resource entity referent and each candidate English entity; Accordingly, taking the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity includes: The maximum value between the target first similarity and the second similarity is used as the target similarity between the low-resource entity referent and each candidate English entity.

3. A low-resource cross-language entity linking device, characterized in that: The device comprises: An acquisition module, used to acquire low-resource entity reference items and a candidate entity set in the text to be processed, wherein the candidate entity set includes multiple candidate English entities; a linking module, for retrieving a pivot model that has been transferred to the low-resource language to which the low-resource entity referent belongs, wherein the pivot model is obtained based on bilingual vocabulary training between a high-resource language and English and transferred to the low-resource language in a zero-transfer manner, the pivot model is a character-level neural network, and the high-resource language is the pivot language of the low-resource language; and determining a target English entity to which the low-resource entity referent is linked from the multiple candidate English entities based on the pivot model; The process of obtaining the pivot model by training the link module based on the bilingual vocabulary between the high resource language and English includes: Based on the bilingual vocabulary, a first data pair and a second data pair are respectively obtained as positive samples, wherein the first data pair includes a high-resource entity and an English entity, and the second data pair includes a high-resource entity reference item and an English entity; a third data pair and a fourth data pair are constructed as negative samples, wherein the third data pair includes the same high-resource entity as the first data pair and a different English entity, and the fourth data pair includes the same high-resource entity reference item as the second data pair and a different English entity; sample data for this training is selected from the positive samples and the negative samples, and the sample data is input into a basic model to achieve: The high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are randomly initialized by the character embedding layer in the basic model to obtain their respective corresponding embedded representations; the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are encoded by the graph convolutional network in the basic model based on the embedded representation to obtain their respective corresponding encoded representations; the similarities between the high-resource entities and English entities / high-resource entity reference items and English entities in the sample data are calculated by the similarity function in the basic model based on the encoded representation; According to the similarity corresponding to the sample data, the loss function value of the basic model is calculated by means of maximum interval negative sampling, the weight parameters of the character embedding layer and the graph convolutional network are reversely adjusted according to the loss function value, and the step of selecting the sample data for this training from the positive samples and the negative samples is returned to be executed until the loss function value meets the end condition, and the adjusted basic model is used as the pivot model; The linking module determines the target English entity to which the low-resource entity referent is linked from the multiple candidate English entities based on the pivot model, including: Determine the high-resource entity corresponding to each candidate English entity; input the low-resource entity reference term and each candidate English entity into the pivot model to obtain a first similarity between the low-resource entity reference term and each candidate English entity through the pivot model; and input the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity into the pivot model to obtain a second similarity between the low-resource entity reference term and the high-resource entity corresponding to each candidate English entity through the pivot model; use the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity reference term and each candidate English entity; and select the candidate English entity with the largest target similarity as the target English entity linked to the low-resource entity reference term.

4. The device according to claim 3, characterized in that The link module is also used to: Determine at least one alias corresponding to each candidate English entity; input the low-resource entity designation and at least one alias corresponding to each candidate English entity into the pivot model respectively, so as to obtain a third similarity between the low-resource entity designation and each alias through the pivot model; and use the maximum value of each third similarity and the first similarity as the target first similarity between the low-resource entity designation and each candidate English entity; Accordingly, the linking module uses the maximum value between the first similarity and the second similarity as the target similarity between the low-resource entity referent and each candidate English entity, including: The maximum value between the target first similarity and the second similarity is used as the target similarity between the low-resource entity referent and each candidate English entity.

5. An electronic device, characterized in that: The electronic device includes: at least one memory and at least one processor; the memory stores an application, the processor calls the application stored in the memory, and the application is used to implement the low-resource cross-language entity linking method described in any one of claims 1-2.

6. A storage medium, characterized in that: The storage medium stores computer executable instructions, and the computer executable instructions are used to execute the low-resource cross-language entity linking method according to any one of claims 1-2.

Images