Method, electronic device and computer readable medium for multi-entity joint linking

Abstract

Embodiments of the present disclosure disclose a method, an electronic device and a computer readable medium for multi-entity joint linking. A specific implementation of the method comprises: obtaining a target entity mention and original text; generating a candidate entity information set based on a predetermined text library and the target entity mention; inputting the candidate entity information set into a pre-trained encoding model to generate a candidate entity feature set; for each candidate entity information in the candidate entity information set, concatenating the candidate entity, the original text and the candidate entity feature set of the candidate entity information to generate a text information feature, so as to obtain a text information feature set; inputting the text information feature set into a pre-trained classification model to generate a discrimination result set; and determining an entity linking result set based on the discrimination result set. This method uses the text information feature set to supplement information in the candidate entity through multi-entity joint linking, thereby improving the entity linking accuracy in short text and information deficiency scenarios.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of natural language processing, in particular to an entity linking method, an electronic device and a computer readable medium. BACKGROUND

[0002] In the context of network big data, it brings great challenges to people on how to exclude redundant and noisy knowledge and accurately mine target information. With the continuous expansion of knowledge base and the emerging development of natural language processing, how to fuse multi-source heterogeneous data and enrich fragmented knowledge relationships has become a difficult problem to be solved at present. Among them, the problem that cannot be avoided is the diversity and ambiguity of natural language expression, specifically, the same entity may have multiple different expressions (polysemy), and the same expression may refer to multiple different entities (monosemy). Therefore, entity linking has become the key to solving this problem. Entity linking refers to the process of linking entity mentions obtained from network big data text to knowledge base entities. Entity linking not only enhances the reading experience and helps people and computers to better understand the meaning of target information, but also forms an accurate information aggregation system centered on entities and promotes the development of domain knowledge base.

[0003] However, when the above method is used for entity linking processing, the following technical problems often exist:

[0004] First, when there are multiple entity mentions in the text, the existing method measures and predicts the entity corresponding to each entity mention in turn, without considering the relationship between multiple entity mentions and multiple entities, and can only get a locally optimal linking result for a certain entity mention, and cannot get a globally optimal linking result in the whole text.

[0005] Second, the relationship between multiple entities is obtained by simple addition and splicing, which cannot handle different degrees of closeness between different entities and target entity mentions, and there is a problem of weakening some key entities, resulting in poor accuracy of the finally found globally optimal linking result. SUMMARY

[0006] Embodiments of the present disclosure propose a multi-entity joint linking method.

[0007] In a first aspect, some embodiments of the present disclosure provide a method for multi-entity joint linking, the method comprising: obtaining a target entity mention and original text; generating a candidate entity information set based on a pre-determined text corpus and the target entity mention; inputting the candidate entity information set into a pre-trained encoding model to generate a candidate entity feature set; for each candidate entity information in the candidate entity information set, concatenating the candidate entity, the original text and the candidate entity feature set of the candidate entity information to generate a text information feature, to obtain a text information feature set; inputting the text information feature set into a pre-trained classification model to generate a discrimination result set; and determining an entity linking result set based on the discrimination result set.

[0008] In a second aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; and a storage device having one or more programs stored thereon, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of the first aspect.

[0009] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method of any of the first aspect.

[0010] The above various embodiments of the present disclosure have the following beneficial effects: the multi-entity joint linking method of some embodiments of the present disclosure can utilize the text information feature set to supplement the information in the candidate entity through multi-entity joint, thereby improving the entity linking accuracy in the short text and information deficiency scenarios. Specifically, the inventors found that the main reason for poor linking accuracy in entity linking is that when there are multiple entity mentions in the text, the existing method measures and predicts the entity corresponding to each entity mention one by one without considering the relationship between multiple entity mentions and multiple entities, and can only obtain a locally optimal linking result for a certain entity mention, and cannot obtain a globally optimal linking result in the entire text. In addition, if only a simple summing and splicing process is performed on multiple entities, there is a problem of weakening some key entities, resulting in poor accuracy of the finally found globally optimal linking result. Based on this, first, some embodiments of the present disclosure determine a text library in advance. Specifically, the pre-determined text library is used to search for a set of entity mentions and entities corresponding to a target entity mention. Second, based on the pre-determined text library and the target entity mention sent by the user, a candidate entity information set is generated. The candidate entity information includes a candidate entity and an entity description text, and the candidate entity information set contains all candidate entities related to the target entity mention, thereby considering the relationship between multiple entities. Third, the candidate entity information set is input into a pre-trained encoding model to generate a candidate entity feature set. The encoding model converts the information corresponding to the entity into a candidate entity feature, and simultaneously introduces an attention mechanism to process the relationship between multiple entities. Then, the candidate entity feature set is used to generate a text information feature set, realizing the splicing of the candidate entity, the original text and the candidate entity feature set. Finally, the text information feature set is input into a pre-trained classification model to generate a discrimination result set, and the entity linking result set is determined according to the discrimination result set. The pre-trained classification model is a binary classification model, which can determine whether the entity is the entity corresponding to the target entity mention. This processing method can consider the information between multiple entities, and simultaneously splice the candidate entity, the original text and the candidate entity feature set to generate a text information feature, thereby obtaining a complete representation of the text information. According to the text information feature, the final classification result can be predicted, which can supplement the information lacking in the original text, and can improve the accuracy of the entity linking result in the short text and information deficiency scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0011] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments made with reference to the accompanying drawings:

[0012] Figure 1 is an architecture diagram of an exemplary system to which some embodiments of the present disclosure can be applied;

[0013] Figure 2 is a flowchart of some embodiments of a method of multi-entity joint linking according to the present disclosure;

[0014] Figure 3 is an exemplary authorization prompt box;

[0015] Figure 4 is a flowchart of one embodiment of a training step for training a pre-trained classification model according to the present disclosure;

[0016] Figure 5 is a structural schematic diagram of a terminal device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only parts related to the present application are shown in the drawings. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0019] It should be noted that the modification of "one" and "multiple" mentioned in the present disclosure is illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0020] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0021] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] It should also be noted that, for ease of description, only parts related to the present application are shown in the drawings. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0023] It should be noted that the terms "first", "second", and the like in the present disclosure are used only to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that "one" or "multiple" should be understood as "one or more" unless otherwise explicitly indicated in the context.

[0025] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0026] Figure 1 An exemplary system architecture 100 of an embodiment to which the method of multi-entity joint linking of the present disclosure can be applied is shown.

[0027] As shown in Figure 1 The system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0028] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as information generation applications, entity linking applications, information extraction applications, etc.

[0029] The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various terminal devices with display screens, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the terminal devices listed above. They can be implemented as multiple software or software modules (such as for providing target entity mentions and original text inputs, etc.), or as a single software or software module. No specific limitation is made herein.

[0030] The server 105 can be a server that provides various services, such as a server that stores target data input by the terminal devices 101, 102, 103, etc. The server can process the received target order sequence and feed back the processing result (such as an entity linking result set) to the terminal device.

[0031] It should be noted that the method for multi-entity joint linking provided by the embodiments of the present disclosure can be executed by the server 105, or by the terminal device.

[0032] It should be noted that the local of the server 105 can also directly store the target entity mention and the original text, and the server 105 can directly extract the target entity mention and the original text in the local to obtain the entity linking result set after processing. At this time, the exemplary system architecture 100 can not include the terminal devices 101, 102, and 103 and the network 104.

[0033] It should also be noted that the method for multi-entity joint linking can also be installed in the terminal devices 101, 102, and 103, and at this time, the processing method can also be executed by the terminal devices 101, 102, and 103. At this time, the exemplary system architecture 100 can not include the server 105 and the network 104.

[0034] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, to provide a method for multi-entity joint linking service), or as a single software or software module. This is not specifically limited here.

[0035] It should be understood that Figure 1 The number of terminal devices, networks, and servers in

[0036] With reference to Figure 2 , flow 200 of some embodiments of the method for multi-entity joint linking according to the present disclosure is shown. The method for multi-entity joint linking includes the following steps:

[0037] Step 201, obtaining a target entity mention and an original text.

[0038] In some embodiments, the subject of execution of the method for multi-entity joint linking (for example Figure 1The terminal device shown) can acquire the target entity designation and the original text in response to receiving the target authorization signal. The original text includes a first number of characters. The target authorization signal can be a signal generated by the user corresponding to the target entity designation and the original text performing a target operation on the target control. The target control can be included in the authorization prompt box. The authorization prompt box can be displayed on the target terminal device. The target terminal device can be a terminal device logged in with an account corresponding to the user. The terminal device can be a "mobile phone" or a "computer". The target operation can be a "click operation" or a "slide operation". The target control can be a "confirmation button".

[0039] As an example, the authorization prompt box can be as shown. Figure 3 The authorization prompt box can include a prompt information display part 301 and a control 302. The prompt information display part 301 can be used to display prompt information. The prompt information can be "whether to allow the acquisition of the target entity designation and the original text". The control 302 can be a "confirmation button" or a "cancel button".

[0040] In step 202, a candidate entity information set is generated based on a predetermined text library and a target entity designation.

[0041] In some embodiments, the execution subject generates a candidate entity information set based on a predetermined text library and a target entity designation. The candidate entity information includes a candidate entity and an entity description text. The predetermined text library is used to search for a target entity corresponding to the target entity designation. The predetermined text library includes an index set and a corpus. Specifically, the predetermined text library can be constructed according to a Wikipedia text library, and the Wikipedia text library can be a corpus. The index set can be determined by manually searching and annotating the Wikipedia text library in advance. Specifically, the content of the index can be an entity.

[0042] Optionally, a candidate entity designation set is determined based on a predetermined text library and a target entity designation. Specifically, the predetermined text library is searched for an entity designation set consistent with the target entity designation. These entity designation sets are determined as the candidate entity designation set. For each candidate entity designation in the candidate entity designation set, the corresponding candidate entity is searched in the predetermined text library, and a candidate entity information is generated to obtain a candidate entity information set. Specifically, the candidate entity information set is obtained by searching and finding in the predetermined text library, and the purpose is to find all entities in the predetermined text library that have similar text meanings with the target entity designation.

[0043] Step 203, input the candidate entity information set into the pre-trained encoding model to generate a candidate entity feature set.

[0044] In some embodiments, the above execution subject inputs the candidate entity information set into the pre-trained encoding model to generate a candidate entity feature set. Specifically, the pre-trained encoding model includes a first encoding network and a second encoding network.

[0045] Optionally, for each candidate entity information in the candidate entity information set, the candidate entity of the candidate entity information is input into the first encoding network to generate a candidate entity first encoding feature to obtain a candidate entity first encoding feature set. The candidate entity first encoding feature set is input into the second encoding network to generate a candidate entity feature set.

[0046] Specifically, the first encoding network can be an encoder network. The encoder network encoder can include 6 identical hierarchical layers, and each hierarchical layer can include two sub-layers. The first sub-layer can be a multi-head self-attention mechanism, and the second sub-layer can be a simple fully connected feedforward network. The two layers are connected by a residual network respectively. The residual network is composed of a second number of residual modules, and the residual module generates an output using the following formula:

[0047] y=F(x,{W i})+x,

[0048] wherein x is the input of the residual module, y is the output of the residual module, F() is the residual function. W is a weight matrix, i is the layer count in the residual module, and W i represents the weight matrix of the i-th layer. {W i} represents the set of weight matrices of all layers in the residual module. In order to optimize the results of these residual networks, all hierarchical layers in the first encoding network generate an output with a dimension of 512.

[0049] Optionally, the second encoding network can include a calculation module and a focusing module. Specifically, the second encoding network can be a decoder network. The candidate entity first encoding feature set is input into the calculation module to generate a candidate entity first encoding feature parameter set. The candidate entity first encoding feature set and the candidate entity first encoding feature parameter set are input into the focusing module to generate a candidate entity feature set. Specifically, the focusing module can include 6 identical hierarchical layers. The focusing module can be an attention mechanism. The candidate entity first encoding feature set and the candidate entity first encoding feature parameter set are concatenated to obtain an initial input sequence. The weight parameters W:{w i = (w i,1 , w i,2 ,..., w i,k). Specifically, W represents a weight parameter, i represents a hierarchical count, k represents an initial input number in an initial input sequence, w i represents a weight parameter of the i-th hierarchical layer. The focusing module includes a first number of hierarchical layers, and the first number can be 6. w i,1 represents a weight parameter of the i-th hierarchical layer corresponding to the first initial input. w i,k represents a weight parameter of the i-th hierarchical layer corresponding to the k-th initial input.

[0050] The intermediate input sequence of the focusing module is calculated by using the following formula:

[0051] C: {C i = w i * Z = (w i,1 , w i,2 , …, w i,k ) * Z,

[0052] wherein Z represents an initial input sequence, C represents an intermediate input sequence, C i represents an intermediate input of the i-th hierarchical layer. The focusing module includes a first number of hierarchical layers. The intermediate input sequence includes a first number of intermediate inputs.

[0053] The output sequence of the focusing module is calculated by using the following formula:

[0054] Y: {y i = f de (C i , y i-1 ),

[0055] wherein f de represents a neuron of the i-th hierarchical layer, C i represents an intermediate input of the i-th hierarchical layer. y i represents an output of the i-th hierarchical layer, y i-1 represents an output of the i-1-th hierarchical layer, and Y represents an output sequence of the focusing module. The last output in the output sequence of the focusing module is determined as a candidate entity feature set. Specifically, f de represents a neuron of the i-th hierarchical layer, C i represents an intermediate input of the i-th hierarchical layer, y i represents an output of the i-th hierarchical layer, y i-1 represents an output of the i-1-th hierarchical layer. f de may be a one-layer perceptron. y i is calculated by using a normalized exponential function (softmax function). The output of the last hierarchical layer is determined as an output of the decoder network to obtain the candidate entity feature set.

[0056] The optional content in step 203, that is, the technical content of "introducing a weight parameter to calculate the candidate entity feature set", is an invention point of an embodiment of the present disclosure, which solves the second technical problem mentioned in the background that "adopting simple addition, splicing and other methods to obtain the relationship between multiple entities cannot handle different degrees of closeness between different entities and the target entity, and there is a problem of weakening some key entities, resulting in poor accuracy of the final global optimal link result". The factors that lead to poor accuracy of the link result are often as follows: the traditional method simply adds or splices multiple entities, and cannot consider the influence of different entities. If the above factors are solved, the effect of improving the accuracy of the link result can be achieved. In order to achieve this effect, the present disclosure introduces a weight parameter to calculate the candidate entity feature set. First, the candidate entity first encoding feature set is input into the calculation module to generate the candidate entity first encoding feature parameter set. The role of the calculation module is to generate the candidate entity first encoding feature parameter set. The specific candidate entity first encoding feature parameter is used to represent the entity. Then, the candidate entity first encoding feature set is input into the second encoding network to generate the candidate entity feature set. The second encoding network includes a focusing module, and the focusing module introduces a weight parameter. The intermediate output is calculated by using each neuron in the hierarchical layer. Due to the existence of the weight parameter, different candidate entity first encoding features have different influences. This method introduces an attention mechanism to handle the relationship between multiple entities, improves the accuracy of the final global optimal link result, and thus solves the second technical problem.

[0057] In step 204, for each candidate entity information in the candidate entity information set, the candidate entity, the original text and the candidate entity feature set of the candidate entity information are spliced to generate a text information feature, so as to obtain a text information feature set.

[0058] In some embodiments, the above execution subject splices the candidate entity, the original text and the candidate entity feature set of each candidate entity information in the candidate entity information set to generate a text information feature, so as to obtain a text information feature set.

[0059] Optionally, for each candidate entity information in the candidate entity information set, a character set is generated based on the original text. The character set includes a first number of characters. Specifically, all characters included in the original text are put into the character set. The character set is input into the first encoding network to generate character features. Specifically, the first encoding network can be an encoder network. The encoder network encoder can include 6 identical hierarchical layers, and each hierarchical layer can include two sub-layers. The first sub-layer can be a multi-head self-attention mechanism, and the second sub-layer can be a simple fully connected feedforward network. The two layers are respectively connected by a residual network. The residual network is composed of a second number of residual modules, and the residual module generates an output using the following formula:

[0060] y = F(x, {W i}) + x,

[0061] where x is the input of the residual module, y is the output of the residual module, F() is the residual function. W is a weight matrix, i is the layer count in the residual module, and W i represents the weight matrix of the i-th layer. {W i} represents the set of weight matrices of all layers in the residual module. In order to optimize the results of these residual networks, all hierarchical layers in the first encoding network generate an output with a dimension of 512.

[0062] The candidate entity of the candidate entity information is input into the first encoding network to generate a candidate entity encoding feature. The candidate entity encoding feature, the character feature, and the candidate entity feature set are spliced to generate a text information feature to obtain a text information feature.

[0063] Step 205: inputting the text information feature set into a pre-trained classification model to generate a discrimination result set.

[0064] In some embodiments, the above-mentioned execution subject inputs the text information feature set into a pre-trained classification model to generate a discrimination result set. Specifically, the pre-trained classification model includes a third encoding network and a first decoding network.

[0065] Optionally, for each text information feature in the text information feature set, the text information feature is input into the third encoding network to generate a first encoded text information feature to obtain a first encoded text information feature set.

[0066] For each first encoded text information feature in the first encoded text information feature set, the first encoded text information feature is input into the first decoding network to generate a discrimination result to obtain a discrimination result set.

[0067] Specifically, the pre-trained classification model can be Bidirectional Encoder Representation from Transformers (BERT). BERT is a bidirectional encoding representation model based on a transformer. The goal of BERT is to obtain the semantic representation of the text. Specifically, the third encoding network can be an encoder network. The third encoding network can include 6 identical layers, and each layer can include two sub-layers. The first sub-layer can be a self-attention mechanism, and the second sub-layer can be a simple fully connected feedforward network. The two layers are connected by a residual network respectively. The residual network is composed of a second number of residual modules, and the residual module generates an output using the following formula:

[0068] y = F(x, {W i}) + x,

[0069] where x is the input of the residual module, y is the output of the residual module, F() is the residual function. W is a weight matrix, i is the layer count in the residual module, and W i represents the weight matrix of the i-th layer. {W i} represents the set of weight matrices of all layers in the residual module. In order to optimize the results of these residual networks, all layers in the first encoding network generate an output with a dimension of 512. The fully connected feedforward network uses the following formula as the activation function:

[0070] FFN(x) = max(0, xH1 + b1}H2 + b2,

[0071] where x represents the input of the fully connected feedforward network, H1 represents the first weight parameter of the fully connected feedforward network, H2 represents the second weight parameter of the fully connected feedforward network, b1 represents the first bias parameter of the fully connected feedforward network, b2 represents the second bias parameter of the fully connected feedforward network, and FFN represents the fully connected feedforward network. Specifically, the fourth encoding network can be a decoder network. The fourth encoding network can include 6 identical layers, and each layer can include three sub-layers. The first sub-layer can be a self-attention mechanism, the second sub-layer can be a multi-head attention mechanism, and the third sub-layer can be a simple fully connected feedforward network. In addition, the fourth encoding network also includes a linear processing layer and a classification layer. The output of the classification layer is "0" or "1". The output of the classification layer is the discrimination result. Specifically, the above execution subject inputs the text information feature set into the pre-trained classification model to obtain a discrimination result set.

[0072] Step 206, based on the discrimination result set, determining an entity linking result set.

[0073] In some embodiments, the execution subject determines the entity linking result set based on the discrimination result set. Optionally, for each discrimination result in the discrimination result set, in response to the discrimination result being "1", the candidate entity corresponding to the discrimination result is determined as a result entity to obtain a result entity set. The result entity set is determined as the entity linking result set. In response to the discrimination result being "0", it indicates that the candidate entity corresponding to the discrimination result is not a result entity matching the target entity denotation.

[0074] Optionally, the entity linking result set is pushed to a target device with a display function, and the target device is controlled to display the entity linking result set. The target device with a display function can be a device in communication connection with the execution subject, and can display according to the received entity linking result set. For example, the execution subject can display the current time entity linking result set to show the entity linking result matching the target entity denotation. The result can assist subsequent natural language processing tasks, improve the efficiency of information extraction, intelligence recognition and other work, and prompt the user to make corresponding operation actions.

[0075] Figure 2 An embodiment provided has the following beneficial effects: obtaining a target entity denotation and an original text; generating a candidate entity information set based on a pre-determined text library and the target entity denotation; inputting the candidate entity information set into a pre-trained encoding model to generate a candidate entity feature set; for each candidate entity information in the candidate entity information set, concatenating the candidate entity of the candidate entity information, the original text and the candidate entity feature set to generate a text information feature, to obtain a text information feature set; inputting the text information feature set into a pre-trained classification model to generate a discrimination result set; and determining an entity linking result set based on the discrimination result set. This method uses the text information feature set to supplement the information in the candidate entity through multi-entity joint, to improve the entity linking accuracy in the short text and information lacking scenario.

[0076] With reference to Figure 4 , a flow 400 of one embodiment of a training step of a pre-trained classification model according to the present disclosure is shown. The training step can include the following steps:

[0077] Step 401, determining the network structure of an initial classification model and initializing the network parameters of the initial classification model.

[0078] In the present embodiment, the execution subject of the training step can be the execution subject of the multi-entity joint linking method (for example Figure 1The network structure information and the parameter values of the network parameters of the trained initial classification model can be stored locally by the execution subject of the training step if the network structure information and the parameter values of the network parameters of the trained initial classification model are the same as the network structure information and the parameter values of the network parameters of the terminal device shown in the figure. If the network structure information and the parameter values of the network parameters of the trained initial classification model are different from the network structure information and the parameter values of the network parameters of the terminal device shown in the figure, the execution subject of the training step can send the network structure information and the parameter values of the network parameters of the trained initial classification model to the execution subject of the multi-entity joint linking method after the trained initial classification model is obtained.

[0079] In this embodiment, the execution subject of the training step can first determine the network structure of the initial classification model. For example, it is necessary to determine which layers are included in the initial classification model, the connection order relationship between the layers, which neurons are included in each layer, the weight and bias corresponding to each neuron, the activation function of each layer, and the like.

[0080] Then, the execution subject of the training step can initialize the network parameters of the initial classification model. In practice, each network parameter (for example, weight parameter and bias parameter) of the initial classification model can be initialized with some different small random numbers. The "small random number" is used to ensure that the network will not enter a saturated state due to excessive weight, thereby causing training failure, and the "different" is used to ensure that the network can normally learn.

[0081] Step 402, obtaining a training sample set.

[0082] In this embodiment, the execution subject of the training step can obtain the training sample set locally or remotely from other terminal devices connected to the above-mentioned execution subject network. The training sample set includes a set of sample entity information and a sample category corresponding to the set of sample entity information.

[0083] Step 403, selecting a training sample from the training sample set, taking the set of training sample entity information included in the training sample as input, taking the corresponding pre-obtained training sample category corresponding to the set of training sample entity information as expected output, and training the initial classification model.

[0084] In this embodiment, the execution subject of the training step can perform step one, training the initial classification model.

[0085] Step one, initial classification model training process.

[0086] First, input the set of training sample entity information included in the selected training sample in the set of selected training samples to the initial classification model to obtain the category of the selected training sample.

[0087] Secondly, the category of the selected training sample is compared with the category of the corresponding training sample. Specifically, the difference between the category of the selected training sample and the category of the corresponding training sample can be calculated first by using a preset loss function. For example, the difference between the category of the selected training sample and the category of the corresponding training sample can be calculated by using a cross-entropy loss function, in which a sigmoid function is used to avoid the problem of reducing the machine learning rate during gradient descent.

[0088] Thirdly, in response to determining that the initial classification model reaches the optimization target, the initial classification model is determined as the pre-trained classification model trained to be completed, and the training is ended. Specifically, the preset optimization target can include, but is not limited to, at least one of the following: the training time exceeds the preset time length; the training times exceeds the preset number of times; and the calculated difference is less than the preset difference threshold.

[0089] Step 404, in response to determining that the initial classification model is not trained to be completed, adjusting the related parameters in the initial classification model, and reselecting the training sample from the training sample set, using the adjusted initial classification model as the initial classification model, and executing the training step again.

[0090] In this embodiment, the execution subject of the training step adjusts the related parameters in the initial classification model in response to determining that the initial classification model is not trained to be completed, specifically, in response to the initial classification model not reaching the optimization target. Specifically, the network parameters of the initial classification model can be adjusted based on the difference between the category of the selected sample and the category of the corresponding sample by using various implementation manners. For example, the network parameters of the initial classification model can be adjusted by using Adam, BP (Back Propagation, back propagation) algorithm or SGD (Stochastic Gradient Descent, Stochastic Gradient Descent) algorithm.

[0091] Optionally, the execution subject reselects the training sample from the training sample set. The set of training sample entity information included in the training sample is taken as input, the corresponding pre-obtained training sample category corresponding to the set of training sample entity information is taken as expected output, the adjusted initial classification model is taken as the initial classification model, and the initial classification model is trained again by using the above step one.

[0092] In this embodiment, the execution subject of the training step determines the initial classification model trained to be the pre-trained classification model.

[0093] Figure 4An embodiment given has the following beneficial effects: a pre-trained classification model is trained based on a sample entity information set and a sample category corresponding to the sample entity information set. The pre-trained classification model can be directly applied to determine the result of whether an input entity information set corresponds to the category. A candidate entity information set is directly input into the pre-classified classification model, without manual intervention, without the need to separately extract features of the candidate entity information, and the result of whether the candidate entity information corresponds to the "0" or "1" category can be automatically obtained.

[0094] Reference will be made to the following description Figure 5 , which shows a structural schematic diagram of a computer system 500 suitable for implementing a terminal device of an embodiment of the present disclosure. Figure 5 The terminal device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0095] As Figure 5 shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or loaded from a storage portion 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the system 500 are also stored in the RAM 503. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0096] The following components are connected to the I / O interface 505: a storage portion 506 including a hard disk and the like; and a communication portion 507 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication portion 507 performs communication processing via a network such as the Internet. A drive 508 is also connected to the I / O interface 505 as needed. A removable medium 509 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 508 as needed, so that a computer program read therefrom is installed in the storage portion 506 as needed.

[0097] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 507, and / or installed from the detachable medium 509. When the computer program is executed by the central processing unit (CPU) 501, the above-described functions defined in the methods of the present disclosure are executed. It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code contained on a computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0098] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0099] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0100] The above description is only preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the application involved in the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

Claims

1. A method of multi-entity joint linking, comprising: obtaining a target entity mention and an original text, wherein the original text comprises a first number of characters; generating a candidate entity information set based on a predetermined text corpus and the target entity mention, wherein the candidate entity information comprises a candidate entity and an entity description text; inputting the candidate entity information set into a pre-trained encoding model to generate a candidate entity feature set; wherein the inputting the candidate entity information set into the pre-trained encoding model to generate the candidate entity feature set comprises: for each candidate entity information in the candidate entity information set, inputting a candidate entity of the candidate entity information into a first encoding network to generate a candidate entity first encoding feature to obtain a candidate entity first encoding feature set, wherein the first encoding network is an encoder network, the encoder network encoder comprises six identical hierarchical layers, each hierarchical layer comprises two sub-layers, the first sub-layer is a multi-head self-attention mechanism, and the second sub-layer is a simple fully connected feedforward network, the two layers are connected by a residual network respectively, the residual network is composed of a second number of residual modules, the residual module generates an output by using the following formula: y = F(x, {Wi}) + x, wherein x is an input of the residual module, y is an output of the residual module, F() is a residual function, W is a weight matrix, i is a layer count in the residual module, Wi is a weight matrix of the i-th layer, {Wi} represents a set of weight matrices of all layers in the residual module, in order to optimize the results of the residual networks, all hierarchical layers in the first encoding network generate an output with a dimension of 512; inputting the candidate entity first encoding feature set into a second encoding network to generate the candidate entity feature set, wherein the second encoding network is a decoder network; inputting the candidate entity first encoding feature set into a calculation module to generate a candidate entity first encoding feature parameter set; inputting the candidate entity first encoding feature set and the candidate entity first encoding feature parameter set into a focusing module to generate the candidate entity feature set, wherein the focusing module comprises six identical hierarchical layers, the focusing module is an attention mechanism, concatenating the candidate entity first encoding feature set and the candidate entity first encoding feature parameter set to obtain an initial input sequence, determining a weight parameter W: {wi = (wi,1, wi,2,..., wi,k)}, wherein W is a weight parameter, i is a hierarchical layer count, k is an initial input number in the initial input sequence, wi is a weight parameter of the i-th hierarchical layer, the focusing module comprises a first number of hierarchical layers, the first number is 6, wi,1 represents a weight parameter of the i-th hierarchical layer corresponding to the first initial input, and wi,k represents a weight parameter of the i-th hierarchical layer corresponding to the k-th initial input; calculating an intermediate input sequence of the focusing module by using the following formula: C: {Ci = wi * Z = (wi,1, wi,2,..., wi,k) * Z}, Wherein, Z is an initial input sequence, C is an intermediate input sequence, Ci is an intermediate input of the i th layer, the focusing module includes a first number of layers, and the intermediate input sequence includes a first number of intermediate inputs; An output sequence of the focusing module is calculated by using the following formula: Y: {yi = fde(Ci, yi-1)}, Wherein, fde is a neuron of the i th layer, Ci is an intermediate input of the i th layer; yi is an output of the i th layer, yi-1 is an output of the i-1 th layer, Y is an output sequence of the focusing module, the last output in the output sequence of the focusing module is determined as a candidate entity feature set, fde is a one-layer perceptron, yi is calculated by using a normalized exponential function, the output of the last layer is determined as an output of the decoder network to obtain the candidate entity feature set; For each candidate entity information in the candidate entity information set, the candidate entity, the original text and the candidate entity feature set of the candidate entity information are spliced to generate a text information feature to obtain a text information feature set; The text information feature set is input into a pre-trained classification model to generate a discrimination result set; Based on the discrimination result set, a result entity set is determined.

2. The method of claim 1, wherein, The candidate entity information set is generated based on the pre-determined text library and the target entity, including: A candidate entity reference set is determined according to the pre-determined text library and the target entity reference; For each candidate entity reference in the candidate entity reference set, a candidate entity corresponding to the candidate entity reference is searched in the pre-determined text library to generate a candidate entity information to obtain a candidate entity information set.

3. The method of claim 2, wherein, The pre-trained classification model includes a third encoding network, a first decoding network; and For each text information feature in the text information feature set, the text information feature is input into the third encoding network to generate a first encoded text information feature to obtain a first encoded text information feature set; For each first encoded text information feature in the first encoded text information feature set, the first encoded text information feature is input into the first decoding network to generate a discrimination result to obtain the discrimination result set.

4. The method of claim 3, wherein, The candidate entity, the original text and the candidate entity feature set of the candidate entity information are spliced to generate a text information feature, including: Based on the original text, a character set is generated, wherein the character set includes a first number of characters; The character set is input into the first encoding network to generate a character feature; The candidate entity of the candidate entity information is input into the first encoding network to generate a candidate entity encoding feature; The candidate entity encoding feature, the character feature and the candidate entity feature set are spliced to generate the text information feature.

5. The method of claim 4, wherein, The result entity set is determined based on the discrimination result set, including: For each discrimination result in the discrimination result set, in response to the discrimination result being "1", the candidate entity corresponding to the discrimination result is determined as a result entity to obtain a result entity set; The result entity set is determined as the entity linking result set.

6. The method of claim 5, wherein, The method further includes: The entity linking result set is pushed to a target device having a display function, and the target device is controlled to display the entity linking result set.

7. A first terminal device comprising: one or more processors; a memory device having stored thereon one or more programs; when the one or more programs are executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.

Images