Entity alias relationship acquisition method, training method, device and storage medium

Abstract

The disclosed embodiments provide an entity alias relationship acquisition method, training method, device and storage medium, which are applied to an entity alias relationship model; based on the model's embedding layer, input text to obtain a text feature vector sequence; based on the feature encoding layer, obtain a text semantic feature vector sequence; based on the sequence annotation layer, obtain a text prediction label sequence, and obtain each entity mention segment in the text; based on the label fusion layer, fuse the text semantic feature vector sequence and the text prediction label sequence to obtain a text enhanced feature vector sequence; based on the multi-head selection layer, obtain an entity alias relationship probability matrix; according to the positions of the characters corresponding to the rows and columns of the screened probability values ​​in the entity mention segments to which they belong, obtain the entity alias relationship acquisition result between the two entity mention segments. The disclosed solution realizes accurate and efficient mining of entity alias relationships, which is conducive to the construction of high-quality knowledge graphs.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an entity alias relationship acquisition method, training method, device and storage medium. Background Art

[0002] Knowledge graph is considered to be an important cornerstone for the transition from perceptual intelligence to cognitive intelligence. As a semantic network, knowledge graph has strong expressive power and modeling flexibility, and can accurately describe entities and relationships between entities in the real world. It is not only easy for humans to understand, but its machine-friendly nature has made this technology widely used in information retrieval, intelligent question and answer, search recommendation, e-commerce and other scenarios in recent years, greatly improving the defects of traditional machine learning algorithms that lack reasoning and association capabilities.

[0003] The construction of high-quality knowledge graphs is a major challenge facing the industry. With more and more knowledge data mined through various methods, the first problem that needs to be solved is how to integrate new and old knowledge, reduce information overload, and avoid entity redundancy. For example, "Guangzhou" and "Yangcheng" can both represent the capital of Guangdong Province, China, but the two are quite different from the literal meaning. If they are not aligned to the same entity, the effects of some downstream tasks such as entity expansion and entity disambiguation will be affected.

[0004] Therefore, automatically mining entity synonyms from massive corpora is an indispensable task for building high-quality knowledge graphs.

[0005] Entity synonym mining usually involves judging how likely two given entity mentions are to refer to the same entity, and its main goal is to learn a metric function that can distinguish synonyms from non-synonyms. Previous methods have mostly focused on feature engineering on the literal amount of entity mentions, and calculated entity similarity by constructing character-level features. However, many semantically similar entity mentions have very little or no overlap at the character level, and it is also difficult to judge to what extent mentions with similar literal features can represent the same entity without a specific context.

[0006] One approach to solving entity synonym mining is to convert it into an entity relationship extraction task, because entity synonyms correspond to entity alias relationships, which can be regarded as a special type of entity relationship. However, there is still a lack of accurate and efficient solutions for extracting entity alias relationships.

[0007] Invention News

[0008] In view of the above shortcomings of the related art, the purpose of the present disclosure is to provide an entity alias relationship acquisition method, training method, device and storage medium to solve the problems in the related art.

[0009] The first aspect of the present disclosure provides an entity alias relationship acquisition method, which is applied to an entity alias relationship acquisition model, wherein the entity alias relationship acquisition model includes: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the method includes: obtaining an input text and inputting it into the entity alias relationship acquisition model; processing the input text based on the embedding layer to generate a character-level token sequence, a character-level position index sequence and a text fragment index sequence, and processing the fusion of feature vectors respectively mapped by the character-level token sequence, the character-level position index sequence and the text fragment index sequence based on the embedding layer to obtain a text feature vector sequence; processing the text feature vector sequence based on the feature encoding layer to obtain a text semantic feature vector sequence; processing the text semantic feature vector sequence based on the sequence annotation layer to obtain a text feature vector sequence. A text prediction label sequence is obtained, and each entity mention segment in the text is obtained according to the text prediction label sequence; the text semantic feature vector sequence and the text prediction label sequence are fused based on a label fusion layer to obtain a text enhanced feature vector sequence; the text enhanced feature vector sequence is processed based on a multi-head selection layer to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between entity mention segments to which each two characters in the input text belong respectively; probability values ​​reaching a preset threshold are filtered from the entity alias relationship probability matrix, and the entity alias relationship acquisition result between the two entity mention segments is obtained according to the positions of the characters corresponding to the rows and columns of the filtered probability values ​​in the entity mention segments to which they belong respectively.

[0010] In an embodiment of the first aspect, the fusion of sequence vectors respectively mapped based on the word-level token sequence, the word-level position index sequence and the text fragment index sequence to obtain a text feature vector sequence includes: encoding the word-level token sequence, the word-level position index sequence and the text fragment index sequence to obtain respective encoding sequences; converting each of the encoding sequences into each of the sequence vectors; summing and normalizing each of the sequence vectors to obtain the text feature vector sequence.

[0011] In an embodiment of the first aspect, the feature encoding layer includes a multi-head self-attention layer and a fully connected layer; the feature encoding layer-based processing of the text feature vector sequence to obtain a text semantic feature vector sequence includes: processing the text feature vector sequence through a multi-head self-attention layer to obtain an intermediate feature vector sequence; processing the intermediate feature vector sequence through a fully connected layer to obtain the text semantic feature vector sequence.

[0012] In an embodiment of the first aspect, the sequence annotation layer is used to process the text semantic feature vector sequence to obtain a text prediction label sequence, and each entity mention segment in the text is obtained according to the text prediction label sequence, including: predicting the position of the word corresponding to each feature value in the text semantic feature vector sequence in the corresponding entity mention segment to obtain a prediction label to form the text prediction label sequence; and obtaining each entity mention segment based on the entity mention boundaries marked by the text prediction label sequence.

[0013] In an embodiment of the first aspect, the label fusion layer includes a gated neural network layer; the label fusion layer-based fusion of the text semantic feature vector sequence and the text prediction label sequence to obtain a text enhancement feature vector sequence includes: fusing the text semantic feature vector sequence and the text prediction label sequence according to word-level position through the gated neural network layer to obtain a text enhancement feature vector sequence.

[0014] In an embodiment of the first aspect, the entity alias relationship acquisition result between two entity mention segments is obtained based on the positions of the characters respectively corresponding to the rows and columns where the filtered probability values ​​are located in the entity mention segments to which they belong, including: in response to the characters respectively corresponding to the rows and columns where the probability values ​​are located being located at the same boundary positions in the entity mention segments to which they belong, determining that an entity alias relationship exists between the two entity mention segments.

[0015] In an embodiment of the first aspect, the entity alias relationship acquisition method further includes: generating a mask sequence corresponding to the input text; and inputting the text feature vector sequence after being acted upon by the mask sequence into the entity alias relationship acquisition model.

[0016] The second aspect of the present disclosure provides a training method for an entity alias relationship acquisition model, wherein the entity alias relationship acquisition model includes: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the training method includes: obtaining a training sample set and inputting it into the entity alias relationship acquisition model, wherein each training sample text in the training sample set has a corresponding text true label sequence and an entity alias relationship label; generating a word-level token sequence, a word-level position index sequence and a text fragment index sequence based on each training sample text, and obtaining a text feature vector sequence based on the fusion of sequence vectors respectively mapped by the word-level token sequence, the word-level position index sequence and the text fragment index sequence; processing the text feature vector sequence based on the feature encoding layer to obtain a text semantic feature vector sequence; processing the text semantic feature vector sequence based on the sequence annotation layer to obtain a text prediction label sequence, and obtaining the text according to the text prediction label sequence. Each entity mention segment in the text; calculate the first loss between the text prediction label sequence and the text real label sequence; based on the label fusion layer, fuse the text semantic feature vector sequence with the text prediction label sequence or the text real label sequence to obtain a text enhanced feature vector sequence; based on the multi-head selection layer, process the text enhanced feature vector sequence to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that there is an entity alias relationship between the entity mention segments to which each two words in the text belong respectively; calculate the second loss based on each probability value in the entity alias relationship probability matrix and the entity alias relationship label; wherein the entity alias relationship label is determined based on whether there is an entity alias relationship between the two words corresponding to the row and column corresponding to each probability value; obtain the overall loss based on the fusion of the first loss and the second loss, and update the entity alias relationship acquisition model according to the overall loss. In an embodiment of the second aspect, in the training round using the first part of the training sample text, the text semantic feature vector sequence and the text prediction label sequence are fused based on the label fusion layer to obtain a text enhanced feature vector sequence; in the training round of the remaining second part of the training sample text, the text semantic feature vector sequence and the text true label sequence are fused based on the label fusion layer to obtain a text enhanced feature vector sequence.

[0017] In an embodiment of the second aspect, the sample number of the training samples in the second part is greater than that of the training samples in the first part.

[0018] In an embodiment of the second aspect, the fusion of sequence vectors respectively mapped based on the word-level token sequence, the word-level position index sequence and the text fragment index sequence obtains a text feature vector sequence, including: encoding the word-level token sequence, the word-level position index sequence and the text fragment index sequence respectively to obtain each encoded sequence; converting each of the encoded sequences into each of the sequence vectors; summing and normalizing each of the sequence vectors to obtain the text feature vector sequence.

[0019] In an embodiment of the second aspect, the feature encoding layer includes a multi-head self-attention layer and a fully connected layer; the feature encoding layer-based processing of the text feature vector sequence to obtain a text semantic feature vector sequence includes: processing the text feature vector sequence through a multi-head self-attention layer to obtain an intermediate feature vector sequence; processing the intermediate feature vector sequence through a fully connected layer to obtain the text semantic feature vector sequence.

[0020] In an embodiment of the second aspect, the sequence-based annotation layer processes the text semantic feature vector sequence to obtain a text prediction label sequence, and obtains each entity mention segment in the text according to the text prediction label sequence, including: predicting the position of the word corresponding to each feature value in the text semantic feature vector sequence in the corresponding entity mention segment to obtain a prediction label to form the text prediction label sequence; and obtaining each entity mention segment based on the entity mention boundaries marked by the text prediction label sequence.

[0021] In an embodiment of the second aspect, the label fusion layer includes a gated neural network layer; the label fusion layer-based fusion of the text semantic feature vector sequence with the text prediction label sequence or the text true label sequence to obtain a text enhanced feature vector sequence includes: fusing the text semantic feature vector sequence and the text prediction label sequence according to the word-level position through the gated neural network layer to obtain a text enhanced feature vector sequence.

[0022] In an embodiment of the second aspect, the position correspondence means that the characters corresponding to the row and column where the probability values ​​are located are located at the same boundary position in the entity mention segment to which they belong.

[0023] In an embodiment of the second aspect, the training method further includes: generating a mask sequence corresponding to the text; and inputting the text feature vector sequence after being acted upon by the mask sequence into an entity alias relationship acquisition model.

[0024] The third aspect of the present disclosure provides an entity alias relationship acquisition device, which is applied to an entity alias relationship acquisition model, wherein the entity alias relationship acquisition model includes: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the device includes: an input module, which is used to obtain input text and input it into the entity alias relationship acquisition model; the embedding layer is used to process the input text to generate a word-level token sequence, a word-level position index sequence and a text fragment index sequence, and obtain a text feature vector sequence based on the fusion of feature vectors respectively mapped by the word-level token sequence, the word-level position index sequence and the text fragment index sequence; the feature encoding layer is used to process the text feature vector sequence to obtain a text semantic feature vector sequence; the sequence annotation layer is used to process the text semantic feature vector sequence To obtain a text prediction label sequence, and obtain each entity mention segment in the text according to the text prediction label sequence; the label fusion layer is used to fuse the text semantic feature vector sequence and the text prediction label sequence to obtain a text enhanced feature vector sequence; the multi-head selection layer is used to process the text enhanced feature vector sequence to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between the entity mention segments to which each two characters in the input text belong respectively; and, the probability values ​​reaching a preset threshold are screened from the entity alias relationship probability matrix, and the entity alias relationship acquisition result between the two entity mention segments is obtained according to the positions of the characters corresponding to the rows and columns of the screened probability values ​​in the entity mention segments to which they belong respectively.

[0025] According to a fourth aspect of the present disclosure, there is provided a training device for an entity alias relationship acquisition model, wherein the entity alias relationship acquisition model comprises: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the training device comprises: an input module, which is used to obtain a training sample set and input the entity alias relationship acquisition model, wherein each training sample text in the training sample set has a corresponding text true label sequence and an entity alias relationship label; the embedding layer, which is used to generate a word-level token sequence, a word-level position index sequence and a text segment index sequence based on each training sample text, and obtain a text feature vector sequence based on the fusion of sequence vectors respectively mapped by the word-level token sequence, the word-level position index sequence and the text segment index sequence; the feature encoding layer, which is used to process the text feature vector sequence to obtain a text semantic feature vector sequence; the sequence annotation layer, which is used to process the text semantic feature vector sequence to obtain a text prediction label sequence, and obtain each entity in the text according to the text prediction label sequence. Mention segment; a loss calculation module, used to calculate the first loss between the text prediction label sequence and the text real label sequence; the label fusion layer, used to fuse the text semantic feature vector sequence with the text prediction label sequence or the text real label sequence to obtain a text enhanced feature vector sequence; the multi-head selection layer, used to process the text enhanced feature vector sequence to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that there is an entity alias relationship between the entity mention segments to which each two words in the text belong respectively; the loss calculation module, used to calculate the second loss based on each probability value in the entity alias relationship probability matrix and the entity alias relationship label; wherein the entity alias relationship label is determined based on whether there is an entity alias relationship between the two words corresponding to the row and column corresponding to each probability value; the loss calculation module, used to obtain the overall loss based on the fusion of the first loss and the second loss, and update the entity alias relationship acquisition model according to the overall loss. The fifth aspect of the present disclosure provides a computer device, including: a communicator, a memory and a processor; the communicator is used to communicate with the outside; the memory stores program instructions; the processor is used to run the program instructions to execute the entity alias relationship acquisition method as described in any one of the first aspect; or, execute the training method as described in any one of the second aspect.

[0026] A sixth aspect of the present disclosure provides a computer-readable storage medium storing program instructions, wherein the program instructions are executed to execute the entity alias relationship acquisition method as described in any one of the first aspect; or, to execute the training method as described in any one of the second aspect.

[0027] As described above, the embodiments of the present disclosure provide an entity alias relationship acquisition method, a training method, an apparatus, and a storage medium, which are applied to an entity alias relationship model; a text feature vector sequence is obtained by inputting text based on the embedding layer of the model; a text semantic feature vector sequence is obtained based on the feature encoding layer; a text prediction label sequence is obtained based on the sequence annotation layer, and each entity mention segment in the text is obtained; the text semantic feature vector sequence and the text prediction label sequence are fused based on the label fusion layer to obtain a text enhanced feature vector sequence; the text enhanced feature vector sequence is processed based on the multi-head selection layer to obtain an entity alias relationship probability matrix; the entity alias relationship acquisition result between the two entity mention segments is obtained according to the positions of the characters corresponding to the rows and columns of the screened probability values ​​in the entity mention segments to which they belong. The present disclosure implements a solution for accurately and efficiently mining entity alias relationships, which is conducive to the construction of high-quality knowledge graphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart showing a method for training an entity alias relationship acquisition model in an embodiment of the present disclosure is shown.

[0029] Figure 2 A schematic diagram showing the architecture of an entity alias relationship acquisition model in an embodiment of the present disclosure.

[0030] Figure 3 A schematic diagram showing an entity alias relationship probability matrix in an example of the present disclosure.

[0031] Figure 4 A schematic diagram showing the training principle of the entity alias relationship acquisition model in a specific implementation architecture in one embodiment of the present disclosure.

[0032] Figure 5 A flowchart of a method for acquiring entity alias relationships in an embodiment of the present disclosure is shown.

[0033] Figure 6 A module schematic diagram showing an entity alias relationship acquisition device in an embodiment of the present disclosure.

[0034] Figure 7 A module diagram showing a training device for an entity alias relationship acquisition model in an embodiment of the present disclosure.

[0035] Figure 8 A schematic diagram showing the structure of a computer device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0036] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the information disclosed in the present disclosure. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in the present disclosure can also be modified or changed in various ways according to different viewpoints and application systems without departing from the spirit of the present disclosure. It should be noted that the embodiments and features in the embodiments in the present disclosure can be combined with each other without conflict.

[0037] The following is a detailed description of the embodiments of the present disclosure with reference to the accompanying drawings so that those skilled in the art can easily implement the present disclosure. The present disclosure can be embodied in many different forms and is not limited to the embodiments described herein.

[0038] In the representations of the present disclosure, the reference terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" and the like mean that the specific features, structures, materials, or characteristics represented in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or characteristics represented may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples represented in the present disclosure and the features of different embodiments or examples, unless they are mutually contradictory.

[0039] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the representation of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0040] In order to clearly describe the present disclosure, components not related to the description are omitted, and the same reference numerals are given to the same or similar components throughout the specification.

[0041] Throughout the specification, when a device is said to be "connected" to another device, this includes not only the case of "direct connection" but also the case of "indirect connection" by placing other elements therebetween. In addition, when a device is said to "include" a certain component, unless otherwise stated, it does not exclude other components, but means that other components may be included.

[0042] Although the terms first, second, etc. are used to represent various elements in some examples, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first interface and the second interface, etc. are represented. Moreover, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, modules, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, modules, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0043] The technical terms used herein are only used to refer to specific embodiments and are not intended to limit the present disclosure. The singular form used herein also includes the plural form unless the sentence clearly indicates the contrary meaning. The meaning of "including" used in the specification is to specify specific characteristics, regions, integers, steps, operations, elements and / or components, and does not exclude the existence or addition of other characteristics, regions, integers, steps, operations, elements and / or components.

[0044] Terms indicating relative space, such as "below" and "above", may be used to more easily describe the relationship of one device illustrated in the drawings relative to another device. Such terms refer not only to the meaning indicated in the drawings, but also to other meanings or executions of the device in use. For example, if the device in the drawings is turned over, a device that was described as being "below" other devices is described as being "above" other devices. Therefore, the exemplary term "below" includes both above and below. The device can be rotated 90° or other angles, and the terms indicating relative space are interpreted accordingly.

[0045] Although not defined differently, all terms, including technical and scientific terms used herein, have the same meaning as those generally understood by those skilled in the art to which the present disclosure belongs. Terms defined in commonly used dictionaries are additionally interpreted as having meanings consistent with relevant technical literature and the message of the present disclosure, and shall not be overly interpreted as ideal or very formal meanings unless defined.

[0046] In actual scenarios, in order to build a high-quality knowledge graph, there is a need to mine entity relationships from a large amount of unstructured corpus. In addition, since entities may have various entity synonyms, such as "Shanghai", "Hu", etc., accurately mining the entity alias relationship between these entity synonyms is crucial to building a high-quality knowledge graph.

[0047] In view of this, in the embodiments of the present disclosure, machine learning technology is used to establish and train an entity alias relationship acquisition model, so that entity mentions with entity alias relationships can be accurately mined from the corpus through the entity alias relationship acquisition model.

[0048] See also Figure 1 As shown, a flow chart showing the training method of the entity alias relationship acquisition model in the embodiment of the present disclosure is shown. Figure 2 As shown, in some embodiments, the entity alias relationship acquisition model 200 may include: an embedding layer 201, a feature encoding layer 202, a sequence annotation layer 203, a label fusion layer 204, and a multi-head selection layer 205. The input text is processed by the above layers in sequence, and a corresponding loss function is constructed to back-propagate the loss to update the model parameters.

[0049] exist Figure 1 In the process, the process includes:

[0050] Step S101: obtaining a training sample set and inputting it into the entity alias relationship acquisition model, wherein each training sample text in the training sample set has a corresponding text true label sequence and an entity alias relationship label.

[0051] In some embodiments, the source of each text in the training sample set may include but is not limited to a known library, an online encyclopedia or a website, etc. Optionally, if the entity alias relationship acquisition model is required to be dedicated to the identification of entity alias relationships in texts related to a certain field, relevant texts in the field may be collected, for example, consulting websites and encyclopedias related to the tourism industry.

[0052] Step S102: Generate a character-level token sequence, a character-level position index sequence and a text segment index sequence based on each training sample text, and obtain a text feature vector sequence based on the fusion of sequence vectors mapped by the character-level token sequence, the character-level position index sequence and the text segment index sequence.

[0053] In some embodiments, a Word Piece Tokenization tokenizer such as a BERT model may be used to generate a word-level token sequence, a word-level position index sequence, and a text segment index sequence based on the text.

[0054] In some embodiments, given the text "Xizi Lake generally refers to the West Lake", the character-level token sequence {[CLS], "X", "i", "z", "i", "L", "a", "k", "e", "g", "e", "n", "e", "r", "a", "l", "l", "y", "r", "e", "f", "e", "r", "s", "t", "o", "t", "h", "e", "W", "e", "s", "t", "L", "a", "k", "e", "[SEP]"} can be obtained. The character-level position index sequence can be a sequence of {0, 1....9} corresponding to 10 tokens. The text segment index sequence refers to the label of the segment (Segment) to which each character belongs. Each segment can be, for example, a sentence, etc. Segments can be separated by "[SEP]". For example, "Xizi Lake generally refers to the West Lake" is a segment, and its label is, for example, A. Then the corresponding text segment index sequence can be expressed as, for example, {A, A, A, A, A, A, A, A, A, A}.

[0055] In some embodiments, a corresponding mask (MASK) sequence can also be generated based on each text, and the text feature vector sequence after the action of the mask sequence is input into the entity alias relationship acquisition model. For example, by masking the "i" in the character-level token sequence {[CLS], "X", "i", "z", "i", "L", "a", "k", "e", "g", "e", "n", "e", "r", "a", "l", "l", "y", "r", "e", "f", "e", "r", "s", "t", "o", "t", "h", "e", "W", "e", "s", "t", "L", "a", "k", "e", "[SEP]"} with the mask sequence, the resulting character-level token sequence after the action is {[CLS], "X", "MASK", "z", "i", "L", "a", "k", "e", "g", "e", "n", "e", "r", "a", "l", "l", "y", "r", "e", "f", "e", "r", "s", "t", "o", "t", "h", "e", "W", "e", "s", "t", "L", "a", "k", "e", "[SEP]"}.

[0056] Exemplarily, the character-level token sequence, the character-level position index sequence, and the text segment index sequence can be encoded respectively (such as one-hot encoding, etc.) to obtain respective encoded sequences, convert each of the encoded sequences into each of the sequence vectors, sum and normalize each of the sequence vectors to obtain the text feature vector sequence.

[0057] The above possible implementation methods can be illustrated through specific application examples. It can be completed through a model embedding layer, which can be exemplarily implemented by adopting the embedding method of BERT to integrate the information of characters, positions, and segments. Taking the character-level token sequence as an example, the character-level token sequence of the input text can be numerically processed and converted into the corresponding index id sequence in the token vocabulary V, and the encoded sequence is obtained through one-hot encoding. Use a word embedding matrix Convert the encoded sequence into an h-dimensional dense vector, that is, the sequence vector:

[0058]

[0059] Similarly, the character-level position index sequence and the text segment index sequence also respectively adopt similar operations to obtain the position information encoding, that is, the sequence vector and the segment information encoding, that is, the sequence vector Then, these three vectors are summed up and fused and layer normalized (LayerNorm) to obtain the following text feature vector sequence:

[0060]

[0061] Step S103: Processing the text feature vector sequence based on the feature coding layer to obtain a text semantic feature vector sequence.

[0062] In some embodiments, the feature coding layer includes a multi-head self-attention layer and a fully connected layer. In step S103, in the feature coding layer, the text feature vector sequence is processed by the multi-head self-attention layer to obtain an intermediate feature vector sequence; and the intermediate feature vector sequence is processed by the fully connected layer to obtain the text semantic feature vector sequence.

[0063] The above principle is specifically explained by examples. The feature encoding layer can use, for example, a Transformer Encoder structure, whose basic unit includes a multi-headed self-attention layer and a fully connected layer. The feature encoding layer includes a multi-headed self-attention layer and a fully connected layer. Multi-headed self-attention refers to the Q, K, and V of the token being self-attentioned multiple times to obtain multiple different outputs, and then these different outputs are connected to obtain the final output of the token.

[0064] Since the text feature vector sequence is composed of the feature vector of each token, it is actually represented as a sequence feature matrix H (0) =[e1,e2,…,e T ], T is the number of tokens.

[0065] Assume that a total of L EncoderBlocks are used. For l = 1, 2, ..., L, the following processing is performed:

[0066] 1) In the multi-head self-attention layer, N attention heads are used to extract context features at different grammatical or semantic levels. The dimension of each head is set to d = h / N, and the weights of Query, Key, Value and proj are In addition, residual connections are added to the sequence features aggregated by the attention mechanism to control the upward flow of low-level information.

[0067]

[0068]

[0069]

[0070] 2) In the fully connected layer, the parameter matrix includes Here, the feature vector is first mapped to a high-dimensional space, activated by a Gaussian error linear unit (gelu), and then projected to the original low-dimensional space. A residual connection is also added. The process is expressed as:

[0071]

[0072]

[0073] S104: Processing the text semantic feature vector sequence based on the sequence annotation layer to obtain a text prediction label sequence, and obtaining each entity mention segment in the text according to the text prediction label sequence.

[0074] In some embodiments, step S104 may specifically include: predicting the position of the word corresponding to each feature value in the text semantic feature vector sequence in the corresponding entity mention segment to obtain a predicted label to form the text prediction label sequence; and obtaining each of the entity mention segments based on the entity mention boundaries marked by the text prediction label sequence.

[0075] Let’s take an example to illustrate. Continuing with the previous example, suppose the text prediction label sequence of length T output by the L encoders in the feature encoding layer is For example, the sequence annotation layer can be implemented using a Softmax classification layer to predict the predicted label (i.e., probability) of each token belonging to the position in the entity mention segment, so as to identify the boundary of the entity mention segment in the sequence. Specifically, by adopting the BIO annotation system, assuming that the token label set is S, the projection matrix is Bias Then the label probability distribution of the token at position t is

[0076]

[0077] By way of example, the annotation BX indicates that the token is in the header of entity X; the annotation IX indicates that the token is inside entity X; and O indicates that the token is outside the entity.

[0078] Therefore, the labels of the corresponding positions of "West Lake generally refers to West Lake" in the real label sequence of the text can be expressed as "West" corresponds to "B", "Zi" corresponds to "I", "Lake" corresponds to "I", "One" corresponds to "O", "General" corresponds to "O", "Refer to" corresponds to "O", "West" corresponds to "B", "Lake" corresponds to "I", that is, BIIOOOBI. According to the closure of two Bs and their connected I, two entity mention fragments "West Lake" and "West Lake" are obtained.

[0079] Ideally, if the text prediction label sequence is the same as the text true label sequence, the prediction is accurate, which is the goal of training. Therefore, the sequence annotation loss between the text prediction label sequence and the text true label sequence is calculated, and the model parameters are updated with the goal of minimizing the loss.

[0080] S105: Calculate the first loss between the text prediction label sequence and the text true label sequence, that is, the sequence labeling loss.

[0081] In a possible example, the first loss L ner The calculation uses the cross entropy function:

[0082]

[0083] in, That is, the value of the tth position in the text true label sequence, That is, the value of the tth position in the text prediction label sequence.

[0084] S106: Based on the label fusion layer, the text semantic feature vector sequence is fused with the text prediction label sequence or the text real label sequence to obtain a text enhanced feature vector sequence.

[0085] In some embodiments, the label fusion layer includes a gated neural network layer, which may be similar to the principle of a gated recurrent unit (GRU). Step S106 may specifically include: fusing the text semantic feature vector sequence and the text prediction label sequence based on the label fusion layer to obtain a text enhancement feature vector sequence, including: fusing the text semantic feature vector sequence and the text prediction label sequence according to the word-level position through the gated neural network layer to obtain a text enhancement feature vector sequence.

[0086] Let's take an example to illustrate the principle. The text true / predicted label sequence represents the position of each token in the entity segment, which has a certain promoting effect on determining the entity relationship. Especially for the table filling type relationship extraction task, the introduction of word-level entity label information helps to reduce noise, so that the entity relationship acquisition model can pay more attention to the interaction information between entity mention segments.

[0087] In the training phase, taking the fusion of text semantic feature vector sequence and text real label sequence as an example, the text real label sequence is y = [y1, y2, ..., y T ],in It can be exemplarily a one-hot vector, which is embedded in the label matrix Processing results:

[0088] v t =W label y t

[0089] The label information is fused through a gated neural network layer similar to GRU. The principle is as follows:

[0090]

[0091]

[0092]

[0093]

[0094] The brackets represent the elements on both sides of the semicolon. is the network parameter, ⊙ represents the Hadamard product operation. t Fusion with y t Related t , g t The information and feature encoding layer output The text enhancement feature vector sequence after integrating label information is H = [h1,h2,…,h T ], t = 1 ~ T; the H sequence can be represented by vectors h1 ~ h T Stacked to form a matrix.

[0095] Similarly, if the text semantic feature vector and the text prediction label sequence are fused, the text prediction label sequence can be used Replace y in the above description t In addition, when the model is actually applied, the predicted text is also used to predict the label sequence To obtain the text enhanced feature vector sequence.

[0096] In some embodiments, when using each training sample of the training set, in the training round using the first part of the training sample text, the text semantic feature vector sequence and the text prediction label sequence can be fused based on the label fusion layer to obtain a text enhanced feature vector sequence; in the training round using the remaining second part of the training sample text, the text semantic feature vector sequence and the text real label sequence can be fused based on the label fusion layer to obtain a text enhanced feature vector sequence. Optionally, the number of training samples of the second part is higher than that of the training samples of the first part. For example, in the training round using 25% of the training sample text The text enhancement feature vector sequence is obtained by fusion, and the remaining training rounds use y t The fusion results in a sequence of text-enhanced feature vectors.

[0097] S107: Process the text enhancement feature vector sequence based on the multi-head selection layer to obtain an entity alias relationship probability matrix.

[0098] Considering that the same subject in the text may be associated with multiple objects, a multi-head selection mechanism is introduced to solve the problem of multi-relationship extraction.

[0099] Among them, each probability value in the entity alias relationship probability matrix represents the probability that there is an entity alias relationship between the entity mention segments to which every two words in the text belong respectively.

[0100] It can be referred to Figure 3 As shown, a schematic diagram of the entity alias relationship probability matrix in an example is presented. Taking "Xizi Lake generally refers to the West Lake" as an example, the rows and columns respectively correspond to the index ids of each word in "Xizi Lake generally refers to the West Lake". The probability at the intersection of the row and the column represents the probability that there is an entity alias relationship between the entity mention segments to which the corresponding two words belong. Due to the relationship of entity synonyms, [CLS] and [SEP] can be not considered here, so they are omitted. Among them, the row index id can be represented by i (0 to 7), and the column index id can be represented by j (0 to 7), where i represents the subject and j represents the object.

[0101] In some embodiments, the probability value in the entity alias relationship probability matrix can be calculated based on the alias relationship score, and the calculation method of the alias relationship score is as follows

[0102] s(x i ,x j )=w T tanh(W s h i +W o h j +b)

[0103] Among them Furthermore, a sigmoid activation function is used to convert the score into a probability value:

[0104] p i,j =σ(s(x i ,x j ))

[0105] Exemplarily, since the tokens at the entity boundary positions are determined in the sequence labeling layer, it can be set that the entity alias relationship only exists at the same boundary positions in the entity mention segments, such as between the head tokens. That is, if the tokens at positions i and j are respectively the heads of two entity mention segments, and there is an alias relationship between these two entity mention segments, then r i,j =r j,i =1, otherwise it is 0.

[0106] Just Figure 3For example, "West Lake" and "West Lake" have an entity alias relationship, "West Lake" is the subject, and "West Lake" is the object. Then the "West" in "West Lake" at row i = 0 and the "West" in "West Lake" at column j = 6 belong to the head position of "West Lake" and "West Lake" respectively. The probability values ​​of i = 0 and j = 6 should ideally be 1, that is, the entity alias relationship label r ij The label value r at this position 06 =1, otherwise 0. If the subject and object are not distinguished, then similarly, r 60 =r 06 =1.

[0107] S108: Calculate a second loss based on each probability value in the entity alias relationship probability matrix and the entity alias relationship label.

[0108] In some embodiments, the second loss, i.e., the long option loss L rel It can also be calculated using cross entropy. For example, as shown in the following formula:

[0109] S109: Obtain an overall loss based on the fusion of the first loss and the second loss, and update the entity alias relationship acquisition model according to the overall loss.

[0110] Exemplarily, the overall loss may be expressed as follows:

[0111] L=L ner +αL rel

[0112] Among them, α>0 is a factor that balances the two parts of loss. For example, 1 indicates that the two parts of loss are equally important.

[0113] Then, back propagation is performed based on the overall loss to update the entity alias relationship to obtain the model parameters. Preferably, an algorithm that accelerates the efficiency of gradient descent, such as the AdamW optimization algorithm, is used to iterate and update the model parameters until convergence.

[0114] To more intuitively explain the principle of the entity alias relationship acquisition model, please refer to the following Figure 4 Example.

[0115] like Figure 4 As shown, a schematic diagram showing the training principle of the entity alias relationship acquisition model in a specific implementation architecture in one embodiment of the present disclosure is shown.

[0116] like Figure 4As shown, {[CLS], "West", "Lake", "generally", "refers", "to", "the", "West", "Lake", [SEP]} is input into the feature encoding layer after being processed by the embedding layer of the model. The feature encoding layer can be implemented based on, for example, BERT, and outputs a sequence of text semantic feature vectors. The sequence labeling layer performs BIO labeling on the tokens of "West", "Lake", "generally", "refers", "to", "the", "West", "Lake", and the prediction results are: "West" (B: 0.8, I: 0.1, O: 0.1, most likely B), "Lake" (B: 0.1, I: 0.7, O: 0.2, most likely I), "Lake" (B: 0.1, I: 0.8, O: 0.1, most likely I), "one" (B: 0.1, I: 0.1, O: 0.8, most likely O), "generally" (B: 0.1, I: 0.1, O: 0.8, most likely O), "refers" (B: 0.2, I: 0.1, O: 0.7, most likely O), "West" (B: 0.9, I: 0.1, O: 0.0, most likely B), "Lake" (B: 0.2, I: 0.7, O: 0.1, most likely I). The text prediction label sequence can be obtained, and the text true label sequence is "BIIOOOBI" with a probability of 1.

[0117] Calculate the first part of the loss L through the cross-entropy function ner :

[0118]

[0119] In addition, the text prediction label sequence and / or the text true label sequence and the text semantic feature vector sequence are input into the label fusion layer for fusion. The fused text enhanced feature vector sequence is input into the multi-head selection layer, and the subject (such as the entity of the subject, h i ) and the object (such as the object of the object, h j ) calculate s(x i , x j ), and obtain p through sigmoid i,j to form an entity alias relationship probability matrix.

[0120] According to the fact that "West Lake" and "the West Lake" have an entity alias relationship, the probability value p at the cross position of the corresponding rows and columns of their head tokens i,j , and the corresponding label values r i,j , r j,i are 1, that is, r i,j = r j,i = 1, and calculate the second loss L using cross-entropy rel :

[0121] Combining the first loss and the second loss, the overall loss is obtained:

[0122] L=L ner +αL rel

[0123] Then update the model.

[0124] A possible implementation of the training method in a practical application example is provided, but the present invention is not limited thereto.

[0125] Exemplarily, the process can be shown as:

[0126] (1) Collect corpus from tourism-related consulting websites or encyclopedias, agree that the maximum sequence length that the model can process is 128, split the original text, manually select some valid samples for annotation, and construct training and validation data sets;

[0127] (2) For the input text, use the BERTWordPiece tokenizer to generate word-level tokenid sequence, positionid sequence, segmentid sequence, and mask sequence;

[0128] (3) Input the tokenid sequence, positionid sequence, and segmentid sequence into the model embedding layer to obtain a sequence embedding matrix, where the vector dimension is h = 768;

[0129] (4) The feature encoding layer is a 12-layer TransformerEncoder, in which the multi-head self-attention mechanism uses, for example, N = 12 heads to extract features, receives the embedding vector sequence of the previous step, and outputs a word sequence representation that incorporates the contextual semantics, that is, a text semantic feature vector sequence;

[0130] (5) The result of the previous step is passed to the sequence annotation layer to obtain the label distribution of the sequence token and calculate the cross entropy loss L between the predicted label distribution and the true label distribution. ner ;

[0131] (6) Randomly select 25% of the training samples and use the predicted label distribution in the previous step. The other samples use the real label distribution and are embedded into a label latent space of, for example, m = 128 dimensions. Then, the output results of the feature encoding layer are passed to the label fusion layer to obtain a sequence representation enhanced with entity labels, i.e., a text enhanced feature vector sequence.

[0132] (7) Input the sequence representation obtained in the previous step into the multi-head selection layer, where the hidden layer dimension is, for example, k = 100, and calculate the probability value of the entity alias relationship between each pair of tokens, i.e., p i,j ;

[0133] (8) Calculate the long selection loss L rel, add it to the sequence annotation loss L in step (5), with the weight factor α = 1, to obtain the overall loss; ner Add them together, with the weight factor α = 1, to obtain the overall loss;

[0134] (9) Use the AdamW optimization algorithm to minimize the overall loss, and continuously iterate and update the model parameters until convergence.

[0135] As Figure 5 shown, a flowchart of the method for obtaining entity alias relationships in an embodiment of the present disclosure is shown. Exemplarily, the method for obtaining entity alias relationships can be obtained by using an entity alias relationship acquisition model trained by the previous training method. Figure 5 Details of the model in the embodiment can refer to the embodiment of the previous training method.

[0136] In Figure 5 the example, the method for obtaining entity alias relationships includes:

[0137] Step S501: Obtain the input text and input it into the entity alias relationship acquisition model.

[0138] Exemplarily, "Xizi Lake generally refers to the West Lake" is input into the model.

[0139] Step S502: Process the input text based on the embedding layer to generate a character-level token sequence, a character-level position index sequence, and a text segment index sequence, and based on the fusion of the feature vectors respectively mapped by the character-level token sequence, the character-level position index sequence, and the text segment index sequence, obtain a text feature vector sequence.

[0140] Exemplarily, the text feature vector sequence is as shown in the previous e t shown.

[0141] Step S503: Process the text feature vector sequence based on the feature encoding layer to obtain a text semantic feature vector sequence.

[0142] Exemplarily, the text semantic feature vector sequence is as shown in the previous H (l) shown.

[0143] Step S504: Process the text semantic feature vector sequence based on the sequence annotation layer to obtain a text prediction label sequence, and obtain each entity mention segment in the text according to the text prediction label sequence.

[0144] Exemplarily, the text prediction label sequence is as shown, and the entity mention segments are "Xizi Lake" and "West Lake".

[0145] Step S505: Fusion the text semantic feature vector sequence and the text prediction label sequence based on the label fusion layer to obtain a text enhanced feature vector sequence.

[0146] Exemplarily, the text enhancement feature vector sequence is like H = [h1, h2, …, h T .

[0147] Step S506: Process the text enhancement feature vector sequence based on a multi-head selection layer to obtain an entity alias relationship probability matrix; wherein, each probability value in the entity alias relationship probability matrix represents the probability that there is an entity alias relationship between the entity mention segments to which each two words in the input text belong respectively.

[0148] Exemplarily, the probability values in the entity alias relationship probability matrix are like p i,j .

[0149] Step S507: Screen out the probability values that reach a preset threshold from the entity alias relationship probability matrix, and obtain the entity alias relationship acquisition result between two entity mention segments according to the positions of the words corresponding to the rows and columns where the screened probability values are located in their respective entity mention segments.

[0150] Exemplarily, the preset threshold can be, for example, 0.5. Take out the probability values that reach 0.5 or more. For example, the probability value corresponding to "西" in "西子湖" and "西" in "西湖" is 0.6, then it can be determined that there is an entity alias relationship between the entity mention segments "西子湖" and "西湖", that is, they belong to entity synonyms.

[0151] In some embodiments, the fusion of the sequence vectors respectively mapped by the word-level token sequence, the word-level position index sequence, and the text segment index sequence to obtain the text feature vector sequence includes: encoding the word-level token sequence, the word-level position index sequence, and the text segment index sequence respectively to obtain respective encoded sequences; converting each of the encoded sequences into each of the sequence vectors; summing and normalizing each of the sequence vectors to obtain the text feature vector sequence.

[0152] In some embodiments, the feature encoding layer includes a multi-head self-attention layer and a fully-connected layer; the processing of the text feature vector sequence based on the feature encoding layer to obtain the text semantic feature vector sequence includes: processing the text feature vector sequence through the multi-head self-attention layer to obtain an intermediate feature vector sequence; processing the intermediate feature vector sequence through the fully-connected layer to obtain the text semantic feature vector sequence.

[0153] In some embodiments, the sequence annotation layer processes the text semantic feature vector sequence to obtain a text prediction label sequence, and obtains each entity mention segment in the text according to the text prediction label sequence, including: predicting the position of the word corresponding to each feature value in the text semantic feature vector sequence in the corresponding entity mention segment to obtain a prediction label to form the text prediction label sequence; and obtaining each entity mention segment based on the entity mention boundaries marked by the text prediction label sequence.

[0154] In some embodiments, the label fusion layer includes a gated neural network layer; the label fusion layer-based fusion of the text semantic feature vector sequence and the text prediction label sequence to obtain a text enhancement feature vector sequence includes: fusing the text semantic feature vector sequence and the text prediction label sequence according to word-level position through the gated neural network layer to obtain a text enhancement feature vector sequence.

[0155] In some embodiments, the entity alias relationship acquisition result between the two entity mention segments is obtained based on the positions of the characters corresponding to the rows and columns where the filtered probability values ​​are located in their respective entity mention segments, including: in response to the characters corresponding to the rows and columns where the probability values ​​are located in the same boundary positions in their respective entity mention segments, determining that an entity alias relationship exists between the two entity mention segments.

[0156] In some embodiments, the entity alias relationship acquisition method further includes: generating a mask sequence corresponding to the input text; and inputting the text feature vector sequence after being acted upon by the mask sequence into the entity alias relationship acquisition model.

[0157] The following is a specific application example to illustrate the process of the entity relationship acquisition method. The specific steps are as follows:

[0158] (1) Normalize the input text, convert full text to half text, convert traditional text to simplified text, and truncate overly long sentences;

[0159] (2) The text is digitized to obtain word-level tokens, position and segment ID sequences and mask sequences, which are input into the entity alias relationship acquisition model for forward propagation calculation;

[0160] (3) Token label distribution predicted by the decoding sequence annotation layer Get each entity mention snippet;

[0161] (4) In the inference phase, the label embedding matrix is ​​directly used to distribute the token labels predicted in the previous step. Projection is performed, and then the result of the feature encoding layer is input into the label fusion layer. Finally, after calculation in the multi-head selection layer, the entity alias relationship probability matrix is ​​obtained.

[0162] like Figure 6 As shown, a schematic diagram of a module of an entity alias relationship acquisition device in an embodiment of the present disclosure is shown. Since the specific implementation of the entity alias relationship acquisition device can refer to the previous entity alias relationship acquisition method embodiment, the technical details will not be repeated here.

[0163] exist Figure 6 In the figure, the entity alias relationship acquisition device 600 is applied to the entity alias relationship acquisition model 601, and the entity alias relationship acquisition model 601 includes: an embedding layer 611, a feature encoding layer 612, a sequence annotation layer 613, a label fusion layer 614 and a multi-head selection layer 615.

[0164] The entity alias relationship acquisition device 600 includes:

[0165] An input module 602 is used to obtain input text and input it into the entity alias relationship acquisition model;

[0166] The embedding layer 611 is used to process the input text to generate a character-level token sequence, a character-level position index sequence and a text segment index sequence, and obtain a text feature vector sequence based on the fusion of feature vectors mapped by the character-level token sequence, the character-level position index sequence and the text segment index sequence respectively;

[0167] The feature encoding layer 612 is used to process the text feature vector sequence to obtain a text semantic feature vector sequence;

[0168] The sequence annotation layer 613 is used to process the text semantic feature vector sequence to obtain a text prediction label sequence, and obtain each entity mention segment in the text according to the text prediction label sequence;

[0169] The label fusion layer 614 is used to fuse the text semantic feature vector sequence and the text prediction label sequence to obtain a text enhancement feature vector sequence;

[0170] The multi-head selection layer 615 is used to process the text enhancement feature vector sequence to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between entity mention segments to which each two characters in the input text belong respectively; and, by filtering probability values ​​reaching a preset threshold from the entity alias relationship probability matrix, and obtaining an entity alias relationship acquisition result between two entity mention segments based on the positions of the characters corresponding to the rows and columns of the filtered probability values ​​in the entity mention segments to which they belong respectively.

[0171] like Figure 7As shown, a module schematic diagram of a training device for acquiring an entity alias relationship model in an embodiment of the present disclosure is shown. Since the specific implementation of the training device 700 for acquiring an entity alias relationship model can refer to the previous embodiment of the training method for acquiring an entity alias relationship model, the technical details will not be repeated here.

[0172] The entity alias relationship acquisition model 701 includes: an embedding layer 711 , a feature encoding layer 712 , a sequence annotation layer 713 , a label fusion layer 714 and a multi-head selection layer 716 .

[0173] The training device 700 comprises:

[0174] An input module 702 is used to obtain a training sample set and input the entity alias relationship acquisition model, wherein each training sample text in the training sample set has a corresponding text true label sequence and an entity alias relationship label;

[0175] The embedding layer 711 is used to generate a character-level token sequence, a character-level position index sequence and a text segment index sequence based on each training sample text, and obtain a text feature vector sequence based on the fusion of sequence vectors mapped by the character-level token sequence, the character-level position index sequence and the text segment index sequence respectively;

[0176] The feature encoding layer 712 is used to process the text feature vector sequence to obtain a text semantic feature vector sequence;

[0177] The sequence annotation layer 713 is used to process the text semantic feature vector sequence to obtain a text prediction label sequence, and obtain each entity mention segment in the text according to the text prediction label sequence;

[0178] A loss calculation module 703, used to calculate a first loss between the text prediction label sequence and the text true label sequence;

[0179] The label fusion layer 714 is used to fuse the text semantic feature vector sequence with the text prediction label sequence or the text real label sequence to obtain a text enhanced feature vector sequence;

[0180] The multi-head selection layer 715 is used to process the text enhancement feature vector sequence to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between entity mention segments to which each two words in the text belong respectively;

[0181] The loss calculation module 703 is used to calculate the second loss based on each probability value in the entity alias relationship probability matrix and the entity alias relationship label; wherein the entity alias relationship label is determined based on whether there is an entity alias relationship between two words corresponding to the row and column corresponding to each probability value;

[0182] The loss calculation module 703 is used to obtain the overall loss based on the fusion of the first loss and the second loss, and update the entity alias relationship acquisition model according to the overall loss.

[0183] It should be noted that in Figure 6 , Figure 7 Each functional module in the embodiment can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a program instruction product. The program instruction product includes one or more program instructions. When the program instruction is loaded and executed on a computer, the process or function according to the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The program instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0184] And, in Figure 6 , Figure 7 The device disclosed in the embodiment can be implemented by other module division methods. The device embodiments shown above are only schematic. For example, the module division is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or modules can be combined or dynamically transferred to another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical or other forms.

[0185] In addition, Figure 6 , Figure 7 Each functional module and submodule in the embodiment can be dynamically in a processing component, or each module can exist physically separately, or two or more modules can be dynamically in one component. The above-mentioned dynamic components can be implemented in the form of hardware or in the form of software functional modules. If the above-mentioned dynamic components are implemented in the form of software functional modules and executed as independent products for sale or use, they can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0186] It should be particularly noted that the process or method represented by the flowchart of the above embodiment of the present disclosure can be understood as a module, fragment or part of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0187] For example, Figure 1 , Figure 5 The order of the steps in the embodiments may be changed in specific scenarios and is not limited to the above.

[0188] like Figure 8 , which shows a schematic diagram of the structure of a computer device in one embodiment of the present disclosure.

[0189] In some embodiments, the computer device is used to load and implement the aforementioned method embodiments (for example Figure 1 , Figure 5 The computer device may be implemented as a server, such as a server in a server cluster in the above-mentioned embodiment, which may be used as a carrier for implementing one or more nodes.

[0190] like Figure 8 As shown, the computer device 800 is in the form of a general computing device. The components of the computer device 800 may include but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0191] The storage unit stores program codes, and the program codes can be executed by the processing unit 810, so that the computer device is used to implement the method steps described in the above embodiments of the present disclosure.

[0192] In some embodiments, the storage unit 820 may include a volatile storage unit, such as a random access storage unit (RAM) 8201 and / or a cache storage unit 8202 , and may further include a read-only storage unit (ROM) 8203 .

[0193] In some embodiments, the storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment.

[0194] In some embodiments, bus 830 may include a data bus, an address bus, and a control bus.

[0195] In some embodiments, the computer device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and such communication may be performed through an input / output (I / O) interface 850. Optionally, the computer device 800 also includes a display unit 840, which is connected to the input / output (I / O) interface 850 for display. In addition, the computer device 800 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 860. As shown, the network adapter 860 communicates with other modules of the computer device 800 through a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0196] In the embodiments of the present disclosure, a computer-readable storage medium may also be provided, which may include program code and may be run on a device, such as a personal computer, to implement the above method embodiments of the present disclosure (such as Figure 1 , Figure 5 In this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0197] The program code may be in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0198] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0199] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0200] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0201] In summary, the embodiments of the present disclosure provide an entity alias relationship acquisition method, a training method, an apparatus, and a storage medium, which are applied to an entity alias relationship model; a text feature vector sequence is obtained by inputting text based on the embedding layer of the model; a text semantic feature vector sequence is obtained based on the feature encoding layer; a text prediction label sequence is obtained based on the sequence annotation layer, and each entity mention segment in the text is obtained; the text semantic feature vector sequence and the text prediction label sequence are fused based on the label fusion layer to obtain a text enhanced feature vector sequence; the text enhanced feature vector sequence is processed based on the multi-head selection layer to obtain an entity alias relationship probability matrix; the entity alias relationship acquisition result between the two entity mention segments is obtained according to the positions of the characters corresponding to the rows and columns of the screened probability values ​​in the entity mention segments to which they belong. The present disclosure implements a solution for accurately and efficiently mining entity alias relationships, which is conducive to the construction of high-quality knowledge graphs.

[0202] The solution proposed in the disclosed embodiment realizes the automatic discovery of a large number of alias relationships from unstructured corpus, and promotes the development of knowledge graph construction and fusion technology in the tourism industry. At the same time, due to the improved fusion of word-level label information (label fusion layer), the limitation of label imbalance in multiple relationship extraction on model learning ability is effectively alleviated, and the accuracy is improved by about 2%.

[0203] Finally, after testing by the applicant, on the test data, the entity recognition f1 value reached 92%, and the relationship extraction f1 value reached 74%. Compared with the traditional text literal matching algorithm and simple CNN, RNN and other network modules, the embodiment of the present disclosure reduces a lot of feature engineering while obtaining higher performance.

[0204] The above embodiments are merely illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present disclosure shall still be covered by the claims of the present disclosure.

Claims

1. A method for obtaining entity alias relationship, It is characterized in that Applied to an entity alias relationship acquisition model, the entity alias relationship acquisition model includes: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the method includes: Obtain input text and input the entity alias relationship acquisition model; Processing the input text based on the embedding layer to generate a character-level token sequence, a character-level position index sequence, and a text segment index sequence, and fusing the feature vectors mapped by the character-level token sequence, the character-level position index sequence, and the text segment index sequence based on the embedding layer to obtain a text feature vector sequence; Processing the text feature vector sequence based on the feature coding layer to obtain a text semantic feature vector sequence; Processing the text semantic feature vector sequence based on the sequence annotation layer to obtain a text prediction label sequence, and obtaining each entity mention segment in the text according to the text prediction label sequence; Based on the label fusion layer, the text semantic feature vector sequence and the text prediction label sequence are fused to obtain a text enhanced feature vector sequence; Processing the text enhancement feature vector sequence based on the multi-head selection layer to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between entity mention segments to which each two characters in the input text belong respectively; The probability values ​​reaching a preset threshold are filtered out from the entity alias relationship probability matrix, and the entity alias relationship acquisition result between the two entity mention segments is obtained according to the positions of the characters corresponding to the rows and columns where the filtered probability values ​​are located in the entity mention segments to which they belong.

2. The entity alias relationship acquisition method according to claim 1, It is characterized in that The fusion of the sequence vectors mapped respectively based on the word-level token sequence, the word-level position index sequence and the text segment index sequence to obtain a text feature vector sequence includes: The character-level token sequence, the character-level position index sequence and the text segment index sequence are respectively encoded to obtain respective encoding sequences; Convert each of the coded sequences into each of the sequence vectors; The sequence vectors are summed and normalized to obtain the text feature vector sequence.

3. The entity alias relationship acquisition method according to claim 1, It is characterized in that The feature coding layer includes a multi-head self-attention layer and a fully connected layer; the feature coding layer is based on processing the text feature vector sequence to obtain a text semantic feature vector sequence, including: Processing the text feature vector sequence through a multi-head self-attention layer to obtain an intermediate feature vector sequence; The intermediate feature vector sequence is processed through a fully connected layer to obtain the text semantic feature vector sequence.

4. The entity alias relationship acquisition method according to claim 1, It is characterized in that The step of processing the text semantic feature vector sequence based on the sequence annotation layer to obtain a text prediction label sequence, and obtaining each entity mention segment in the text according to the text prediction label sequence, includes: Predicting the position of the word corresponding to each feature value in the text semantic feature vector sequence in the entity mention segment to which it belongs, to obtain a predicted label, so as to form the text prediction label sequence; Based on the entity mention boundaries marked by the text prediction label sequence, each entity mention segment is obtained.

5. The entity alias relationship acquisition method according to claim 1, It is characterized in that The label fusion layer includes a gated neural network layer; the label fusion layer fuses the text semantic feature vector sequence and the text prediction label sequence to obtain a text enhancement feature vector sequence, including: The text semantic feature vector sequence and the text prediction label sequence are fused according to word-level positions through a gated neural network layer to obtain a text enhanced feature vector sequence.

6. The entity alias relationship acquisition method according to claim 1, It is characterized in that The step of obtaining the entity alias relationship between two entity mention segments based on the positions of the words corresponding to the rows and columns of the selected probability values ​​in the entity mention segments to which they belong respectively comprises: In response to the characters corresponding to the row and column where the probability values ​​are located respectively being located at the same boundary position in the entity mention segments to which they belong, it is determined that an entity alias relationship exists between the two entity mention segments.

7. The entity alias relationship acquisition method according to claim 1, It is characterized in that Also includes: Generating a mask sequence corresponding to the input text; The text feature vector sequence after being acted upon by the mask sequence is input into the entity alias relationship acquisition model.

8. A training method for an entity alias relationship acquisition model, It is characterized in that The entity alias relationship acquisition model includes: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the training method includes: Obtaining a training sample set and inputting it into the entity alias relationship acquisition model, wherein each training sample text in the training sample set has a corresponding text true label sequence and an entity alias relationship label; Generate a word-level token sequence, a word-level position index sequence, and a text segment index sequence based on each training sample text, and obtain a text feature vector sequence based on the fusion of sequence vectors mapped by the word-level token sequence, the word-level position index sequence, and the text segment index sequence; Processing the text feature vector sequence based on the feature coding layer to obtain a text semantic feature vector sequence; Processing the text semantic feature vector sequence based on the sequence annotation layer to obtain a text prediction label sequence, and obtaining each entity mention segment in the text according to the text prediction label sequence; Calculating a first loss between the text prediction label sequence and the text true label sequence; Based on the label fusion layer, the text semantic feature vector sequence is fused with the text prediction label sequence or the text real label sequence to obtain a text enhanced feature vector sequence; Processing the text enhancement feature vector sequence based on the multi-head selection layer to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between entity mention segments to which each two characters in the text belong respectively; Calculating a second loss based on each probability value in the entity alias relationship probability matrix and the entity alias relationship label; wherein the entity alias relationship label is determined based on whether there is an entity alias relationship between two characters corresponding to the row and column corresponding to each probability value; An overall loss is obtained based on the fusion of the first loss and the second loss, and the entity alias relationship acquisition model is updated according to the overall loss.

9. The training method according to claim 8, It is characterized in that In a training round using the first part of the training sample text, the text semantic feature vector sequence and the text prediction label sequence are fused based on the label fusion layer to obtain a text enhanced feature vector sequence; In the remaining training rounds of the second part of the training sample text, the text semantic feature vector sequence and the text real label sequence are fused based on the label fusion layer to obtain a text enhanced feature vector sequence.

10. The training method according to claim 9, It is characterized in that The sample quantity of the training samples in the second part is greater than that of the training samples in the first part.

11. The training method according to claim 8, It is characterized in that The fusion of the sequence vectors mapped respectively based on the word-level token sequence, the word-level position index sequence and the text segment index sequence to obtain a text feature vector sequence includes: The character-level token sequence, the character-level position index sequence and the text segment index sequence are respectively encoded to obtain respective encoding sequences; Convert each of the coded sequences into each of the sequence vectors; The sequence vectors are summed and normalized to obtain the text feature vector sequence.

12. The training method according to claim 8, It is characterized in that The feature coding layer includes a multi-head self-attention layer and a fully connected layer; the feature coding layer is based on processing the text feature vector sequence to obtain a text semantic feature vector sequence, including: Processing the text feature vector sequence through a multi-head self-attention layer to obtain an intermediate feature vector sequence; The intermediate feature vector sequence is processed through a fully connected layer to obtain the text semantic feature vector sequence.

13. The training method according to claim 8, It is characterized in that The step of processing the text semantic feature vector sequence based on the sequence annotation layer to obtain a text prediction label sequence, and obtaining each entity mention segment in the text according to the text prediction label sequence, includes: Predicting the position of the word corresponding to each feature value in the text semantic feature vector sequence in the entity mention segment to which it belongs, to obtain a predicted label, so as to form the text prediction label sequence; Based on the entity mention boundaries marked by the text prediction label sequence, each entity mention segment is obtained.

14. The training method according to claim 8, It is characterized in that The label fusion layer includes a gated neural network layer; the label fusion layer fuses the text semantic feature vector sequence with the text prediction label sequence or the text real label sequence to obtain a text enhanced feature vector sequence, including: The text prediction label sequence or the text true label sequence is fused according to the word-level position through a gated neural network layer to obtain a text enhanced feature vector sequence.

15. The training method according to claim 8, It is characterized in that The position correspondence means that the characters corresponding to the row and column where the probability value is located are located at the same boundary position in the entity mention segment to which they belong.

16. The training method according to claim 8, It is characterized in that Also includes: generating a mask sequence corresponding to the text; The text feature vector sequence after being acted upon by the mask sequence is input into the entity alias relationship acquisition model.

17. An entity alias relationship acquisition device, It is characterized in that Applied to an entity alias relationship acquisition model, the entity alias relationship acquisition model includes: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the device includes: An input module, used to obtain input text and input it into the entity alias relationship acquisition model; The embedding layer is used to process the input text to generate a word-level token sequence, a word-level position index sequence and a text segment index sequence, and obtain a text feature vector sequence based on the fusion of feature vectors respectively mapped by the word-level token sequence, the word-level position index sequence and the text segment index sequence; The feature encoding layer is used to process the text feature vector sequence to obtain a text semantic feature vector sequence; The sequence annotation layer is used to process the text semantic feature vector sequence to obtain a text prediction label sequence, and obtain each entity mention segment in the text according to the text prediction label sequence; The label fusion layer is used to fuse the text semantic feature vector sequence and the text prediction label sequence to obtain a text enhancement feature vector sequence; The multi-head selection layer is used to process the text enhancement feature vector sequence to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between entity mention segments to which each two characters in the input text belong respectively; and, by filtering probability values ​​reaching a preset threshold from the entity alias relationship probability matrix, and obtaining an entity alias relationship acquisition result between two entity mention segments based on the positions of the characters corresponding to the rows and columns of the filtered probability values ​​in the entity mention segments to which they belong respectively.

18. A training device for an entity alias relationship acquisition model, It is characterized in that The entity alias relationship acquisition model includes: an embedding layer, a feature encoding layer, a sequence annotation layer, a label fusion layer and a multi-head selection layer; the training device includes: An input module, used to obtain a training sample set and input it into the entity alias relationship acquisition model, wherein each training sample text in the training sample set has a corresponding text true label sequence and an entity alias relationship label; The embedding layer is used to generate a word-level token sequence, a word-level position index sequence and a text segment index sequence based on each training sample text, and obtain a text feature vector sequence based on the fusion of sequence vectors mapped by the word-level token sequence, the word-level position index sequence and the text segment index sequence respectively; The feature encoding layer is used to process the text feature vector sequence to obtain a text semantic feature vector sequence; The sequence annotation layer is used to process the text semantic feature vector sequence to obtain a text prediction label sequence, and obtain each entity mention segment in the text according to the text prediction label sequence; A loss calculation module, used to calculate a first loss between the text prediction label sequence and the text true label sequence; The label fusion layer is used to fuse the text semantic feature vector sequence with the text prediction label sequence or the text real label sequence to obtain a text enhanced feature vector sequence; The multi-head selection layer is used to process the text enhancement feature vector sequence to obtain an entity alias relationship probability matrix; wherein each probability value in the entity alias relationship probability matrix represents the probability that an entity alias relationship exists between entity mention segments to which each two words in the text belong respectively; The loss calculation module is used to calculate the second loss based on each probability value in the entity alias relationship probability matrix and the entity alias relationship label; wherein the entity alias relationship label is determined based on whether there is an entity alias relationship between two words corresponding to the row and column corresponding to each probability value; The loss calculation module is used to obtain the overall loss based on the fusion of the first loss and the second loss, and update the entity alias relationship acquisition model according to the overall loss.

19. A computer device, It is characterized in that include: Communicator, memory and processor; The communicator is used for communicating with the outside; The memory stores program instructions; The processor is used to run the program instructions to execute the entity alias relationship acquisition method as described in any one of claims 1 to 7; or, to execute the training method as described in any one of claims 8 to 16.

20. A computer-readable storage medium, It is characterized in that Program instructions are stored, and the program instructions are executed to execute the entity alias relationship acquisition method as described in any one of claims 1 to 7; or, to execute the training method as described in any one of claims 8 to 16.

Images