Semantic decomposition relation triple combined extraction method based on double-level contrast learning

Abstract

The invention is suitable for the technical field of natural language processing, and particularly relates to a semantic decomposition relation triple combined extraction method based on double-level contrast learning, and the method comprises the steps: marking a given sentence based on a preset marking strategy; the marked sentences are input into a preset encoder, format standardization processing is carried out, and an encoded text is obtained; performing entity-level comparative learning and relation-level comparative learning on the coded text in sequence, and extracting semantic information; and performing scoring processing on the semantic information through a score-based classifier, inputting a score vector into a preset function, and performing prediction to obtain a relation triple. According to the method, the relation triad is rapidly and accurately extracted.

Description

Technical Field

[0001] The present application belongs to the field of natural language processing technology, and specifically relates to a method for jointly extracting semantic decomposition relationship triplets based on two-level contrastive learning. Background Art

[0002] Integrating human knowledge is a key area of ​​research in artificial intelligence. Inspired by human problem-solving, intelligent systems can solve complex tasks by representing and reasoning about knowledge. Therefore, the importance of knowledge representation to artificial intelligence is self-evident. Among the many knowledge representation methods, knowledge graphs, with their powerful expressive power and modeling flexibility, have attracted significant attention from both academia and industry.

[0003] Currently, driven by the rapid development of big data and deep learning, knowledge graphs have become one of the core driving forces behind the development of the internet and artificial intelligence. A key component of knowledge graphs is relational facts, which are essentially two entities connected by a semantic relationship. Facts are formally represented as a head entity, a tail entity, and a relationship, known as relation triples. Therefore, extracting relation triples quickly and accurately is not only a key step in building large-scale knowledge graphs, but also a crucial task in information extraction.

[0004] However, in practice, it is found that the relation triple extraction in the existing technology has the following technical problems:

[0005] 1. Error accumulation: In early research on relation triple extraction, researchers typically employed pipeline models. This model divides the information extraction process into two independent steps: named entity recognition and relation classification, with each step processing and passing data sequentially. However, due to the error accumulation effect, errors in earlier steps can affect the results of subsequent steps, ultimately leading to a decrease in the quality of the overall output.

[0006] 2. Entity overlap: Based on the overlap of triples, sentences can be divided into three types: no entity overlap, single entity overlap, and entity pair overlap. The presence of multiple overlapping triples in the same sentence increases the difficulty of the model in extracting relation triples.

[0007] 3. Insufficient interactivity between entities and relationships: There is a strong correlation between entities and relationships (e.g., some relationships only exist between specific entity types), but existing models cannot fully utilize this bidirectional dependency. Summary of the Invention

[0008] In view of this, an embodiment of the present application provides a method for jointly extracting semantic decomposition relationship triplets based on two-level contrastive learning to solve the above-mentioned technical problems in the prior art.

[0009] A first aspect of an embodiment of the present application provides a method for jointly extracting semantic decomposition relationship triplets based on two-level contrastive learning, comprising:

[0010] Tag the given sentence based on the preset tagging strategy;

[0011] Input the marked sentences into the preset encoder for normalization and formatting to obtain the encoded text;

[0012] The encoded text undergoes entity-level contrastive learning and relation-level contrastive learning in turn to extract semantic information;

[0013] The semantic information is scored by a scoring-based classifier, and the score vector is input into a preset function to predict the relationship triples.

[0014] In some embodiments, the given sentence is marked based on a preset marking strategy, specifically:

[0015] Based on the Re1-Spec corner tagging strategy, the head entity and tail entity of a given sentence are combined with the entity's start position and entity's end position to obtain four types of tag texts.

[0016] In some embodiments, the Rel-Spec corner tagging strategy combines the head entity and tail entity of a given sentence with the entity's start position and entity's end position to obtain four types of tag text, specifically:

[0017] Based on the table filling model, the Rel-Spec corner tagging strategy is used as a tagging scheme. This tagging strategy combines the head and tail entity tags with the BIOES tagging strategy. 'H' and 'T' are used to represent the head and tail entities, B represents the start position of the entity, and E represents the end position of the entity. The following four types of tags are obtained by combining them:

[0018] (1) HB-TB: indicates that the position is the starting position of the head entity and the tail entity of the relation triple;

[0019] (2) HB-TE: indicates that the position is the starting position of the head entity and the ending position of the tail entity of the relation triple;

[0020] (3) HE-TB: indicates the end position of the head entity and the start position of the tail entity of the relation triple;

[0021] (4) HE-TE: Indicates that the position is the end position of the head entity and the tail entity of the relation triple.

[0022] In some embodiments, the marked sentence is input into a preset encoder for normalization to obtain an encoded text, specifically:

[0023] The labeled sentences are pre-padded to keep all sentences of uniform length, and then the pre-trained BERT is used as the sentence encoder to obtain the d-dimensional embedding of each entity to obtain the encoded text.

[0024] In some embodiments, the encoded text is sequentially subjected to entity-level contrastive learning and relation-level contrastive learning to extract semantic information, specifically:

[0025] Based on the preset entity-level contrastive learning module, contrastive learning is performed on the head entity and the tail entity in the encoded text respectively, enhancing the distinction between the head entity and the tail entity in the embedding space, so as to independently learn the semantic information of each entity;

[0026] On the basis of entity-level contrastive learning, based on the preset relationship-level contrastive learning module, the relationship triples are mapped to the high-dimensional embedding space. By defining the similarity measure in the high-dimensional embedding space, the distinction between entities and relationships is enhanced, and the semantic structure of the relationship triples is captured.

[0027] In some embodiments, the preset entity-level contrastive learning module performs contrastive learning on the head entity and the tail entity in the encoded text, respectively, to enhance the distinction between the head entity and the tail entity in the embedding space, so as to independently learn the semantic information of each entity, including the following:

[0028] Get the d-dimensional embeddings of all entities contained in the original sequence, where

[0029] For the head entity, the entity extracted from the original input sentence is used as the anchor sample, the head entity in the relation triple extracted from the label is used as the positive sample, and the entity different from the target relation triple is selected as the negative sample to obtain the contrastive learning loss function of the head entity;

[0030] For the tail entity, the entity extracted from the original input sentence is used as the anchor sample, the tail entity in the relation triple extracted from the label is used as the positive sample, and the tail entity different from the target relation triple is selected as the negative sample to obtain the contrastive learning loss function of the tail entity;

[0031] The contrastive learning loss function of the head entity and the contrastive learning loss function of the tail entity are weighted and summed to obtain the total loss function, and the semantic information of each entity is obtained according to the total loss function.

[0032] In some embodiments, based on the entity-level contrastive learning, the relationship triples are mapped to a high-dimensional embedding space based on a preset relationship-level contrastive learning module. By defining a similarity measure in the high-dimensional embedding space, the distinction between entities and relationships is enhanced, and the semantic structure of the relationship triples is captured, including the following:

[0033] The relation triples are mapped to a high-dimensional embedding space, and a similarity measure is defined in the high-dimensional embedding space. The positive and negative sample pairs are subjected to comparative learning optimization so that the distance between the positive sample pairs in the embedding space is as close as possible and the distance between the negative sample pairs in the embedding space is as far as possible. This enhances the distinction between entities and relations and captures the semantic structure of the relation triples.

[0034] In some embodiments, during the training phase of the model, for each input sequence, all relationship triplets in the training data are first sampled as positive samples, and negative samples are generated by randomly replacing the relationship between the head and tail entities, all entity pairs to be matched in the input sequence are obtained, and each entity pair is assigned a pending relationship to constitute an anchor sample, and each relationship triplet is spliced ​​and further mapped to a high-dimensional space through a neural network.

[0035] In some embodiments, the semantic information is scored by a scoring-based classifier, and the score vector is input into a preset function to predict the relationship triples, wherein the preset function is a softmax function.

[0036] A second aspect of an embodiment of the present application provides a device for jointly extracting semantic decomposition relationship triplets based on two-level contrastive learning, comprising:

[0037] The tagging module is used to tag a given sentence based on a preset tagging strategy;

[0038] The encoding module is used to input the marked sentences into a preset encoder for normalization and formatting to obtain encoded text;

[0039] A two-level contrastive learning module is used to encode text, which undergoes entity-level contrastive learning and relation-level contrastive learning in turn to extract semantic information;

[0040] The extraction module is used to score the semantic information through a scoring-based classifier, and input the score vector into a preset function to predict the relationship triples.

[0041] A third aspect of an embodiment of the present application provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0042] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0043] A fifth aspect of the present application provides a computer program product, which, when executed on a terminal, enables the terminal to execute the steps of the method described in the first aspect.

[0044] As can be seen from the above, the embodiment of the present application introduces the idea of ​​contrastive learning and sets entity-level contrastive learning, so that the model can make full use of the entity information in the semantic environment, thereby improving the model's extraction effect on overlapping entities; and, on the basis of entity-level contrastive learning, further creates relationship-level contrastive learning, which can improve the model's utilization of the strong correlation between entities and relationships by integrating the connection between entities and relationships in relationship triples, that is: by introducing two-level contrastive learning, the model's ability to understand complex semantic information is improved, and the model's extraction effect in complex semantic scenes is greatly improved; at the same time, a scoring-based classifier is used as the decoder of the model, which effectively reduces the computational complexity of the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 1 is a flowchart of a method for joint extraction of semantic decomposition relationship triplets based on two-level contrastive learning provided in an embodiment of the present application;

[0047] Figure 2 Schematic diagram of a device for joint extraction of semantic decomposition relationship triples based on dual-level contrastive learning provided in an embodiment of the present application;

[0048] Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0050] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0051] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0052] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0053] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0054] It should be understood that the size of the serial numbers of each step in this embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of this application.

[0055] In view of the relevant technical problems in the prior art raised in the background technology of this application specification, as well as other technical problems, such as: due to the separation of entity recognition and relationship classification in the pipeline method, error propagation and entity-relationship interactivity are missing, and the model has a low recall rate for overlapping triples; because the joint model relies on complex decoding strategies, such as sequence generation or graph traversal, the model has high computational complexity and low decoding efficiency, which makes it difficult to meet real-time requirements; because the prior art lacks implicit relational reasoning, it is unable to capture logical chains that rely on background knowledge or cross-sentences, and the extraction accuracy in complex semantic scenes is insufficient, etc., the embodiment of the present application proposes a semantic decomposition relationship triple joint extraction method based on two-level contrastive learning to solve at least one of the many technical problems in the prior art.

[0056] For ease of understanding, the following briefly describes the relationship triple extraction solution described in this application:

[0057] Relation triple extraction can be understood as the joint probability of named entity recognition and relation extraction: P((h, r, t)|S), where S represents the text, h and t represent the head entity and tail entity respectively, and r represents the relationship between the two.

[0058] Using a pipeline model, the probability of named entity recognition is expressed as P((h,t)|S). Based on the recognized entities, the relationship between them is further predicted with the probability P(r|h,t,S). By combining these conditional probabilities, the complete triple extraction problem can be expressed as follows:

[0059] P((h,r,t)|S)=P((h,t)|S)·P(r|h,t,S)

[0060] The embodiment of the present application mainly focuses on joint extraction, models the joint probability, and directly outputs the probability distribution of triples through the network structure, namely:

[0061] P((h,r,t)|S)=softmax(f(h,r,t,S))

[0062] In the above formula, f(h,r,t,S) is a scoring function calculated by a neural network, which is used to measure whether the triple (h,r,t) is semantically consistent with the text S.

[0063] Joint extraction methods can be roughly divided into two categories: joint decoding and parameter sharing. The joint decoding method uniformly marks the entities and relations in the input text by using table filling, sequence labeling or sequence-to-sequence methods, and then uses a unified decoding mechanism to generate the relationship triplets contained in the text. Unlike the joint decoding method, the parameter sharing method mostly adopts a multi-task learning method to share part or all of the parameters within the model, thereby reducing the amount of parameters that need to be learned independently for a single task.

[0064] TPLinker is a relation triple extraction model based on joint decoding. It solves the problems of entity overlap and relationship overlap by introducing the handshake annotation method and matrix decomposition strategy. The model regards the head entity, tail entity and relationship as a whole, constructs a head and tail entity matching matrix for each relationship type, and directly identifies triple boundaries and relationships through joint decoding of three sub-matrices (head entity start / end, tail entity start / end, relationship type), avoiding the error propagation problem of traditional pipeline models. Its core advantage is that it does not rely on a predefined entity order, can efficiently process multiple overlapping triplets in the same sentence, and improves decoding efficiency through lightweight design, showing high recall and robustness in complex language environments.

[0065] The following describes one embodiment of a method for jointly extracting semantic decomposition relationship triplets based on two-level contrastive learning described in this application:

[0066] See also Figure 1 , Figure 1 This is a flow chart of a method for joint extraction of semantic decomposition relationship triples based on dual-level contrastive learning provided by an embodiment of the present application. Figure 1 As shown, a semantic decomposition relation triple joint extraction method based on two-level contrastive learning includes the following steps:

[0067] Step 1: tag the given sentence based on the preset tagging strategy;

[0068] Step 2: Input the marked sentence into a preset encoder for normalization to obtain the encoded text;

[0069] Step 3: The encoded text undergoes entity-level contrastive learning and relation-level contrastive learning in turn to extract semantic information;

[0070] Step 4: The semantic information is scored by a scoring-based classifier, and the score vector is input into a preset function to predict the relationship triples.

[0071] Obviously, by introducing the idea of ​​contrastive learning and setting up entity-level contrastive learning, the model can fully utilize the entity information in the semantic environment and improve the model's extraction effect on overlapping entities;

[0072] Furthermore, building on entity-level contrastive learning, we further developed relation-level contrastive learning. By integrating the connections between entities and relations in relational triples, we can improve the model's utilization of the strong associations between entities and relations. Specifically, by introducing two-level contrastive learning, we enhance the model's ability to understand complex semantic information, significantly improving the model's extraction performance in complex semantic scenarios.

[0073] At the same time, a scoring-based classifier is used as the decoder of the model, which effectively reduces the computational complexity of the existing technology.

[0074] In some embodiments, step 1 is to mark a given sentence based on a preset marking strategy, which may be:

[0075] Based on the Re1-Spec corner tagging strategy, the head entity and tail entity of a given sentence are combined with the entity's start position and entity's end position to obtain four types of tag texts.

[0076] Obviously, the embodiment of the present application effectively solves the problems of entity overlap, semantic ambiguity and insufficient relationship mutual exclusivity in traditional relationship triple extraction through innovative labeling strategies and multi-level comparative learning mechanisms.

[0077] Specifically, based on the Re1-Spec corner marking strategy, four combination types of "H / T" and position marking (B / E) are combined to accurately locate the boundaries of overlapping entities and avoid confusion in entity pair matching.

[0078] For example, given a sentence S = {x1, x2, ..., x n} and the relationship R contained in the sentence = {r1,r2,...,r k}, where n is the length of the words in the sentence and k is the number of relations in the sentence;

[0079] The goal of relation triple extraction is to identify all possible relation triplets T = {(h, r, t) | h, t∈E, r∈R} contained in a sentence, where E contains all entities in the sentence, and different relation triplets may share overlapping entities;

[0080] Based on the table filling model, the Rel-Spec corner tagging strategy is used as a tagging scheme. This tagging strategy combines the head and tail entity tags with the BIOES tagging strategy. 'H' and 'T' are used to represent the head and tail entities, B represents the start position of the entity, and E represents the end position of the entity. The following four types of tags are obtained by combining them:

[0081] (1) HB-TB: indicates that the position is the starting position of the head entity and the tail entity of the relation triple;

[0082] (2) HB-TE: indicates that the position is the starting position of the head entity and the ending position of the tail entity of the relation triple;

[0083] (3) HE-TB: indicates the end position of the head entity and the start position of the tail entity of the relation triple;

[0084] (4) HE-TE: Indicates that the position is the end position of the head entity and the tail entity of the relation triple.

[0085] In addition, cells other than the above cases are marked with '-'.

[0086] Obviously, the embodiment of the present application eliminates the error accumulation effect by adopting an encoder-decoder structure based on the table filling principle to merge the two subtasks of named entity recognition and relation classification into a relation triple joint extraction task.

[0087] In some embodiments, in step 2, the marked sentence is input into a preset encoder for normalization to obtain an encoded text, which may be:

[0088] The labeled sentences are pre-padded to keep all sentences of uniform length, and then the pre-trained BERT is used as the sentence encoder to obtain the d-dimensional embedding of each entity to obtain the encoded text.

[0089] Obviously, using the pre-trained BERT model to semantically model the input sentence and generate a unified d-dimensional token embedding, combined with token embedding and position embedding, can enhance the semantic representation ability and thus fully capture the deep semantic features of the sentence.

[0090] For example, in order to better semantically model each input sentence, for each input sentence, we first pre-fill the sentence to keep all input sentences of the same length, and then use a pre-trained BERT as the sentence encoder to obtain the d-dimensional embedding of each entity (token), which can be expressed as follows:

[0091] {e1,e2,...,e L}=BERT{(x1,x2,...,x L )}

[0092] In the above formula, ei is represented as the sum of the corresponding entity embedding and position embedding, and L is the length of the sentence sequence after pre-filling.

[0093] Obviously, the encoded text is in a format that can be recognized by the entity-level contrastive learning and relationship-level contrastive learning modules. Therefore, the encoded text is sent to the entity-level contrastive learning and relationship-level contrastive learning modules respectively, which can effectively obtain semantic information.

[0094] In some embodiments, in step 3, the encoded text is sequentially subjected to entity-level contrastive learning and relation-level contrastive learning to extract semantic information, which may be specifically:

[0095] Based on the preset entity-level contrastive learning module, contrastive learning is performed on the head entity and the tail entity in the encoded text respectively, enhancing the distinction between the head entity and the tail entity in the embedding space, so as to independently learn the semantic information of each entity;

[0096] On the basis of entity-level contrastive learning, based on the preset relationship-level contrastive learning module, the relationship triples are mapped to the high-dimensional embedding space. By defining the similarity measure in the high-dimensional embedding space, the distinction between entities and relationships is enhanced, and the semantic structure of the relationship triples is captured.

[0097] Obviously, by setting entity-level contrastive learning to perform contrastive learning on the head entity and the tail entity respectively, and calculating the contrastive loss through cosine similarity, the discriminability of entity embedding can be optimized, thereby enhancing the model's adaptability to complex semantics.

[0098] More specifically, based on the preset entity-level contrastive learning module, contrastive learning is performed on the head entity and the tail entity in the encoded text, enhancing the distinction between the head entity and the tail entity in the embedding space, so as to independently learn the semantic information of each entity. This may include the following:

[0099] Get the d-dimensional embeddings of all entities contained in the original sequence, where

[0100] For the head entity, the entity extracted from the original input sentence is used as the anchor sample, the head entity in the relation triple extracted from the label is used as the positive sample, and the entity different from the target relation triple is selected as the negative sample to obtain the contrastive learning loss function of the head entity;

[0101] For the tail entity, the selection of positive and negative sample pairs is similar to that of the head entity. That is, the entity extracted from the original input sentence is used as the anchor sample, the tail entity in the relation triple extracted from the label is used as the positive sample, and the tail entity different from the target relation triple is selected as the negative sample to obtain the contrastive learning loss function of the tail entity.

[0102] The contrastive learning loss function of the head entity and the contrastive learning loss function of the tail entity are weighted and summed to obtain the total loss function, and the semantic information of each entity is obtained according to the total loss function.

[0103] Since the existing technology relies solely on the combination of encoders and decoders, it is impossible to make the joint extraction model adapt to complex semantic environments. In recent years, contrastive learning has demonstrated its powerful ability in learning and understanding complex semantic environments by focusing on positive and negative samples for comparison, and has achieved good results in many natural language processing tasks.

[0104] However, traditional contrastive learning learns semantic knowledge in sentences by optimizing the similarity goal between sentence-level representations. In contrast, the entity-level contrastive learning (ECL) proposed in the embodiment of the present application can independently learn the semantic information of each entity by performing contrastive learning on the head entity and the tail entity respectively.

[0105] For example, for each input sequence, we first obtain the d-dimensional embeddings e of all entities contained in the original sequence. i ∈R d ;

[0106] Since the goal of contrastive learning is to narrow the distance between the anchor sample and the positive sample, and to increase the distance between the anchor sample and the negative sample, the selection of positive and negative sample pairs is crucial for contrastive learning;

[0107] Based on this, for the head entity, the entity extracted from the original input sentence is used as the anchor sample, and the head entity in the relation triple extracted from the label is used as the positive sample. In order to construct the negative sample, the entity different from the target relation triple is selected as the negative sample. The contrastive learning loss function of the head entity can be defined as:

[0108]

[0109] In the above formula, Represented as positive head entity embedding, from target relation triples, It is represented as the negative sample head entity embedding, which comes from other relation triples. τ1 is the temperature hyperparameter used to control the smoothness of the distribution. N is the number of negative samples. sim(·) is the cosine similarity used to measure the similarity of entity embeddings, which can be formulated as follows:

[0110]

[0111] In the above formula, ||·|| represents the Euclidean norm of entity embedding;

[0112] For the tail entity, the selection of positive and negative sample pairs is similar to that of the head entity. The tail entity in the relation triple extracted from the label is used as the positive sample, and the tail entity different from the target relation triple is selected as the negative sample. The contrastive learning loss function of the tail entity can be defined as follows:

[0113]

[0114] In the above formula, Represented as positive sample tail entity embedding, It is represented as negative sample tail entity embedding, τ1 is the temperature hyperparameter;

[0115] At the same time, in entity-level contrastive learning, in order to simultaneously optimize the representation of the head entity contrastive learning loss function and the tail entity contrastive learning loss function, a total loss function is set. The total loss function is the weighted sum of the head entity contrastive loss function and the tail entity contrastive loss function, which can be defined as follows:

[0116] ζ ecl =ζ head +ζ tail

[0117] More specifically, based on entity-level contrastive learning and a preset relation-level contrastive learning module, relation triplets are mapped to a high-dimensional embedding space. By defining a similarity measure in the high-dimensional embedding space, the distinction between entities and relations is enhanced, and the semantic structure of relation triplets is captured. This can include the following:

[0118] The relation triples are mapped to a high-dimensional embedding space, and a similarity measure is defined in the high-dimensional embedding space. The positive and negative sample pairs are subjected to comparative learning optimization so that the distance between the positive sample pairs in the embedding space is as close as possible and the distance between the negative sample pairs in the embedding space is as far as possible. This enhances the distinction between entities and relations and captures the semantic structure of the relation triples.

[0119] Although the representations of the two entities are independently optimized and their distinguishability in the embedding space is enhanced by performing entity-level contrastive learning, this method cannot utilize the effective relational information in the semantics. Therefore, in an embodiment of the present application, relation-level contrastive learning (RCL) is performed on the basis of entity-level contrastive learning. By mapping relation triplets to a high-dimensional embedding space and defining a similarity measure in the space, contrastive learning is optimized for positive and negative sample pairs, so that the distance between the positive sample pairs in the embedding space is as close as possible, and the distance between the negative sample pairs in the embedding space is as far as possible, thereby enhancing the distinguishability of entities and relationships and capturing the semantic structure of relation triplets.

[0120] For example, during the training phase of the model, for each input sequence, all the relation triplets (h, r i ,t) as positive samples, and generate negative samples (h,r i * ,t), in addition, all the entity pairs to be matched in the input sequence are obtained, and each entity pair is assigned a pending relationship to form an anchor sample (h i ,r i ,t i ), each relation triple is concatenated and further mapped to a high-dimensional space through a neural network as follows:

[0121] z h,r,t =MLP([e h ;r;e t ])

[0122] In the above formula, [·] is a concatenation operation, and MLP is a fully connected network used to capture complex interactive features. Similarly, the similarity between embeddings can be measured by using cosine similarity, which can be expressed as follows:

[0123]

[0124] In order to maximize the distance between positive sample pairs, we can adopt the idea of ​​InfoNCE and define the loss function of relation-level contrastive learning as follows:

[0125]

[0126] In the above formula, τ2 is the temperature parameter. By minimizing the loss, the model maximizes the similarity of positive samples and minimizes the similarity of negative samples, effectively utilizing the relationship information between entity pairs and enhancing the distinguishability of entity and relationship representations.

[0127] It can be seen that entity-level contrastive learning enhances the distinguishability of entities in complex semantics by independently optimizing the embedding representations of head and tail entities. By setting up relationship-level contrastive learning to map complete relationship triplets into a high-dimensional embedding space (through MLP), using InfoNCE loss to maximize the similarity of positive samples and minimize the similarity of negative samples, and generating negative samples by randomly replacing relationships, it can enhance the semantic structure distinguishability of relationship triplets and strengthen the discriminative ability of relationship structures.

[0128] In some embodiments, in step 4, the semantic information is scored using a scoring-based classifier, and the score vector is input into a preset function to predict a relationship triple.

[0129] Although the model has integrated the semantic information in the sentence after obtaining the loss function extracted from ECL and RCL, since the goal of the relation triple extraction task is to obtain all triples in the text, the embodiment of the present application introduces the decoding stage at the end.

[0130] For example, in decoder design, in order to enumerate all possible relation triplets (e i ,r k ,e j ), and assign high confidence labels to each entity pair, set up a score-based classifier, and define the final score function as follows:

[0131]

[0132] In the above formula, b represents the trainable weights and biases, de represents the dimension of entity pair representation, [;] represents the concatenation operation, (·) represents the ReLU activation function, and Drop(·) represents the dropout strategy used to prevent overfitting. The classifier can be seamlessly integrated with the encoder and can capture the correlation and mutual exclusivity of the relationship between entities.

[0133] Finally, (w i ,r k ,w j ) is input into the softmax function to predict the corresponding label:

[0134]

[0135] The loss function of the entire decoding process is defined as follows:

[0136]

[0137] In the above formula, represents the gold label obtained from the annotation;

[0138] During the entire training process of the model, we choose to optimize the following model by jointly minimizing the contrastive training loss and the relation triplet decoding loss:

[0139] ζ all =ζ triple +ζ ecl +ζ rcl

[0140] Obviously, by jointly optimizing the triplet decoding loss, entity-level contrastive loss, and relationship-level contrastive loss, and balancing the optimization goals of triplet extraction and contrastive learning through multi-task learning, the model can significantly improve the accuracy and robustness of entity-relationship joint extraction while reducing overfitting, especially in scenarios with multiple relations and overlapping entities.

[0141] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a semantic decomposition relationship triple joint extraction device based on double-level contrastive learning provided in an embodiment of the present application. Figure 2 As shown, a semantic decomposition relationship triple joint extraction device based on two-level contrastive learning, the device includes:

[0142] The tagging module 1 is used to tag a given sentence based on a preset tagging strategy;

[0143] Encoding module 2 is used to input the marked sentences into a preset encoder for normalization and formatting to obtain encoded text;

[0144] The two-level contrastive learning module 3 is used to encode text and extract semantic information through entity-level contrastive learning and relation-level contrastive learning.

[0145] The extraction module 4 is used to score the semantic information through a scoring-based classifier, and input the score vector into a preset function to predict the relationship triples.

[0146] Obviously, by introducing the idea of ​​contrastive learning and setting up entity-level contrastive learning, the model can fully utilize the entity information in the semantic environment and improve the model's extraction effect on overlapping entities;

[0147] Furthermore, building on entity-level contrastive learning, we further developed relation-level contrastive learning. By integrating the connections between entities and relations in relational triples, we can improve the model's utilization of the strong associations between entities and relations. Specifically, by introducing two-level contrastive learning, we enhance the model's ability to understand complex semantic information, significantly improving the model's extraction performance in complex semantic scenarios.

[0148] At the same time, a scoring-based classifier is used as the decoder of the model, which effectively reduces the computational complexity of the existing technology.

[0149] Figure 3 This is a structural diagram of a terminal provided by an embodiment of the present application. As shown in the figure, the terminal 3 of this embodiment includes: at least one processor 30 ( Figure 3 Only one is shown), a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 implements the steps of any of the above-mentioned method embodiments when executing the computer program 32.

[0150] The terminal 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that Figure 3 It is only an example of terminal 3 and does not constitute a limitation on terminal 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0151] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0152] The memory 31 may be an internal storage unit of the terminal 3, such as a hard disk or memory of the terminal 3. The memory 31 may also be an external storage device of the terminal 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the terminal 3. Furthermore, the memory 31 may include both an internal storage unit of the terminal 3 and an external storage device. The memory 31 is used to store the computer program and other programs and data required by the terminal. The memory 31 may also be used to temporarily store data that has been output or is about to be output.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0154] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0155] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0156] In the embodiments provided in this application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0158] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0159] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0160] The present application implements all or part of the processes in the above-mentioned embodiment method, and can also be implemented through a computer program product. When the computer program product runs on a terminal, the terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0161] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for joint extraction of semantic decomposition relation triplets based on two-level contrastive learning, characterized by: include: Tag the given sentence based on the preset tagging strategy; Input the marked sentences into the preset encoder for normalization and formatting to obtain the encoded text; The encoded text undergoes entity-level contrastive learning and relation-level contrastive learning in turn to extract semantic information; The semantic information is scored by a scoring-based classifier, and the score vector is input into a preset function to predict the relationship triples.

2. The method according to claim 1, characterized in that The given sentence is marked based on the preset marking strategy, specifically: Based on the Re1-Spec corner tagging strategy, the head entity and tail entity of a given sentence are combined with the entity's start position and entity's end position to obtain four types of tag texts.

3. The method according to claim 2, characterized in that The Rel-Spec corner tagging strategy combines the head entity and tail entity of a given sentence with the entity's start position and entity's end position to obtain four types of tag text, specifically: Based on the table filling model, the Rel-Spec corner tagging strategy is used as a tagging scheme. This tagging strategy combines the head and tail entity tags with the BIOES tagging strategy. 'H' and 'T' are used to represent the head and tail entities, B represents the start position of the entity, and E represents the end position of the entity. The following four types of tags are obtained by combining them: (1) HB-TB: indicates that the position is the starting position of the head entity and the tail entity of the relation triple; (2) HB-TE: indicates that the position is the starting position of the head entity and the ending position of the tail entity of the relation triple; (3) HE-TB: indicates the end position of the head entity and the start position of the tail entity of the relation triple; (4) HE-TE: Indicates that the position is the end position of the head entity and the tail entity of the relation triple.

4. The method according to claim 1, wherein The marked sentence is input into a preset encoder for normalization to obtain the encoded text, specifically: The labeled sentences are pre-padded to keep all sentences of uniform length, and then the pre-trained BERT is used as the sentence encoder to obtain the d-dimensional embedding of each entity to obtain the encoded text.

5. The method according to claim 1, wherein The encoded text is sequentially subjected to entity-level contrastive learning and relation-level contrastive learning to extract semantic information, specifically: Based on the preset entity-level contrastive learning module, contrastive learning is performed on the head entity and the tail entity in the encoded text respectively, enhancing the distinction between the head entity and the tail entity in the embedding space, so as to independently learn the semantic information of each entity; On the basis of entity-level contrastive learning, based on the preset relationship-level contrastive learning module, the relationship triples are mapped to the high-dimensional embedding space. By defining the similarity measure in the high-dimensional embedding space, the distinction between entities and relationships is enhanced, and the semantic structure of the relationship triples is captured.

6. The method according to claim 5, characterized in that The preset entity-level contrastive learning module performs contrastive learning on the head entity and the tail entity in the encoded text, enhancing the distinction between the head entity and the tail entity in the embedding space, so as to independently learn the semantic information of each entity, including the following: Get the d-dimensional embeddings of all entities contained in the original sequence, where For the head entity, the entity extracted from the original input sentence is used as the anchor sample, the head entity in the relation triple extracted from the label is used as the positive sample, and the entity different from the target relation triple is selected as the negative sample to obtain the contrastive learning loss function of the head entity; For the tail entity, the entity extracted from the original input sentence is used as the anchor sample, the tail entity in the relation triple extracted from the label is used as the positive sample, and the tail entity different from the target relation triple is selected as the negative sample to obtain the contrastive learning loss function of the tail entity; The contrastive learning loss function of the head entity and the contrastive learning loss function of the tail entity are weighted and summed to obtain the total loss function, and the semantic information of each entity is obtained according to the total loss function.

7. The method according to claim 5, characterized in that Based on entity-level contrastive learning and a preset relation-level contrastive learning module, the relation triples are mapped to a high-dimensional embedding space. By defining a similarity measure in the high-dimensional embedding space, the distinction between entities and relations is enhanced, and the semantic structure of the relation triples is captured. This includes the following: The relation triples are mapped to a high-dimensional embedding space, and a similarity measure is defined in the high-dimensional embedding space. The positive and negative sample pairs are subjected to comparative learning optimization so that the distance between the positive sample pairs in the embedding space is as close as possible and the distance between the negative sample pairs in the embedding space is as far as possible. This enhances the distinction between entities and relations and captures the semantic structure of the relation triples.

8. The method according to claim 7, characterized in that in, During the training phase of the model, for each input sequence, all relation triplets in the training data are first sampled as positive samples, and negative samples are generated by randomly replacing the relationship between the head and tail entities. All entity pairs to be matched in the input sequence are obtained, and each entity pair is assigned a pending relationship to form an anchor sample. Each relation triple is spliced ​​and further mapped to a high-dimensional space through a neural network.

9. The method according to claim 8, characterized in that The semantic information is scored by a scoring-based classifier, and the score vector is input into a preset function to predict the relationship triples, wherein the preset function is a softmax function.

10. A semantic decomposition relation triple joint extraction device based on two-level contrastive learning, characterized in that: include: The tagging module is used to tag a given sentence based on a preset tagging strategy; The encoding module is used to input the marked sentences into a preset encoder for normalization and formatting to obtain encoded text; A two-level contrastive learning module is used to encode text, which undergoes entity-level contrastive learning and relation-level contrastive learning in turn to extract semantic information; The extraction module is used to score the semantic information through a scoring-based classifier, and input the score vector into a preset function to predict the relationship triples.

Images