Relation extraction method and related equipment

Abstract

The present application provides a relationship extraction method and related equipment, which obtains a target entity pair to be predicted in a document sample to be predicted, and then inputs the target entity pair to be predicted into a pre-trained relationship extraction model. In the process of training to obtain the relationship extraction model, the information of the previous training cycle corresponding to the current training cycle is used to provide more informative supervision in the training process, thereby avoiding the problem of ignoring the differences between entities, so that a more accurate relationship corresponding to the target entity pair to be trained can be obtained through the relationship extraction model, thereby ensuring the accuracy of relationship extraction.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a relationship extraction method and related equipment. Background Art

[0002] Relation Extraction (RE) aims to identify the relationships between entities from text, which is of great significance for many real-world applications, such as knowledge base construction, information retrieval and question answering systems.

[0003] Based on the above situation, the relationship extraction method for documents in the prior art often ignores the differences between entities, thereby reducing the accuracy of relationship extraction. Summary of the invention

[0004] In view of this, the purpose of this application is to propose a relationship extraction method and related equipment to solve or partially solve the above technical problems.

[0005] Based on the above purpose, the first aspect of the present application provides a relationship extraction method, comprising:

[0006] Obtaining target application entity pairs to be predicted in the application document sample to be predicted;

[0007] The target entity pair to be predicted is input into a pre-trained relationship extraction model to obtain a relationship corresponding to the target entity pair to be predicted.

[0008] Optionally, the relationship extraction model is trained through the following process:

[0009] Obtaining information divergence between a target training entity pair in a training document sample and a pre-built initial model in a current training cycle and a previous training cycle corresponding to the current training cycle, and a relationship extraction loss of the target training entity pair and the initial model in the current training cycle;

[0010] Using the information divergence and the relationship extraction loss, combined with the training document sample, to construct a loss trade-off function;

[0011] The initial model is trained and adjusted according to the minimization result obtained in the process of minimizing the loss trade-off function to obtain a relationship extraction model.

[0012] Optionally, obtaining information divergence of a target training entity pair in the training document sample with respect to a pre-built initial model in a current training cycle and a previous training cycle corresponding to the current training cycle includes:

[0013] Obtaining a first training entity relationship prediction probability of the target training entity pair with respect to a pre-built initial model in the current training cycle, and a second training entity relationship prediction probability of the target training entity pair with respect to the initial model in the previous training cycle;

[0014] Relative entropy processing is performed on the first training entity relationship prediction probability and the second training entity relationship prediction probability to obtain the information divergence.

[0015] Optionally, the obtaining of the first training entity relationship prediction probability of the target training entity pair with respect to the pre-built initial model in the current training cycle includes:

[0016] Obtaining a first training entity relationship prediction score of the target training entity pair with respect to the initial model in the current training cycle, and a distillation temperature corresponding to the initial model in the current training cycle;

[0017] Based on the distillation temperature corresponding to the current training cycle, the first training entity relationship prediction score is normalized to obtain a normalized result in the current training cycle, and the normalized result in the current training cycle is used as the first training entity relationship prediction probability.

[0018] Optionally, the obtaining of the predicted probability of a second training entity relationship of the target training entity pair with respect to the initial model in the previous training cycle includes:

[0019] Obtaining a second training entity relationship prediction score of the target training entity pair with respect to the initial model in the previous training cycle, and a distillation temperature corresponding to the initial model in the previous training cycle;

[0020] Based on the distillation temperature corresponding to the previous training cycle, the second training entity relationship prediction score is normalized to obtain the normalized result in the previous training cycle, and the normalized result in the previous training cycle is used as the second training entity relationship prediction probability.

[0021] Optionally, the process of obtaining the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle includes:

[0022] Obtaining all training cycles of the initial model;

[0023] Using all the training cycles, performing difference processing with the current training cycle or the previous training cycle to obtain a first difference processing result;

[0024] Performing a difference processing using all the training cycles and a preset distillation constant to obtain a second difference processing result;

[0025] performing a difference processing on the current training cycle or the previous training cycle and the preset distillation constant to obtain a third difference processing result;

[0026] Performing ratio processing on the first difference processing result and the second difference processing result to obtain a first ratio processing result, and performing ratio processing on the third difference processing result and the second difference processing result to obtain a second ratio processing result;

[0027] Performing product processing on a preset first distillation temperature threshold and the first ratio processing result to obtain a product processing result, obtaining a first product processing result, and performing product processing on a preset second distillation temperature threshold and the second ratio processing result to obtain a second product processing result;

[0028] The first product processing result and the second product processing are added together to obtain a first addition processing result, and the first addition processing result is used as the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle.

[0029] Optionally, obtaining the relationship extraction loss of the target training entity pair with respect to the initial model in the current training cycle includes:

[0030] Obtaining a first training entity relationship prediction score of the target training entity pair in a current training cycle, a positive relationship corresponding to the target training entity pair, and a negative relationship corresponding to the target training entity pair;

[0031] The relation extraction loss is determined based on the first training entity relation prediction score, the positive relation, and the negative relation.

[0032] Optionally, the using the information divergence and the relationship extraction loss in combination with the training document sample to construct a loss trade-off function includes:

[0033] Obtaining a loss trade-off parameter in a current training cycle and the number of document samples for training;

[0034] Using the loss trade-off parameter in the current training cycle, multiplying the information divergence to obtain a third product processing result;

[0035] Performing a summing process on the relationship extraction loss and the third product processing result to obtain a second summing processing result;

[0036] The second sum processing result and the ratio between the sum and the number of the training document samples are processed to obtain a third ratio processing result, and the third ratio processing result is used as the loss trade-off function.

[0037] Optionally, obtaining the loss trade-off parameter in the current training cycle includes:

[0038] Obtaining all training cycles of the initial model;

[0039] Performing a difference processing on the current training cycle and the loss trade-off constant to obtain a fourth difference processing result, performing a difference processing on the total number of training cycles and a preset loss trade-off constant to obtain a fifth difference processing result;

[0040] performing ratio processing on the fourth difference processing result and the fifth difference processing result to obtain a fourth ratio processing result;

[0041] The fourth ratio processing result is multiplied by a preset loss trade-off parameter threshold to obtain a fourth product processing result, and the fourth product processing result is used as the loss trade-off parameter in the current training cycle.

[0042] Based on the same inventive concept, the second aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable by the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0043] From the above, it can be seen that the relationship extraction method and related equipment provided by the present application obtain the target entity pairs to be predicted in the document samples to be predicted, and then input the target entity pairs to be predicted into a pre-trained relationship extraction model. Since the information of the previous training cycle corresponding to the current training cycle is used in the process of training to obtain the relationship extraction model, more informative supervision is provided in the training process, thereby avoiding the problem of ignoring the differences between entities, so that the relationship extraction model can obtain a more accurate relationship corresponding to the target entity pairs to be trained, thereby ensuring the accuracy of relationship extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1A flowchart of a relationship extraction method according to an embodiment of the present application;

[0046] Figure 2A A schematic diagram of a unified framework for document-level relationship extraction according to an embodiment of the present application;

[0047] Figure 2B A schematic diagram of the training process of the relationship extraction model in an embodiment of the present application;

[0048] Figure 3 is a structural block diagram of a relationship extraction device according to an embodiment of the present application;

[0049] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0051] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0052] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0053] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0054] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0055] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0056] Relation Extraction (RE) aims to identify the relationship between entities from plain text, which is of great significance for many real-world applications such as knowledge base construction, information retrieval and question-answering systems. Previous research has mainly focused on sentence-level relation extraction, which can only predict the relationship between two entities within a single sentence.

[0057] Extending relation extraction to the document level and identifying the relations between all entities in the entire document makes the task more practical but also more challenging. Document-level relation extraction is usually designed as a relation classification task with pre-detected entities (if there is no pre-defined relation between two entities, it is recorded as NA), which can be achieved in practical applications by using ready-made named entity recognition toolkits in advance.

[0058] The key to document-level relation extraction is to effectively model the rich and complex interactions between entities in the entire document. The current mainstream methods for this task can be roughly divided into two categories according to their model architecture: sequence-based and graph-based. Sequence-based methods use various sequence encoders, such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) and Bi-directional Long Short-Term Memory (BiLSTM) to encode the entire document to implicitly capture the long-distance dependencies between entities. Graph-based methods further construct homogeneous or heterogeneous graphs to explicitly model various types of entity dependencies in the entire document, and reason about these graphs through Graph Neural Networks (GNNs). Recently, with the popularity of language models based on self-attention neural network models (Transformer) in various natural language processing (NLP) tasks, especially information extraction tasks, the boundaries between sequence-based and graph-based methods have become blurred. Graph-based methods begin to build their graph reasoning modules on top of increasingly complex and powerful sequence encoders, while sequence-based methods can also directly incorporate graph structures as priors directly into their self-attention-based neural network model encoding architectures.

[0059] In addition to designing the model architecture, another is to design a loss function that is more suitable for the document-level relation extraction (RE) task. Binary cross entropy loss is probably the most straightforward loss function for this task, treating all relations (including NA) as independent classes. Later, some more advanced loss functions were proposed to better constrain the probabilities of predefined relations and NA classes, such as adaptive threshold loss, which uses the confidence score of the NA class as a learnable adaptive threshold to separate positive and negative relation classes; adaptive focal loss, which serves as an enhancement to further deal with the class imbalance problem; adaptive margin loss, which encourages more separation between positive and negative classes in a margin-based manner; and classless ranking loss, which encourages a larger margin between the confidence scores of each positive class and the NA class.

[0060] Despite the differences in model architectures and loss functions, almost all previous methods follow the hard-label training regime. However, this typical hard-label training regime ignores the following information:

[0061] (1) Semantic diffusion of NA categories.

[0062] (2) Semantic relevance of other relation categories.

[0063] For example, there are four training entities that are all labeled NA, but clearly convey different meanings, such as ("MightyWings, Mark Spiro") implies a potential relationship between a song and a person, while ("Berlin, American") implies a relationship between a band and a country. This difference cannot be captured by using a hard NA label. In the training examples of document-level relationship extraction, the proportion of NA entity pairs is usually high (e.g., 97.1% in DocRED and 94.0% in Re-DocRED), and ignoring the differences will inevitably lead to a considerable loss of information. In addition, in addition to NA, other categories of relationships are not independent, but are often correlated with each other. For example, composer is considered to be related to the author, but not to the nationality. This correlation cannot always be captured by hard labels, resulting in another loss of information.

[0064] Task paradigm: In the text, we follow all previous works and define document-level relation extraction as a relation classification task, where entities are pre-detected in advance. In some practical applications, if entities are not pre-detected, we can use an off-the-shelf named entity recognition toolkit to extract entities first and then perform the relation classification task. More specifically, given a document consisting of a sequence of tokens d = [x1, x2, …, x l ], and a set of entities are pre-detected The document-level relation extraction task is to predict the relationship between each pair of entities {(e s , e o )|s, o=1,2…,n,s≠o} select a part of relations from R∪{NA}, where R is a set of predefined relations, e s , e o are identified as subject and object entities respectively. i May appear multiple times in a document, via entity mentions in, Refers to entity e i The i-th mention of N i is the total number of mentions of the entity. If any predefined relationship is expressed by any pair of their mentions, then the entity (e s , e o ) between them. Entity pairs that do not represent any predefined relationship belong to the NA class.

[0065] Although there are many solutions for document-level relation extraction, most of them can be roughly summarized into a unified framework, such as Figure 2AAs shown, it includes the continuous levels of document encoding, entity representation, entity pair representation and relation classification.

[0066] Entity Representation: After document encoding, this module builds representations for each entity in the input document. Typically, entity representations are obtained by applying some specific pooling strategy, such as average pooling or logsumexp pooling, where the representation of each mention is averaged over the representations of each token in that mention after applying the tagging strategy for the tokens in that mention. One can further build graph structures to explicitly model the dependencies between tokens, mentions, and entities, and then apply graph neural networks on these graphs to update the entity representations so that structured dependencies can also be incorporated.

[0067] Entity Pair Representation: Different from the previous module, this module not only builds representations of individuals, but also further builds representations of entity pairs. Given two entities, the entity pair representation summarizes the semantic clues that can infer the association between the two entities, such as the local context embeddings related to the two entities or the selective fusion of the paths connecting the two entities on the entity graph. These entity pair representations help better describe the interactions between entities and are therefore very useful for determining entity relationships.

[0068] Relation classification: This module finally predicts the possible relationship between each pair of entities. Given a pair of entities (e s , e o ),let Represents entity representation, h s，o represents the entity pair representations obtained in the first two modules respectively. These entities and entity pair representations are combined and transformed and then input into a multi-class classifier to produce predictions about all relations (sometimes including NA):

[0069]

[0070] Among them, Z is the final prediction result, and each element Z r is a valid score, indicating that relation r exists in entity e s and e o To train the classifier, a loss function is designed that usually forces the effective score of the positive relationship class to be higher than that of the negative relationship class:

[0071] L RE= RELass(Z,P s，o , N s，o )

[0072] Here, Z is given by: The model prediction results obtained are Contains entity (e s , e o ) has a positive relationship, if (e s , e o ) is marked as NA, then And N s，o =R\P s，o Contains negative relations. There are various specific forms of relation losses, such as binary cross entropy loss, adaptive threshold loss, adaptive focal loss, etc. Despite their differences, these loss functions all treat the annotated relations as hard labels and use them as the only target during training, ignoring the differences of NA classes and the correlations between other classes.

[0073] An embodiment of the present application provides a relationship extraction method, which obtains a target entity pair to be predicted in a document sample to be predicted, and then inputs the target entity pair to be predicted into a pre-trained relationship extraction model. In the process of training to obtain the relationship extraction model, the information of the previous training cycle corresponding to the current training cycle is used to provide more informative supervision in the training process, thereby avoiding the problem of ignoring the differences between entities, so that a more accurate relationship corresponding to the target entity pair to be trained can be obtained through the relationship extraction model, thereby ensuring the accuracy of relationship extraction.

[0074] like Figure 1 As shown, the method of this embodiment includes:

[0075] Step 101: Obtain target to-be-predicted application entity pairs in to-be-predicted application document samples.

[0076] In this step, the present application is directed to document-level relationship extraction to identify the relationships between all entities in the entire document.

[0077] Entities can be people, organizations, places, etc., and relationships are words or phrases that describe the connections between entities.

[0078] When the application document sample to be predicted includes multiple application entities to be predicted, two application entities to be predicted whose relationship needs to be predicted among the multiple application entities to be predicted are taken as target application entity pairs to be predicted.

[0079] Step 102: input the target entity pair to be predicted into a pre-trained relationship extraction model to obtain a relationship corresponding to the target entity pair to be predicted.

[0080] In this step, due to the complex structure and sophisticated language expression of the document samples to be predicted, manual reading and analysis of the document samples to be predicted would be very time-consuming and inefficient. Therefore, it is necessary to automatically identify and extract the relationships between the target entity pairs to be predicted from the document samples to be predicted through relationship extraction.

[0081] Since the models used in the prior art take hard labels as the only target during training, the differences between NA categories and the correlations between other categories are ignored, resulting in a considerable loss of information, thereby reducing the accuracy of relationship extraction.

[0082] This application adopts a completely online approach, using the information of the previous training cycle corresponding to the current training cycle to provide more information supervision during the training process to gradually soften the hard labels, thereby avoiding the problem of ignoring the differences between entities, so that the relationship extraction model can obtain a more accurate relationship corresponding to the target entity pair to be trained, thereby ensuring the accuracy of relationship extraction.

[0083] In some embodiments, in step 102, the relationship extraction model is trained by the following process:

[0084] Step A1, obtaining the information divergence between the target training entity pairs in the training document sample and the pre-built initial model in the current training cycle and the previous training cycle corresponding to the current training cycle, as well as the relationship extraction loss of the target training entity pairs with respect to the initial model in the current training cycle.

[0085] Step A2: construct a loss trade-off function by using the information divergence and the relationship extraction loss in combination with the training document sample.

[0086] Step A3: training and adjusting the initial model according to the minimization result obtained in the process of minimizing the loss trade-off function to obtain a relationship extraction model.

[0087] In the above scheme, in order to ensure that the initial model corresponding to the current training cycle should produce temperature scaling predictions consistent with the soft labels, it is necessary to obtain the information divergence between the target training entity pairs in the training document samples and the pre-built initial model in the current training cycle and the previous training cycle corresponding to the current training cycle, and to achieve this by minimizing the information divergence.

[0088] On the other hand, the initial model corresponding to the current training cycle should produce predictions consistent with the hard labels. It is necessary to obtain the relationship extraction loss of the target training entity pair with respect to the initial model in the current training cycle, which is achieved by minimizing the relationship extraction loss.

[0089] The total loss of updating the initial model is the combined average of information divergence and relationship extraction loss. Therefore, the information divergence and relationship extraction loss are used to construct a loss trade-off function in combination with training document samples. The initial model is trained and adjusted based on the minimization result obtained in the process of minimizing the loss trade-off function to obtain a relationship extraction model.

[0090] Since the loss trade-off function uses a trade-off between information divergence and relation extraction loss in the current training cycle, using this loss trade-off function can provide more informative supervision during the training process, thereby making the relation extraction model more accurate.

[0091] In some embodiments, in step A1, obtaining information divergence of a target training entity pair in a training document sample with respect to a pre-built initial model in a current training cycle and a previous training cycle corresponding to the current training cycle includes:

[0092] Step B1, obtaining the first training entity relationship prediction probability of the target training entity pair with respect to the pre-built initial model in the current training cycle, and the second training entity relationship prediction probability of the target training entity pair with respect to the initial model in the previous training cycle.

[0093] Step B2: performing relative entropy processing on the first training entity relationship prediction probability and the second training entity relationship prediction probability to obtain the information divergence.

[0094] In the above scheme, in order to make the initial model produce temperature scaling predictions consistent with the labels, the first training entity relationship prediction probability and the second training entity relationship are processed by relative entropy to obtain information divergence (Kullback-Leibler, KL), which is achieved by minimizing the information divergence. The information divergence can be expressed as follows:

[0095]

[0096] in, represents information divergence, p t represents the prediction probability of the first training entity relationship in the current training cycle t, p t-1 Represents the predicted probability of the second training entity relationship in the previous training cycle t-1.

[0097] In some embodiments, in step B1, obtaining the first training entity relationship prediction probability of the target training entity pair with respect to the pre-built initial model in the current training cycle includes:

[0098] Step C1, obtaining the first training entity relationship prediction score of the target training entity pair with respect to the initial model in the current training cycle, and the distillation temperature corresponding to the initial model in the current training cycle.

[0099] Step C2, based on the distillation temperature corresponding to the current training cycle, normalize the first training entity relationship prediction score to obtain the normalized processing result under the current training cycle, and use the normalized processing result under the current training cycle as the first training entity relationship prediction probability.

[0100] In the above scheme, by obtaining the distillation temperature corresponding to the initial model in the current training cycle, applying the temperature-scaled normalization processing (softmax) operation, the target training entity pair with respect to the first training entity relationship prediction score of the initial model in the current training cycle is converted into a K-dimensional soft label vector p t .

[0101] The second training entity relationship prediction probability p in the previous training cycle t-1 t-1 And the first training entity relationship prediction probability p in the current training cycle t t These are soft labels generated by a normalization (softmax) operation with temperature scaling applied.

[0102] To generate the second training entity relationship prediction probability p t-1 For example, p t-1 Expressed as

[0103] The soft label p is generated by t-1 :

[0104]

[0105] in, represents the target training entity pair e predicted by the initial model s and e o The probability that there is a relationship r between them, and τ t is the distillation temperature used in the current training epoch t. Dynamic temperature scheduling is used to adaptively adjust the temperature during training. These soft labels are used together with the original hard labels to update the initial model.

[0106] In addition, the first training entity relationship prediction score is obtained in the following way: For example, the target entity pair is (e s , e o ), Represents entity representation, h s，oThese entities and entity pair representations are combined and transformed and then fed into a multi-class classifier to produce predictions about all relations (sometimes including NA):

[0107]

[0108] Among them, Z is the final prediction result, and each element Z r is a valid score, indicating that relation r exists in entity e s and e o The possibilities between.

[0109] In some embodiments, in step B1, obtaining the predicted probability of the second training entity relationship of the target training entity pair with respect to the initial model in the previous training cycle includes:

[0110] Step D1, obtaining a second training entity relationship prediction score of the target training entity pair with respect to the initial model in the previous training cycle, and a distillation temperature corresponding to the initial model in the previous training cycle.

[0111] Step D2, based on the distillation temperature corresponding to the previous training cycle, normalize the second training entity relationship prediction score to obtain the normalized result under the previous training cycle, and use the normalized result under the previous training cycle as the second training entity relationship prediction probability.

[0112] In the above scheme, by obtaining the distillation temperature corresponding to the initial model in the previous training cycle, applying the temperature-scaled normalization processing (softmax) operation, the target training entity pair with respect to the second training entity relationship prediction score of the initial model in the previous training cycle is converted into a K-dimensional soft label vector p t-1 .

[0113] The second training entity relationship prediction probability p in the previous training cycle t-1 t-1 And the first training entity relationship prediction probability p in the current training cycle t t These are soft labels generated by a normalization (softmax) operation with temperature scaling applied.

[0114] To generate the second training entity relationship prediction probability p t-1 For example, p t-1 Expressed as

[0115] The soft label p is generated by t-1 :

[0116]

[0117] in, represents the target training entity pair e predicted by the initial model s and e o The probability that there is a relationship r between them, and τ t is the distillation temperature used at training epoch t. Dynamic temperature scheduling is used to adaptively adjust the temperature during training. These soft labels are used together with the original hard labels to update the initial model.

[0118] In addition, the second training entity relationship prediction score is obtained in the following way: For example, the target entity pair is (e s , e o ), Represents entity representation, h s，o These entities and entity pair representations are combined and transformed and then fed into a multi-class classifier to produce predictions about all relations (sometimes including NA):

[0119]

[0120] Among them, Z is the final prediction result, and each element Z r is a valid score, indicating that relation r exists in entity e s and e o The possibilities between.

[0121] In some embodiments, in step C1 or step D1, the process of obtaining the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle includes:

[0122] Step E1, obtaining all training cycles of the initial model.

[0123] Step E2, using all the training cycles to perform difference processing with the current training cycle or the previous training cycle to obtain a first difference processing result.

[0124] Step E3, performing difference processing using all the training cycles and a preset distillation constant to obtain a second difference processing result.

[0125] Step E4, performing a difference processing on the current training cycle or the previous training cycle and the preset distillation constant to obtain a third difference processing result.

[0126] Step E5, performing ratio processing on the first difference processing result and the second difference processing result to obtain a first ratio processing result, and performing ratio processing on the third difference processing result and the second difference processing result to obtain a second ratio processing result.

[0127] Step E6, multiplying the preset first distillation temperature threshold by the first ratio processing result to obtain a first product processing result, and multiplying the preset second distillation temperature threshold by the second ratio processing result to obtain a second product processing result.

[0128] Step E7, performing sum processing on the first product processing result and the second product processing to obtain a first sum processing result, and using the first sum processing result as the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle.

[0129] In the above scheme, the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle is not a fixed static distillation temperature, but a dynamic temperature scheduling is adopted, which adaptively adjusts the temperature during the training process.

[0130] In this application, the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle determines the soft label p t-1 Relative to its original effectiveness score z t-1 (ie with the soft label p t-1 The corresponding hard label is the smoothness (or sharpness) of the second training entity relationship prediction score). Higher temperatures lead to smoother soft labels, and the information extracted from them will have less original information from the initial model. Considering the ability of the initial model in this application, its reliability usually increases with training, and less knowledge should be transferred from the initial model in the early stages of training, and more knowledge should be transferred in the later stages. Therefore, this application designs a dynamic scheduler that gradually increases the temperature from τ to τ during training. max (i.e. the first distillation temperature threshold) is reduced to τ min (i.e., the second distillation temperature threshold), can be expressed as follows:

[0131]

[0132] Among them, τ t is the distillation temperature used in training cycle t, T is the total number of training cycles (i.e., all training cycles), 1≤t≤T. As the temperature decreases, the intensity of distillation gradually increases.

[0133] Tt represents the first difference processing result, t-1 represents the third difference processing result, T-1 represents the second difference processing result, represents the result of the first product processing, represents the result of the second product processing, represents the first ratio processing result, represents the second ratio processing result,

[0134] In some embodiments, in step A1, obtaining the relationship extraction loss of the target training entity pair with respect to the initial model in the current training cycle includes:

[0135] Step F1, obtaining the first training entity relationship prediction score of the target training entity pair in the current training cycle, the positive relationship corresponding to the target training entity pair, and the negative relationship corresponding to the target training entity pair.

[0136] Step F2, determining the relationship extraction loss based on the first training entity relationship prediction score, the positive relationship and the negative relationship.

[0137] In the above scheme, the initial model corresponding to the current training cycle should produce predictions consistent with the hard labels. It is necessary to obtain the relationship extraction loss of the target training entity pair with respect to the initial model in the current training cycle, which is achieved by minimizing the relationship extraction loss, which can be expressed as follows:

[0138]

[0139] in, represents the relation extraction loss, z t It consists of the original validity score of the initial model predicting the relationship (i.e., the first training entity relationship prediction score), P s，o and N s，o They respectively contain the target training entity pairs (e s , e o ) between positive and negative relationships.

[0140] In some embodiments, step A2 comprises:

[0141] Step G1, obtaining the loss trade-off parameter in the current training cycle and the number of document samples for training.

[0142] Step G2, using the loss trade-off parameter in the current training cycle to perform a product process with the information divergence to obtain a third product processing result.

[0143] Step G3, summing the relationship extraction loss and the third product processing result to obtain a second summing processing result.

[0144] Step G4, processing the ratio between the second sum processing result and the number of the training document samples to obtain a third ratio processing result, and using the third ratio processing result as the loss trade-off function.

[0145] In the above scheme, the total loss of updating the initial model is the combined average of information divergence and relation extraction loss. Therefore, the information divergence and relation extraction loss are used to construct a loss trade-off function in combination with training document samples. The initial model is trained and adjusted according to the minimization result obtained in the process of minimizing the loss trade-off function, and the relation extraction model can be obtained, which can be expressed as follows:

[0146]

[0147] Among them, L t represents the loss trade-off function, λ t is the trade-off between the two losses used in training epoch t, ​​using dynamic loss trade-off scheduling to adaptively adjust the loss trade-off during training.

[0148] also, represents the result of the third product processing, represents the result of the second summation process, Indicates the third ratio processing result.

[0149] In some embodiments, in step G1, obtaining the loss trade-off parameter in the current training cycle includes:

[0150] Step H1, obtaining all training cycles of the initial model.

[0151] Step H2, using the current training cycle and the loss trade-off constant to perform a difference processing to obtain a fourth difference processing result, and performing a difference processing on the total number of training cycles and the preset loss trade-off constant to obtain a fifth difference processing result.

[0152] Step H3, performing ratio processing on the fourth difference processing result and the fifth difference processing result to obtain a fourth ratio processing result.

[0153] Step H4, multiplying the fourth ratio processing result by a preset loss trade-off parameter threshold to obtain a fourth product processing result, and using the fourth product processing result as the loss trade-off parameter in the current training cycle.

[0154] In the above scheme, the loss trade-off determines the contribution of the knowledge distillation loss to the overall loss. A larger trade-off indicates a higher contribution. Similarly, when the capabilities of the initial model and the reliability of its predictions are relatively weak, the contribution of knowledge distillation must be weakened in the early stages of training. Therefore, another dynamic scheduler is used to gradually increase the loss trade-off during training, which can be expressed as follows:

[0155]

[0156] Among them, λmax is the tradeoff at the last training cycle T, and 1≤t≤T. As the tradeoff increases, the contribution of knowledge distillation to the overall loss gradually increases.

[0157] In addition, t-1 represents the fourth difference processing result, T-1 represents the fifth difference processing result, represents the fourth ratio processing result, represents the result of the fourth product processing, λ t represents the loss trade-off parameter.

[0158] Based on the same inventive concept, the training process of the relation extraction model corresponding to the relation extraction method of the above embodiment is described in detail. Figure 2B As shown, the details are as follows:

[0159] A fully online approach is used to gradually soften the hard labels using self-knowledge distillation and provide more information supervision during the training process. The key idea is to use past predictions during the training process, or more precisely, to use the predictions of the document-level relation extraction model in the previous period to gradually soften the hard labels.

[0160] Specifically, suppose there is a training set (i.e., training document samples) D = {((e s , e o ), P s，o , N s，o )}, where (e s , e o ) is the entity pair obtained from the document (i.e., the target training entity pair), P s，o and N s，o Contains entity e s and entity e o In each period t, the student document-level RE model (student, i.e., the initial model) obtained from the last period is denoted as Θ t-1 , become a teacher.

[0161] For each entity pair in the training set, Θ t-1 Generate a K-dimensional vector It contains the validity scores of all relation categories, where K = |R|, or K = |R| + 1 is the total number of predefined relation categories, sometimes including the special NA class. These validity scores are then converted into a K-dimensional soft label vector by applying a temperature-scaled softmax operation The softmax operation with temperature scaling applied looks like this:

[0162]

[0163] in, is t Predicted entity e s and entity e o The probability that there is a relationship r between them, and τ t is the distillation temperature used in the tth epoch. It should be noted that this application does not use a fixed static distillation temperature, but a dynamic temperature schedule that adaptively adjusts the temperature during training. These soft labels are used together with the original hard labels to update the student network of the current epoch, denoted as Θ t .

[0164] On the one hand, Θ t should produce temperature-scaled predictions that are consistent with the soft labels. This is done by minimizing p for each training instance. t and p t-1 This is achieved by the Kullback-Leibler (KL) divergence (i.e., information divergence) between them, which can be expressed as follows:

[0165]

[0166] Here, p t-1 is based on By Θ t-1 The soft labels generated by p t is t The temperature-scaled predictions (temperature τ) generated in a similar manner t ). On the other hand, Θ t It should produce predictions that are consistent with the hard labels. This is done by s , e o ), P s，o N s，o )∈D minimize It is implemented using the RE loss (i.e., relation extraction loss) defined in .

[0167] z t By θ t The raw validity scores of the predicted relationships consist of (predictions), for (e s , e o ). Update Θ t The total loss (i.e., loss trade-off function) is the combined average of RE loss and KD loss on the training set, which can be expressed as follows:

[0168]

[0169] Among them, λ tis the trade-off between the two losses used in period t (i.e., loss trade-off parameter). This application also uses dynamic loss trade-off scheduling to adaptively adjust the loss trade-off during training.

[0170] Dynamic distillation temperature adjustment: The distillation temperature used in determines the teacher soft label p t-1 Relative to its original effectiveness score z t-1 The smoothness (or sharpness) of the soft labels. Higher temperatures lead to smoother soft labels, from which the information extracted will be less original from the teacher. Considering that the teacher's ability, and therefore its reliability, usually grows with training, we should transfer less knowledge from the teacher in the early stages of training and more knowledge in the later stages. Therefore, we design a dynamic regulator that gradually adjusts the temperature from τ to τ during training. max Down to τ min , which can be expressed as follows:

[0171]

[0172] T is the total number of training epochs, 1≤t≤T. As the temperature decreases, the intensity of distillation gradually increases.

[0173] Dynamic loss function weight adjustment: The loss tradeoff in determines the contribution of the knowledge distillation loss to the overall loss. A larger tradeoff indicates a higher contribution. Similarly, when the teacher's ability and the reliability of its predictions are relatively weak, the contribution of knowledge distillation must be weakened in the early stages of training. Therefore, this application uses another dynamic scheduler to gradually increase the loss tradeoff during training, which can be expressed as follows:

[0174]

[0175] Among them, λ max is the trade-off in the last period T, and 1≤t≤T. As the trade-off increases, the contribution of knowledge distillation in the overall loss gradually increases.

[0176] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0177] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0178] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a relationship extraction device.

[0179] refer to Figure 3 , the relationship extraction device comprises:

[0180] An acquisition module 301 is configured to acquire a target entity pair to be predicted in a document sample to be predicted;

[0181] The input module 302 is configured to input the target entity pair to be predicted into a pre-trained relationship extraction model to obtain a relationship corresponding to the target entity pair to be predicted.

[0182] In some embodiments, the relationship extraction device further includes a training module, and the training module includes:

[0183] An acquisition unit is configured to acquire information divergence between a target training entity pair in a training document sample and a pre-built initial model in a current training cycle and a previous training cycle corresponding to the current training cycle, and a relationship extraction loss of the target training entity pair with respect to the initial model in the current training cycle;

[0184] A function construction unit, configured to construct a loss trade-off function by using the information divergence and the relationship extraction loss in combination with the training document sample;

[0185] The training adjustment unit is configured to perform training adjustment on the initial model according to the minimization result obtained in the process of minimizing the loss trade-off function to obtain a relationship extraction model.

[0186] In some embodiments, the acquisition unit includes:

[0187] A first acquisition subunit is configured to acquire a first training entity relationship prediction probability of the target training entity pair with respect to the pre-built initial model in the current training cycle, and a second training entity relationship prediction probability of the target training entity pair with respect to the initial model in the previous training cycle;

[0188] The relative entropy processing subunit is configured to perform relative entropy processing on the first training entity relationship prediction probability and the second training entity relationship prediction probability to obtain the information divergence.

[0189] In some embodiments, the first acquisition subunit is specifically configured to:

[0190] Obtaining a first training entity relationship prediction score of the target training entity pair with respect to the initial model in the current training cycle, and a distillation temperature corresponding to the initial model in the current training cycle;

[0191] Based on the distillation temperature corresponding to the current training cycle, the first training entity relationship prediction score is normalized to obtain a normalized result in the current training cycle, and the normalized result in the current training cycle is used as the first training entity relationship prediction probability.

[0192] In some embodiments, the first acquisition subunit is specifically configured to:

[0193] Obtaining a second training entity relationship prediction score of the target training entity pair with respect to the initial model in the previous training cycle, and a distillation temperature corresponding to the initial model in the previous training cycle;

[0194] Based on the distillation temperature corresponding to the previous training cycle, the second training entity relationship prediction score is normalized to obtain the normalized result in the previous training cycle, and the normalized result in the previous training cycle is used as the second training entity relationship prediction probability.

[0195] In some embodiments, the relationship extraction device further includes a distillation temperature obtaining module, and the distillation temperature obtaining module is specifically configured as follows:

[0196] Obtaining all training cycles of the initial model;

[0197] Using all the training cycles, performing difference processing with the current training cycle or the previous training cycle to obtain a first difference processing result;

[0198] Performing a difference processing using all the training cycles and a preset distillation constant to obtain a second difference processing result;

[0199] Performing a difference processing on the current training cycle or the previous training cycle and the preset distillation constant to obtain a third difference processing result;

[0200] Performing ratio processing on the first difference processing result and the second difference processing result to obtain a first ratio processing result, and performing ratio processing on the third difference processing result and the second difference processing result to obtain a second ratio processing result;

[0201] Performing product processing on a preset first distillation temperature threshold and the first ratio processing result to obtain a product processing result, obtaining a first product processing result, and performing product processing on a preset second distillation temperature threshold and the second ratio processing result to obtain a second product processing result;

[0202] The first product processing result and the second product processing are added together to obtain a first addition processing result, and the first addition processing result is used as the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle.

[0203] In some embodiments, the acquisition unit includes:

[0204] A second acquisition subunit is configured to acquire a loss trade-off parameter in a current training cycle and the number of training document samples;

[0205] A product processing unit is configured to perform a product processing on the information divergence using the loss trade-off parameter in the current training cycle to obtain a third product processing result;

[0206] an addition processing unit, configured to perform addition processing on the relationship extraction loss and the third product processing result to obtain a second addition processing result;

[0207] The ratio processing unit is configured to process the second sum processing result and the ratio between the second sum processing result and the number of the training document samples to obtain a third ratio processing result, and use the third ratio processing result as the loss trade-off function.

[0208] In some embodiments, the second acquisition subunit is specifically configured to:

[0209] Obtaining all training cycles of the initial model;

[0210] Performing a difference processing on the current training cycle and the loss trade-off constant to obtain a fourth difference processing result, performing a difference processing on the total number of training cycles and a preset loss trade-off constant to obtain a fifth difference processing result;

[0211] performing ratio processing on the fourth difference processing result and the fifth difference processing result to obtain a fourth ratio processing result;

[0212] The fourth ratio processing result is multiplied by a preset loss trade-off parameter threshold to obtain a fourth product processing result, and the fourth product processing result is used as the loss trade-off parameter in the current training cycle.

[0213] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0214] The device of the above embodiment is used to implement the corresponding relationship extraction method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0215] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the relationship extraction method described in any of the above embodiments is implemented.

[0216] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 401, a memory 402, an input / output interface 403, a communication interface 404, and a bus 405. The processor 401, the memory 402, the input / output interface 403, and the communication interface 404 are connected to each other in communication within the device through the bus 405.

[0217] The processor 401 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0218] The memory 402 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 402 and called and executed by the processor 401.

[0219] The input / output interface 403 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0220] The communication interface 404 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0221] The bus 405 comprises a pathway for transmitting information between the various components of the device (eg, the processor 401 , the memory 402 , the input / output interface 403 , and the communication interface 404 ).

[0222] It should be noted that, although the above device only shows the processor 401, the memory 402, the input / output interface 403, the communication interface 404 and the bus 405, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0223] The electronic device of the above embodiment is used to implement the corresponding relationship extraction method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0224] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the relationship extraction method described in any of the above embodiments.

[0225] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0226] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the relationship extraction method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0227] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0228] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0229] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0230] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A relationship extraction method, characterized in that: include: Obtain a target application entity pair to be predicted in the application document sample to be predicted, where the entity is a person, an organization, or a location. The application document sample to be predicted includes multiple application entities to be predicted, and two application entities to be predicted that need to predict a relationship among the multiple application entities to be predicted are taken as the target application entity pair to be predicted; Inputting the target entity pair to be predicted into a pre-trained relation extraction model to obtain a relation corresponding to the target entity pair to be predicted, where the relation is a word or phrase that describes the connection between entities; The relationship extraction model is trained through the following process: Obtaining information divergence between a target training entity pair in a training document sample and a pre-built initial model in a current training cycle and a previous training cycle corresponding to the current training cycle, and a relationship extraction loss of the target training entity pair and the initial model in the current training cycle; Using the information divergence and the relationship extraction loss, combined with the training document sample, to construct a loss trade-off function; Training and adjusting the initial model according to the minimization result obtained in the process of minimizing the loss trade-off function to obtain a relationship extraction model; The obtaining of information divergence of a target training entity pair in a training document sample with respect to a pre-built initial model between a current training cycle and a previous training cycle corresponding to the current training cycle includes: Obtaining a first training entity relationship prediction probability of the target training entity pair with respect to a pre-built initial model in the current training cycle, and a second training entity relationship prediction probability of the target training entity pair with respect to the initial model in the previous training cycle; Performing relative entropy processing on the first training entity relationship prediction probability and the second training entity relationship prediction probability to obtain the information divergence; The obtaining of the first training entity relationship prediction probability of the target training entity pair with respect to the pre-built initial model in the current training cycle includes: Obtaining a first training entity relationship prediction score of the target training entity pair with respect to the initial model in the current training cycle, and a distillation temperature corresponding to the initial model in the current training cycle; Based on the distillation temperature corresponding to the current training cycle, normalizing the first training entity relationship prediction score to obtain a normalized result in the current training cycle, and using the normalized result in the current training cycle as the first training entity relationship prediction probability; The obtaining of the second training entity relationship prediction probability of the target training entity pair with respect to the initial model in the previous training cycle includes: Obtaining a second training entity relationship prediction score of the target training entity pair with respect to the initial model in the previous training cycle, and a distillation temperature corresponding to the initial model in the previous training cycle; Based on the distillation temperature corresponding to the previous training cycle, normalizing the second training entity relationship prediction score to obtain a normalized result in the previous training cycle, and using the normalized result in the previous training cycle as the second training entity relationship prediction probability; The process of obtaining the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle includes: Obtaining all training cycles of the initial model; Using all the training cycles, performing difference processing with the current training cycle or the previous training cycle to obtain a first difference processing result; Performing a difference processing using all the training cycles and a preset distillation constant to obtain a second difference processing result; Performing a difference processing on the current training cycle or the previous training cycle and the preset distillation constant to obtain a third difference processing result; Performing ratio processing on the first difference processing result and the second difference processing result to obtain a first ratio processing result, and performing ratio processing on the third difference processing result and the second difference processing result to obtain a second ratio processing result; Performing product processing on a preset first distillation temperature threshold and the first ratio processing result to obtain a first product processing result, and performing product processing on a preset second distillation temperature threshold and the second ratio processing result to obtain a second product processing result; Performing sum processing on the first product processing result and the second product processing result to obtain a first sum processing result, and using the first sum processing result as the distillation temperature corresponding to the current training cycle or the distillation temperature corresponding to the previous training cycle; The obtaining of the target training entity pair with respect to the relation extraction loss of the initial model in the current training cycle includes: Obtaining a first training entity relationship prediction score of the target training entity pair in a current training cycle, a positive relationship corresponding to the target training entity pair, and a negative relationship corresponding to the target training entity pair; Determine the relationship extraction loss based on the first training entity relationship prediction score, the positive relationship, and the negative relationship; The using the information divergence and the relationship extraction loss in combination with the training document sample to construct a loss trade-off function includes: Obtaining a loss trade-off parameter in a current training cycle and the number of document samples for training; Using the loss trade-off parameter in the current training cycle, multiplying the information divergence to obtain a third product processing result; Performing a summing process on the relationship extraction loss and the third product processing result to obtain a second summing processing result; Performing ratio processing on the second sum processing result and the number of the training document samples to obtain a third ratio processing result, and using the third ratio processing result as the loss trade-off function; The obtaining of the loss trade-off parameter in the current training cycle includes: Obtaining all training cycles of the initial model; Performing a difference processing on the current training cycle and the loss trade-off constant to obtain a fourth difference processing result, performing a difference processing on the total number of training cycles and a preset loss trade-off constant to obtain a fifth difference processing result; performing ratio processing on the fourth difference processing result and the fifth difference processing result to obtain a fourth ratio processing result; The fourth ratio processing result is multiplied by a preset loss trade-off parameter threshold to obtain a fourth product processing result, and the fourth product processing result is used as the loss trade-off parameter in the current training cycle.

2. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to claim 1 is implemented.

Images