Relation extraction model training method and device, electronic equipment and storage medium

Abstract

The application provides a relationship extraction model training method and device, electronic equipment and storage medium. From a sample data set, noise sample text and pure sample text are determined. Based on an initial relationship extraction model, noise hidden features of the noise sample text and pure hidden features of the pure sample text are determined respectively, and based on the noise hidden features, predicted entity relationships of the noise sample text are determined. Based on the predicted entity relationships of the noise sample text, the pure hidden features of the pure sample text and sample entity relationships, parameter iteration is performed on the initial relationship extraction model to obtain a relationship extraction model. The defects of loss of useful information caused by discarding of noise samples, insufficient model training and poor effect are overcome. Information in the noise sample text is fully utilized through pseudo labels to enable the model to better learn relationship features, thereby optimizing the training effect and improving the performance of the model.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a training method, apparatus, electronic device, and storage medium for a relation extraction model. Background Technology

[0002] Relation extraction, a crucial step in knowledge graph construction, mines knowledge by predicting relationships between entities. However, most supervised relation extraction techniques require large amounts of labeled data, which is difficult to obtain manually. To address this issue, remote supervision methods have been proposed, automatically generating labeled text corpora by aligning plain text with a knowledge base. However, due to the annotation process, remote supervision often faces high label noise in the training data.

[0003] To mitigate the impact of noise in remote supervision, current methods employ multi-instance learning frameworks or modifications to train relation extraction models to recognize bag-level relation labels. However, these models still struggle to accurately map each sentence within a bag to a specific sentence label. To address this, remote supervision relation extraction models focused on sentence-level improvements have been proposed. However, these models suffer from a significant problem: when faced with noisy samples, they discard all of them. Directly filtering out noisy samples leads to the loss of useful information, resulting in insufficient model learning and poor training performance. Summary of the Invention

[0004] This invention provides a training method, apparatus, electronic device, and storage medium for a relation extraction model, which addresses the shortcomings of existing technologies where the discarding of noisy samples leads to the loss of useful information, resulting in insufficient model training and poor performance. By generating pseudo-labels and using them to fully utilize the information in the noisy sample text, the model can better learn relation features, thereby optimizing training results and improving model performance.

[0005] This invention provides a training method for a relation extraction model, comprising:

[0006] A sample dataset is determined, and noisy sample text and clean sample text are identified from the sample dataset. The sample dataset contains multiple sample texts and sample entity relationships corresponding to each sample text.

[0007] Based on the initial relation extraction model, the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text are determined respectively, and the predicted entity relations of the noisy sample text are determined based on the noise hidden features.

[0008] Based on the predicted entity relationships of the noisy sample text, and the clean hidden features and entity relationships of the clean sample text, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model.

[0009] According to the training method of the relation extraction model provided by the present invention, the initial relation extraction model includes a feature extraction model, a first classification model and a second classification model, and the predicted entity relationship includes a first predicted entity relationship and a second predicted entity relationship.

[0010] The initial relation extraction model determines the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text, and determines the predicted entity relations of the noisy sample text based on the noise hidden features, including:

[0011] Based on the feature extraction model in the initial relation extraction model, the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text are determined respectively.

[0012] Based on the first classification model in the initial relation extraction model and the noise hiding features, the first predicted entity relation of the noise sample text is determined;

[0013] Based on the second classification model in the initial relation extraction model and the noise hiding features, the second predicted entity relation of the noise sample text is determined.

[0014] According to a training method for a relation extraction model provided by the present invention, the first predicted entity relation of the noisy sample text is determined based on a first classification model in the initial relation extraction model and the noise hiding features; the second predicted entity relation of the noisy sample text is determined based on a second classification model in the initial relation extraction model and the noise hiding features, comprising:

[0015] Based on the first classification model and the first enhanced feature, the first predicted entity relationship of the noisy sample text is determined;

[0016] Based on the second classification model and the second enhanced features, the second predicted entity relationship of the noisy sample text is determined;

[0017] The first enhancement feature and the second enhancement feature are respectively the noise hidden features determined by the feature extraction model based on the noise sample text under the first neuron loss rate and the second neuron loss rate;

[0018] The loss rate of the first neuron is less than the loss rate of the second neuron.

[0019] According to a training method for a relation extraction model provided by the present invention, the initial relation extraction model is iterated by parameters based on the predicted entity relations of the noisy sample text, and the clean hidden features and sample entity relations of the clean sample text to obtain the relation extraction model, including:

[0020] Based on the first and second predicted entity relationships in the predicted entity relationships, the unlabeled sample loss is determined;

[0021] By performing feature mixing on the pure hidden features of different pure sample texts, multiple pure mixed features are obtained;

[0022] Based on each pure mixture feature and its corresponding sample entity relationship, determine the contrast loss;

[0023] Based on the unlabeled sample loss and the contrast loss, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model.

[0024] According to the training method of the relation extraction model provided by the present invention, the unlabeled sample loss is determined based on the following formula:

[0025]

[0026] in, The loss is for unlabeled samples, where μB is the number of noisy text samples u. Denotes the first predicted entity relationship, θ s,t To use the feature extraction model for the t-th iteration with the second neuron loss rate, ψ t For the second classification model in the t-th iteration, Let l represent the second predicted entity relationship, and l be the cross-entropy loss function. σ t (c) represents the number of noisy sample texts with a prediction confidence greater than the threshold τ in the first predicted entity relation corresponding to relation category c.

[0027] According to the training method of the relation extraction model provided by the present invention, the contrastive loss is determined based on the following formula:

[0028]

[0029]

[0030] z i =λz a +(1-λ)z b

[0031]

[0032] yi ∈{y a ,y b}

[0033] in, To compare the loss, N is the number of clean sample texts, and λ can be randomly sampled from the Beta distribution, λ∈[0,1]~Beta(α) m ,α m ), α m For hyperparameters, z i For z a and z b The mixed characteristics, i.e., the pure mixed characteristics, z a and z b The clean hidden features of different clean sample texts, y a To hide the pure feature z a The corresponding pure sample text's sample entity relations, y b To hide the pure feature z b The corresponding pure sample text's sample entity relations, y a and y b All are pure mixed characteristics z i The corresponding sample entity relationship, Let y represent the entity relationship of a sample in a batch. i The number of clean sample texts, z j Let y be the entity relationship of the sample. j Pure hidden features of pure sample text, z r Let y be the entity relationship of the sample. r The pure hidden features of the pure sample text, y r =y i or y r ≠y i τ is the temperature hyperparameter.

[0034] According to a training method for a relation extraction model provided by the present invention, determining noisy sample text and clean sample text from the sample dataset includes:

[0035] Based on the initial relation extraction model, the hidden features of each sample text in the sample dataset are determined;

[0036] Based on the hidden features of each sample text, the K nearest neighbor graph corresponding to each sample text is determined;

[0037] Based on the K-nearest neighbor graph, noisy sample texts and clean sample texts are determined from each sample text, and the sample entity relationships of the noisy sample texts are discarded.

[0038] The present invention also provides a training apparatus for a relation extraction model, comprising:

[0039] A determining unit is used to determine a sample dataset and, from the sample dataset, determine noisy sample text and clean sample text, wherein the sample dataset contains multiple sample texts and sample entity relationships corresponding to each sample text;

[0040] The prediction unit is used to determine the noise hidden features of the noise sample text and the clean hidden features of the clean sample text based on the initial relation extraction model, and to determine the predicted entity relations of the noise sample text based on the noise hidden features.

[0041] The training unit is used to perform parameter iteration on the initial relation extraction model based on the predicted entity relations of the noisy sample text, as well as the clean hidden features and sample entity relations of the clean sample text, to obtain the relation extraction model.

[0042] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a training method for a relation extraction model as described above.

[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method of the relation extraction model as described above.

[0044] The present invention provides a training method, apparatus, electronic device, and storage medium for a relation extraction model. From a sample dataset, noisy sample text and clean sample text are identified. An initial relation extraction model is used to determine the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text, respectively. Based on the noise hidden features, predicted entity relationships of the noisy sample text are determined. Based on the predicted entity relationships of the noisy sample text, the clean hidden features of the clean sample text, and the sample entity relationships, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model. This overcomes the shortcomings of current methods that discard noisy samples, leading to the loss of useful information and insufficient model training, resulting in poor performance. By using pseudo-labels, the information in the noisy sample text is fully utilized, enabling the model to better learn relation features, thereby optimizing training effects and improving model performance. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the training method for the relation extraction model provided by the present invention;

[0047] Figure 2 This is a schematic diagram of the training process of the relation extraction model provided by the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of the training device for the relation extraction model provided by the present invention;

[0049] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] Currently, most supervised relation extraction techniques require large amounts of labeled data, which is often difficult to obtain manually. To address this issue, remote supervision methods have been proposed, which automatically generate labeled text corpora by aligning plain text with a knowledge base. For example, if a sentence contains relation triples...<s,r,o> If the subject(s) and object(o) of a sentence are in the form of a relational triple (<subject, relation, object>), then the remote supervision method will consider the sentence to be a relational triple.<s,r,o> The sentence is labeled as "NA" if it is a valid sample; otherwise, if no applicable relation triplet exists, the sentence is labeled as "NA". However, due to the annotation process, remote supervised datasets often face high label noise in the training data.

[0052] To mitigate the impact of noisy labels caused by remote supervision, multiple instance learning (MIL) frameworks or modifications of MIL are commonly used to train relation extraction models. However, while MIL-based techniques can identify package-level relation labels, they are still not adept at accurately mapping each sentence within each package to a specific sentence label. Some studies have focused on sentence-level improvements and empirically demonstrated the shortcomings of package-level methods in sentence-level evaluation. However, these methods still suffer from a significant problem: when faced with noisy samples, simply discarding all noisy samples and using only selected samples to train the relation extraction model leads to the loss of useful information, resulting in insufficient model learning and poor training performance.

[0053] To address this, the present invention provides a training method for a relation extraction model under semi-supervised learning and remote supervision. The method aims to generate pseudo-labels for noisy sample text, and to make full use of the information in the noisy sample text through pseudo-labels, so as to enable the model to learn relation features better, thereby optimizing the training effect and improving the performance of the model. Figure 1 This is a flowchart illustrating the training method for the relation extraction model provided by the present invention, as shown below. Figure 1 As shown, the method includes:

[0054] Step 110: Determine the sample dataset, and from the sample dataset, determine the noisy sample text and the clean sample text. The sample dataset contains multiple sample texts and the sample entity relationships corresponding to each sample text.

[0055] Step 120: Based on the initial relation extraction model, determine the noise hidden features of the noise sample text and the clean hidden features of the clean sample text respectively, and determine the predicted entity relations of the noise sample text based on the noise hidden features.

[0056] Step 130: Based on the predicted entity relationships of noisy sample text, and the clean hidden features and entity relationships of clean sample text, perform parameter iteration on the initial relation extraction model to obtain the relation extraction model.

[0057] Specifically, before training the model, it is necessary to first determine the training data, that is, the sample text required for model training, and the corresponding labels of the sample text. Here, the labels are the entity relations in the relation triples that refer to the relationship between entity pairs, that is, the sample entity relations.

[0058] Here, the sample text required for training can be obtained through a remote supervision method, that is, the text corpus can be collected in advance, and then the text corpus can be aligned with the knowledge base to obtain text with labels, which are the sample texts, and the labels are the sample entity relations of the sample texts.

[0059] The process of aligning text corpora using a pre-built knowledge base essentially involves labeling the text with named entity tags. That is, matching the text corpora with existing relation triples in the knowledge base to assign the entity relations in the relation triples to the text, thereby obtaining labeled text, i.e., sample text and its sample entity relations.

[0060] Here, the text corpus can be current news, research reports, etc., which can be collected from news events, journals, Wikipedia, etc. The text corpus can be a single type of corpus from a single domain, or it can be a corpus from multiple types of corpus from multiple domains. This embodiment of the invention does not make specific limitations in this regard.

[0061] After obtaining the sample texts and their entity relationships, a training dataset can be constructed to train the model, resulting in a trained relation extraction model. Here, the training dataset is the sample dataset, which contains multiple sample texts and the label for each sample text, i.e., the entity relationship.

[0062] Furthermore, considering that sample datasets obtained through remote supervision often suffer from high-noise labels—that is, the annotations during remote supervision introduce noise, such as multiple entity relationships or no relationship between two entities—this leads to noisy labels, which negatively impact subsequent model training. Therefore, in this embodiment of the invention, to remove the influence of noise labels caused by remote supervision, the obtained sample dataset can be processed to distinguish between noisy and clean data, thereby obtaining noisy sample text and clean sample text.

[0063] Here, noisy sample text refers to sample text where the entity relationships are biased or incorrectly corresponded; clean sample text, on the other hand, is the sample text where the entity relationships are correctly corresponded, that is, the entity relationships are completely consistent with the sample text.

[0064] In traditional solutions, when faced with noisy data, all noisy samples are typically discarded to avoid interference during subsequent model training. However, directly filtering out noisy samples leads to the loss of useful information, resulting in insufficient learning and poor training performance. Therefore, in this embodiment of the invention, after distinguishing between noisy sample text and clean sample text, the labels of the noisy sample text can be selectively discarded to remove the labels and retain the samples, thus obtaining noisy samples without labels. This approach avoids interference while fully preserving the effective information in the noisy samples.

[0065] To distinguish between noisy and clean sample texts, the K-nearest neighbor method can be used. This method separates noisy and clean data in the sample dataset and divides the dataset into labeled and unlabeled datasets.

[0066] Then, noise-robust semi-supervised learning can be used to learn relation extraction capabilities from labeled and unlabeled datasets.

[0067] Understandably, after obtaining noisy sample text and clean sample text, the initial model can learn based on these noisy sample text and clean sample text to learn better relation features, thereby enabling more accurate output and predicting more accurate entity relations to complete model training and obtain a trained relation extraction model.

[0068] Specifically, after distinguishing between noisy and clean samples, the initial model can be used to process them. That is, the hidden features of the noisy and clean sample texts can be extracted by the initial model. The initial model here is the initial relation extraction model, which is built on the relation extraction model based on the semi-supervised learning framework. Moreover, it is oriented towards sentence-level remote supervised relation extraction.

[0069] Specifically, the initial relation extraction model is used to extract features from both noisy and clean sample texts to extract the effective textual information contained therein for subsequent relation prediction, thereby obtaining their respective hidden features. That is, the noisy sample text and the clean sample text can be input into the initial relation extraction model, and the initial relation extraction model extracts features from the input sample texts to obtain the hidden features of the noisy sample text, i.e., the noise hidden features, and the hidden features of the clean sample text, i.e., the clean hidden features.

[0070] Considering that the labels of the noisy sample text have been discarded during the aforementioned discrimination process, and it is unlabeled data, in order to ensure that the useful information in the noisy sample text can be used in the subsequent model training process, in this embodiment of the invention, pseudo-labels can be generated for the noisy sample text, so as to make full use of the information in the noisy sample text through pseudo-labels, thereby better performing model training.

[0071] Specifically, after extracting the noise hidden features, pseudo-labels can be generated based on these noise hidden features to obtain pseudo-labels for the noise sample text, i.e., the predicted entity relationships of the noise sample text. Here, specifically, after obtaining the noise hidden features of the noise sample text, an initial relation extraction model can be applied to predict the relationship between entities in the noise sample text, thereby obtaining the predicted entity relationships.

[0072] Then, based on the predicted entity relationships of the noisy sample text, and the pure hidden features and entity relationships of the pure sample text, the model can be trained to obtain the trained entity relationship extraction model. Specifically, supervised learning is used to learn from the labeled dataset. Supervised learning makes the model more robust to noise. Unsupervised learning is used to learn from the unlabeled dataset, that is, using pseudo-labels to learn, so that the model can make full use of the information in the noisy sample text, thereby fully learning the relationship features, optimizing model training, and improving model performance.

[0073] Specifically, the predicted entity relationships from noisy sample texts, as well as the clean hidden features and entity relationships from clean sample texts, can be used to measure the model's learning loss from labeled and unlabeled datasets, respectively, thus obtaining supervised and unsupervised losses. Based on these two losses, the initial relation extraction model is iterated to minimize the loss of the model in both supervised and unsupervised learning processes, enabling it to learn more accurate relation features. This allows for the output of entity relationships corresponding to the input text corpus in subsequent applications using accurate relation features.

[0074] The relation extraction model training method provided by this invention identifies noisy sample text and clean sample text from a sample dataset. Using an initial relation extraction model, it determines the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text, respectively. Based on the noise hidden features, it determines the predicted entity relationships of the noisy sample text. Based on the predicted entity relationships of the noisy sample text, the clean hidden features of the clean sample text, and the sample entity relationships, it iterates the parameters of the initial relation extraction model to obtain the relation extraction model. This overcomes the current shortcomings of discarding noisy samples, which leads to the loss of useful information and insufficient model training, resulting in poor performance. By using pseudo-labels, it fully utilizes the information in the noisy sample text, enabling the model to better learn relation features, thereby optimizing the training effect and improving model performance.

[0075] Based on the above embodiments, the initial relationship extraction model includes a feature extraction model, a first classification model, and a second classification model; the predicted entity relationship includes a first predicted entity relationship and a second predicted entity relationship; step 120 includes:

[0076] Based on the feature extraction model in the initial relation extraction model, the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text are determined respectively.

[0077] Based on the first classification model in the initial relation extraction model and the noise hidden features, the first predicted entity relation of the noise sample text is determined.

[0078] Based on the second classification model in the initial relation extraction model and the noise hiding features, the second predicted entity relation of the noise sample text is determined.

[0079] Specifically, in step 120, the process of determining the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text based on the initial relation extraction model, and determining the predicted entity relations of the noisy sample text based on the noise hidden features, specifically includes:

[0080] Figure 2 This is a schematic diagram illustrating the training process of the relation extraction model provided by the present invention, as shown below. Figure 2 As shown, the initial relation extraction model includes a feature extraction model θ, a first classification model φ, and a second classification model ψ. The feature extraction model θ is used to extract the hidden features of the noisy sample text y and the clean sample text x, so as to obtain the noise hidden features of the noisy sample text u and the clean hidden features of the clean sample text x.

[0081] Correspondingly, compared to the three classification heads of traditional models, the initial relation extraction model in this embodiment of the invention includes two different classification models, namely the first classification model φ and the second classification model ψ. φ is used for training labeled data and generating pseudo-labels for unlabeled data, while ψ is used for training unlabeled data using pseudo-labels.

[0082] Here, the first classification model φ is used to determine the first predicted entity relationship of the noisy sample text u based on the noise hiding features. The second classification model ψ is used to determine the second predicted entity relation of the noisy sample text u based on the noise hiding features. The first predicted entity relation... Together with the second predicted entity relation, they constitute the predicted entity relation of the noisy sample text u.

[0083] Specifically, from the sample dataset After identifying the noisy sample text u and the clean sample text x, the feature extraction model θ in the initial relation extraction model can be used to determine the noise hidden features of the noisy sample text u and the clean hidden features of the clean sample text x. That is, the noisy sample text u and the clean sample text x are respectively input into the feature extraction model θ so that θ can extract features from the input sample text, thereby obtaining the noise hidden features (Weak augmented feature and Strong augmented feature) and the clean hidden feature (Hidden feature) output by θ.

[0084] Then, entity relationship prediction can be performed based on the noise-hidden features output by θ. That is, based on the noise-hidden features, the first classification model φ is applied to predict entity relationships, so as to obtain the first predicted entity relationship of the noise sample text u output by φ. Simultaneously, based on the noise-hiding features, a second classification model χ can be applied to predict entity relationships, thereby obtaining the second predicted entity relationships for the output noise sample text u. Combining the first and second predicted entity relationships, the predicted entity relationships for the noise sample text u can be determined.

[0085] Based on the above embodiments, based on the first classification model in the initial relation extraction model and the noise hiding features, the first predicted entity relation of the noise sample text is determined; based on the second classification model in the initial relation extraction model and the noise hiding features, the second predicted entity relation of the noise sample text is determined, including:

[0086] Based on the first classification model and the first enhanced feature, the first predicted entity relationship of the noisy sample text is determined;

[0087] Based on the second classification model and the second enhanced features, the second predicted entity relationship of the noisy sample text is determined;

[0088] The first enhanced feature and the second enhanced feature are noise hidden features determined by the feature extraction model based on noisy sample text under the first neuron loss rate and the second neuron loss rate, respectively; the first neuron loss rate is less than the second neuron loss rate.

[0089] Specifically, the process of determining the first predicted entity relationship of the noisy sample text based on the first classification model and the noise hiding features, and determining the second predicted entity relationship of the noisy sample text based on the second classification model and the noise hiding features, specifically includes:

[0090] In the process of extracting features from the noisy sample text u using the feature extraction model θ to obtain noise-hidden features, in order to enhance the robustness of the model, make the model more robust to noise, and improve its generalization performance, data augmentation can be performed in this embodiment of the invention. That is, multiple different representations of the noisy sample text can be obtained by using the Dropout technique, that is, multiple different noise-hidden features can be obtained. Furthermore, the intensity of the augmentation can be adjusted by adjusting the Dropout rate, thereby obtaining noise-hidden features of different intensities.

[0091] Specifically, when using the feature extraction model θ to extract noisy hidden features, different neuron dropout rates can be set so that the feature extraction model θ outputs different noise hidden features for the input noisy sample text under different neuron dropout rates, i.e., noise hidden features with different enhancement intensities, which are the first augmented feature (Weak augmented feature) and the second augmented feature (Strong augmented feature) in this case. The enhancement intensity of the first augmented feature is weaker than that of the second augmented feature. In other words, the first neuron dropout rate of the feature extraction model θ corresponding to the first augmented feature is less than the second neuron dropout rate of the feature extraction model θ corresponding to the second augmented feature.

[0092] After this, entity relationship prediction can be performed using the first and second enhancement features respectively to obtain the first and second predicted entity relationships. That is, based on the first enhancement features, the first classification model φ can be applied to predict entity relationships to obtain the first predicted entity relationship of the noisy sample text u output by the first classification model φ. Based on the second enhanced features, the second classification model ψ is applied to predict entity relationships, so as to obtain the second predicted entity relationships of the noise sample text u output by the second classification model ψ.

[0093] Based on the above embodiments, step 130 includes:

[0094] Based on the first and second predicted entity relationships in the predicted entity relationships, determine the unlabeled sample loss;

[0095] By performing feature mixing on the pure hidden features of different pure sample texts, multiple pure mixed features are obtained;

[0096] Based on each pure mixture feature and its corresponding sample entity relationship, determine the contrast loss;

[0097] Based on unlabeled sample loss and contrastive loss, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model.

[0098] Specifically, in step 130, the process of iterating the parameters of the initial relation extraction model based on the predicted entity relationships of the noisy sample text and the clean hidden features and entity relationships of the clean sample text to obtain the relation extraction model may include:

[0099] First, the loss of the initial relation extraction model in the unsupervised learning process can be measured based on the predicted entity relationships in the noisy sample text, thus obtaining the unsupervised loss, which is the loss of learning from the unlabeled dataset using unsupervised learning, also known as the unlabeled sample loss. Specifically, this can be achieved by measuring the difference between the first predicted entity relationship predicted by the first classification model and the second predicted entity relationship predicted by the second classification model in the unsupervised learning process, thereby obtaining the unlabeled sample loss of the model.

[0100] Meanwhile, the pure hidden features and entity relationships of the pure sample text can be used to measure the loss of the initial relation extraction model in the supervised learning process, thereby obtaining the supervised loss, which is the loss of learning from the labeled dataset in a supervised manner. However, considering that the pure sample text obtained by screening is not perfect, that is, it is not a truly noise-free sample text, but may still contain a small amount of noise, and in the presence of noise, directly training the model will lead to noise memory.

[0101] In this embodiment of the invention, when performing supervised learning, hybrid supervised contrastive learning can be used to learn relation features from labeled data, and the loss function of supervised contrastive learning can be used to measure the loss of the initial relation extraction model in the supervised learning process, thereby obtaining the contrastive loss of the initial relation extraction model.

[0102] Specifically, this can involve first performing feature mixing to obtain multiple different mixed features. That is, feature mixing can be performed on the pure hidden features of different pure sample texts to obtain multiple distinct pure mixed features. The sample entity relationships of the pure sample texts corresponding to each pure hidden feature involved in the feature mixing can be used as the sample entity relationships corresponding to the pure mixed features obtained from the feature mixing. In short, the sample entity relationships corresponding to each pure mixed feature include the sample entity relationships of the pure sample texts corresponding to each pure hidden feature before mixing. Subsequently, based on each pure mixed feature and its corresponding sample entity relationships, the loss of the model during the mixed-supervised contrastive learning process can be measured, thus obtaining the model's contrastive loss.

[0103] Then, based on the model's loss in supervised and unsupervised learning processes, the initial relation extraction model can be trained to obtain a trained relation extraction model. This involves combining the unlabeled sample loss and the contrastive loss to determine the overall loss during model training. Specifically, the overall loss of the model can be calculated by weighted summation based on the unlabeled sample loss and the contrastive loss. Then, based on this overall loss, the parameters of the initial relation extraction model can be iterated. Specifically, the standard gradient descent method can be used to update the parameters so that the loss of the updated model in supervised and unsupervised learning processes can be minimized as much as possible, enabling it to learn more accurate relation features. Finally, a trained relation extraction model can be obtained, which can then be used in subsequent applications to output entity relations corresponding to the input text corpus using the accurate relation features learned by the relation extraction model.

[0104] Furthermore, it should be noted that after training the relation extraction model using the sample dataset, in order to verify the performance of the trained relation extraction model, this embodiment of the invention can also perform testing and verification. The specific process includes the following: S1, obtaining a validation dataset and a test dataset, wherein the validation dataset can be obtained through a remote supervision method, and the test dataset can be determined through manual annotation, thus ensuring the accuracy of the test; S2, using the validation dataset to validate the model, and selecting the model with the best performance on the validation dataset as the final model based on the accuracy of the prediction on the validation dataset; S3, testing this model on the test set, and evaluating the prediction accuracy and precision. The following table shows the test results of the relation extraction model:

[0105]

[0106] In this context, Prec (Precision) and Rec (Recall) represent the model's precision and recall, respectively, while the F1 score is the harmonic mean of precision and recall.

[0107] Here, accuracy measures the model's accuracy in predicting positive classes; recall measures the model's ability to find all positive class samples; and the F1 score is used to balance accuracy and recall. When the F1 score is high, the model can maintain high accuracy while having good recall.

[0108] As can be seen, the relation extraction model provided by this invention outperforms current mainstream prediction models on most metrics of the test datasets NYT10m and Wiki20m.

[0109] Based on the above embodiments, the unlabeled sample loss is determined using the following formula:

[0110]

[0111] in, For unlabeled samples, μB represents the number of noisy text samples u in each batch. Denotes the first predicted entity relationship, θ s,t To use the feature extraction model for the t-th iteration with the second neuron loss rate, ψ t For the second classification model in the t-th iteration, Let l represent the second predicted entity relationship, and l be the cross-entropy loss function. σ t (c) represents the number of noisy sample texts with a prediction confidence greater than the threshold τ in the first predicted entity relation corresponding to relation category c.

[0112] Based on the above embodiments, the first predicted entity relationship is determined based on the following formula:

[0113]

[0114] Where φ is the first classification model, θ w,t This is a feature extraction model for the t-th iteration using the first neuron loss rate, where u is the noisy sample text. This is a function composition operator. Function composition refers to using the output of one function as the input of another function. For example, for functions f(m) and g(m), This means first calculating f(m) with respect to q, and then calculating g(f(m)) with respect to the result f(m) from the previous step.

[0115] Based on the above embodiments, the comparison loss is determined using the following formula:

[0116]

[0117]

[0118] z i =λz a +(1-λ)z b

[0119]

[0120] y i ∈{y a ,y b}

[0121] in, To compare the loss, N is the number of clean sample texts, and λ can be randomly sampled from the Beta distribution, λ∈[0,1]~Beta(α) m ,α m), α m For hyperparameters, z i For z a and z b The mixed characteristics, i.e., the pure mixed characteristics, z a and z b The clean hidden features of different clean sample texts, y a To hide the pure feature z a The corresponding pure sample text's sample entity relations, y b To hide the pure feature z b The corresponding pure sample text's sample entity relations, y a and y b All are pure mixed characteristics z i The corresponding sample entity relationship, Let y represent the entity relationship of a sample in a batch. i The number of clean sample texts, z j Let y be the entity relationship of the sample. j The pure hidden features of the pure sample text, y j With y i Same, z r Let y be the entity relationship of the sample. r The pure hidden features of the pure sample text, y r Can be with y i They can be the same or different; τ is the temperature hyperparameter.

[0122] Based on the above embodiments, step 110, which involves determining noisy sample text and clean sample text from the sample dataset, includes:

[0123] Based on the initial relation extraction model, the hidden features of each sample text in the sample dataset are determined;

[0124] Based on the hidden features of each sample text, determine the K nearest neighbor graph corresponding to each sample text;

[0125] Based on the K-nearest neighbor graph, noisy sample text and clean sample text are identified from each sample text, and the sample entity relations of the noisy sample text are discarded.

[0126] Specifically, step 110, the process of determining noisy sample text and clean sample text from the sample dataset, may include:

[0127] First, the initial relation extraction model can be used to determine the hidden features of each sample text in the sample dataset. Specifically, the initial relation extraction model is used to extract features from all sample texts in the sample dataset to extract the effective text information contained therein, thereby obtaining the hidden features of each sample text. That is, each sample text can be input into the initial relation extraction model, and the initial relation extraction model extracts features from each input sample text to obtain the hidden features of each sample text.

[0128] Subsequently, the K-nearest neighbor graph corresponding to each sample text can be determined through the hidden features of each sample text. That is, the K-NN (K-Nearest Neighbor Graph) graph of all sample texts can be constructed using the hidden features of each sample text. Then, the noisy sample text and clean sample text can be separated from each sample text through the K-NN graph. In other words, the noisy sample text and clean sample text can be obtained by identifying the noisy sample text and clean sample text from the graph structure of the K-NN graph.

[0129] After obtaining the noisy sample text and the clean sample text, the labels of the noisy sample text can be filtered out from the sample dataset, while the labels of the clean sample text can be retained. That is, the sample entity relations of the noisy sample text are discarded to construct an unlabeled dataset, while the sample entity relations of the clean sample text are retained to construct a labeled dataset.

[0130] Based on the above embodiments, the process of screening clean sample texts can be represented by the following formula:

[0131]

[0132] In the formula, Based on sample texts i The entity relationships are predicted from the hidden features. For sample texts i K's nearest neighbor, s k For sample texts i Sample text from the K nearest neighbors, For sample texts k The sample entity relationships, where c represents the relationship category, are calculated using the cross-entropy loss function. and The difference between noisy and clean samples is used to distinguish them, and a labeled dataset is constructed, which is represented as follows:

[0133]

[0134] in, The labeled dataset represents the pure sample text corresponding to relation category c. For sample texts i The sample entity relationships, l is the cross-entropy loss function, γ c is the threshold corresponding to relation category c, and is a hyperparameter. C represents the set of all relation categories, which includes multiple relation categories.

[0135] The training apparatus for the relation extraction model provided by the present invention will be described below. The training apparatus for the relation extraction model described below and the training method for the relation extraction model described above can be referred to in correspondence.

[0136] Figure 3 This is a schematic diagram of the structure of the training device for the relation extraction model provided by the present invention, as shown below. Figure 3 As shown, the device includes:

[0137] The determining unit 310 is used to determine a sample dataset and to determine noisy sample text and clean sample text from the sample dataset. The sample dataset contains multiple sample texts and sample entity relationships corresponding to each sample text.

[0138] The prediction unit 320 is used to determine the noise hidden features of the noise sample text and the clean hidden features of the clean sample text based on the initial relation extraction model, and to determine the predicted entity relations of the noise sample text based on the noise hidden features.

[0139] Training unit 330 is used to perform parameter iteration on the initial relation extraction model based on the predicted entity relations of the noisy sample text, as well as the clean hidden features and sample entity relations of the clean sample text, to obtain the relation extraction model.

[0140] The relation extraction model training device provided by this invention identifies noisy sample text and clean sample text from a sample dataset. Using an initial relation extraction model, it determines the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text, respectively. Based on the noise hidden features, it determines the predicted entity relationships of the noisy sample text. Based on the predicted entity relationships of the noisy sample text, the clean hidden features of the clean sample text, and the sample entity relationships, it iterates the parameters of the initial relation extraction model to obtain the relation extraction model. This overcomes the current shortcomings of discarding noisy samples, which leads to the loss of useful information and insufficient model training, resulting in poor performance. By using pseudo-labels, it fully utilizes the information in the noisy sample text, enabling the model to better learn relation features, thereby optimizing the training effect and improving model performance.

[0141] Based on the above embodiments, the initial relationship extraction model includes a feature extraction model, a first classification model, and a second classification model; the predicted entity relationship includes a first predicted entity relationship and a second predicted entity relationship; the prediction unit 320 is used for:

[0142] Based on the feature extraction model in the initial relation extraction model, the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text are determined respectively.

[0143] Based on the first classification model in the initial relation extraction model and the noise hiding features, the first predicted entity relation of the noise sample text is determined;

[0144] Based on the second classification model in the initial relation extraction model and the noise hiding features, the second predicted entity relation of the noise sample text is determined.

[0145] Based on the above embodiments, the prediction unit 320 is used for:

[0146] Based on the first classification model and the first enhanced feature, the first predicted entity relationship of the noisy sample text is determined;

[0147] Based on the second classification model and the second enhanced features, the second predicted entity relationship of the noisy sample text is determined;

[0148] The first enhancement feature and the second enhancement feature are respectively the noise hidden features determined by the feature extraction model based on the noise sample text under the first neuron loss rate and the second neuron loss rate;

[0149] The loss rate of the first neuron is less than the loss rate of the second neuron.

[0150] Based on the above embodiments, the training unit 330 is used for:

[0151] Based on the first and second predicted entity relationships in the predicted entity relationships, the unlabeled sample loss is determined;

[0152] By performing feature mixing on the pure hidden features of different pure sample texts, multiple pure mixed features are obtained;

[0153] Based on each pure mixture feature and its corresponding sample entity relationship, determine the contrast loss;

[0154] Based on the unlabeled sample loss and the contrast loss, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model.

[0155] Based on the above embodiments, the unlabeled sample loss is determined using the following formula:

[0156]

[0157] in, The loss is for unlabeled samples, where μB is the number of noisy text samples u. Denotes the first predicted entity relationship, θ s,t To use the feature extraction model for the t-th iteration with the second neuron loss rate, ψ t For the second classification model in the t-th iteration, Let l represent the second predicted entity relationship, and l be the cross-entropy loss function. σ t (c) represents the number of noisy sample texts with a prediction confidence greater than the threshold τ in the first predicted entity relation corresponding to relation category c.

[0158] Based on the above embodiments, the comparison loss is determined using the following formula:

[0159]

[0160]

[0161] x i =λz a +(1-λ)z b

[0162]

[0163] y i ∈{y a ,y b}

[0164] in, To compare the loss, N is the number of clean sample texts, and λ can be randomly sampled from the Beta distribution, λ∈[0,1]~Beta(α) m ,α m ), α m For hyperparameters, Z i For z a and z b The mixed characteristics, i.e., the pure mixed characteristics, z a and z b The clean hidden features of different clean sample texts, y a To hide the pure feature z a The corresponding pure sample text's sample entity relations, y b To hide the pure feature z b The corresponding pure sample text's sample entity relations, y a and y b All are pure mixed characteristics z iThe corresponding sample entity relationship, Let y represent the entity relationship of a sample in a batch. i The number of clean sample texts, z j Let y be the entity relationship of the sample. j Pure hidden features of pure sample text, z r Let y be the entity relationship of the sample. r The pure hidden features of the pure sample text, y r =y i or y r ≠y i τ is the temperature hyperparameter.

[0165] Based on the above embodiments, the determining unit 310 is used for:

[0166] Based on the initial relation extraction model, the hidden features of each sample text in the sample dataset are determined;

[0167] Based on the hidden features of each sample text, the K nearest neighbor graph corresponding to each sample text is determined;

[0168] Based on the K-nearest neighbor graph, noisy sample texts and clean sample texts are determined from each sample text, and the sample entity relationships of the noisy sample texts are discarded.

[0169] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a training method for a relation extraction model. This method includes: determining a sample dataset, and from the sample dataset, determining noisy sample text and clean sample text, wherein the sample dataset contains multiple sample texts and corresponding sample entity relations for each sample text; based on an initial relation extraction model, determining the noise hiding features of the noisy sample text and the clean hiding features of the clean sample text, and based on the noise hiding features, determining the predicted entity relations of the noisy sample text; and based on the predicted entity relations of the noisy sample text, the clean hiding features of the clean sample text, and the sample entity relations, performing parameter iteration on the initial relation extraction model to obtain a relation extraction model.

[0170] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0171] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute a training method for the relation extraction model provided by the above methods, the method comprising: determining a sample dataset, and determining noisy sample text and clean sample text from the sample dataset, the sample dataset containing multiple sample texts and sample entity relations corresponding to each sample text; determining, based on an initial relation extraction model, the noise hiding features of the noisy sample text and the clean hiding features of the clean sample text, and determining the predicted entity relations of the noisy sample text based on the noise hiding features; and performing parameter iteration on the initial relation extraction model based on the predicted entity relations of the noisy sample text, the clean hiding features of the clean sample text, and the sample entity relations, to obtain a relation extraction model.

[0172] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a training method for the relation extraction model provided by the methods described above. This method includes: determining a sample dataset, and from the sample dataset, determining noisy sample text and clean sample text, wherein the sample dataset contains multiple sample texts and sample entity relations corresponding to each sample text; based on an initial relation extraction model, determining the noise hiding features of the noisy sample text and the clean hiding features of the clean sample text, and based on the noise hiding features, determining the predicted entity relations of the noisy sample text; and based on the predicted entity relations of the noisy sample text, the clean hiding features of the clean sample text, and the sample entity relations, performing parameter iteration on the initial relation extraction model to obtain a relation extraction model.

[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a relation extraction model, characterized in that, include: A sample dataset is determined, and noisy sample text and clean sample text are identified from the sample dataset. The sample dataset contains multiple sample texts and sample entity relationships corresponding to each sample text. Based on the initial relation extraction model, the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text are determined respectively, and the predicted entity relations of the noisy sample text are determined based on the noise hidden features. Based on the predicted entity relationships of the noisy sample text, and the clean hidden features and entity relationships of the clean sample text, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model. The method of predicting entity relationships based on the noisy sample text, and obtaining a relation extraction model by iterating the parameters of the initial relation extraction model using the clean hidden features and entity relationships of the clean sample text, includes: Based on the first and second predicted entity relationships in the predicted entity relationships, the unlabeled sample loss is determined; By performing feature mixing on the pure hidden features of different pure sample texts, multiple pure mixed features are obtained; Based on each pure mixture feature and its corresponding sample entity relationship, determine the contrast loss; Based on the unlabeled sample loss and the contrast loss, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model; The unlabeled sample loss is determined based on the following formula: ； in, For unlabeled sample loss, For noise sample text The number, Indicates the first predicted entity relationship. To use the second neuron loss rate as the first The feature extraction model of the next iteration. For the first The second classification model in the next iteration Indicates the second predicted entity relationship. Let cross-entropy be the loss function. , Represents relation categories In the corresponding first predicted entity relationship, the prediction confidence is greater than the threshold. The number of noisy sample texts; The contrast loss is determined based on the following formula: ； ； ； ； ； in, To compare the losses, The number of clean sample texts. From Obtained by random sampling from the distribution. , For superparameters, for and The mixed characteristics, i.e., the pure mixed characteristics, and These are the clean hidden features of different clean sample texts. To hide features for purity The corresponding pure sample text's sample entity relationships, To hide features for purity The corresponding pure sample text's sample entity relationships, and All are characterized by pure mixture. The corresponding sample entity relationship, This indicates that the entity relationships in a batch are... The number of clean sample texts, The entity relationship of the sample is Pure hidden features of pure sample text The entity relationship of the sample is Pure hidden features of pure sample text or , This is for temperature hyperparameters.

2. The training method for the relation extraction model according to claim 1, characterized in that, The initial relationship extraction model includes a feature extraction model, a first classification model, and a second classification model; the predicted entity relationship includes a first predicted entity relationship and a second predicted entity relationship. The initial relation extraction model determines the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text, and determines the predicted entity relations of the noisy sample text based on the noise hidden features, including: Based on the feature extraction model in the initial relation extraction model, the noise hidden features of the noisy sample text and the clean hidden features of the clean sample text are determined respectively. Based on the first classification model in the initial relation extraction model and the noise hiding features, the first predicted entity relation of the noise sample text is determined; Based on the second classification model in the initial relation extraction model and the noise hiding features, the second predicted entity relation of the noise sample text is determined.

3. The training method for the relation extraction model according to claim 2, characterized in that, The determination of the first predicted entity relationship of the noisy sample text based on the first classification model in the initial relation extraction model and the noise hiding features; and the determination of the second predicted entity relationship of the noisy sample text based on the second classification model in the initial relation extraction model and the noise hiding features, including: Based on the first classification model and the first enhanced feature, the first predicted entity relationship of the noisy sample text is determined; Based on the second classification model and the second enhanced features, the second predicted entity relationship of the noisy sample text is determined; The first enhancement feature and the second enhancement feature are respectively the noise hidden features determined by the feature extraction model based on the noise sample text under the first neuron loss rate and the second neuron loss rate; The loss rate of the first neuron is less than the loss rate of the second neuron.

4. The training method for the relation extraction model according to any one of claims 1 to 3, characterized in that, The step of determining noisy sample text and clean sample text from the sample dataset includes: Based on the initial relation extraction model, the hidden features of each sample text in the sample dataset are determined; Based on the hidden features of each sample text, the K nearest neighbor graph corresponding to each sample text is determined; Based on the K-nearest neighbor graph, noisy sample texts and clean sample texts are determined from each sample text, and the sample entity relationships of the noisy sample texts are discarded.

5. A training device for a relation extraction model, characterized in that, include: A determining unit is used to determine a sample dataset and, from the sample dataset, determine noisy sample text and clean sample text, wherein the sample dataset contains multiple sample texts and sample entity relationships corresponding to each sample text; The prediction unit is used to determine the noise hidden features of the noise sample text and the clean hidden features of the clean sample text based on the initial relation extraction model, and to determine the predicted entity relations of the noise sample text based on the noise hidden features. The training unit is used to perform parameter iteration on the initial relation extraction model based on the predicted entity relations of the noisy sample text, as well as the clean hidden features and sample entity relations of the clean sample text, to obtain the relation extraction model. The method of predicting entity relationships based on the noisy sample text, and obtaining a relation extraction model by iterating the parameters of the initial relation extraction model using the clean hidden features and entity relationships of the clean sample text, includes: Based on the first and second predicted entity relationships in the predicted entity relationships, the unlabeled sample loss is determined; By performing feature mixing on the pure hidden features of different pure sample texts, multiple pure mixed features are obtained; Based on each pure mixture feature and its corresponding sample entity relationship, determine the contrast loss; Based on the unlabeled sample loss and the contrast loss, the parameters of the initial relation extraction model are iterated to obtain the relation extraction model; The unlabeled sample loss is determined based on the following formula: ； in, For unlabeled sample loss, For noise sample text The number, Indicates the first predicted entity relationship. To use the second neuron loss rate as the first The feature extraction model of the next iteration. For the first The second classification model in the next iteration Indicates the second predicted entity relationship. Let cross-entropy be the loss function. , Represents relation categories In the corresponding first predicted entity relationship, the prediction confidence is greater than the threshold. The number of noisy sample texts; The contrast loss is determined based on the following formula: ； ； ； ； ； in, To compare the losses, The number of clean sample texts. From Obtained by random sampling from the distribution. , For superparameters, for and The mixed characteristics, i.e., the pure mixed characteristics, and These are the clean hidden features of different clean sample texts. To hide features for purity The corresponding pure sample text's sample entity relationships, To hide features for purity The corresponding pure sample text's sample entity relationships, and All are characterized by pure mixture. The corresponding sample entity relationship, This indicates that the entity relationships in a batch are... The number of clean sample texts, The entity relationship of the sample is Pure hidden features of pure sample text The entity relationship of the sample is Pure hidden features of pure sample text or , This is for temperature hyperparameters.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the training method for the relation extraction model as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method for the relation extraction model as described in any one of claims 1 to 4.

Images