Cross-domain small sample relationship extraction method, device and electronic equipment

Abstract

The present invention provides a method, apparatus, and electronic device for extracting cross-domain small-sample relationships. The method comprises: performing domain sampling on multiple source domains to partition each domain into a pseudo-visible domain and a pseudo-invisible domain; applying a global parameter update strategy to update the meta-knowledge of the pseudo-visible domain; performing transfer learning based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge; and predicting relationship labels for a target domain based on the meta-knowledge and the updated transferable knowledge. Through the above approach, the present invention can adapt to distribution differences between source and target domains and improve generalization capabilities for unseen target domains.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and specifically relates to a cross-domain small sample relationship extraction method, device and electronic equipment. Background Art

[0002] The goal of relation extraction (RE) is to predict the relationship between two entities in a given context. For example, given the sentence "London is the capital of the UK," capital of represents the relation label, London represents the head entity, and the UK represents the tail entity. A relation extraction model can recognize that the relationship between London and the UK is capital of. Relation extraction is a cornerstone of natural language processing, providing fundamental support for downstream tasks such as intelligent search, semantic analysis, and automated question answering.

[0003] Traditional kernel-, feature-vector-, and neural-network-based relationship extraction methods require large amounts of labeled data for training, making them difficult to generalize to real-world applications. Furthermore, the ever-increasing number of new relationships and entities easily makes relationship extraction models face the challenge of data scarcity. To address these issues, meta-learning (ML) is proposed to guide models to rapidly learn new concepts using previously learned knowledge. Specifically, ML models first construct a series of richly labeled meta-tasks and then generalize to data-scarce test tasks by learning similarity metrics or better parameters.

[0004] However, traditional ML-based relation extraction models aim to address the small sample problem in the same domain, where both training and testing relation categories come from the same domain. Unfortunately, in some specialized domains, such as biomedicine, data annotation requires significant expertise and time, making it difficult to collect sufficient labeled data for meta-learning methods. Therefore, transferring meta-knowledge from the data-rich source domain to the data-scarce target domain is a practical challenge for relation extraction. The cross-domain few-shot (CrossFew) relation extraction (RE) task can be applied in two scenarios: domain generalization (DG) and domain adaptation (DA). In the DG scenario, the relation extraction model cannot obtain any data from the target domain during training. In the DG scenario, the relation extraction model can obtain unlabeled samples or a small number of labeled samples from the target domain during training. Traditionally, cross-domain problems have been addressed by directly employing adversarial training or simply using meta-learning methods. However, these methods are not effective for cross-domain small-sample relation extraction tasks, treating each task as coming from the same data distribution in the same domain and failing to generalize to unseen domains. Existing relation extraction models crudely combine the relationship labels of the source and target domains into a large set of relationship labels, which is then trained using the data setting of a label-level meta-task, ignoring the distribution differences between data from different domains. Furthermore, all methods directly update all large-scale parameters to achieve a global optimum. However, this parameter update strategy is not suitable for cross-domain small-sample relation extraction tasks because it causes the relation model to converge to the optimal point in the source domain with a larger amount of data, resulting in an optimization gap with the target domain. Summary of the Invention

[0005] The present invention provides a cross-domain small sample relationship extraction method, device and electronic device to solve the problem that the existing cross-domain small sample relationship extraction regards each task as coming from the same data distribution in the same field and fails to generalize it to unseen fields.

[0006] Based on the above objectives, the present invention proposes a cross-domain small sample relationship extraction method, comprising: performing domain sampling on multiple source domains to divide them into a pseudo-visible domain and a pseudo-invisible domain respectively; applying a global parameter update strategy to update the meta-knowledge of the pseudo-visible domain; performing transfer learning operations based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge; and predicting relationship labels of the target domain based on the meta-knowledge and the updated transferable knowledge.

[0007] Optionally, performing domain sampling on multiple source domains to respectively divide a pseudo-visible domain and a pseudo-invisible domain includes: performing domain sampling on multiple source domains and randomly selecting two candidate domains therefrom; performing domain sampling on the two candidate domains to respectively divide a pseudo-visible domain and a pseudo-invisible domain.

[0008] Optionally, the application of a global parameter update strategy to update the meta-knowledge of the pseudo-visible domain includes: performing label-level meta-task sampling on the pseudo-visible domain and the pseudo-invisible domain to construct label-level meta-tasks, wherein the label-level meta-tasks include pseudo-visible meta-tasks and pseudo-invisible meta-tasks; and applying a relation extraction model according to the pseudo-visible meta-tasks to update the meta-knowledge of the pseudo-visible domain.

[0009] Optionally, the pseudo-visible meta-task includes a support set and a query set, and the updating of the meta-knowledge of the pseudo-invisible domain by applying the relation extraction model according to the pseudo-visible meta-task includes: applying the encoder in the relation extraction model to map the support set instances and query set instances in the pseudo-visible meta-task to a low-dimensional vector space; applying the metric function in the relation extraction model to obtain the representation of the support set instances and query set instances in the pseudo-visible meta-task in the low-dimensional vector space, and predicting the relation labels of the query set instances in the pseudo-visible meta-task; calculating the cross-entropy loss function of the relation extraction model based on the predicted relation labels of the query set instances; and updating the meta-knowledge according to the cross-entropy loss function.

[0010] Optionally, the pseudo-invisible meta-task includes a support set and a query set, and the transfer learning operation is performed based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge, including: dividing the updated meta-knowledge into weight parameters and bias parameters; performing a transfer learning operation on the weight parameters and the bias parameters based on a feedforward neural network to obtain transferable knowledge of the pseudo-invisible domain; calculating the generalization ability of the relationship extraction model on the pseudo-invisible meta-task based on the updated meta-knowledge and the transferable knowledge of the pseudo-invisible domain; and updating the transferable knowledge according to the generalization ability.

[0011] Optionally, the transfer learning operation is performed on the weight parameters and the bias parameters based on the feedforward neural network to obtain transferable knowledge of the pseudo invisible domain, including: and the deviation parameter Apply the following relationship to calculate the transferable knowledge θ of the pseudo-unseen domain pu ;

[0012]

[0013] Among them, f trans(·) represents the transfer meta-learning operation, ⊙ represents vector multiplication, represents the scaling parameter, Indicates migration parameters.

[0014] Optionally, predicting the relationship label of the target domain based on the meta-knowledge and the updated transferable knowledge includes: predicting the relationship label corresponding to the query set instance of the target domain based on the meta-knowledge and the updated transferable knowledge, satisfying the following relationship:

[0015]

[0016] Among them, y pred is the predicted relation label, and represent the support instance and query instance of the test meta-task respectively, express The corresponding relationship label.

[0017] Based on the same inventive concept, the present invention also proposes a cross-domain small sample relationship extraction device, including: a network generation unit, used to obtain a drug-target symbol heterogeneous graph, and obtain a symbolic bipartite network, a drug interaction network and a target interaction network based on the drug-target symbol heterogeneous graph; an embedding result acquisition unit, used to obtain the node embedding results of drugs and targets based on the symbolic graph neural network model trained after the application of the symbolic bipartite network; a component analysis unit, used to perform principal component analysis on the drug interaction network and the target interaction network to obtain auxiliary information; a result prediction unit, used to splice the node embedding results with the auxiliary information, and apply a discriminator to predict the symbols between the drug and the target to obtain the cross-domain small sample relationship extraction results.

[0018] Based on the same inventive concept, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the cross-domain small sample relationship extraction method as described above is implemented.

[0019] Based on the same inventive concept, the present invention also proposes a computer storage medium, in which at least one executable instruction is stored, and the executable instruction enables a processor to execute the cross-domain small sample relationship extraction method as described above.

[0020] From the above, it can be seen that the beneficial effects of the technical solution provided by the present invention are: the present invention provides a cross-domain small sample relationship extraction method, device and electronic device, the method including: domain sampling of multiple source domains, respectively dividing a pseudo-visible domain and a pseudo-invisible domain; applying a global parameter update strategy to update the meta-knowledge of the pseudo-visible domain; performing transfer learning operations based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge; predicting the relationship label of the target domain based on the meta-knowledge and the updated transferable knowledge, which can adapt to the distribution differences between the source domain and the target domain, and improve the generalization ability of unseen target domains. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 Schematic diagram of the process of extracting cross-domain small sample relationships according to an embodiment of the present invention;

[0023] Figure 2 Schematic diagram of the structure of the generalized meta-learning framework of an embodiment of the present invention;

[0024] Figure 3 Schematic diagram of the phase update of the cross-domain small sample relationship extraction method according to an embodiment of the present invention;

[0025] Figure 4 Schematic diagram of the structure of a cross-domain small sample relationship extraction device according to an embodiment of the present invention;

[0026] Figure 5 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0028] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0029] The embodiment of the present invention implements a cross-domain small sample relationship extraction method, such as Figure 1 As shown in Figure 2, the cross-domain small sample relationship extraction method includes:

[0030] Step S11: performing domain sampling on multiple source domains to respectively divide a pseudo visible domain and a pseudo invisible domain.

[0031] The goal of relation extraction is to identify relationships between entities within a given sentence. Most traditional relation extraction models extract relations in a supervised setting and can be broadly categorized into three categories: feature vector-based methods, kernel function-based methods, and neural network-based methods. Feature vector-based methods typically focus on generating a series of features for the relation classifier, such as word features, syntactic features, and semantic features. Feature vector-based methods explicitly construct feature vectors, while kernel function-based methods implicitly calculate the similarity between two relation instances. These kernel functions include grammatical sequence kernels, tree kernels, dependency tree kernels, dependency graph path kernels, and composite kernels. Unlike feature vector-based or kernel function-based methods, which require manual intervention, neural network-based models often focus on extracting relational features using neural networks, such as convolutional neural networks (CNNs), graph convolutional networks, and recurrent neural networks, using end-to-end training. Generally, these methods can achieve good classification results with the help of large-scale annotated data. However, in some specialized domains, collecting large-scale data is time-consuming, labor-intensive, or even impossible. The present invention addresses the problem of relation extraction in scenarios where data is extremely scarce.

[0032] The embodiment of the present invention applies the generalized meta-learning framework (Geml) to perform cross-domain small sample relationship extraction tasks, and the structure is as follows: Figure 2As shown in the figure, the traditional label-level meta-task is expanded into a domain-level meta-task; this domain-level meta-task consists of two sub-tasks extracted from two visible source domains, namely the pseudo-visible task and the pseudo-unseen task. During training, the cross-domain process is simulated, and by explicitly learning the transfer process from the source domain to the target domain, the relation extraction model can adapt to the distribution difference between the source and target domains. The pseudo-visible task is then used to update the large-scale meta-learning parameters, and then the meta-learning parameters are transferred to the pseudo-unseen task through the transfer meta-learning operation, avoiding the model converging only to the source domain with a large amount of data and ignoring the target domain.

[0033] In the embodiment of the present invention, relation extraction can be expressed as a classification task, which is to identify the corresponding relation labels from the predefined relation label set R according to the entities in the given sentence. head ,…,e tail ,…,w n}, a relation extraction model A relation ternary {e head , e tail ,y}, that is

[0034] Data processing complies with the following conditions. The training, validation and test set data (X trian , X val and X test ) corresponding relation label R trian , R val and R test They are constructed from sets of relation labels from the same field:

[0035]

[0036] P(·) represents the feature distribution of the input data. As can be seen from the above formula (1), in traditional relation extraction tasks, the training, validation, and test set data come from the same relation labels and have the same data feature distribution.

[0037] In addition to the conditions in formula (1), meta-learning methods also need to meet more stringent conditions:

[0038]

[0039] Even though the relationship label sets corresponding to the training, validation, and test datasets are pairwise disjoint, the data is presented in the form of meta-tasks, that is, each meta-task T contains a support set S and a query set Q. The support set S contains N types of relationship labels, and each type has K instances (N-way K-shot), as follows:

[0040] S={X s ,Ys}={(x i,j ,y i )}, i=1,…,N, j=1,…,K (3)

[0041] Query set Q = {X q ,Y q} is constructed in the same way as the support set S. It is worth noting that the feature distributions of Q and S still obey the conditions in formula (1), and this meta-task is called a label-level meta-task.

[0042] Generally speaking, cross-domain small sample relation extraction can be divided into two types: domain generalization (DG) and domain adaptation (DA). Different from traditional relation extraction and small sample relation extraction, in cross-domain scenarios, X trian , X val and X test From different fields.

[0043] In the domain generalization (DG) scenario, in addition to formula (2), the model cannot access any target domain data during training and validation, and the data distribution follows the following formula:

[0044]

[0045] In cross-domain conditions, X train It may be composed of multiple source domain data, that is, X train ={D1,…D V}.

[0046] In the context of domain adaptation (DA), the model can be exposed to some unlabeled data or a small amount of labeled data from the target domain during training. Therefore, the condition in Equation (4) can be relaxed to

[0047] P(X trian )∩P(X test ),P(X val )∩P(X test )=P(X aux ), (5)

[0048] P(X aux ) represents unlabeled or labeled auxiliary data from the target domain. In summary, the generalized meta-learning framework of the present invention primarily improves the model's ability to generalize to unseen target domains in a domain generalization context. Furthermore, the performance of the generalized meta-learning framework is also evaluated in a domain adaptation context to verify its robustness.

[0049] In step S11, optionally, domain sampling is first performed on multiple source domains, and two candidate domains are randomly selected from them. Specifically, given V source domains {D1,…,D V}, using the domain sampler Randomly select two candidate domains from and

[0050]

[0051] Then, domain sampling is performed on the two candidate domains to respectively divide a pseudo visible domain and a pseudo invisible domain. The candidate domain and Divide into a pseudo visible domain D ps and a pseudo-invisible domain D pu ,Right now

[0052]

[0053] This allows the generalization meta-learning framework to explicitly simulate the process of migrating from the visible domain to the unseen target domain. ps and pseudo-invisible domain D pu The data distribution should satisfy formula (4), that is,

[0054] Step S12: applying a global parameter updating strategy to update the meta-knowledge of the pseudo visible domain.

[0055] In the embodiment of the present invention, firstly, label-level meta-task sampling is performed on the pseudo-visible domain and the pseudo-invisible domain to construct a label-level meta-task, wherein the label-level meta-task includes a pseudo-visible meta-task and a pseudo-invisible meta-task. Construct label-level meta-tasks:

[0056]

[0057] Tag-level meta-task sampler First, from D ps Extract N types of relationship labels, and then select K instances from each type of relationship label data to construct S ps .Q ps The construction method and S ps According to the pseudo visible domain D ps and pseudo-invisible domain D pu A domain-level meta-task can be constructed There are two label-level meta-tasks: pseudo-visible meta-task T ps ={S ps ,Q ps} and pseudo-invisible meta-task T pu ={S pu ,Q pu}composition.

[0058] Then, the meta-knowledge of the pseudo-visible domain is updated based on the pseudo-visible meta-task application relation extraction model. The parameter updates of the cross-domain small sample relation extraction model in the pseudo-visible domain and the pseudo-invisible domain cannot be treated equally. Therefore, the meta-knowledge θ is introduced. meta and transferable knowledge θ trans Represent the changes in pseudo-visible domain and pseudo-invisible domain respectively. In order to obtain general meta-knowledge, θ meta Through the large pre-trained language model BERT base initialization.

[0059] In an embodiment of the present invention, an encoder in the relationship extraction model is applied to map the support set instances and query set instances in the pseudo-visible meta-task to a low-dimensional vector space; a metric function in the relationship extraction model is applied to obtain the representation of the support set instances and query set instances in the pseudo-visible meta-task in the low-dimensional vector space, and predict the relationship labels of the query set instances in the pseudo-visible meta-task; the cross-entropy loss function of the relationship extraction model is calculated based on the predicted relationship labels of the query set instances; and the meta-knowledge is updated based on the cross-entropy loss function.

[0060] Meta-knowledge θ meta Only comes from pseudo-visible meta-task T ps , without considering cross-domain issues. Therefore, the relationship extraction model In the pseudo-visible domain T ps The model training can be regarded as a traditional small sample relationship extraction task, and the following Figure 3 The global parameter update strategy shown below updates the parameter θ meta , where w and b are weight parameters and d is the dimension. Specifically, the relation extraction model It consists of an encoder E and a metric function M. The encoder E first transforms T ps Support set instance in and queryset instance Mapping to a low-dimensional vector space, the metric function uses the support set instance and queryset instance Representing predicted query set instances in a low-dimensional vector space Relationship tags:

[0061]

[0062] express Belongs to relationship label The probability of cross entropy loss function L ps Used to evaluate the classification in the query set

[0063] mistake:

[0064]

[0065] So far, meta-knowledge θ meta The update can be expressed as

[0066]

[0067] α represents the learning rate of meta-knowledge gradient update, Represents meta-knowledge at t th The parameter value of the step.

[0068] Step S13: performing a transfer learning operation based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge.

[0069] In theory, the updated meta-knowledge And the global parameter update strategy can be directly used for the pseudo-unseen domain D pu However, the gap between the pseudo-visible domain and the pseudo-invisible domain in gradient distribution is large, and this global update strategy cannot guarantee the relationship extraction model Converges to the pseudo-invisible domain D pu Therefore, we introduce transferable knowledge θ trans Go to indicate domain migration.

[0070] The pseudo invisible meta-task includes a support set and a query set. In step S13, optionally, firstly, the updated meta-knowledge is divided into weight parameters and bias parameters And based on the feedforward neural network, the weight parameters and the bias parameters are transferred to learn the transferable knowledge of the pseudo invisible domain. meta It can be divided into two categories: weight and deviation Therefore, the transfer meta-learning operation is also divided into two steps: by scaling and transferring parameters θ meta Adapt it to the parameter distribution of the pseudo invisible domain. and the deviation parameter Apply the following relationship to calculate the transferable knowledge θ of the pseudo-unseen domain pu :

[0071]

[0072] f trans (·) represents the transfer meta-learning operation, ⊙ represents vector multiplication, represents the scaling parameter, Denotes the migration parameter. Using θ meta and θ trans Generate pseudo invisible meta-task Tpu The parameter θ pu ; and are respectively and Parameters for performing scaling and migration operations. Specifically, the relation extraction model As an example, a feedforward neural network layer is given the output h of the previous layer. in ∈R c×d , then the forward propagation process can be described as:

[0073] h out =(w⊙s1)h in +(b+s2),(13)

[0074] h out ∈P c×d Represents the output of the feedforward neural network of this layer, w∈R d×d and b∈R d Represent the weight and bias of the layer respectively, s1∈P d and s2∈P d denote the scaling and migration parameters of the layer respectively.

[0075] Then, the generalization ability of the relation extraction model on the pseudo-invisible meta-task is calculated based on the updated meta-knowledge and the transferable knowledge of the pseudo-invisible domain.

[0076] By parameters and Can evaluate relation extraction models Generalization ability on pseudo-unseen meta-tasks:

[0077]

[0078] and They represent the pseudo invisible meta-task T pu The support set and query set of and Represent their corresponding relationship labels respectively.

[0079] Finally, the transferable knowledge is updated according to the generalization ability. pu It reflects the effect of the transfer meta-learning operation, so it is optimized by the following formula parameter:

[0080]

[0081] β represents the learning rate. It is worth noting that this back propagation does not update the parameters

[0082] Figure 3 shows different parameter update processes, in the pseudo-visible meta-task T ps The global parameter update strategy is used to update the meta-knowledge; in the pseudo-unseen meta-task T pu In order to avoid the problems of underfitting and catastrophic forgetting, the meta-knowledge can be frozen and the parameters of the transfer learning operation can be updated using a local update strategy. Figure 3 For example, compared to the global parameter update which requires learning 3*d*(d+1) parameters, the staged update in the embodiment of the present invention can reduce the number of parameters to 6*d.

[0083] Step S14: predicting the relationship labels of the target domain based on the meta-knowledge and the updated transferable knowledge.

[0084] After the training phase, the optimized meta-knowledge θ is obtained meta and transferable knowledge θ trans , meta-knowledge θ meta Captures the generalized semantic information in the relation extraction model and transfers knowledge θ trans The two parameters can be converted into parameters θ for the test target domain data using formula (12): test .

[0085] In the testing phase, the test data is generated by formula (8): T t ={S t ,Q t = Z3(D t ).

[0086] Based on the parameter θ test , predict the relationship labels corresponding to the query set instances of the target domain based on the meta-knowledge and the updated transferable knowledge, satisfying the following relationship:

[0087]

[0088] Among them, y pred is the predicted relation label, and represent the support instance and query instance of the test meta-task respectively, express The corresponding relationship label.

[0089] The cross-domain small-shot relation extraction method of this embodiment employs a generalized meta-learning framework (GEML) to solve the cross-domain few-shot relation extraction task. This method involves domain-level task generation and staged updates. Domain-level task generation simulates the domain shift process to improve generalization to unseen target domains. Staged updates employ lightweight transfer meta-operations, enabling the model to quickly converge to the target domain. Extensive experiments on typical relation extraction meta-tasks demonstrate that the generalized meta-learning framework (GEML) can significantly improve the performance of models for cross-domain few-shot regularization tasks.

[0090] The following is a test of the cross-domain small sample relation extraction method of the embodiment of the present invention. In order to explore the performance of the cross-domain small sample relation extraction method and the baseline of the cross-domain small sample relation extraction (CrossFew RE) task, the dataset is extracted from a multi-domain corpus. A publicly available regular benchmark dataset, namely FewRel 2.0, is selected, consisting of four domain names: NYT, Wiki, SemEval, and PubMed. The data statistics of FewRel 2.0 are reported in Table 1. #relation, #sentence, and vocabulary describe the number of relation labels, sentences, and words in the corresponding domain, respectively. The average length refers to the average sentence length of each domain. Some statistics of the PubMed test are not disclosed by FewRel 2.0, as shown in Table 1. Specifically, in the DG scenario, Wiki and NYT are used as the initial source domain data, and SemEval is used as the verification data. In the DA scenario, since the RE model can access the auxiliary labeled data PubMed aux , our proposal will be PubMed aux It is divided into two parts: pseudo invisible domain data and verification data, and Wiki and NYT are used as pseudo visible domain data.

[0091] Table 1 Dataset

[0092]

[0093] First, we analyze the baselines. Table 2 shows the performance of each model in the domain generalization (DG) scenario, where the best baseline and best-performing results are underlined and bolded, respectively. Among these baselines, the traditional transfer learning method Fine-tune performs the worst on four typical meta-tasks: 5-way 1-shot, 5-way 5-shot, 10-way 1-shot, and 10-way 5-shot. This suggests that traditional transfer learning strategies are not suitable for data-sparse scenarios. Meta-learning (ML) methods such as Snail, Proto, MTB, and PAIR generally perform moderately well. Notably, PAIR performs the best on the 10-way 1-dimensional meta-task. This is likely because PAIR effectively alleviates the few-shot problem of the Cross-Domain Few Relation Extraction (CrossFew RE) task by pairing each query sentence with all support sentences. The CrossFew method HCRP performs the best on all meta-tasks except the 10-way 1-dimensional meta-task. This can be attributed to the underfitting risk of HCRP in the most challenging 10-way 1-dimensional task.

[0094] Table 2 Performance in DG scenario

[0095]

[0096] Next, we compare the proposed results with the baselines. After integrating Geml, the accuracy scores of Snail, Proto, MTB, and PAIR continue to increase. Geml 、Proto Geml , MTB Geml and PAIR Geml All of them beat the corresponding initial models, namely Snai, Proto, MTB and PAIR. Taking the 5-way 1-shot meta-task as an example, Snail Geml 、Proto Geml , MTB Geml and PAIR Geml The accuracy of Snail, Proto, MTB, and PAIR were improved by 5.75%, 16.71%, 4.06%, and 12.09%, respectively, which shows that the Geml framework of the embodiment of the present invention can indeed improve the generalization ability of the corresponding ML methods.

[0097] In addition, PAIR Geml It performs best among all the compared models. Specifically, in 5-way 1-shot, 5-way 5-shot, 10-way 1-shot and 10-way 5-shot meta tasks, PAIR GemlThe accuracy of the proposed method is 11.68%, 5.85%, 12.35% and 3.67% higher than the best baseline, respectively. This shows that the Geml framework of the embodiment of the present invention can better generalize to new relation labels in the unseen target domain by explicitly learning domain shift, rather than these cross-domain few-shot methods. It is worth noting that the Geml framework of the embodiment of the present invention makes PAIR Geml The largest improvement (i.e., 12.35%) is achieved on the most challenging 10-way 1-dimensional meta-task, which means that Geml can handle data-limited problems well when integrated into meta-learning-based methods.

[0098] Table 3 shows the performance of each model in the domain adaptation (DA) scenario, where in each column, the best baseline and best performance results are underlined and bolded, respectively.

[0099] Table 3 Performance in domain adaptation (DA) scenario

[0100]

[0101]

[0102] As shown in Table 3, the results in the DA scenario are roughly similar to those in the DG scenario. For the baseline, HCRP consistently performs best in the four meta-tasks. This suggests that an appropriate amount of target domain data can alleviate the underfitting problem. Overall, the performance of each model in the DA scenario is better than that of the corresponding model in the DG scenario. For example, PAIR Geml The accuracy of the proposed method was improved by 14.06%, from 68.34% in the DG scenario to 82.40% in the DA scenario. In addition, the PAIR accuracy score was improved by 9.06%, from 56.25% in the DG scenario to 65.31% in the DA scenario. This means that the Geml framework can help the corresponding meta-learning-based model better utilize the target domain data.

[0103] In summary, the cross-domain small sample relationship extraction method of the embodiment of the present invention performs domain sampling on multiple source domains to divide them into a pseudo-visible domain and a pseudo-invisible domain respectively; applies a global parameter update strategy to update the meta-knowledge of the pseudo-visible domain; performs transfer learning operations based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge; predicts the relationship labels of the target domain based on the meta-knowledge and the updated transferable knowledge, which can adapt to the distribution differences between the source domain and the target domain, and improve the generalization ability of unseen target domains.

[0104] The foregoing description is of specific embodiments of the present invention. In some cases, the actions or steps described in the embodiments of the present invention may be performed in an order different from that shown in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] The embodiment of the present invention also provides a cross-domain small sample relationship extraction device, such as Figure 4 As shown, the cross-domain small sample relationship extraction device includes: a domain sampling unit, a pseudo visible domain updating unit, a pseudo invisible domain updating unit and a label prediction unit.

[0106] A domain sampling unit is used to perform domain sampling on multiple source domains, dividing them into a pseudo visible domain and a pseudo invisible domain respectively;

[0107] a pseudo-visible domain updating unit, configured to apply a global parameter updating strategy to update the meta-knowledge of the pseudo-visible domain;

[0108] a pseudo-invisible domain updating unit, configured to perform a transfer learning operation based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge;

[0109] A label prediction unit is used to predict the relationship label of the target domain based on the meta-knowledge and the updated transferable knowledge.

[0110] For the convenience of description, the above device is described as being divided into various units according to their functions. Of course, when implementing the embodiments of the present invention, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0111] The apparatus of the above embodiment is applied to the corresponding method of the above embodiment and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0112] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the method described in any one of the above embodiments is implemented.

[0113] An embodiment of the present invention provides a non-volatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the method described in any one of the above embodiments.

[0114] Figure 5A more specific hardware structure diagram of an electronic device provided in this embodiment is shown. The device may include: a processor 501, a memory 502, an input / output interface 503, a communication interface 504, and a bus 505. The processor 501, the memory 502, the input / output interface 503, and the communication interface 504 are communicatively connected to each other within the device via the bus 505.

[0115] The processor 501 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the method embodiments of the present invention.

[0116] The memory 502 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 502 can store an operating system and other application programs. When the technical solutions provided by the method embodiments of the present invention are implemented through software or firmware, the relevant program codes are stored in the memory 502 and called and executed by the processor 501.

[0117] The input / output interface 503 is used to connect to the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0118] The communication interface 504 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0119] The bus 505 comprises a pathway for transmitting information between the various components of the device (eg, the processor 501 , the memory 502 , the input / output interface 503 , and the communication interface 504 ).

[0120] It should be noted that although the above device only shows the processor 501, memory 502, input / output interface 503, communication interface 504, and bus 505, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the embodiments of the present invention, and does not necessarily include all the components shown in the figure.

[0121] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0122] The embodiments of the present invention are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the scope of protection of this disclosure.

Claims

1. A cross-domain small sample relationship extraction method, characterized by: The cross-domain small sample relationship extraction method includes: Perform domain sampling on multiple source domains to divide them into a pseudo-visible domain and a pseudo-invisible domain respectively; Applying a global parameter update strategy to update the meta-knowledge of the pseudo-visible domain includes: sampling label-level meta-tasks on the pseudo-visible domain and the pseudo-invisible domain to construct label-level meta-tasks, wherein the label-level meta-tasks include pseudo-visible meta-tasks and pseudo-invisible meta-tasks; and applying a relation extraction model according to the pseudo-visible meta-tasks to update the meta-knowledge of the pseudo-visible domain. The pseudo-visible meta-task includes a support set and a query set, and the updating of the meta-knowledge of the pseudo-invisible domain by applying a relation extraction model according to the pseudo-visible meta-task includes: applying an encoder in the relation extraction model to map the support set instances and the query set instances in the pseudo-visible meta-task to a low-dimensional vector space; applying a metric function in the relation extraction model to obtain representations of the support set instances and the query set instances in the pseudo-visible meta-task in the low-dimensional vector space, and predicting relation labels of the query set instances in the pseudo-visible meta-task; calculating a cross-entropy loss function of the relation extraction model according to the predicted relation labels of the query set instances; and updating the meta-knowledge according to the cross-entropy loss function; performing a transfer learning operation based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge; Relation labels of a target domain are predicted based on the meta-knowledge and the updated transferable knowledge.

2. The cross-domain small sample relationship extraction method according to claim 1, characterized in that: The domain sampling of the multiple source domains to respectively divide a pseudo visible domain and a pseudo invisible domain includes: Perform domain sampling on multiple source domains and randomly select two candidate domains from them; Domain sampling is performed on the two candidate domains to respectively divide a pseudo visible domain and a pseudo invisible domain.

3. The cross-domain small sample relationship extraction method according to claim 1, characterized in that: The pseudo-invisible meta-task includes a support set and a query set, and performing a transfer learning operation based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge includes: Divide the updated meta-knowledge into weight parameters and bias parameters; Performing a transfer learning operation on the weight parameters and the bias parameters based on a feedforward neural network to obtain transferable knowledge of the pseudo-invisible domain; Calculating the generalization ability of the relation extraction model on the pseudo-invisible meta-task based on the updated meta-knowledge and the transferable knowledge of the pseudo-invisible domain; The transferable knowledge is updated according to the generalization capability.

4. The cross-domain small sample relationship extraction method according to claim 3 is characterized in that: The performing transfer learning operations on the weight parameters and the bias parameters based on a feedforward neural network to obtain transferable knowledge of the pseudo-invisible domain includes: According to the weight parameter and the deviation parameter Apply the following relationship to calculate the transferable knowledge of the pseudo-invisible domain ; in, represents the transfer meta-learning operation, represents vector multiplication, represents the scaling parameter, Indicates migration parameters.

5. The cross-domain small sample relationship extraction method according to claim 1, characterized in that: The predicting the relationship label of the target domain according to the meta-knowledge and the updated transferable knowledge includes: The relationship labels corresponding to the query set instances of the target domain are predicted based on the meta-knowledge and the updated transferable knowledge, satisfying the following relationship: in, is the encoder, is the metric function, is the predicted relation label, and represent the support instance and query instance of the test meta-task respectively, express The corresponding relationship label.

6. A cross-domain small sample relationship extraction device, characterized by: The cross-domain small sample relationship extraction device includes: A domain sampling unit is used to perform domain sampling on multiple source domains, dividing them into a pseudo visible domain and a pseudo invisible domain respectively; a pseudo-visible domain updating unit, configured to apply a global parameter updating strategy to update the meta-knowledge of the pseudo-visible domain, comprising: sampling label-level meta-tasks for the pseudo-visible domain and the pseudo-invisible domain to construct label-level meta-tasks, wherein the label-level meta-tasks include pseudo-visible meta-tasks and pseudo-invisible meta-tasks; and applying a relation extraction model based on the pseudo-visible meta-tasks to update the meta-knowledge of the pseudo-visible domain; The pseudo-visible meta-task includes a support set and a query set, and the updating of the meta-knowledge of the pseudo-invisible domain by applying a relation extraction model according to the pseudo-visible meta-task includes: applying an encoder in the relation extraction model to map the support set instances and the query set instances in the pseudo-visible meta-task to a low-dimensional vector space; applying a metric function in the relation extraction model to obtain representations of the support set instances and the query set instances in the pseudo-visible meta-task in the low-dimensional vector space, and predicting relation labels of the query set instances in the pseudo-visible meta-task; calculating a cross-entropy loss function of the relation extraction model according to the predicted relation labels of the query set instances; and updating the meta-knowledge according to the cross-entropy loss function; a pseudo-invisible domain updating unit, configured to perform a transfer learning operation based on the pseudo-invisible domain and the updated meta-knowledge to obtain updated transferable knowledge; A label prediction unit is used to predict the relationship label of the target domain based on the meta-knowledge and the updated transferable knowledge.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images