Zero sample event relation extraction method for agrometeorological disasters
By applying the zero-sample event relationship extraction method of BERT encoder and self-attention mechanism in the field of agricultural meteorological disasters, combined with relationship description and synonyms, the problem of insufficient generalization ability of agricultural meteorological disaster events in the existing technology is solved, and the problem of insufficient generalization ability of agricultural meteorological disaster events in the existing technology is realized. Accurate prediction of unseen relationships and effective capture of critical information.
Patent Information
- Application Number
- CN202410979552.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
AI Technical Summary
The existing agricultural meteorological disaster event relationship extraction methods lack generalization ability in unseen relationship predictions, especially the semantic matching method failed to deeply explore relationship attribute information and key information attention.
The zero-sample event relationship extraction method based on the BERT encoder and self-attention mechanism is adopted. Relational attribute coding and sentence attribute coding are constructed, and relationship description and synonyms are combined, and relationship prediction is used to use multi-task learning and nearest neighbor search algorithms.
Significantly improves the accuracy of prediction of new relationships between undefined or unobserved agrometeorological disaster events, reduces data annotation costs, and provides a more accurate and robust solution.
Smart Images

Figure CN118940751A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a zero-sample event relationship extraction method for agricultural meteorological disasters. Background Art
[0002] Event relation extraction is an important task in the field of natural language processing, which aims to infer the semantic relationship between two events in a given sentence. For example, from the sentence "Continuous snowfall and sudden drop in temperature in ** and other places have led to the collapse and damage of vegetable greenhouses and other facilities", the relationship between the event "continuous snowfall and sudden drop in temperature" and the event "collapse and damage of vegetable greenhouses and other facilities" can be extracted as "cause". Correctly extracting the relationship between agricultural meteorological disaster events is a key link in constructing an agricultural meteorological disaster event map. Existing methods usually define the relationship extraction task as a multi-classification problem. Based on training models with existing labeled data, they need to rely on a large amount of manually annotated data, lack generalization ability, and are limited in application in the extraction of unseen relationships. Although some recent studies generate labeled data based on remote supervision, when the relationship extraction task is applied to actual scenarios, the existing supervised models cannot well identify the relationships between events that are extremely rare or even never covered in the training data.
[0003] In order to avoid expensive manual labeling of data related to agricultural meteorological disasters, researchers began to explore zero-shot relationship extraction methods, that is, models that can predict new relationships that are not defined or observed. Existing zero-shot methods for extracting relationships between agricultural meteorological disaster events include methods based on prompt templates, methods based on textual implications, and methods based on semantic matching. Prompt technology provides prompts to pre-trained models to stimulate the hidden knowledge required for processing downstream tasks, but prompt templates need to be constructed manually, which is time-consuming and laborious. At the same time, manually constructed templates cannot cover all situations, resulting in limited generalization of the model. Textual implications reveal the inherent logical relationship between two texts, but require a high level of understanding of the text. If the relationship implied by the text is complex or ambiguous, the model may not be able to accurately infer the correct relationship. The semantic matching method not only learns the correspondence between labels and texts, but also evaluates the semantic similarity between texts, and has gradually become the mainstream method for zero-shot event relationship extraction in the field of agricultural meteorological disasters.
[0004] However, the existing zero-shot event relation extraction methods based on semantic matching fail to deeply mine relation attribute information and pay insufficient attention to the key information of sentences. Based on this, the present invention proposes a zero-shot event relation extraction method based on self-attention mechanism and semantic matching for agricultural meteorological disasters. Summary of the invention
[0005] In order to solve the above problems, the present invention proposes a zero-sample event relationship extraction method for agricultural meteorological disasters. Based on semantic matching, BERT encoder and self-attention mechanism, a zero-sample event relationship prediction model for agricultural meteorological disasters is constructed. This model can be used to predict new relationships between undefined or unobserved agricultural meteorological disaster events.
[0006] The technical solution of the present invention is as follows:
[0007] A zero-sample event relationship extraction method for agricultural meteorological disasters includes the following steps:
[0008] Step 1: Obtain agricultural meteorological disaster sentences, obtain the relationship descriptions and synonyms corresponding to the relationships using online open resource libraries based on the relationships between events in the sentences, construct a data set and perform preprocessing;
[0009] Step 2: Construct a zero-sample event relationship extraction model for agricultural meteorological disasters based on semantic matching and self-attention mechanism;
[0010] Step 3: Use the dataset to train and test the model. Train the model under a multi-task learning structure and use the nearest neighbor search algorithm to perform zero-sample relationship prediction on new input sentences during the test phase.
[0011] Furthermore, in step 1, the process of constructing a data set and performing preprocessing is as follows: agricultural meteorological disaster sentences with the same relationship are placed in the same set, and four special tags [EH], [\EH], [ET] and [\ET] are inserted into the input sentence, [EH] and [\EH] respectively mark the beginning and end of the head event; [ET] and [\ET] respectively mark the beginning and end of the tail event; the relationship description and synonyms are processed as a whole sequence, and if the relationship between agricultural meteorological disaster events has no corresponding relationship description or no synonym, the relationship description and the entire sequence of the relationship description are filled with "DEFAULT".
[0012] Furthermore, the specific process of step 2 is as follows:
[0013] Step 2.1, construct the encoding module, which includes two parts: relation attribute encoding and sentence attribute encoding;
[0014] Step 2.2: Construct the objective function for model training.
[0015] Furthermore, in step 2.1, the specific process of encoding the relationship attributes is as follows:
[0016] First, the natural language relation description D corresponding to the relation r r ={d r1 ,d r2 ,...,drk} is input into the BERT encoder to generate a sentence-level representation as the relation description vector d corresponding to relation r r ; k is the number of words contained in the relationship description; d rk is the kth word in the relation description of relation r;
[0017] Then, the synonym S corresponding to the relation r r ={s r1 ,s r2 ,...,s rl} Input into BERT encoder to obtain synonym vector s r ; l is the number of synonyms that relation r has; s rl is the lth synonym of relation r;
[0018] Finally, d r With s r Sum and get the relationship attribute vector corresponding to the final relationship r The formula is as follows:
[0019]
[0020] in, The dimension is
[0021] Furthermore, in step 2.1, the specific process of sentence attribute encoding is as follows:
[0022] First, the agricultural meteorological disaster sentence X corresponding to the relationship r s ={x s1 ,x s2 ,...,x sn} is input into the BERT encoder to generate a vector representation of each word; n is the sentence X s The number of words contained in x sn For sentence X s The nth word of ; the formula is as follows:
[0023] H = BERT(x s1 ,x s2 ,...,x sn ) (2);
[0024] Where H is the sentence X generated by the BERT encoder s The semantic vector of ; BERT(·) is calculated by the BERT encoder;
[0025] Then, the self-attention mechanism is used for context representation, and the calculation formula is as follows:
[0026] Q=W q*H (3);
[0027] V=W v *H (4);
[0028] K=W k *H (5);
[0029]
[0030] H′=β*V (7);
[0031] Where Q is the new query matrix; V is the new value matrix; K is the new key matrix; W q , W k and W v are the parameter matrices for calculating Q, K and V respectively; β is the sentence X s The weight value of ; softmax(·) is the softmax function; T is the transposition symbol; is the dimension of the hidden layer; H′ is the context vector;
[0032] Then, by averaging the vectors of each word, we get the hidden state vectors of the two events. The formula is as follows:
[0033]
[0034] in, is the hidden state vector of event c; when c is 1, it means the hidden state vector of event 1 When c is 2, it represents the hidden state vector of event 2 W e , b e are different parameters; tanh(·) is the tanh activation function; en is the upper limit of the index range; st is the lower limit of the index range; t is the index number; H t is the semantic vector of the word with index t;
[0035] Finally, by splicing H′, and To learn the sentence attribute vector, the formula is as follows:
[0036]
[0037] in, For sentence X s The corresponding sentence attribute vector; W and b are different learnable parameters; is the concatenation operator.
[0038] Furthermore, the specific process of step 2.2 is as follows:
[0039] Combining the relation attribute vector with the softmax layer generates a classification probability distribution over the visible relations, as shown in the following formula:
[0040]
[0041] Among them, p(·) is the classification probability distribution; y s ∈Y s is a visible relationship, Y s are all visible relations; θ is the model parameter, W * is the parameter matrix; b * is the bias term;
[0042] The objective function L is as follows:
[0043]
[0044] Among them, α is the balance coefficient; i is the sample number; N is the total number of samples; γ is the threshold; is the sentence attribute vector of sample i; j is the negative sample number; is the sentence attribute vector of negative sample j; is the relationship of sample i; is the predicted relationship of sample i.
[0045] Furthermore, the specific process of step 3 is as follows:
[0046] Step 3.1, randomly select 5 relations from the data set as the validation set, select 5, 10 and 15 relations as invisible relations to form the test set, and use the remaining relations as visible relations to form the training set;
[0047] Step 3.2: Input the training set into the constructed zero-sample event relationship extraction model for agricultural meteorological disaster events under the multi-task learning structure, and train the model according to the objective function constructed in step 2.2;
[0048] Step 3.3: Input the relationship descriptions and synonyms in the test set into the trained model, and the model's encoding module generates the corresponding relationship attribute vector is the mth relation attribute vector of all unseen relations in the embedding space. These vectors will be used as representations of new relations for subsequent matching and prediction. k When it arrives, generate the kth sentence attribute vector according to the process of step 2.1.2 The sentence attribute vector captures the key information related to the target event in the sentence, as shown in the following formula:
[0049]
[0050] Step 3.4: Since this is a zero-shot relation extraction method, the training set and the test set are different, so during the test, the nearest neighbor search is used to take the input test set sentence as a new sentence for zero-shot prediction; for the attribute vector of the vth input sentence in the test set The nearest relationship attribute vector The corresponding relationship is regarded as the predicted invisible relationship, as shown in the following formula:
[0051]
[0052] Among them, C(X k ) means returning the vth input sentence X v The predictive relationship of is the oth relation attribute vector among all invisible relations; argmin means finding the minimum value; dist(·) is the distance function.
[0053] Beneficial technical effects brought by the present invention: The zero-sample event relationship extraction method for agricultural meteorological disasters of the present invention first uses online open resources to obtain relationship descriptions and synonyms corresponding to the relationships; secondly, a zero-sample relationship classification model is designed, which can predict new undefined or unobserved relationships, that is, even if the relationships in the data set used for model training are different from those in the test set, they can still be identified. The model of the present invention makes full use of the advantages of contextual information and semantic matching, and provides a more accurate and robust solution for the zero-sample event relationship extraction task. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 The present invention is a flowchart of the zero-sample event relationship extraction method for agricultural meteorological disasters.
[0055] Figure 2 This is a framework diagram of the zero-sample event relationship prediction model for agricultural meteorological disasters of the present invention. DETAILED DESCRIPTION
[0056] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0057] like Figure 1As shown, the present invention provides a zero-sample event relationship extraction method for agricultural meteorological disasters, and the model corresponding to the method has three inputs, namely agricultural meteorological disaster sentences, synonyms and relationship descriptions. First, the agricultural meteorological disaster sentences are encoded using the BERT encoder, and then the key information of the sentence context is captured through the self-attention mechanism, and then the sentence attribute vector is obtained by splicing with the agricultural meteorological disaster event vector. At the same time, BERT is used to encode the relationship description and its synonyms, and the two are summed to form a rich relationship attribute vector. Five relationships are randomly selected from the data set as a validation set; the relationships are divided into visible relationships and invisible relationships, and 5, 10 and 15 relationships are selected as invisible relationships to form a test set, and the remaining relationships are selected as visible relationships to form a training set. Finally, the model is trained under a multi-task learning structure, and the nearest neighbor search is used in the test phase to perform zero-sample relationship prediction on the new input sentences. The present invention can handle unseen relationship types, significantly reduce the data annotation cost, and show the real-time advantage of processing data. The detailed steps of the present invention are as follows:
[0058] Step 1: Obtain agricultural meteorological disaster sentences, obtain the relationship descriptions and synonyms corresponding to the relationships in the sentences using online open resource libraries, construct a data set and perform preprocessing. The specific process is as follows:
[0059] Step 1.1, since this method has three inputs, in addition to the input of agricultural meteorological disaster sentences, the relationship descriptions and synonyms corresponding to the relationships between agricultural meteorological disaster events are also necessary. Therefore, we first use online open resources such as Baidu Encyclopedia and "Da Ci Lin" to obtain the relationship descriptions and synonyms corresponding to the relationships.
[0060] Table 1 shows examples of four agricultural meteorological disaster event relationships and their corresponding relationship descriptions and synonyms. Taking the relationship "caused" as an example, if you enter "caused" in Baidu Encyclopedia and search, you can get its corresponding relationship description as "cause or cause", and the corresponding synonyms include "become, brew, form". It can be observed that the synonyms are very close to "caused" in semantics, which can provide rich supplementary information for the relationship attribute vector.
[0061] Table 1 Relationship description and synonym examples
[0062]
[0063]
[0064] Step 1.2, the process of constructing the data set and preprocessing is as follows: put the agricultural meteorological disaster sentences with the same relationship in the same set, and insert four special tags [EH], [\EH], [ET] and [\ET] into the input sentence, [EH] and [\EH] mark the beginning and end of the head event respectively; [ET] and [\ET] mark the beginning and end of the tail event respectively; because when designing the model, the relationship description and the synonyms corresponding to the relationship are processed as a whole sequence, so if the relationship between agricultural meteorological disaster events has no corresponding relationship description or no synonym, the entire sequence of relationship description and synonyms will be filled with "DEFAULT".
[0065] Step 2: Construct a zero-sample event relationship extraction model for agricultural meteorological disasters based on semantic matching and self-attention mechanism. The model framework is shown in the figure below: Figure 2 shown.
[0066] Step 2.1, constructing a coding module; the coding module is the core component of the zero-sample event relation extraction method proposed in the present invention, and includes two parts: relation attribute coding and sentence attribute coding.
[0067] In the relationship attribute encoding, each relationship is equipped with a relationship description, which elaborates on the specific meaning of the relationship. However, if we only rely on the relationship description and the input sentence for semantic matching, we will find that the relationship attribute information is thin and insufficient. Therefore, the present invention proposes to combine the relationship description and synonyms to construct a rich relationship attribute vector. This vector not only contains the basic meaning of the relationship, but also incorporates the additional semantic information brought by synonyms, thereby enhancing the accuracy of semantic matching. In this way, the relationship information contained in the text can be more accurately identified in the zero-sample relationship extraction task. BERT is a pre-trained language representation model. The BERT pre-training model is used to convert relationship descriptions and synonyms into vector representations rich in semantic information. By performing a summation operation, a vector that fully reflects the relationship attributes is obtained, which provides a solid foundation for subsequent relationship extraction.
[0068] In the sentence attribute encoding, the BERT encoder is also used to convert the input agricultural meteorological disaster sentence into a context representation. In order to extract important features from the context that are conducive to zero-shot relationship extraction, a self-attention mechanism is also introduced. This mechanism can dynamically adjust the weight distribution of different words or phrases in the encoding process according to the context information of the current position. Then it is spliced with the agricultural meteorological disaster event vector to obtain the sentence attribute vector. In this way, the model can accurately capture the key information related to the target event, providing strong support for subsequent relationship prediction.
[0069] The specific working process of the encoding module is:
[0070] Step 2.1.1: Encode the relationship attributes to obtain the relationship attribute vector. The process of relationship attribute encoding is as follows:
[0071] First, the natural language relation description D corresponding to the relation r r ={d r1 ,d r2 ,...,d rk} is input into the BERT encoder to generate a sentence-level representation as the relation description vector d corresponding to relation r r ; k is the number of words contained in the relationship description; d rk is the kth word in the relation description of relation r;
[0072] Then, the synonym S corresponding to the relation r r ={s r1 ,s r2 ,...,s rl} Input into BERT encoder to obtain synonym vector s r ; l is the number of synonyms that relation r has; s rl is the lth synonym of relation r;
[0073] Finally, d r With s r Sum and get the relationship attribute vector corresponding to the final relationship r As shown in formula (1):
[0074]
[0075] in, The dimension is It contains not only the descriptive information of the relationship itself, but also the synonym information related to the relationship, thus enriching the attribute representation of the relationship.
[0076] Step 2.1.2: Encode the sentence attributes to obtain a sentence attribute vector. The process of sentence attribute encoding is as follows:
[0077] In order to obtain important features related to zero-shot relation extraction, the self-attention mechanism is applied to the features obtained from the context representation, so that the model focuses on important features that are conducive to relation extraction. The final context vector is then concatenated with the two events, allowing the model to understand the text content more comprehensively. The sentence attribute vector obtained in this way not only contains the information of a single event, but also reflects contextual information such as the position of the event in the sentence or paragraph and its relationship with other events.
[0078] First, the agricultural meteorological disaster sentence X corresponding to the relationship r s ={x s1,x s2 ,...,x sn} is input into the BERT encoder to generate a vector representation of each word; n is the sentence X s The number of words contained in x sn For sentence X s The vector captures the semantic information and contextual information of the word, as shown in formula (2):
[0079] H = BERT(x s1 ,x s2 ,...,x sn ) (2);
[0080] Where H is the sentence X generated by the BERT encoder s The semantic vector of BERT(·) is calculated by the BERT encoder.
[0081] Then, the self-attention mechanism uses the mapping of query, key and value to calculate the attention weight. For each word, its query, key and value representation are extracted from its vector representation through linear transformation. Finally, the similarity between the query of each word and the keys of all words is calculated to obtain the attention weight. These weights reflect the importance of each word in the global context. Specifically, the new matrices Q, K, and V are calculated in the self-attention mechanism, which are the results of multiplying the vector of each input word with a randomly initialized matrix. Then, the obtained Q and K are normalized by the softmax function and multiplied by the matrix V to obtain the weighted context vector H′. The detailed calculation process is shown in formulas (3) to (7):
[0082] Q=W q *H (3);
[0083] V=W v *H (4);
[0084] K=W k *H (5);
[0085]
[0086] H′=β*V (7);
[0087] Where Q is the new query matrix; V is the new value matrix; K is the new key matrix; W q , W k and W v are the parameter matrices for calculating Q, K and V respectively; β is the sentence X s The weight value of ; softmax(·) is the softmax function; T is the transposition symbol; is the dimension of the hidden layer;
[0088] In addition, four special tags [EH], [\EH], [ET], and [\ET] are inserted into the input sentence. [EH] and [\EH] are used to mark the beginning and end of the head event, respectively; [ET] and [\ET] are used to mark the beginning and end of the tail event, respectively;
[0089] Then, by averaging the vectors of each word, we get and Hidden state vectors of two events. For example, an event e consists of multiple words with indexes ranging from st to en, the average hidden state vectors, and an activation operation with a fully connected layer is also added to generate the representation of the event, as shown in formula (8):
[0090]
[0091] in, is the hidden state vector of event c; when c is 1, it means the hidden state vector of event 1 When c is 2, it represents the hidden state vector of event 2 W e , b e For different parameters, and Share the same parameter W e and b e ; tanh(·) is the tanh activation function; en is the upper limit of the index range; st is the lower limit of the index range; t is the index number; H t is the semantic vector of the word with index t;
[0092] Finally, by splicing H′, and To learn the sentence attribute vector, as shown in formula (9):
[0093]
[0094] in, For sentence X s The corresponding sentence attribute vector; W and b are different learnable parameters; is the concatenation operator.
[0095] The concatenation of the sentence context vector and the event vector ensures that the sentence attribute vector can comprehensively reflect the information of the event and the sentence context, so that the model can consider both factors at the same time when performing relationship extraction. This helps the model understand the text content more comprehensively and improves the accuracy of relationship extraction.
[0096] Step 2.2: Construct the objective function of model training; model training focuses on two core objectives: one is to optimize the matching degree between the sentence attribute vector and the corresponding relationship attribute vector, and the other is to improve the accuracy of relationship classification. Specifically, the first objective is reflected in the first term in formula (11), and the task is to minimize the attribute vector of the agricultural meteorological disaster sentence. and the corresponding relation attribute vector The distance between the positive pairs (i.e., the distance between the agricultural meteorological disaster sentences and the unmatched relations (i.e., the negative pairs) is ensured, while the embeddings between the agricultural meteorological disaster sentences and the unmatched relations (i.e., the negative pairs) are farther away from each other. The positive sample pairs are the combination of the agricultural meteorological disaster sentences and their corresponding relations, while the negative sample pairs are the combination of the agricultural meteorological disaster sentences and other relations selected randomly. The second goal is reflected in the second term in formula (11), and the task is to maximize the accuracy of relation classification based on the seen relations using cross entropy loss. The relation attribute vector is combined with the softmax layer to generate a classification probability distribution over the seen relations, as shown in formula (10):
[0097]
[0098] Among them, p(·) is the classification probability distribution; y s ∈Y s is a visible relationship, Y s are all visible relations; θ is the model parameter, W * is the parameter matrix; b * is the bias term.
[0099] The objective function L is as follows, as shown in formula (11):
[0100]
[0101] Among them, α is the balance coefficient; i is the sample number; N is the total number of samples; γ is the threshold; is the sentence attribute vector of sample i; j is the negative sample number; is the sentence attribute vector of negative sample j; is the relationship of sample i; is the predicted relationship of sample i;
[0102] In the first term of the above formula (11), set γ>0 so that the ) must be higher than the maximum value of negative samples (i.e. ) exceeds a predetermined threshold γ. With the introduction of γ, the loss will increase due to the difference between the positive pair and the closest negative pair. In addition, γ is also used to avoid the collapse of the embedding space. If only the distance of the positive pair is minimized using losses such as mean squared error, the optimization may cause each vector in the embedding space to be too close to each other. In order to keep the computational complexity low, only the unmatched relations within a batch are considered as negative samples j.
[0103] The second term in the above formula (11) is a commonly used cross entropy loss. During the training process, these two objectives are optimized alternately to ensure that the model can accurately match sentences and relations and effectively classify relations. Through continuous iteration and optimization, this method eventually obtains a model with good performance.
[0104] Step 3: Use the dataset to train and test the model. Train the model under a multi-task learning structure and use the nearest neighbor search algorithm to perform zero-sample relationship prediction on new input sentences during the test phase.
[0105] Step 3.1, randomly select 5 relations from the data set as the validation set, select 5, 10 and 15 relations as invisible relations to form the test set, and use the remaining relations as visible relations to form the training set;
[0106] Step 3.2: Input the training set into the constructed zero-sample event relationship extraction model for agricultural meteorological disaster events under the multi-task learning structure, and train the model according to the objective function constructed in step 2.2;
[0107] Step 3.3: Input the relationship descriptions and synonyms in the test set into the trained model, and the model's encoding module generates the corresponding relationship attribute vector is the mth relation attribute vector of all unseen relations in the embedding space, which will be used as the representation of new relations for subsequent matching and prediction. k When it arrives, generate the kth sentence attribute vector according to the process of step 2.1.2 The parameters used are the same, and the sentence attribute vector captures the key information related to the target event in the sentence. As shown in formula (12):
[0108]
[0109] Step 3.4: Since it is a zero-sample relation extraction method, the training set and the test set are different, so the classifier cannot be used for classification during the test. Finally, the present invention uses the nearest neighbor search to take the input test set sentence as a new sentence for zero-sample prediction. The nearest relationship attribute vector The corresponding relationship is regarded as the predicted invisible relationship, as shown in formula (13):
[0110]
[0111] Among them, C(X k ) means returning the vth input sentence X v The predictive relationship of is the oth relation attribute vector among all invisible relations; argmin means finding the minimum value; dist(·) is the distance function.
[0112] The advantage of this method is that it does not require additional annotation or training of new relationships, and only requires the use of existing models and encoding methods to make predictions. In addition, the inner product function is used as the distance function in the method of the present invention.
[0113] In order to prove the feasibility and superiority of the present invention, the following experiment was conducted.
[0114] The present invention uses Python 3.8 as the programming language and PyTorch 1.4.0 as the deep learning framework. In addition, precision, recall and F1 value are used as evaluation indicators. Five relations are randomly selected from the data set as the validation set; 5, 10 and 15 relations are selected as invisible relations to form the test set, and the remaining relations are used as visible relations to form the training set. The present invention uses the training set of data to train the model under multi-task learning, and uses the nearest neighbor search method to predict the relationship.
[0115] Table 2 shows in detail the final experimental results of the four comparison methods of the present invention and R-BERT (this method combines the BERT pre-trained language model and the target entity information to solve the relationship classification task), ESIM (this method defines zero-shot relationship extraction as a reading comprehension task), REPrompt (this method synthesizes relationship examples by prompting the language model to generate structured text), and ZS-BERT (this method uses the semantic matching method for zero-shot relationship extraction. In addition to the classification loss, ZS-BERT also introduces a metric-based loss to optimize the representation space) on the agricultural meteorological disaster dataset. In order to more comprehensively evaluate the performance of each method and refer to other zero-shot relationship extraction comparison methods, the cases where the number of invisible relations num is 5, 10 and 15 are considered respectively. These specific numbers are selected to simulate different degrees of complexity in the experiment to more comprehensively evaluate the model's ability to face different challenges. As the number of invisible relations increases, the prediction difficulty of the model will also increase accordingly. Therefore, by comparing the performance of the model under different numbers of invisible relations, we can better understand the model's ability to handle complex situations.
[0116] Table 2 Comparison of results of different methods
[0117]
[0118] As can be seen from Table 2, when num=5 and num=10, the proposed method is superior to other comparison methods, indicating that when the number of invisible relations is relatively small, the method can perform zero-sample relationship extraction more accurately. This may be because the method proposed in the present invention uses the attention mechanism to effectively highlight the key information in the sentence and emphasizes the information related to the relationship attributes, thereby improving the matching accuracy. Although when num=15, the present invention is inferior to the REPrompt model method, its performance is also better than the other three models. In general, the model method proposed in the present invention has improved effects on zero-sample relationship extraction, especially when the number of invisible relations is relatively small. Such results not only verify the effectiveness of using the attention mechanism to enhance the important information of the sentence and the relationship attribute information for semantic matching, but also prove that the simultaneous use of synonyms and relationship descriptions can better obtain rich relationship information.
[0119] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.
Claims
1. A zero-sample event relationship extraction method for agricultural meteorological disasters, characterized in that: The steps include: Step 1: Obtain agricultural meteorological disaster sentences, obtain the relationship descriptions and synonyms corresponding to the relationships using online open resource libraries based on the relationships between events in the sentences, construct a data set and perform preprocessing; Step 2: Construct a zero-sample event relationship extraction model for agricultural meteorological disasters based on semantic matching and self-attention mechanism; Step 3: Use the dataset to train and test the model. Train the model under a multi-task learning structure and use the nearest neighbor search algorithm to perform zero-sample relationship prediction on new input sentences during the test phase.
2. The zero-sample event relationship extraction method for agricultural meteorological disasters according to claim 1 is characterized in that: In step 1, the process of constructing a data set and performing preprocessing is as follows: putting agricultural meteorological disaster sentences with the same relationship in the same set, and inserting four special tags [EH], [\EH], [ET] and [\ET] into the input sentence, [EH] and [\EH] mark the beginning and end of the head event respectively; [ET] and [\ET] mark the beginning and end of the tail event respectively; The relationship descriptions and synonyms are processed as a whole sequence. If the relationship between agricultural meteorological disaster events has no corresponding relationship description or no synonym, the entire sequence of relationship descriptions and synonyms is filled with "DEFAULT".
3. The zero-sample event relationship extraction method for agricultural meteorological disasters according to claim 1 is characterized in that: The specific process of step 2 is as follows: Step 2.1, construct the encoding module, which includes two parts: relation attribute encoding and sentence attribute encoding; Step 2.2: Construct the objective function for model training.
4. The zero-sample event relationship extraction method for agricultural meteorological disasters according to claim 3 is characterized in that: In step 2.1, the specific process of relationship attribute encoding is as follows: First, the natural language relation description D corresponding to the relation r r ={d r1 ,d r2 ,...,d rk } is input into the BERT encoder to generate a sentence-level representation as the relation description vector d corresponding to relation r r ; k is the number of words contained in the relationship description; d rk is the kth word in the relation description of relation r; Then, the synonym S corresponding to the relation r r ={s r1 ,s r2 ,...,s rl } Input into BERT encoder to obtain synonym vector s r ; l is the number of synonyms that relation r has; s rl is the lth synonym of relation r; Finally, d r With s r Sum and get the relationship attribute vector corresponding to the final relationship r The formula is as follows: in, The dimension is 5. The zero-sample event relationship extraction method for agricultural meteorological disasters according to claim 4 is characterized in that: In step 2.1, the specific process of sentence attribute encoding is as follows: First, the agricultural meteorological disaster sentence X corresponding to the relationship r s ={x s1 ,x s2 ,...,x sn } is input into the BERT encoder to generate a vector representation of each word; n is the sentence X s The number of words contained in x sn For sentence X s The nth word of ; the formula is as follows: H=BERT(x s1 ,x s2 ,...,x sn ) (2); Where H is the sentence X generated by the BERT encoder s The semantic vector of ; BERT(·) is calculated by the BERT encoder; Then, the self-attention mechanism is used for context representation, and the calculation formula is as follows: Q=W q *H (3); V=W v *H (4); K=W k *H (5); H′=β*V (7); Where Q is the new query matrix; V is the new value matrix; K is the new key matrix; W q , W k and W v are the parameter matrices for calculating Q, K and V respectively; β is the sentence X s The weight value of ; softmax(·) is the softmax function; T is the transposition symbol; is the dimension of the hidden layer; H′ is the context vector; Then, by averaging the vectors of each word, we get the hidden state vectors of the two events. The formula is as follows: in, is the hidden state vector of event c; when c is 1, it means the hidden state vector of event 1 When c is 2, it represents the hidden state vector of event 2 W e 、b e are different parameters; tanh(·) is the tanh activation function; en is the upper limit of the index range; st is the lower limit of the index range; t is the index number; H t is the semantic vector of the word with index t; Finally, by splicing H′, and To learn the sentence attribute vector, the formula is as follows: in, For sentence X s The corresponding sentence attribute vector; W and b are different learnable parameters; ⊕ is the concatenation operator.
6. The zero-sample event relationship extraction method for agricultural meteorological disasters according to claim 5 is characterized in that: The specific process of step 2.2 is as follows: Combining the relation attribute vector with the softmax layer generates a classification probability distribution over the visible relations, as shown in the following formula: Among them, p(·) is the classification probability distribution; y s ∈Y s is a visible relationship, Y s are all visible relations; θ is the model parameter, W * is the parameter matrix; b * is the bias term; The objective function L is as follows: Among them, α is the balance coefficient; i is the sample number; N is the total number of samples; γ is the threshold; is the sentence attribute vector of sample i; j is the negative sample number; is the sentence attribute vector of negative sample j; is the relationship of sample i; is the predicted relationship of sample i.
7. The zero-sample event relationship extraction method for agricultural meteorological disasters according to claim 6 is characterized in that: The specific process of 3 is as follows: Step 3.1, randomly select 5 relations from the data set as the validation set, select 5, 10 and 15 relations as invisible relations to form the test set, and use the remaining relations as visible relations to form the training set; Step 3.2: Input the training set into the constructed zero-sample event relationship extraction model for agricultural meteorological disaster events under the multi-task learning structure, and train the model according to the objective function constructed in step 2.2; Step 3.3: Input the relationship descriptions and synonyms in the test set into the trained model, and the model's encoding module generates the corresponding relationship attribute vector is the mth relation attribute vector of all invisible relations in the embedding space. These vectors will be used as representations of new relations for subsequent matching and prediction. With the kth input sentence X in the test set k When it arrives, generate the kth sentence attribute vector according to the process of step 2.1.2 The sentence attribute vector captures the key information related to the target event in the sentence, as shown in the following formula: Step 3.4: Since this is a zero-shot relation extraction method, the training set and the test set are different, so during the test, the nearest neighbor search is used to take the input test set sentence as a new sentence for zero-shot prediction; For the vth input sentence attribute vector in the test set The nearest relationship attribute vector The corresponding relationship is regarded as the predicted invisible relationship, as shown in the following formula: Among them, C(X k ) means returning the vth input sentence X v The predictive relationship of is the oth relation attribute vector among all invisible relations; argmin means finding the minimum value; dist(·) is the distance function.
Citation Information
Cited By
Science and education resource relationship prediction method and system based on large model zero sample prediction
CN119597936A
Method and system for predicting relations of science and education resources based on large model zero sample prediction
CN119597936B
