Hyperspherical multi-prototype and optimal transmission method and device for event argument extraction
Through the hyperspherical multi-prototype and optimal transmission method, the problems of intra-class semantic differences and large-segment separation of inter-classes are ignored in the event argument extraction model, and the model performance and decision-making accuracy are improved.
Patent Information
- Application Number
- CN202410975054.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-19
AI Technical Summary
The existing event argument extraction model ignores intra-class semantic differences and large-segment separations, resulting in difficulty in model decision making and limited performance.
Adopting hyperspherical multi-prototype and optimal transmission method for event arguments, optimized training through prototype initialization, inter-class prototype positioning, intra-class prototype positioning, prototype sorting based on label semantics and argument extraction modules, combined with the optimal transmission module The process is to achieve large-space separation of inter-class arguments and effectively characterize intra-class variance.
The performance of event argument extraction is improved, effective separation of inter-class arguments and effective representation of intra-class semantic variance is realized, and the difficulty of model decision is reduced.
Smart Images

Figure CN118982018A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of deep learning and natural language processing, and specifically relates to a hypersphere multi-prototype and optimal transmission method and device for event argument extraction. Background Art
[0002] An event is an event or state change that is triggered by a verb and occurs at a specific time point or time period, within a specific geographical area, and consists of one or more actions involving one or more roles. Event Argument Extraction (EE) aims to extract arguments (such as time, place, and person) of a specified event from natural language text. It is an important research task in the field of Natural Language Processing (NLP), and has a wide range of applications in intelligent question answering, information retrieval, automatic summarization, recommendation, and other fields.
[0003] Existing research focuses on solving the long-distance dependency of arguments and modeling the co-occurrence relationship between roles and events, but ignores the potential inductive bias; the semantic differences between arguments of the same type and the large interval separation between arguments of different types limit the performance of the model in extracting event arguments. Specifically, the semantic differences between arguments of the same type refer to the fact that the arguments of the same role will be aggregated into multiple sub-clusters in the feature space, which will make the model affected by the intra-class variance; ignoring the large interval separation between arguments of different types will lead to blurred boundaries between arguments of different types, making it difficult for the model to make decisions. Summary of the invention
[0004] In view of the problems of fuzzy event argument boundary extraction, difficult decision making and limited performance in current models, the present invention provides a
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A hypersphere multi-prototype and optimal transmission device for event argument extraction, the device comprising a prototype initialization module, an inter-class prototype positioning module, an intra-class prototype positioning module, a prototype sorting module based on label semantics, an argument extraction module and an optimal transmission module;
[0007] The prototype initialization module is used to initialize multiple prototype vectors for each role category; the inter-class prototype positioning module is used to locate the inter-class prototype vectors based on the inter-class relationship; the intra-class prototype positioning module is used to locate the intra-class prototype vectors based on the intra-class relationship; the prototype sorting module based on label semantics is used to locate the inter-class prototype vectors based on the label semantic similarity; the argument extraction module is used to extract the arguments of the specified event from the text; the optimal transmission module is used to align the arguments with their multiple prototypes to optimize the model training and guide the model to learn the representation of the arguments, thereby achieving large-margin separation of inter-class arguments and effectively representing the intra-class variance, and improving the performance of event argument extraction.
[0008] A hypersphere multi-prototype and optimal transmission method for event argument extraction based on the above device comprises the following steps:
[0009] Step 1: Use the prototype initialization module to randomly initialize multiple prototype vectors for each category;
[0010] Step 2: Use the inter-class prototype positioning module to maximize the inter-class prototype similarity as the optimization goal to locate the prototype vector in step 1;
[0011] Step 3: Use the intra-class prototype positioning module to minimize the intra-class prototype similarity as the optimization goal to locate the prototype vector in step 1;
[0012] Step 4: Use the prototype sorting module based on label semantics to further locate the prototype vector in step 1 using label semantic similarity as a guide;
[0013] Step 5, extracting arguments of the event from the text using the argument extraction module;
[0014] Step 6: Use the optimal transmission module to realize the allocation of arguments and prototypes and optimize the model.
[0015] Furthermore, the specific method of step 1 is to randomly initialize multiple prototype vectors for each category by using the prototype initialization module: using the prototype initialization module, randomly initialize N prototypes for each category Where P i is the prototype set of the i-th category, P i The j-th prototype vector in represents a set of real numbers, D represents the dimension of the hidden layer 1024, and the prototypes of all K categories can be expressed as
[0016] Furthermore, the specific method of locating the prototype vector in step 1 by using the inter-class prototype positioning module to maximize the inter-class prototype similarity as the optimization goal is: in order to achieve the maximum interval separation between classes, in each round of iteration during training, the prototype similarity with the greatest similarity to the current prototype is minimized to maximize the difference between classes, and the loss is calculated as follows:
[0017]
[0018] in, represents the loss of the prototype stage between classes, b(i) represents the category represented by the i-th prototype, d(·) represents the metric function of cosine similarity, for The transposed matrix of , E is the identity matrix, and M represents the prototype similarity matrix.
[0019] Furthermore, the specific method of using the intra-class prototype positioning module in step 3 to minimize the intra-class prototype similarity as the optimization goal to locate the prototype vector in step 1 is: maximizing the similarity of the intra-class prototypes as the objective function, and maximizing the minimum similarity between these prototypes, which is specifically expressed as:
[0020]
[0021] in, represents the loss of the prototype stage within the localization class, and k represents the kth category.
[0022] Furthermore, the step 4 uses the prototype ranking module based on label semantics to take label semantic similarity as a guide to further locate the prototype vector in step 1. The specific method is: the distance between prototypes with similar category labels should be smaller than the distance between prototypes with different semantics, and an additional ranking loss is introduced to achieve this goal:
[0023]
[0024] o ijk =d(p i ,p j )-d(p i ,p k )
[0025]
[0026] in, The true labels for the sorted role triplets, w i represents the embedded representation of the i-th category label encoded by the pre-trained model Bert, where G is the set of triplets of all roles, [[·]] is the indicator function, and O ijk Represents the prototype distance difference, pi , p j , p k Respectively represent the prototypes of the i-th, j-th and k-th categories, S ijk Represents the probability value of sorting; represents the sorting loss.
[0027] By adding the three losses as the final loss And optimized by SGD optimizer:
[0028]
[0029] Finally, we get a set of pre-trained prototypes in, is the prototype set of the i-th category, represents the jth prototype of the i-th category.
[0030] Furthermore, the specific method of extracting event arguments from text using the argument extraction module in step 5 is: given an article containing k event trigger words Use the special marker " <t-i>"and"< / t-i> "For the trigger word t i Mark, where i represents the i-th event type; then obtain the input sequence X, and send X to the RoBERTa encoder for encoding Contextual representation of And send it to the decoder to get the event-oriented context embedding representation H X :
[0031]
[0032] Among them, Encoder(·) represents the RoBERTa encoder, and Decoder(·) represents the Transformer decoder;
[0033] In order to establish the co-occurrence relationship between events in the text, argument extraction constructs a slotted table and uses it as the input of the decoder;
[0034] The prompt for each event is fed into the encoder in parallel, resulting in an initialized column header representation:
[0035]
[0036] in, is the jth prompt at the input, is the embedding representation of the jth prompt, E i represents the i-th event, H CHis the connection of all prompts; the i-th row in the slot table starts with the i-th trigger word, followed by the parameter slot S of the i-th prompt i ; The initialization representation of the slot table is obtained by the following formula:
[0037]
[0038] in, is the embedding of the i-th trigger word, is the average embedding of the corresponding roles and triggers;
[0039] Then the non-autoregressive table decoder is used for decoding; the non-autoregressive table decoder uses the mask M Tab Make each element in the table focus on the area related to it; M Tab The specific design is as follows:
[0040] (1) All tokens in the column header follow each other;
[0041] (2) All tokens in the column header are related to the event trigger word;
[0042] (3) Each role and the corresponding argument slot pay attention to each other;
[0043] (4) Each event trigger word and the corresponding argument slot pay attention to each other;
[0044] M Tab Integrate it into the self-attention calculation process of the decoder to obtain the final slot table embedding:
[0045]
[0046] Embedding of argument slots from extracted from is the kth argument slot embedding, N a To explain the original number of slots; use linear transformation to transform each argument slot into a fragment selector:
[0047]
[0048] Among them, w s and w e is a trainable parameter; Together they form a fragment selector with k argument slots, which is used to extract the corresponding text fragment from the context embedding:
[0049]
[0050] in, Represents the probability distribution of each token as the starting position of the segment, represents the probability distribution of each token as the end position of the segment, l c is the length of the context, and are the indices of the start and end of the best segment;
[0051] Finally, the Hungarian algorithm is used to solve the assignment problem between the predicted argument fragments and the golden argument fragments, and this process is optimized by the bipartite matching loss, which is expressed as:
[0052]
[0053] Among them, (s k , e k ) is the golden fragment index assigned to the kth argument slot.
[0054] Furthermore, the step 6 uses the optimal transmission module to realize the allocation of arguments and prototypes. The specific method of optimizing the model is:
[0055] Using the prototypes obtained in step 4, the argument extraction in step 5 is guided to learn the representation of the arguments during training; since each type of argument is represented by multiple prototypes, assigning each argument to the appropriate prototype is necessary to optimize the parameter representation. to represent the M d-dimensional prototypes of category r, Represents the set of arguments predicted to be r; is the embedding of the ith argument, obtained by average pooling:
[0056] The goal is to calculate the allocation matrix Here the problem is modeled as an optimal transmission problem for solution:
[0057]
[0058] in, is the cost matrix, indicating and There are two constraints here:
[0059] (1) The dot product with a vector of all 1s is equal to a, ensuring that Each row of has only one value of 1, so that each argument is assigned exactly one prototype;
[0060] (2) The dot product with a vector of all ones is equal to b, which ensures that γ rEach column of is not all zero, which avoids the situation where all arguments are assigned to only one prototype;
[0061] Then, the sinkhorn-knopp algorithm is used to find the optimal allocation matrix Figure 5 This is the sinkhorn-knopp pseudocode, and uses standard cross entropy to achieve that the arguments are relatively close to the prototype:
[0062]
[0063] Among them, p c To assign to The best prototype vector; in order to achieve compactness of argument features within the same subset cluster, the absolute distance between the argument features and their prototypes is further optimized:
[0064]
[0065] Add the three losses together for joint optimization training:
[0066]
[0067] where λ a and λ c is the loss weight hyperparameter. Update each prototype according to the parameter features assigned to each prototype, and at each training step t, use the exponential moving average EMA to update the prototype:
[0068]
[0069] where α is the EMA update rate; in this way, the learned prototype can be considered as a representation of a subcluster in the embedding space.
[0070] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned hypersphere multi-prototype and optimal transmission method for event argument extraction is implemented.
[0071] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned hypersphere multi-prototype and optimal transmission method for event argument extraction.
[0072] Compared with the prior art, the present invention has the following advantages:
[0073] The current event argument extraction ignores the semantic differences within the class and the large separation between classes, which will make the model affected by the intra-class variance and increase the difficulty of model decision-making. The present invention pre-trains a set of category prototypes based on the above two inductive biases, and guides the event argument extraction to learn the embedded representation of the argument during the training process. Each type of argument is represented by multiple prototypes. In order to align the argument with the prototype, the present invention regards it as an optimal transmission problem for modeling and solving, and uses the cross entropy loss to make the argument relatively close to the prototype, and uses the compact loss to make the sub-cluster more compact, thereby achieving large separation of arguments between classes and effectively representing the semantic variance within the class. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] Figure 1 It is a model framework diagram of the present invention;
[0075] Figure 2 Data samples used in the present invention;
[0076] Figure 3 It is a structural diagram of the argument extraction module of the present invention;
[0077] Figure 4 This is a structural diagram of the optimal transmission module of the present invention;
[0078] Figure 5 is the optimal transmission module algorithm of the present invention;
[0079] Figure 6 A schematic diagram of the structure of a hypersphere multi-prototype and optimal transmission method and device for event argument extraction provided by an embodiment of the present invention;
[0080] Figure 7 A schematic diagram of the overall structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0081] In order to gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description of the present invention. However, the present invention has multiple implementations and is not limited to the specific examples listed herein. The presentation of these examples is intended to deepen the comprehensive understanding of the disclosure of the present invention.
[0082] The present invention provides a hypersphere multi-prototype and optimal transmission device for event argument extraction, such as Figure 1 As shown in the model architecture diagram, it includes a prototype initialization module, an inter-class prototype positioning module, an intra-class prototype positioning module, a prototype sorting module based on label semantics, an argument extraction module and an optimal transmission module.
[0083] The prototype initialization module is used to initialize multiple prototype vectors for each character category;
[0084] The inter-class prototype positioning module is used to locate the inter-class prototype vector based on the inter-class relationship;
[0085] The intra-class prototype positioning module is used to locate the intra-class prototype vector based on the intra-class relationship;
[0086] The prototype ranking module based on label semantics is used to locate the prototype vectors between classes based on label semantic similarity;
[0087] The argument extraction module is used to extract the arguments of the specified event from the text;
[0088] The optimal transfer module is used to align arguments with their multiple prototypes to optimize the model training and guide the model to learn the representation of arguments, thereby achieving large-margin separation of arguments between classes and effectively representing intra-class variance, and improving the performance of event argument extraction.
[0089] Example 1
[0090] like Figure 2 The figure shows an example of event argument extraction from the RAMS dataset, which is derived from real news events.
[0091] Among them, the article involves the event type of "conflict-attack-self-directed conflict", "attacker", "target" and "location" are event roles, and "Team D", "Army E" and "C Valley in Country B" are event arguments.
[0092] A hypersphere multi-prototype and optimal transmission method for event argument extraction based on the above device comprises the following steps:
[0093] Step 1: Use the prototype initialization module to randomly initialize multiple prototype vectors for each category;
[0094] Using the prototype initialization module, randomly initialize N prototypes for each category Where P i is the prototype set of the i-th category, P i The j-th prototype vector in represents a set of real numbers, D represents the dimension of the hidden layer 1024, and the prototypes of all K categories can be expressed as
[0095] Step 2: Use the inter-class prototype positioning module to locate the inter-class prototype, and maximize the similarity between the inter-class prototype vectors as the optimization goal to locate the prototype vector in step 1;
[0096] In order to achieve the maximum margin separation between classes, at each iteration during training, the difference between classes is maximized by minimizing the similarity of the prototype with the greatest similarity to the current prototype. The loss is calculated as follows:
[0097]
[0098] in, represents the loss of the prototype stage of locating classes, b(i) represents the category represented by the i-th prototype, d(·) represents the metric function of cosine similarity, for The transposed matrix of , E is the identity matrix, and M represents the prototype similarity matrix.
[0099] Step 3: Use the intra-class prototype positioning module to minimize the intra-class prototype similarity as the optimization goal to locate the prototype vector in step 1;
[0100] The objective function is to maximize the similarity of prototypes within a class and maximize the minimum similarity between these prototypes, which can be expressed as:
[0101]
[0102] in, represents the loss of the prototype stage of locating the class, and k represents the kth class. Step 4, using the prototype ranking module based on label semantics to further locate the prototype vector in step 1 using label semantic similarity as a guide;
[0103] The distance between prototypes with similar semantics should be smaller than that between prototypes with different semantics. An additional ranking loss is introduced to achieve this goal:
[0104]
[0105] o ijk =d(p i ,p j )-d(p i ,p k )
[0106]
[0107] in, The true labels for the sorted role triplets, w i represents the embedded representation of the i-th category label encoded by the pre-trained model Bert, where G is the set of triplets of all roles, [[·]] is the indicator function, and o ijk Represents the prototype distance difference, p i , p j , p k Respectively represent the prototypes of the i-th, j-th and k-th categories, S ijk Represents the probability value of sorting; represents the sorting loss.
[0108] By adding the three losses as the final loss And optimized by SGD optimizer:
[0109]
[0110] Finally, we get a set of pre-trained prototypes in, is the prototype vector set of the i-th category, Represents the jth prototype vector of the ith category.
[0111] Step 5, extracting arguments of the event from the text using the argument extraction module; Figure 3 is the structural diagram of this module. Given an article containing k event trigger words Use the special marker " <t-i> "and"< / t-i> "For the trigger word t i Mark, where i represents the i-th event type; then obtain the input sequence X, send X to the RoBERTa encoder for encoding, and get Contextual representation of And send it to the decoder to get the event-oriented context embedding representation H X :
[0112]
[0113] Among them, Encoder(·) represents the RoBERTa encoder, and Decoder(·) represents the Transformer decoder;
[0114] In order to establish the co-occurrence relationship between events in the text, argument extraction constructs a slotted table and uses it as the input of the decoder;
[0115] The prompt for each event is fed into the encoder in parallel, resulting in an initialized column header representation:
[0116]
[0117] in, is the jth prompt at the input, is the embedding representation of the jth prompt, E i represents the i-th event, H CH is the connection of all prompts; the i-th row in the slot table starts with the i-th trigger word, followed by the parameter slot S of the i-th prompt i ; The initialization representation of the slot table is obtained by the following formula:
[0118]
[0119] in, is the embedding of the i-th trigger word, is the average embedding of the corresponding roles and triggers;
[0120] Then the non-autoregressive table decoder is used for decoding; the non-autoregressive table decoder uses the mask M Tab Make each element in the table focus on the area related to it; M Tab The specific design is as follows:
[0121] (1) All tokens in the column header follow each other;
[0122] (2) All tokens in the column header are related to the event trigger word;
[0123] (3) Each role and the corresponding argument slot pay attention to each other;
[0124] (4) Each event trigger word and the corresponding argument slot pay attention to each other;
[0125] M Tab Integrate it into the self-attention calculation process of the decoder to obtain the final slot table embedding:
[0126]
[0127] Embedding of argument slots from extracted from is the kth argument slot embedding, N a To explain the original number of slots; use linear transformation to transform each argument slot into a fragment selector:
[0128]
[0129] Among them, w s and w e is a trainable parameter; Together they form a fragment selector with k argument slots, which is used to extract the corresponding text fragment from the context embedding:
[0130]
[0131] in, Represents the probability distribution of each token as the starting position of the segment, represents the probability distribution of each token as the end position of the segment, l c is the length of the context, and are the indices of the start and end of the best segment;
[0132] Finally, the Hungarian algorithm is used to solve the assignment problem between the predicted argument fragments and the golden argument fragments, and this process is optimized by the bipartite matching loss, which is expressed as:
[0133]
[0134] Among them, (s k ,e k ) is the golden fragment index assigned to the kth argument slot.
[0135] Step 6: Use the optimal transmission module to realize the allocation of arguments and prototypes and optimize the model. Figure 4 For this module structure diagram, the prototype obtained in step 4 is used to guide the argument extraction in step 5 to learn the representation of the argument during training; since each type of argument is represented by multiple prototypes, assigning each parameter to the appropriate prototype is necessary to optimize the parameter representation. to represent the M d-dimensional prototypes of category r, Represents the set of arguments predicted to be r; is the embedding of the ith argument, obtained by average pooling:
[0136] The goal is to calculate the allocation matrix Here the problem is modeled as an optimal transmission problem for solution:
[0137]
[0138] in, is the cost matrix, indicating and There are two constraints here:
[0139] (1) The dot product with a vector of all 1s is equal to a, ensuring that Each row of has only one value of 1, so that each argument is assigned exactly one prototype;
[0140] (2) The dot product with a vector of all ones is equal to b, which ensures that γ r Each column of is not all zero, which avoids the situation where all arguments are assigned to only one prototype;
[0141] Then, the sinkhorn-knopp algorithm is used to find the optimal allocation matrix Figure 5 This is the sinkhorn-knopp pseudocode, and uses standard cross entropy to achieve that the arguments are relatively close to the prototype:
[0142]
[0143] Among them, pc To assign to The best prototype vector; in order to achieve compactness of argument features within the same subset cluster, the absolute distance between the argument features and their prototypes is further optimized:
[0144]
[0145] Add the three losses together for joint optimization training:
[0146]
[0147] where λ a and λ c is the loss weight hyperparameter. Update each prototype according to the parameter features assigned to each prototype, and at each training step t, use the exponential moving average EMA to update the prototype:
[0148]
[0149] where α is the EMA update rate; in this way, the learned prototype can be considered as a representation of a subcluster in the embedding space.
[0150] Example 2
[0151] Figure 6 The present invention is a schematic diagram of a hypersphere multi-prototype and optimal transmission method and device for event argument extraction, which mainly includes: a prototype initialization module, a prototype positioning module, an argument extraction module and an optimal transmission module.
[0152] The prototype initialization module is used to initialize multiple prototype vectors for each role type; the prototype positioning module is used to locate the prototype vector based on three aspects: inter-class relationship, intra-class relationship and category label semantic relationship; the argument extraction module is used to extract event arguments from the input text; the optimal transfer module assigns the extracted arguments to the most suitable prototype to guide the model to learn the embedded representation of the arguments.
[0153] The embodiment of the present invention provides a method and device for hypersphere multi-prototype and optimal transmission for event argument extraction, and specifically executes the above-mentioned embodiment process of the hypersphere multi-prototype and optimal transmission method for event argument extraction. For details, please refer to the content of the above-mentioned embodiment of the hypersphere multi-prototype and optimal transmission method for event argument extraction, which will not be repeated here.
[0154] This embodiment provides an electronic device, Figure 7The overall structure diagram of the electronic device provided in the embodiment of the present invention includes: a processor, a memory, a communication bus and a communication interface; wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining multiple prototype vectors for each category through unsupervised learning of hypersphere multi-prototypes; encoding text and prompt templates through a pre-trained model to obtain semantic representations of text and prompts; decoding text and prompts using a decoder to obtain event-oriented text semantic representations; extracting corresponding argument fragments from text based on a fragment selector; aligning prototypes with arguments through optimal transmission problem modeling, and guiding the model to learn embedded representations of arguments.
[0155] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and other media that can store program codes.
[0156] The present embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining multiple prototype vectors for each category through unsupervised learning of hypersphere multi-prototypes; encoding text and prompt templates through a pre-trained model to obtain semantic representations of the text and prompts; decoding the text and prompts using a decoder to obtain event-oriented text semantic representations; extracting corresponding argument fragments from the text based on a fragment selector; aligning the prototype with the argument through optimal transmission problem modeling, and guiding the model to learn the embedded representation of the argument.
[0157] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. Although the illustrative specific embodiments of the present invention are described above to facilitate the understanding of the present invention by the technical personnel in the field, it should be clear that the present invention is not limited to the scope of the specific embodiments. For the ordinary technical personnel in the field, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.
Claims
1. A hypersphere multi-prototype and optimal transmission device for event argument extraction, characterized in that: The device includes a prototype initialization module, an inter-class prototype positioning module, an intra-class prototype positioning module, a prototype sorting module based on label semantics, an argument extraction module and an optimal transmission module; The prototype initialization module is used to initialize multiple prototype vectors for each role category; the inter-class prototype positioning module is used to locate the inter-class prototype vectors based on the inter-class relationship; the intra-class prototype positioning module is used to locate the intra-class prototype vectors based on the intra-class relationship; the prototype sorting module based on label semantics is used to locate the inter-class prototype vectors based on the label semantic similarity; the argument extraction module is used to extract the arguments of the specified event from the text; the optimal transmission module is used to align the arguments with their multiple prototypes to optimize the model training and guide the model to learn the representation of the arguments, thereby achieving large-margin separation of inter-class arguments and effectively representing the intra-class variance, and improving the performance of event argument extraction.
2. A hypersphere multi-prototype and optimal transmission method for event argument extraction based on the device of claim 1, characterized in that: The following steps are involved: Step 1: Use the prototype initialization module to randomly initialize multiple prototype vectors for each category; Step 2: Use the inter-class prototype positioning module to maximize the inter-class prototype similarity as the optimization goal to locate the prototype vector in step 1; Step 3: Use the intra-class prototype positioning module to minimize the intra-class prototype similarity as the optimization goal to locate the prototype vector in step 1; Step 4: Use the prototype sorting module based on label semantics to further locate the prototype vector in step 1 using label semantic similarity as a guide; Step 5, extracting arguments of the event from the text using the argument extraction module; Step 6: Use the optimal transmission module to realize the allocation of arguments and prototypes and optimize the model.
3. The hypersphere multi-prototype and optimal transmission method for event argument extraction according to claim 2, characterized in that: The specific method of step 1 is to randomly initialize multiple prototype vectors for each category by using the prototype initialization module: using the prototype initialization module to randomly initialize N prototypes for each category Where P i is the prototype set of the i-th category, P i The j-th prototype vector in represents a set of real numbers, D represents the dimension of the hidden layer 1024, and the prototypes of all K categories can be expressed as 4. The hypersphere multi-prototype and optimal transmission method for event argument extraction according to claim 2, characterized in that: The specific method of locating the prototype vector in step 1 by using the inter-class prototype positioning module to maximize the inter-class prototype similarity as the optimization goal is: in each round of iteration during training, the prototype similarity with the greatest similarity to the current prototype is minimized to maximize the difference between classes, and the loss is calculated as follows: in, represents the loss of the prototype stage of locating classes, b(i) represents the category represented by the i-th prototype, d(·) represents the metric function of cosine similarity, for The transposed matrix of , E is the identity matrix, and M represents the prototype similarity matrix.
5. The hypersphere multi-prototype and optimal transmission method for event argument extraction according to claim 2, characterized in that: The specific method of using the intra-class prototype positioning module to minimize the intra-class prototype similarity as the optimization goal to locate the prototype vector in step 1 is: maximizing the similarity between the intra-class prototype vectors as the objective function, and maximizing the minimum similarity between these prototype vectors, which is specifically expressed as: in, represents the loss of the prototype stage within the localization class, and k represents the kth category.
6. The hypersphere multi-prototype and optimal transmission method for event argument extraction according to claim 2, characterized in that: The specific method of using the prototype ranking module based on label semantics to further locate the prototype vector in step 1 by taking label semantic similarity as a guide is that the distance between prototypes with similar category labels should be smaller than the distance between prototypes with different semantics, and an additional ranking loss is introduced to achieve this goal: o ijk =d(p i ,p j )-d(p i ,p k ) in, The true labels for the sorted role triplets, w i represents the embedded representation of the i-th category label encoded by the pre-trained model Bert, where G is the set of triplets of all roles, [[·]] is the indicator function, and o ijk Represents the prototype distance difference, p i , p j , p k Respectively represent the prototypes of the i-th, j-th and k-th categories, S ijk Represents the probability value of sorting; represents the sorting loss. By adding the three losses as the final loss And optimized by SGD optimizer: Finally, we get a set of pre-trained prototypes in, is the prototype set of the i-th category, represents the jth prototype of the i-th category.
7. The hypersphere multi-prototype and optimal transmission method for event argument extraction according to claim 2, characterized in that: The specific method of extracting event arguments from text using the argument extraction module in step 5 is: given an article containing k event trigger words Use the special marker " <t-i> "and" "For the trigger word t i Mark, where i represents the i-th event type; then obtain the input sequence X, send X to the RoBERTa encoder for encoding, and get Contextual representation of And send it to the decoder to get the event-oriented context embedding representation H X : Among them, Encoder(·) represents the RoBERTa encoder, and Decoder(·) represents the Transformer decoder; To establish the co-occurrence relationship between events in the text, argument extraction constructs a slot table and uses it as the input of the decoder; The prompt for each event is fed into the encoder in parallel, resulting in an initialized column header representation: in, is the jth prompt at the input, is the embedding representation of the jth prompt, E i represents the i-th event, H CH is the connection of all prompts; the i-th row in the slot table starts with the i-th trigger word, followed by the parameter slot S of the i-th prompt i ; The initialization representation of the slot table is obtained by the following formula: in, is the embedding of the i-th trigger word, is the average embedding of the corresponding roles and triggers; Then the non-autoregressive table decoder is used for decoding; the non-autoregressive table decoder uses the mask M Tab Make each element in the table focus on the area related to it; M Tab The specific design is as follows: (1) All tokens in the column header follow each other; (2) All tokens in the column header are related to the event trigger word; (3) Each role and the corresponding argument slot pay attention to each other; (4) Each event trigger word and the corresponding argument slot pay attention to each other; M Tab Integrate it into the self-attention calculation process of the decoder to obtain the final slot table embedding: Embedding of argument slots from extracted from is the kth argument slot embedding, N a To explain the original number of slots; use linear transformation to transform each argument slot into a fragment selector: Among them, w s and w e is a trainable parameter; Together they form a fragment selector with k argument slots, which is used to extract the corresponding text fragment from the context embedding: in, Represents the probability distribution of each token as the starting position of the fragment, represents the probability distribution of each token as the end position of the segment, l c is the length of the context, and are the indices of the start and end of the best segment; Finally, the Hungarian algorithm is used to solve the assignment problem between the predicted argument fragments and the golden argument fragments, and this process is optimized by the bipartite matching loss, which is expressed as: Among them, (s k , e k ) is the golden fragment index assigned to the kth argument slot.
8. The hypersphere multi-prototype and optimal transmission method for event argument extraction according to claim 2, characterized in that: The step 6 uses the optimal transmission module to achieve the allocation of arguments and prototypes. The specific method for optimizing the model is: Using the prototypes obtained in step 4, the argument extraction in step 5 is guided to learn the representation of the arguments during training; since each type of argument is represented by multiple prototypes, assigning each argument to the appropriate prototype is necessary to optimize the parameter representation. to represent the M d-dimensional prototypes of category r, Represents the set of arguments predicted to be r; is the embedding of the ith argument, obtained by average pooling: The goal is to calculate the allocation matrix Here the problem is modeled as an optimal transmission problem for solution: in, is the cost matrix, indicating and There are two constraints here: (1) The dot product with a vector of all 1s is equal to a, ensuring that Each row of has only one value of 1, so that each argument is assigned exactly one prototype; (2) The dot product with a vector of all 1s is equal to b, which ensures that γ r Each column of is not all zero, which avoids the situation where all arguments are assigned to only one prototype; Then, the sinkhorn-knopp algorithm is used to find the optimal allocation matrix Figure 5 shows the sinkhorn-knopp pseudocode, and uses standard cross entropy to achieve that the argument is relatively close to the prototype: Among them, p c To assign to The best prototype vector; in order to achieve compactness of argument features within the same subset cluster, the absolute distance between the argument features and their prototypes is further optimized: Add the three losses together for joint optimization training: where λ a and λ c is the loss weight hyperparameter. Update each prototype according to the parameter features assigned to each prototype, and at each training step t, use the exponential moving average EMA to update the prototype: where α is the EMA update rate; in this way, the learned prototype can be considered as a representation of a subcluster in the embedding space.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the hypersphere multi-prototype and optimal transmission method for event argument extraction as described in any one of claims 2 to 8 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for hypersphere multi-prototype and optimal transmission for event argument extraction as described in any one of claims 2 to 8 is implemented.
Citation Information
Cited By
Equipment state detection method and device, equipment and medium
CN122241197A
A device state detection method, apparatus, device and medium
CN122241197B
Systems and methods for federated computer processing architecture and coordinated broadcast data messaging
US12743278B2
