Small sample event extraction method based on T5 and knowledge graph data enhancement
By combining T5 and knowledge graphs, the problems of insufficient data distribution and semantic diversity in small sample event extraction are solved, and efficient event extraction is achieved under low resource conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INST OF AUTOMATION CHINESE ACAD OF SCI
- Filing Date
- 2026-01-06
- Publication Date
- 2026-05-01
Smart Images
Figure CN121958566A_ABST
Abstract
Description
A method for small-sample event extraction based on T5 and knowledge graph data augmentation Technical Field
[0001] This invention relates to the field of natural language processing technology, specifically to a method for small sample event extraction based on T5 and knowledge graph data augmentation. Background Technology
[0002] Event extraction is one of the core tasks of information extraction. It aims to transform semi-structured or unstructured text data into structured data, accurately extract key event information from the text, and mine the implicit relationships in the text to provide support for decision analysis. It can be used in fields such as knowledge graph construction, scientific literature mining, historical event analysis, medical event monitoring, and electromechanical equipment fault diagnosis.
[0003] With the development of deep learning technology, event extraction models have achieved good results when the sample size is sufficient. However, in real-world applications, high-quality labeled data for specific domains is often difficult to obtain, leading to the challenge of few-shot learning for these models. To address the problem of data scarcity, data augmentation techniques are widely used to expand the training set.
[0004] However, traditional data augmentation methods still have limitations in existing Chinese small-sample event extraction research. On the one hand, existing augmentation methods struggle to balance enhancing the distribution consistency and semantic diversity of the data. Simple synonym replacement is insufficient to provide adequate contextual variation, while generative model-based methods, if lacking constraints, can easily generate texts with significant stylistic or semantic deviations from the original, resulting in augmented samples that cannot effectively simulate the distribution of real data and hindering the improvement of model generalization ability. On the other hand, existing augmentation strategies typically lack protection mechanisms for the core elements of the event. Rewriting or replacing text can easily alter the original event structure, rendering the original annotation information invalid and introducing noisy data into the training set. Manual re-annotation to correct labels increases the cost of data construction. For these reasons, the event extraction accuracy of existing methods under low-resource conditions still needs improvement. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides a small-sample event extraction method based on T5 and knowledge graph data augmentation. This method solves the problems in existing small-sample event extraction methods for the Chinese domain, such as insufficient balance between the distribution similarity and semantic diversity of data before and after augmentation, which leads to limited sample richness and poor event extraction performance under low resource conditions.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A few-sample event extraction method based on T5 and knowledge graph data augmentation includes: using a masked fragment selection module to segment the original text according to preset delimiters and selecting the longest fragment as the masked object; using a fine-tuning model generation module to replace the masked object in the original text with special markers and construct a reading comprehension fill-in-the-blank question; using a fine-tuned T5 model to predict the reading comprehension fill-in-the-blank question, generating new fragments and backfilling them to obtain preliminary augmented text; using a knowledge graph augmentation module based on a Chinese concept knowledge graph to replace nouns or pronouns in the preliminary augmented text with similar entities except for key information to obtain new samples; using a Chinese event extraction module to merge the new samples with the original text to form an augmented dataset; and using the augmented dataset to drive the training of the event extraction model to extract trigger words and arguments.
[0008] Preferably, when segmenting the original text, the process of determining the delimiter includes: identifying trigger words and arguments in the original text, and defining them as a set of trigger words and a set of arguments, respectively; taking the union of the set of trigger words and the set of arguments to obtain an immutable set; and combining the immutable set with a preset set of punctuation marks to form a delimiter for segmentation.
[0009] Preferably, the process of selecting the longest segment includes: calculating the length of each segment after segmentation using a preset formula, determining the segment with the longest length as the candidate masking object; and determining whether the masking object simultaneously meets the preset boundary and content constraints to confirm the final selection result.
[0010] Preferably, the process of constructing reading comprehension fill-in-the-blank questions includes: replacing the masked objects in the original text with special markers to form a sequence to be completed; splicing a preset natural language prompt at the end of the sequence to be completed; and generating reading comprehension fill-in-the-blank questions using preset construction logic.
[0011] Preferably, the process of using the fine-tuned T5 model for prediction includes: selecting an external dataset that is similar to the original text in distribution or style; using the external dataset to construct a supervised training task to update the parameters of the pre-trained model, thereby obtaining the fine-tuned T5 model; and using the fine-tuned T5 model in conjunction with a Top-N decoding strategy to answer reading comprehension questions and generate new segments that meet the preset prediction constraints.
[0012] Preferably, the process of backfilling to obtain the preliminary enhanced text includes: selecting the generated text fragment as the new fragment from multiple candidate results output by the Top-N decoding strategy; and using a preset replacement logic to fill the generated text fragment back into the original text at the location of the masked object, thereby obtaining the preliminary enhanced text.
[0013] Preferably, the construction process of the Chinese concept knowledge graph includes: extracting triples that conform to hierarchical or attribution relationships from the basic data source; constructing a hierarchical structure based on the triples that includes entities, specific concepts, and the attribution of the top-level concept; and using the hierarchical structure to generate a Chinese concept knowledge graph that supports path queries from entities and concepts to similar entities.
[0014] Preferably, the process of replacing similar entities includes: defining the trigger words and arguments marked in the original text as key information; performing part-of-speech tagging on the preliminarily enhanced text to identify nouns and pronouns, removing words belonging to key information, and obtaining a candidate word set; using a Chinese concept knowledge graph to retrieve the parent concept node of any candidate word in the candidate word set; using the parent concept node as an anchor point to find sibling nodes of the same level as replacement words; and using a preset replacement logic to replace the candidate words with replacement words to generate new samples.
[0015] Preferably, the process of constructing an augmented dataset includes: defining original samples containing original text and corresponding event annotation information; configuring new samples to directly inherit the event annotation information of the original samples; and using a preset construction logic to merge all original samples with multiple corresponding new samples to construct an augmented dataset.
[0016] Preferably, the process of training the event extraction model includes: establishing a mapping relationship from the definition layer to the processing layer, defining the cross-entropy loss function and gradient optimization algorithm; converting the augmented dataset into a vector matrix and inputting it into the event extraction model to obtain prediction results; calculating the prediction error between the prediction results and the true labels based on the cross-entropy loss function; and updating the model parameters according to the prediction error using the gradient optimization algorithm.
[0017] This invention provides a method for small-sample event extraction based on T5 and knowledge graph data augmentation. It has the following beneficial effects:
[0018] 1. This invention uses a fine-tuned T5 model to predict and backfill the longest masked object selected, maintaining the consistency of the distribution of the enhanced text and the original text in terms of language style and syntactic structure. At the same time, it uses Chinese concept knowledge graph to replace nouns or pronouns of the same type with non-critical information, introducing external knowledge entities. This two-stage strategy combining generative models and knowledge graphs solves the problem that existing methods are difficult to balance data distribution similarity and semantic diversity in small sample enhancement, and enriches the semantic features of the samples.
[0019] 2. In the process of masking object selection and entity replacement, the present invention pre-constructs an immutable set containing trigger words and arguments to ensure that the generated preliminary enhanced text and the final new sample retain the core structure of the original event. This allows the enhanced sample to directly inherit the event labeling information of the original sample without manual re-labeling, reducing data processing costs and avoiding the introduction of noisy data into the training set due to the enhancement operation destroying the core semantics of the event.
[0020] 3. This invention selects the longest segment as the masking object, prompting the model to recover long-distance semantic information using the remaining context. Combined with the entity expansion of the knowledge graph, it constructs an enhanced dataset containing multi-source features. This dataset is used to drive the training of the event extraction model, enabling the model to learn more generalized context features under small sample conditions. This improves the extraction effect of trigger words and arguments in unstructured text and is applicable to fields such as knowledge graph construction and equipment fault record analysis. Attached Figure Description
[0021] Figure 1 is a diagram of the overall algorithm framework of the present invention;
[0022] Figure 2 is an example diagram of the problem construction of the fine-tuning model generation module of the present invention;
[0023] Figure 3 is a framework diagram of the knowledge graph enhancement module of the present invention;
[0024] Figure 4 is a flowchart of the training process of the Chinese event extraction model using the enhanced samples of the present invention. Detailed Implementation
[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] Referring to Figure 1, the present invention provides a small sample event extraction system based on T5 and knowledge graph data augmentation. The system mainly consists of four core processing modules: a mask fragment selection module, a fine-tuning model generation module, a knowledge graph augmentation module, and a Chinese event extraction module.
[0027] The mask fragment selection module receives the original text from the sample data to be enhanced, segments the original text according to preset delimiters (including punctuation marks and unmodifiable trigger words and arguments), selects the longest fragment from the segmented fragment set as the object to be masked, and outputs text data containing masking position information.
[0028] The fine-tuning model generation module is connected to the mask fragment selection module. It receives mask location information, replaces the mask fragment with the special identifier "[ANS]", constructs a reading comprehension fill-in-the-blank question, and then uses the fine-tuned T5 model to predict the question, generating new text fragments to replace the special identifier, and outputting the initially enhanced text.
[0029] The knowledge graph enhancement module is connected to the fine-tuning model generation module. It receives the initially enhanced text and uses the pre-built Chinese concept knowledge graph (CN-DBpedia) to perform entity replacement at the concept level for nouns or pronouns in the text, except for key information, and outputs new samples with diverse semantics.
[0030] The Chinese event extraction module is connected to the knowledge graph enhancement module. It receives new samples and merges them with the original samples to form an enhanced dataset. This dataset drives the training of a preset event extraction model, and finally outputs a model with event extraction capabilities.
[0031] This invention provides a method for small sample event extraction based on T5 and knowledge graph data augmentation, comprising the following steps:
[0032] S1, the mask fragment selection module is used to segment each original text in the sample data to be enhanced. The segmentation delimiters are selected as Chinese punctuation marks, English punctuation marks, and trigger words and arguments marked in the original samples. The original text is cut into a set of text fragments as a set of fragments to be selected, and the longest fragment is selected from the fragment set as the mask fragment of the event text.
[0033] S2, using the fine-tuning model generation module to receive the masked fragment of the event text and the original text, using the special symbol "[ANS]" to replace the masked fragment of the event text in the original text, adding the corresponding question description to convert the text into a reading comprehension fill-in-the-blank question, and using the T5 model fine-tuned with data similar to the sample style to answer the question, generating a new text fragment and backfilling it into the masked position, thus obtaining the new text after T5 conversion;
[0034] S3: Receive the new text after T5 transformation using the knowledge graph enhancement module. Use Chinese knowledge graph to extract nodes and relationships that conform to the theme or concept definition to construct a conceptual knowledge graph. Filter out nouns and some pronouns as replacement words from the remaining text in the new text after T5 transformation, excluding trigger words, arguments, and replaced text fragments. Use the conceptual knowledge graph to find corresponding similar words and randomly replace them to obtain the final new sample.
[0035] S4. The Chinese event extraction module is used to merge the final new sample output by the knowledge graph enhancement module with the original sample containing the original text to form a new dataset. The Chinese event extraction model is selected and the loss function and gradient optimization algorithm are defined. The event extraction model is trained with this new dataset as the driving force to achieve the extraction of trigger words and arguments in unstructured text.
[0036] The following section will provide a detailed explanation of steps S1 to S4, combining specific implementation details and formula derivation.
[0037] Referring to Figure 1, step S1 describes in detail how to use the mask fragment selection module to segment and filter each original text in the sample data to be enhanced, specifically including the following sub-steps:
[0038] S101 defines the original text data and the set of data that cannot be modified. The mask fragment selection module first reads the small sample dataset to be augmented. Each sample contains a raw sentence to be processed. Define the original sentence. Let be a sequence consisting of several characters, whose total length is denoted as . Original sentence The character sequence is represented as:
[0039] ;
[0040] in, Indicates the first in the sentence Characters, index The value range is from 0 to .
[0041] To maintain the syntactic structural integrity of the enhanced text, the mask fragment selection module pre-sets a set of delimiters. This set It includes Chinese and English punctuation marks, specifically commas, periods, exclamation marks, question marks, colons, semicolons, and quotation marks.
[0042] The mask fragment selection module utilizes a set of delimiters. and the immutable set defined in step S101 Together, they serve as the segmentation boundary for the original sentence. Perform a scan. The specific segmentation logic is as follows: when the scanned character belongs to... or When a character is selected, it is treated as a delimiter and not included in the candidate segment; the continuous character sequence between two adjacent delimiters is extracted to form an independent text segment. The original sentence... All the text fragments obtained after segmentation constitute the set of fragments to be selected, denoted as . .
[0043] Taking the text shown in Figure 1 as an example, the original sentence The sentence reads: "Yesterday, the company released its financial report, and after the release, the stock price fell below the issue price." The words "released" and "stock price" are used interchangeably. , and ." belong to The mask segment selection module obtains Seg based on the above delimiters, including: "Yesterday the company released its financial report", "released", "after the financial report", "stock price", "fell below the issue price".
[0044] S103, Select the longest masking fragment. To ensure that the generated new samples are highly distinguishable from the original samples in terms of non-critical information, the masking fragment selection module calculates a set. The character length of each segment is calculated, and the longest segment is selected as the target segment to be masked. The longest selected masking segment is defined as... The mask fragment selection module uses the longest mask fragment definition formula to determine the longest mask fragment. The longest mask fragment definition formula is:
[0045] ;
[0046] At the same time, the selected segments must satisfy the masked segment selection constraint formula:
[0047] ;
[0048] in:
[0049] Indicates from set The text segment with the longest length value selected from the options; Represents the original sentence From index Start to Index The ending character subsequence;
[0050] and These represent the segments within the original sentence. The start character index and the end character index in the text;
[0051] This represents the set of fragments to be selected generated in step 5102;
[0052] Represents the original sentence The total length;
[0053] This indicates that the selected segment index must be within the valid sentence length range, and the segment length must be at least 1;
[0054] This indicates a logical constraint relationship, meaning that the selected segment must satisfy both the preceding and following conditions.
[0055] This indicates the currently selected candidate segment;
[0056] This represents the union of trigger words and arguments;
[0057] The union operation represents the set operation;
[0058] This indicates that the element does not belong to the set;
[0059] Indicates the selected segment It does not contain any trigger words, arguments, or punctuation marks.
[0060] As a specific implementation of this embodiment, if the set If there are multiple longest segments of the same length, the mask segment selection module can randomly select one of them, or select one from the original sentence. The one with the first index position is used as As shown in Figure 1, the fragment "Yesterday the company released its financial report" has a length of 9, which is the longest in the set, so it was selected as the masking object.
[0061] Referring to Figures 1 and 2, step S2 details how the fine-tuning model generation module generates semantically coherent new text fragments through a reading fill-in-the-blank task, specifically including the following sub-steps:
[0062] S201, Construct a reading comprehension fill-in-the-blank question.
[0063] The fine-tuning model generation module receives the original sentence output by the mask fragment selection module. And the longest masking segment determined In order to recover the masked information using a generative model, the fine-tuning model generation module first performs a format conversion operation.
[0064] Specifically, the fine-tuning model generation module uses preset special identifiers. (In the specific example in Figure 2, this is represented by "[ANS]") Replace the original sentence text fragments After the replacement is complete, in order to guide the model to focus on the masked area, the fine-tuning model generation module will append a natural language prompt to the end of the replaced text sequence.
[0065] As shown in the red dotted box at the bottom of Figure 2, taking a financial text as an example, the input sequence formed after processing the original text is: "On October 21, Guangzheng Group opened lower and continued to decline, eventually hitting the daily limit down with high volume, [ANS].", followed by the prompt "The text at the [ANS] position is: ."
[0066] The construction process uses the formula for constructing reading comprehension and fill-in-the-blank problems. The formula for constructing reading comprehension and fill-in-the-blank problems is as follows:
[0067] ;
[0068] in:
[0069] This represents the completed sequence of fill-in-the-blank reading comprehension questions, which serves as the input to the model.
[0070] This represents the original sentence;
[0071] This represents the longest masking segment determined in step S1;
[0072] Indicates a placeholder for the answer (e.g., "[ANS]");
[0073] Indicates will substring in Replace with Operation;
[0074] This represents a function that encapsulates the problem, used to combine the replaced text with a preset prompt.
[0075] S202, configure and fine-tune the T5 generation model.
[0076] As shown in Figure 2, the fine-tuning model generation module uses the T5 (Text-to-Text Transfer Transformer) model as its base. The model architecture contains a large blue solid-line box in the middle, and its internal logic is divided into two parts, corresponding to the encoder and decoder, respectively.
[0077] In Figure 2, the English technical terms involved and their meanings in this embodiment are explained as follows:
[0078] Self-Attention: In Chinese, it means self-attention mechanism, which is used to capture long-distance dependencies within the input sequence;
[0079] Add & Normalize: This refers to residual connections and layer normalization, used to solve the gradient vanishing problem in deep networks and accelerate convergence.
[0080] Feed Forward: In Chinese, it means feedforward neural network, which is responsible for performing nonlinear transformations on features;
[0081] DECODER: In Chinese, it means decoder. In the Transformer architecture, it is the component responsible for generating target text based on encoded features.
[0082] Top-N Decoding Strategy: This refers to a strategy that samples from the N candidates with the highest probability when generating text.
[0083] During the fine-tuning phase, the model generation module for fine-tuning is selected based on the original small sample dataset. External datasets that are similar in distribution or style A supervised training task is constructed to update the parameters, resulting in a fine-tuned model. This process is represented by the T5 model fine-tuning formula:
[0084] ;
[0085] in:
[0086] This represents the set of parameters for the T5 model after fine-tuning and training.
[0087] This represents the initial T5 model after pre-training;
[0088] This indicates the selected external augmentation dataset;
[0089] This indicates the model fine-tuning training operation.
[0090] S203, Text Generation and Reconstruction Based on Top-N Strategy.
[0091] The fine-tuning model generation module will construct the problem for S201. The data is input into the fine-tuned model. As shown in Figure 2, the data flows into the model along the wide gray arrow at the bottom and is processed along the following path:
[0092] Encoding path (Encoder):
[0093] The fine-tuning model generation module will construct the problem for S201. Input the vector sequence into the fine-tuned model. The sequence enters the encoder layer, where it sequentially passes through a self-attention layer to capture internal dependencies, a residual connection and layer normalization (Add&Normalize) layer for numerical stabilization, and a feedforward neural network layer for feature transformation, ultimately outputting a high-dimensional feature vector containing contextual semantics.
[0094] Decoder path:
[0095] The decoder layer receives the output features from the encoder, aggregates the feature information through multi-layer self-attention and cross-attention mechanisms, and then processes them through residual connections and layer normalization. The processed feature vector is input to the output layer of the model and combined with the Top-N decoding strategy module for probability calculation and sampling.
[0096] To ensure the effectiveness of the predictions and introduce sample diversity, the fine-tuning model upon which the generation process relies must satisfy preset data distribution constraints. These constraints are defined by the new text fragment generation prediction formula, which is as follows:
[0097] ;
[0098] in: This represents the style distribution similarity between datasets, i.e., the external dataset used for fine-tuning. Needs to match the target original dataset Having similar language styles or domain characteristics (e.g., both being in the financial news field) ensures that the text fragments generated by the model are suitable for the original context.
[0099] During the generation process, the model outputs multiple candidate sequences based on the calculated probability distribution. As shown in Figure 2, the model outputs a list of multiple candidate answers, including:
[0100] Some investors expressed relief that they hadn't bought in.
[0101] Shareholders who bought the stock are full of complaints.
[0102] The price was 8.9 yuan.
[0103] The fine-tuning model generation module selects a generated text fragment (e.g., option (2)) from the Top-N candidate results as the model. Then, a reverse replacement operation is performed to generate the fragment. Fill in the original sentence Central Plains The location is specified. This process uses a generative enhanced text replacement formula for calculation. The generative enhanced text replacement formula is:
[0104] ;
[0105] in: This indicates the enhanced intermediate text generated after the model has been fine-tuned; This represents a new text fragment generated by the model based on a Top-N strategy, used to fill in the masked positions. This indicates a backfilling operation, which means covering up the original sentence. The location content has been updated to .
[0106] Through the above steps, the fine-tuning model generation module has achieved diversified reconstruction of the background, modifiers or non-critical descriptions of the event without changing the core elements of the event (such as trigger words and arguments).
[0107] Referring to Figures 1 and 3, step S3 details how the knowledge graph enhancement module introduces external explicit knowledge and further expands the diversity of samples through entity replacement, specifically including the following sub-steps:
[0108] S301, Constructing a Chinese conceptual knowledge graph.
[0109] The knowledge graph enhancement module first establishes an external knowledge base for entity association queries. In this embodiment, CN-DBpedia is selected as the basic data source. Since the original CN-DBpedia contains a massive amount of triple data, many attribute relationships (such as "date of birth" and "height") are not suitable for semantic replacement of similar entities. Therefore, the knowledge graph enhancement module performs preprocessing and filtering operations.
[0110] As shown in the conceptual CN-DBpedia structure on the left side of Figure 3, the knowledge graph enhancement module extracts triples that conform to "hierarchical relationship" or "attribution relationship" from CN-DBpedia to construct a hierarchical graph. Specifically, the graph contains multiple levels of parent-child node connections:
[0111] Level 1 (Entities and Specific Concepts): As shown on the left side of Figure 3, the central parent node "Price" is connected to the child nodes "Issuance Price," "Opening Price," "Closing Price," and "Lowest Price" via lines marked "Belongs to." This indicates that the above child nodes are all specific instances under the "Price" indicator system.
[0112] Level 2 (Market Affiliation): The node "A-shares" points to the parent node "market" through the line marked "belongs to"; specific securities nodes "stocks", "funds", and "bonds" point to the parent node "securities" through the line marked "belongs to".
[0113] Level 3 (Top-level concept attribution): Both the nodes "Market" and "Securities" are connected by lines marked "belong to", ultimately pointing to the higher-level root concept node "Finance".
[0114] The knowledge graph enhancement module uses the above structure to reorganize unstructured encyclopedia data into a structured graph that supports "entity-concept-similar entity" path queries. .
[0115] S302, Filter the set of candidate words to be replaced.
[0116] The knowledge graph enhancement module receives intermediate text output by the fine-tuning model generation module. To enhance the data without compromising the core semantic structure of the events, the knowledge graph enhancement module first... Part-of-speech tagging is performed to identify all nouns and pronouns in the text.
[0117] Subsequently, the knowledge graph enhancement module executes a filtering strategy: from the identified nouns and pronouns, it removes those belonging to the immutable set defined in step S1. The vocabulary (i.e., trigger words and arguments). After filtering, the remaining nouns and pronouns constitute the candidate word set to be replaced, denoted as . .
[0118] S303, Entity replacement based on conceptual consistency.
[0119] The knowledge graph enhancement module traverses the candidate word set. For each word in the set, a graph constructed using S301 is used for retrieval and replacement. As shown on the right side of Figure 3, this process presents a bottom-up workflow, supported by data from the graph on the left via wide gray arrows:
[0120] Input Phase: As shown in the red dotted box at the bottom of Figure 3, the knowledge graph enhancement module 30 reads the text to be processed: "Yesterday, the company released its financial report, and after the release, the stock price fell below the issue price." The words "released" (trigger word) and "stock price" (argument) in red are locked for protection, while "issue price" in black is selected as the word to be replaced. .
[0121] Conceptualization stage: The knowledge graph enhancement module utilizes conceptualized knowledge graphs. The hierarchical indexing capability allows for the retrieval of the corresponding parent concept node within the graph, using the candidate term "issue price" as the query object. The search results form a concept tuple "<issue price, price>", clearly defining the concept category to which "issue price" belongs as "price".
[0122] In the entity matching phase (Instantiation): The knowledge graph enhancement module 30 uses the retrieved parent concept node "price" as an anchor point to traverse downwards in the graph and search backwards for sibling nodes belonging to the same parent node. The search results generate a list of similar entities "[closing price, opening price, lowest price...]".
[0123] Output Phase (Replacement): Knowledge Graph Enhancement Module 30 randomly selects the entity "closing price" as the replacement word from the above list. , regarding the original word The replacement is performed. As shown in the red dashed box at the top of Figure 3, the final output is the enhanced sample. "The company released its financial report yesterday, and the stock price fell below the closing price after the report was released."
[0124] This process uses the knowledge graph entity replacement formula for calculation. The knowledge graph entity replacement formula is as follows:
[0125] ;
[0126] in:
[0127] This represents the final enhanced text sample generated after entity replacement from the knowledge graph.
[0128] This represents the intermediate text output from step S2, which has been filled by the T5 model.
[0129] Indicates from The set of candidate words to be replaced selected from the data, and satisfying the following conditions: ;
[0130] Representing conceptual knowledge graphs Searched and selected from, and Similar entity nodes that share a common parent node;
[0131] This represents the Chinese conceptual knowledge graph constructed in step S301;
[0132] This indicates that an element belongs to a set;
[0133] Indicates text vocabulary in Replace with entity The operation.
[0134] Through the above steps, the knowledge graph enhancement module replaces "issue price" with "closing price" while ensuring the logical coherence of sentences and concept consistency. This effectively introduces external knowledge entities, increases the diversity of entity objects in the samples, and thus improves the model's ability to generalize perception of different entities.
[0135] Referring to Figures 1 and 4, step S4 details how to construct the final training set and optimize the event extraction model, specifically including the following sub-steps:
[0136] S401, Merging and constructing multi-source augmented datasets.
[0137] The Chinese event extraction module receives a small sample dataset to be augmented. (Includes original samples) (and the final enhanced samples output by the knowledge graph enhancement module) To fully utilize the generated semantically diverse data, the Chinese event extraction module performs a data mixing and recombination operation. In this embodiment, since the sets of trigger words and arguments are located in the preceding steps S1 to S3... The sample is protected and has not been modified; therefore, it is an enhanced sample. It maintained the same consistency with the original sample Consistent event semantic structure. Enhanced sample configuration for the Chinese event extraction module. Directly inherit the corresponding original sample The event annotation information (including event type labels, trigger word position indexes, argument role labels, etc.) does not require manual re-annotation. Subsequently, the Chinese event extraction module merges all original samples with multiple corresponding augmented samples to construct the final augmented dataset. The process uses an augmented dataset construction formula for calculation. The augmented dataset construction formula is as follows:
[0138] ;
[0139] in: This represents the completed multi-source augmented dataset containing both original and augmented data, corresponding to the "Augmented Sample Dataset" in the upper right corner of Figure 4; This represents the original text samples in the original small sample dataset; This represents the enhanced text sample obtained after fine-tuning the model generation and knowledge graph replacement. (Upsilon) represents the maximum index value of a sample in the original dataset (i.e., the total number of samples minus one); (Beta) represents the number of augmented samples generated for each original sample (i.e., the augmentation factor). Indicates the first One original sample; Indicates the first The first original sample corresponding to the first One enhanced sample.
[0140] S402, Parameter configuration and initialization of the event extraction model.
[0141] Before training, the Chinese event extraction module needs to establish the initialization logic shown in the upper configuration layer of Figure 4, and establish the mapping relationship from the definition layer to the processing layer:
[0142] Model architecture definition: The Chinese event extraction module performs the "Select Event Extraction Model" operation, choosing a preset deep learning model (such as BERT or RoBERTa) as the base. As shown in Figure 4, this operation points down to "Model Output", indicating that the structure of the selected model determines the dimension and format of the output tensor.
[0143] Loss function definition: The Chinese event extraction module executes the "Define Loss Function" operation, specifying the algorithm used to measure prediction error (such as cross-entropy loss). As shown in Figure 4, this operation leads down to "Loss Function Calculation," providing a calculation basis for subsequent training.
[0144] Optimization Algorithm Definition: The Chinese event extraction module executes the "Define Gradient Optimization Algorithm" operation to configure the optimizer (such as AdamW). As shown in Figure 4, this operation points down to "Gradient Optimization," which controls the step size and direction of parameter updates.
[0145] Data access definition: The Chinese event extraction module loads the "enhanced sample dataset" constructed in step S401. As shown in Figure 4, this operation points down to "enhanced sample input", converting the static dataset into an input stream that can be read by the model.
[0146] S403, Model-Driven Training and Gradient Optimization.
[0147] The Chinese event extraction module will enhance the dataset. The data is input into the configured event extraction model, and iterative training is performed. As shown in Figure 4, this process forms a closed-loop optimization path, specifically including the following three stages:
[0148] Enhance the sample input stream:
[0149] The Chinese event extraction module reads batch data from the augmented sample dataset, including the original text and multiple generated augmented texts (such as "original sentence" and "new sentence 1" to "new sentence N"). The module converts the text data into a vector matrix form that the model can process. Specifically, taking the data shown in Figure 4 as an example, the input text is: "Yesterday the company released its financial report, and after the release, the stock price fell below the closing price." Here, "released" and "stock price" are pre-labeled trigger words and arguments, while "closing price" reflects the financial entity features augmented by the knowledge graph. This vectorized data is then fed into the event extraction model.
[0150] Model forward propagation and computation:
[0151] The event extraction model receives an input vector and performs encoding and decoding calculations to extract text features. During this process, the model parameters are updated based on a pre-defined training strategy.
[0152] Error calculation: The difference between the model's current predicted value and the true label is calculated using the cross-entropy loss function;
[0153] Parameter update: Using the gradient optimization algorithm, backpropagation is performed based on the calculated gradient values to update the weight parameters inside the model.
[0154] Structured Event Output: After forward propagation, the event extraction model outputs the prediction results. As shown in Figure 4, for the input text above, the structured information finally output by the model specifically includes...
[0155] Event Type: Finance / Enterprise - Release;
[0156] Trigger word: Publish;
[0157] Argument Role: Variable Indicators;
[0158] Stock price: a measure of a stock's value.
[0159] Through the above process, the Chinese event extraction module 40 uses enhanced samples containing diverse entities such as "closing price" to drive training, enabling the model to accurately capture core semantics even in low-resource scenarios.
[0160] This invention provides a non-transitory computer-readable storage medium on which a computer program or instructions are stored. The computer-readable storage medium can be any device capable of storing code, including but not limited to: electrical storage devices (such as random access memory (RAM) and read-only memory (ROM), magnetic storage devices (such as hard disk drives (HDDs), optical storage devices, and semiconductor storage devices (such as solid-state drives (SSDs)).
[0161] When a computer program or instruction is read and executed by the processor of an electronic device, the electronic device performs the steps described above for the low-resource Chinese event extraction method based on generative fine-tuning and knowledge graph enhancement, specifically including:
[0162] Read the original text from the sample data to be augmented, segment the original text according to the preset delimiters, and select the longest masked segment. Replace the masked fragment with a special identifier. And construct reading comprehension fill-in-the-blank questions New text fragments were generated using a finely tuned T5 model. The text is then backfilled to obtain preliminary enhanced text; a pre-constructed Chinese concept knowledge graph (CN-DBpedia) is used to perform entity replacement at the concept level on nouns or pronouns in the text, generating new samples with diverse semantics. ; and merging new samples with original samples to form an augmented dataset. And drive the event extraction model to be trained.
[0163] The electronic device is intended to serve as a hardware platform for executing the aforementioned algorithm, and it mainly includes: a processor 501, a memory 502, a communication interface 503, and a communication bus 504.
[0164] The processor 501 is the control center of the electronic device and may include one or more central processing units (CPUs), and is preferably configured with a graphics processing unit (GPU) or tensor processing unit (TPU) for accelerating deep learning model inference and training. The processor 501 is connected to and controls other components via a communication bus 504.
[0165] The memory 502 is used to store computer programs and various types of data required to execute the above methods. In this embodiment, the memory 502 specifically stores:
[0166] Operating system and application instructions;
[0167] Original small sample dataset ;
[0168] Weight parameters of the T5 model after pre-training and fine-tuning;
[0169] Structured Chinese concept knowledge graph (CN-DBpedia) data;
[0170] Preset event extraction model parameters.
[0171] The communication interface 503 is used for electronic devices to interact with external devices or networks, such as receiving external input of a text stream to be extracted or downloading updated knowledge graph data.
[0172] During operation, processor 501 retrieves and executes computer program instructions from memory 502, thereby forming and running the following functional modules at the logical level:
[0173] Mask Fragment Selection Module: This module corresponds to the logical state of the processor 501 when executing text processing instructions. The processor 501 is configured to receive the original text in the sample data to be enhanced, segment the original text according to preset delimiters (including punctuation marks and unmodifiable trigger words and arguments), and use a comparison algorithm to select the longest fragment from the segmented fragment set as the object to be masked, outputting text data containing masking position information.
[0174] Fine-tuning model generation module: This corresponds to the logical state of the processor 501 using the T5 model for inference. The processor 501 is configured to receive masking location information, replace the masked fragment with the special identifier "[ANS]", construct a reading comprehension fill-in-the-blank question, then load the fine-tuned T5 model parameters stored in memory 502 to predict the question, use a Top-N sampling strategy to generate new text fragments to replace the special identifier, and output the preliminarily enhanced text.
[0175] Knowledge Graph Enhancement Module: Corresponds to the logical state of processor 501 executing graph query and replacement instructions. Processor 501 is configured to receive preliminarily enhanced text, access the Chinese Concept Knowledge Graph (CN-DBpedia) stored in memory 502, perform concept-level retrieval and entity replacement on nouns or pronouns in the text except for key information (trigger words and arguments), and output semantically diverse new samples.
[0176] Chinese event extraction module: This corresponds to the logical state of processor 501 executing model training and gradient update instructions. Processor 501 is configured to receive new samples and merge them with the original samples to form an augmented dataset. This dataset drives the iterative training of a pre-defined event extraction model. By calculating the loss function and backpropagating, the model parameters are updated, and finally, a model with event extraction capabilities is output.
Claims
1. A method for extracting small sample events based on T5 and knowledge graph data augmentation, characterized in that, Includes the following steps: S1. The mask fragment selection module is used to segment the original text according to the preset delimiters and select the longest fragment as the masking object. S2. Using the fine-tuning model generation module, the masked object in the original text is replaced with a special identifier, and a reading comprehension fill-in-the-blank question is constructed. The fine-tuned T5 model is used to predict the reading comprehension fill-in-the-blank question, generate new segments, and fill them in to obtain the preliminary enhanced text. S3. Using the knowledge graph enhancement module based on the Chinese concept knowledge graph, nouns or pronouns in the preliminary enhanced text, except for key information, are replaced with similar entities to obtain new samples. S4. The new sample is merged with the original text using the Chinese event extraction module to form an enhanced dataset. The enhanced dataset is then used to drive the training of the event extraction model to extract trigger words and arguments.
2. The method for small sample event extraction based on T5 and knowledge graph data augmentation according to claim 1, characterized in that, In step S1, the process of determining the delimiter includes: identifying trigger words and arguments in the original text, and defining them as a set of trigger words and a set of arguments, respectively; taking the union of the set of trigger words and the set of arguments to obtain an unmodifiable set; and combining the unmodifiable set with a preset set of punctuation marks to form the delimiter used for segmentation.
3. The method for extracting small sample events based on T5 and knowledge graph data augmentation according to claim 1, characterized in that, In step S1, the process of selecting the longest segment includes: calculating the length of each segment after segmentation using the longest masking segment definition formula, determining the segment with the longest length as the candidate masking object; determining whether the masking object simultaneously satisfies the boundary and content constraints defined by the masking segment selection constraint formula, and confirming the final selection result.
4. The method for small sample event extraction based on T5 and knowledge graph data augmentation according to claim 1, characterized in that, In step S2, the process of constructing the reading comprehension fill-in-the-blank question includes: replacing the masked object in the original text with the special identifier to form a sequence to be completed; concatenating a preset natural language prompt at the end of the sequence to be completed; and generating the reading comprehension fill-in-the-blank question using a reading comprehension fill-in-the-blank question construction formula.
5. The method for extracting small sample events based on T5 and knowledge graph data augmentation according to claim 1, characterized in that, In step S2, the process of using the fine-tuned T5 model for prediction includes: selecting an external dataset that is similar to the original text in distribution or style; using the external dataset to construct a supervised training task to update the parameters of the pre-trained model, thereby obtaining the fine-tuned T5 model; and using the fine-tuned T5 model in conjunction with a Top-N decoding strategy to solve the reading comprehension fill-in-the-blank question, thereby generating the new fragment that satisfies the constraints of the new text fragment generation prediction formula.
6. The method for extracting small sample events based on T5 and knowledge graph data augmentation according to claim 5, characterized in that, In step S2, the process of backfilling to obtain the preliminary enhanced text includes: selecting a generated text fragment as the new fragment from multiple candidate results output by the Top-N decoding strategy; and using a generative enhanced text replacement formula to fill the generated text fragment back into the position of the masked object in the original text to obtain the preliminary enhanced text.
7. The method for extracting small sample events based on T5 and knowledge graph data augmentation according to claim 1, characterized in that, In step S3, the construction process of the Chinese concept knowledge graph includes: extracting triples that conform to hierarchical or attribution relationships from the basic data source; constructing a hierarchical structure containing entities, specific concepts, and the attribution of top-level concepts based on the triples; and using the hierarchical structure to generate the Chinese concept knowledge graph that supports path queries from entities and concepts to similar entities.
8. A method for extracting small sample events based on T5 and knowledge graph data augmentation according to claim 7, characterized in that, In step S3, the process of replacing similar entities includes: defining the trigger words and arguments marked in the original text as the key information; performing part-of-speech tagging on the preliminary enhanced text to identify nouns and pronouns, removing words belonging to the key information, and obtaining a candidate word set; using the Chinese concept knowledge graph to retrieve the parent concept node of any candidate word in the candidate word set; using the parent concept node as an anchor point to find sibling nodes of the same level as replacement words; and using the knowledge graph entity replacement formula to replace the candidate words with the replacement words to generate the new sample.
9. A method for extracting small sample events based on T5 and knowledge graph data augmentation according to claim 1, characterized in that, In step S4, the process of constructing the augmented dataset includes: defining original samples containing the original text and corresponding event annotation information; configuring the new samples to directly inherit the event annotation information of the original samples; and merging all the original samples with the corresponding multiple new samples using the augmented dataset construction formula to construct the augmented dataset.
10. A method for extracting small sample events based on T5 and knowledge graph data augmentation according to claim 1, characterized in that, In step S4, the process of training the driving event extraction model includes: establishing a mapping relationship from the definition layer to the processing layer, defining the cross-entropy loss function and the gradient optimization algorithm; converting the augmented dataset into a vector matrix and inputting it into the event extraction model to obtain the prediction result; calculating the prediction error between the prediction result and the true label based on the cross-entropy loss function; and updating the model parameters according to the prediction error using the gradient optimization algorithm.