Method, apparatus and device for generative event detection oriented to large language model

Abstract

The application relates to a generative event detection method, device and equipment for a large language model. The method comprises the following steps: obtaining semantic perception narrative criteria according to the narrative description and typical trigger word examples of each event type; marking the trigger word in the input text in situ by using a preset symbol, and constructing an output sequence after associating the corresponding event type; embedding the task instruction, the semantic perception narrative criteria, the input text and the output sequence into a pre-constructed supervision prompt template, inputting the large language model after performing noise-based regularization processing on the semantic perception narrative criteria, iteratively optimizing the preset loss function, and obtaining a trained large language model; processing the to-be-detected text by using the trained large language model to generate a target output sequence; and extracting the event type, the trigger word and the position information of the trigger word in the to-be-detected text from the generated target output sequence. The method can improve the event detection accuracy and efficiency.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a generative event detection method, apparatus, and device for large language models. Background Technology

[0002] With the development of natural language processing (NLP) technology, event detection, as a core task of information extraction, has received widespread attention. It aims to identify event trigger words in text and classify corresponding event types, and has been successfully applied in various scenarios such as knowledge base construction, question answering systems, information retrieval, and narrative understanding. The emergence of large-scale language models has driven the field of event detection towards a generative research paradigm. Current advanced methods are beginning to introduce code-based hint strategies, defining events as structured objects such as Python classes. By strengthening structural constraints through formal patterns, these methods assist models in performing structured reasoning, providing a new solution path for event detection.

[0003] However, such instruction forms that rely on code syntax have strict syntactic constraints, which often make it difficult to fully capture the rich narrative information necessary to understand complex events. This results in limited semantic expression, an inability to effectively bridge the semantic gap between general pre-training and specific event detection tasks, and consequently, difficulty in fully stimulating the inherent reasoning potential of large language models. Furthermore, the disambiguation effect is poor when dealing with complex event types with semantic overlap, ultimately limiting the accuracy and overall effectiveness of structured event extraction. Summary of the Invention

[0004] Therefore, it is necessary to provide a generative event detection method, apparatus, and device for large-scale language models to address the aforementioned technical problems.

[0005] A generative event detection method for large-scale language models, the method comprising:

[0006] Training samples are constructed based on the input text and the event type and trigger words corresponding to the input text.

[0007] Based on the narrative description and typical trigger word examples of each event type, semantically aware narrative criteria are obtained;

[0008] The system uses preset symbols to mark trigger words in situ in the input text and constructs the output sequence by associating them with the text sequence following the corresponding event type.

[0009] The task instructions, the semantically aware narrative criteria, the input text, and the output sequence are embedded into a pre-built supervision prompt template. After performing noise-based regularization on the semantically aware narrative criteria, the results are input into a large language model. The preset loss function is iteratively optimized to obtain a trained large language model. The task instructions clarify the role and execution requirements of the large language model. The execution requirements include identifying event trigger words in the input text and associating them with corresponding predefined event types.

[0010] The task instruction, the semantic-aware narrative criteria, and the text to be detected are embedded into the reasoning prompt template and then input into the trained large language model to generate the target output sequence.

[0011] Event detection is completed by extracting the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence.

[0012] A generative event detection device for large-scale language models, the device comprising:

[0013] The sample acquisition module is used to construct training samples based on the input text and the event type and trigger words corresponding to the input text;

[0014] The narrative criteria construction module is used to obtain semantically aware narrative criteria based on the narrative description and typical trigger word examples of each event type;

[0015] The output sequence construction module is used to mark trigger words in situ in the input text using preset symbols and construct the output sequence by associating the text sequence with the corresponding event type.

[0016] The model training module is used to embed the task instructions, the semantically aware narrative criteria, the input text, and the output sequence into a pre-built supervision prompt template. After performing noise-based regularization on the semantically aware narrative criteria, the module is input into the large language model and iteratively optimized using a preset loss function to obtain a trained large language model. The task instructions clarify the role and execution requirements of the large language model. The execution requirements include identifying event trigger words in the input text and associating them with corresponding predefined event types.

[0017] The model inference module is used to acquire the text to be detected, embed the task instructions, the semantic-aware narrative criteria and the text to be detected into the inference prompt template, and then input it into the trained large language model to generate the target output sequence.

[0018] The event detection module is used to extract the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence, so as to complete the event detection.

[0019] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0020] Training samples are constructed based on the input text and the event type and trigger words corresponding to the input text.

[0021] Based on the narrative description and typical trigger word examples of each event type, semantically aware narrative criteria are obtained;

[0022] The system uses preset symbols to mark trigger words in situ in the input text and constructs the output sequence by associating them with the text sequence following the corresponding event type.

[0023] The task instructions, the semantically aware narrative criteria, the input text, and the output sequence are embedded into a pre-built supervision prompt template. After performing noise-based regularization on the semantically aware narrative criteria, the results are input into a large language model. The preset loss function is iteratively optimized to obtain a trained large language model. The task instructions clarify the role and execution requirements of the large language model. The execution requirements include identifying event trigger words in the input text and associating them with corresponding predefined event types.

[0024] The task instruction, the semantic-aware narrative criteria, and the text to be detected are embedded into the reasoning prompt template and then input into the trained large language model to generate the target output sequence.

[0025] Event detection is completed by extracting the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence.

[0026] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0027] Training samples are constructed based on the input text and the event type and trigger words corresponding to the input text.

[0028] Based on the narrative description and typical trigger word examples of each event type, semantically aware narrative criteria are obtained;

[0029] The system uses preset symbols to mark trigger words in situ in the input text and constructs the output sequence by associating them with the text sequence following the corresponding event type.

[0030] The task instructions, the semantically aware narrative criteria, the input text, and the output sequence are embedded into a pre-built supervision prompt template. After performing noise-based regularization on the semantically aware narrative criteria, the results are input into a large language model. The preset loss function is iteratively optimized to obtain a trained large language model. The task instructions clarify the role and execution requirements of the large language model. The execution requirements include identifying event trigger words in the input text and associating them with corresponding predefined event types.

[0031] The task instruction, the semantic-aware narrative criteria, and the text to be detected are embedded into the reasoning prompt template and then input into the trained large language model to generate the target output sequence.

[0032] Event detection is completed by extracting the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence.

[0033] The aforementioned generative event detection methods, devices, and equipment for large-scale language models construct semantically aware narrative criteria by fusing narrative descriptions of event types with typical trigger word examples. This breaks the grammatical constraints of traditional code-based prompts, fully captures the rich narrative information required for events, effectively clarifies the semantic boundaries of each event type, and resolves semantic overlap and ambiguity between different events. By marking trigger words in situ in the input text with preset symbols and associating them with event types, the integrity of the original text context can be maintained, establishing a direct mapping between semantic understanding and structured extraction, avoiding information loss caused by abstract encoding. By performing noise-based regularization on the semantically aware narrative criteria, coupled with unified supervised prompt templates and inference prompt templates, the model's adherence to the criteria can be strengthened, improving the model's robustness and generalization ability. Combined with iteratively optimized loss functions and accurate result extraction, the inference potential of large-scale language models can be fully stimulated, achieving efficient and accurate identification of event types, trigger words, and their location information, significantly improving the efficiency of structured event extraction. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a generative event detection method for a large language model in one embodiment.

[0035] Figure 2 This is a schematic diagram of an instruction architecture based on semantic awareness criteria in one embodiment;

[0036] Figure 3 This is a schematic diagram comparing the performance of the method of the present invention with multiple strong baseline models in one embodiment;

[0037] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] Modern large-scale language models have demonstrated remarkable emergent reasoning and semantic understanding capabilities, enabling them to handle complex language structures. Instruction fine-tuning plays a crucial role in effectively bridging the gap between general pre-training and specific event detection tasks. This process connects the model's broad semantic understanding capabilities with the specific details of different event types, ensuring that it can accurately perform event detection according to the task specifications.

[0040] To address the limitations of existing methods in semantic representation and fully unleash the reasoning potential of large language models, this invention proposes the SENSE framework, a semantic enhancement method based on narrative and symbolic encoding. This framework reconstructs traditional event pattern paradigms into narrative descriptions rich in semantics. Unlike relying on rigid templates, these descriptive narratives aim to enable the model to grasp the essential conceptual connotations of each event type, thereby establishing a more logical mapping between the input text and task-specific patterns, and ultimately achieving more accurate trigger word recognition and classification.

[0041] In one embodiment, such as Figure 1 As shown, a generative event detection method for large-scale language models is provided, including the following steps:

[0042] Step 102: Construct training samples based on the input text and the event type and trigger words corresponding to the input text.

[0043] The input text is the raw text to be processed, which is the core analysis object of event detection. The event type is a predefined category of events to be identified (e.g., personnel events, business events). Trigger words are core words in the text that can trigger a specific event; they are indicative words of the event's occurrence. The training samples are a dataset containing the input text, its corresponding event type, and trigger word annotations, used for supervised learning of the model.

[0044] Understandably, this step provides standardized supervised learning data for large language models, ensuring that the model establishes an association between event types and trigger words through samples.

[0045] Step 104: Based on the narrative description and typical trigger word examples of each event type, obtain the semantically aware narrative criteria.

[0046] Narrative description is a fluent expression in natural language that defines the essential meaning of an event type, focusing on the linguistic essence of the event rather than isolated labels. Typical trigger word examples are a representative set of real-world trigger words corresponding to each event type, providing concrete semantic anchors for abstract descriptions. Semantic-aware narrative guidelines are unified guiding rules formed by integrating narrative descriptions of all event types with typical trigger word examples; their core function is to clarify the semantic boundaries of each event type.

[0047] Understandably, this step can transform abstract event patterns into semantically rich natural language rules, which fully aligns with the pre-training characteristics of large language models that excel at understanding natural language. It effectively stimulates the model's semantic reasoning ability, helps resolve semantic overlap and ambiguity between different event types, and provides core guidance for accurate event detection.

[0048] Step 106: Use preset symbols to mark trigger words in the input text in situ, and construct the output sequence by associating the text sequence with the corresponding event type.

[0049] Preset symbols are special characters (such as square brackets "[]") pre-defined to identify trigger words. In-situ markings are symbol annotations placed at the original positions of trigger words in the input text, without changing the original context and structure of the text. The output sequence is a structured text formed by concatenating the event type label with the in-situ marked input text according to a preset format, and serves as the target reference result for model training.

[0050] It is understandable that this step can establish a direct mapping relationship between trigger words and event types while preserving the spatiotemporal context integrity of the input text, avoiding information loss caused by abstract encoding. It is beneficial for the model to learn the semantic and positional features of trigger words at the same time, providing clear and structured supervision signals for model training, and improving the targeting and efficiency of training.

[0051] Step 108 involves embedding the task instructions, semantically aware narrative criteria, input text, and output sequence into a pre-built supervised prompt template. After performing noise-based regularization on the semantically aware narrative criteria, the data is input into the large language model. The preset loss function is iteratively optimized to obtain the trained large language model. The task instructions clearly define the role and execution requirements of the large language model. The execution requirements include identifying event trigger words in the input text and associating them with corresponding predefined event types.

[0052] Task instructions are functional commands that define the core objectives of event detection, clarify the model's role and execution requirements, and are used to anchor the model's generation direction. Supervision prompt templates are a unified input structure containing task instructions, semantically aware narrative criteria, input text, and output sequences, providing a fixed format specification for model training. Noise-based regularization is an optimization technique that introduces noise by rearranging the order of criteria, randomly discarding some criteria, restating event descriptions, and sampling trigger word examples. The large language model is a core model with powerful pre-trained knowledge reserves and instruction compliance capabilities. Preset loss functions (such as cross-entropy loss functions) measure the difference between the model's prediction results and the output sequence, and are used to guide model parameter updates.

[0053] In this framework, event detection is formalized as a conditional sequence generation task. Given a given sequence... n The input sentence consists of 100 words. and a group m Predefined event types The goal of event detection is to identify a set of trigger words. and each trigger word Classify them according to their corresponding event types .

[0054] To achieve this goal, this invention transforms the extraction process into a controlled sequence generation task. Specifically, the model learns to generate a target sequence Y, which is annotated with event-triggered words by inserting symbolic brackets (i.e., "[]") into the original sentence structure. This is done with prompts containing task-related contextual information. I Given input text X, the model is trained in an autoregressive manner to maximize the conditional probability. The training objective is defined by the standard cross-entropy loss function as follows:

[0055] ;

[0056] Among them, y <t This represents the sequence of words generated before the current time step t, which serves as the prediction. Historical context information.

[0057] It is understandable that this step can provide the model with a complete training context through supervised prompt templates, effectively improve the model's robustness and generalization ability by combining noise-based regularization, and continuously adjust the model parameters by iteratively optimizing the loss function, so that the model can accurately master the event detection rules, and finally obtain a dedicated model with stable performance and high detection accuracy.

[0058] Step 110: Obtain the text to be detected, embed the task instructions, semantic-aware narrative criteria and the text to be detected into the reasoning prompt template, and then input it into the trained large language model to generate the target output sequence.

[0059] The text to be detected is the unlabeled raw text for which event detection is required. The inference hint template is a fixed input structure used for model inference, containing no output sequence, and supports the model in generating results autonomously. The target output sequence is a structured result generated by the model based on the inference hint template, containing the event type and the labeled text to be detected.

[0060] Understandably, this step provides standardized inference input to the trained model, ensuring that the model can autonomously complete detection under unified rule constraints and efficiently generate structured output without human intervention. This helps improve the automation and processing efficiency of event detection, while maintaining consistent output format, which facilitates subsequent information extraction.

[0061] Step 112: Extract the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence to complete the event detection.

[0062] Location information refers to the character-level offset of the trigger word in the text to be detected, specifically including the start character index and the end character index. Event detection is the core task of identifying event trigger words in text and classifying their corresponding event types, and it is a key link in the field of information extraction.

[0063] It is understandable that this step can accurately separate event types, trigger words, and their location information from the target output sequence, achieving accurate location of trigger words and clear classification of event types, directly completing the core objective of event detection. This is beneficial for providing structured event information for downstream applications such as knowledge base construction, intelligent question answering, and information retrieval, thereby improving the convenience and practical value of information utilization.

[0064] The generative event detection method for large-scale language models described above constructs semantically aware narrative criteria by fusing narrative descriptions of event types with typical trigger word examples. This breaks the grammatical constraints of traditional code-based prompts, fully captures the rich narrative information required for events, effectively clarifies the semantic boundaries of each event type, and resolves semantic overlap and ambiguity between different events. By marking trigger words in situ in the input text with preset symbols and associating them with event types, the integrity of the original text context can be maintained, establishing a direct mapping between semantic understanding and structured extraction, avoiding information loss caused by abstract encoding. By performing noise-based regularization on the semantically aware narrative criteria, coupled with unified supervised prompt templates and inference prompt templates, the model's adherence to the criteria can be strengthened, improving the model's robustness and generalization ability. Combined with iteratively optimized loss functions and accurate result extraction, the inference potential of large-scale language models can be fully stimulated, achieving efficient and accurate identification of event types, trigger words, and their location information, and significantly improving the efficiency of structured event extraction.

[0065] In one embodiment, the input text is a sequence containing multiple lexical units, each of which is a basic semantic unit constituting the input text, and the set of lexical units covers the complete semantic information of the input text.

[0066] In this embodiment, the lexical units of the input text can be words, characters, or phrases. The segmentation method is consistent with the pre-training lexicalization rules of the large language model, ensuring that each lexical unit can carry independent semantic information, and that all lexical units together constitute the complete semantic expression of the input text, avoiding semantic loss due to improper lexical unit segmentation. It can be understood that this implementation provides a complete and accurate semantic foundation for subsequent event trigger word recognition and event type classification, which is beneficial for the model to accurately capture event-related information in the text, providing a prerequisite guarantee for the accuracy of event detection.

[0067] In one embodiment, the process of using preset symbols to in-situ mark trigger words in the input text and constructing an output sequence by associating the text sequence with the corresponding event type includes: obtaining the event type and the corresponding trigger word of the input text; using preset symbols to in-situ mark the lexical units corresponding to the trigger words in the input text; and constructing an output sequence based on the label corresponding to the event type and the input text after in-situ marking the trigger words.

[0068] In this embodiment, the preset symbol is preferably square brackets "[]", which completely enclose all the lexical units corresponding to the trigger word during marking, without changing the original position and context of the lexical units in the input text. The event type label adopts a predefined standardized name, which is concatenated with the marked input text in the format of "event type label + colon + marked text" to form a structured output sequence. It can be understood that this implementation method can establish a direct mapping between trigger words and event types while preserving the spatiotemporal context integrity of the input text, avoiding information loss caused by abstract encoding, providing clear and intuitive supervision signals for model training, and significantly improving the model's learning efficiency of event patterns.

[0069] In one embodiment, the method further includes: if the input text has multiple event types, then performing in-situ tagging of trigger words for each event type to obtain multiple output subsequences corresponding to different event types; using a line-by-line enumeration method, combining all output subsequences to form an output sequence, and ensuring that the trigger word tags of different output subsequences do not overlap.

[0070] In this embodiment, each output subsequence corresponds to only one event type and its associated trigger word. The annotation process focuses on the accuracy of matching the trigger word with the event type. It can be understood that this implementation can efficiently handle complex text scenarios containing multiple events. By enumerating line by line, the output structure becomes clear and easy to understand. It also adapts to scenarios where the same lexical unit corresponds to multiple events, which helps the model accurately learn event differentiation rules in scenarios where multiple events coexist, thus improving the comprehensiveness and flexibility of event detection.

[0071] In one embodiment, obtaining semantically aware narrative criteria based on the narrative description and typical trigger word examples for each event type includes: for each event type, obtaining the narrative description and corresponding typical trigger word examples, wherein the narrative description defines the event in natural language form; fusing the narrative description and typical trigger word examples according to a preset splicing method to obtain semantically aware sub-criteria for the event type; and summarizing the semantically aware sub-criteria for all event types to obtain semantically aware narrative criteria.

[0072] In this embodiment, the narrative description follows the template "an event e refers to...", constructed based on the official guidelines of the dataset, accurately defining the essential connotation and applicable scenarios of the event; typical trigger word examples are preferentially extracted from the official guidelines, and if not available, they are obtained from the training set through word frequency statistics or manual screening, serving as specific semantic anchors for the abstract narrative description; the preset splicing method is "narrative description first, typical trigger word examples second", separated by a preset separator to ensure the semantic coherence of the sub-rules. It can be understood that this implementation method can transform abstract event types into semantically rich natural language rules, fully matching the pre-training characteristics of large language models, effectively clarifying the semantic boundaries of each event type, and significantly improving the model's ability to resolve event ambiguities and its semantic reasoning capabilities.

[0073] In one embodiment, the supervision prompt template is {P,G,X,Y}, and the reasoning prompt template is {P,G,X}, where P is the task instruction, G is the semantically aware narrative criterion, X is the input text, and Y is the output sequence.

[0074] In this embodiment, the component order of the supervision prompt template strictly follows "task instruction → semantically aware narrative criteria → input text → output sequence", while the component order of the reasoning prompt template is "task instruction → semantically aware narrative criteria → text to be detected". The components are separated by line breaks or specific symbols as preset separators, making the template structure clear and logically coherent. The task instruction clearly defines the model's role as an event detection expert, and the system elaborates on the core requirements for identifying trigger words, associating event types, and the output format specifications.

[0075] This invention employs a large language model to complete the final event detection task in a single-stage manner. During the training phase, unified prompts are used for supervised optimization. I It consists of {P, G, X, Y}. During the reasoning phase, hints are provided. I Only {P,G,X} are included to support autonomous generation. In this setting, the model's task is to simultaneously identify all potential trigger words and their corresponding event types during a single inference iteration. The process strictly follows the predefined output format Y, thereby ensuring that the text position offset of the trigger word and its category classification result can be recovered simultaneously.

[0076] It is understood that this implementation method can provide a standardized input structure for model training and inference, ensuring that the model performs event detection under unified rule constraints, guiding the model to focus on task requirements and semantic criteria rather than relying on irrelevant features such as text position, which is conducive to improving the relevance of model training and the consistency of inference results.

[0077] In one embodiment, performing noise-based regularization on the semantically aware narrative criteria includes: performing one or more regularization noise-adding operations on the semantically aware narrative criteria to complete the noise-based regularization; the regularization noise-adding operation includes: using a random sorting algorithm to rearrange the semantically aware sub-criteria of each event type in the semantically aware narrative criteria; randomly selecting some semantically aware sub-criteria of some event types and discarding them according to a preset probability threshold, and the corresponding output sequence no longer contains instances corresponding to the discarded event types; obtaining the original narrative description of each event type, using a preset generation model to perform restatement processing on the original narrative description to obtain restatement versions with diversified expressions, and replacing the original narrative description; predefining a trigger word candidate pool for each event type, and in each training cycle, randomly selecting a subset of trigger words from the trigger word candidate pool corresponding to the event type as typical trigger word examples for the event type.

[0078] In this embodiment, the preset generation model is Vicuna 33B v1.3, and the paraphrasing process generates a version that is semantically consistent with the original description but with different wording. The preset probability threshold is adaptively adjusted according to the training data scale and the number of event types. The typical trigger word candidate pool is constructed by combining extraction from official guidelines, word frequency statistics of the training set, and manual screening.

[0079] This invention introduces several noise-based regularization methods during training to force the model to follow guidelines rather than simply memorizing labels. Specifically, it employs pattern reordering to break positional biases and random pattern discarding (randomly deleting type patterns from inputs) to reduce illusion generation and strengthen pattern adherence. To further enhance robustness, this invention applies guideline paraphrasing to prevent over-reliance on specific wording and uses representative candidate word sampling, selecting a subset from a predefined candidate pool in each training cycle to provide diverse example vocabulary for each event. For event descriptions, this invention uses Vicuna 33B v1.3 to automatically generate paraphrased versions; for representative candidate words, this invention primarily extracts them from the official guidelines when available, otherwise sampling from the training set partitions based on word frequency or manual selection.

[0080] It is understandable that this implementation method can effectively force the model to deeply understand the core meaning of semantic rules, rather than simply memorizing text wording or location information, significantly improving the robustness and generalization ability of the model, reducing the generation of illusions, and ensuring that the model can stably follow the event detection rules in different scenarios.

[0081] In one specific embodiment, such as Figure 2As shown, a schematic diagram of an instruction architecture based on semantic-aware guidelines is provided. The top module is "Task Instruction", the upper middle module is "Semantic-aware Narrative Guidelines", the lower middle module is "Input Text", and the bottom module is "Output". Specifically, the prompt I input to the large language model is structured into a unified sequence containing four core components: Task Instruction (P), Semantic-aware Narrative Guidelines (G), Input Text (X), and Output Label (Y).

[0082] Task Instructions (P): The task instructions define the core objectives of the task and clarify the model's role as a specialized information extraction tool. They systematically outline the task's basic requirements and expected output format. As the starting point of the prompting information, this instruction acts as a functional anchor, ensuring that the model's generation process aligns with the objectives of the event detection task. Figure 2 The task instruction reads, “You are an expert in event detection. Your task is to identify and classify all event triggers in the text according to predefined event types. Please mark the event triggers within brackets “[]”. If there are multiple events, please list them one by one.” This task instruction, as the starting point of the prompt, clarifies the model's role and task requirements, serving as a functional anchor and ensuring that the model generation process aligns with the event detection objective.

[0083] Semantic Aware Narrative Criterion (G): This section embodies the core strategy of the present invention, which is to transform event patterns into fluent natural language descriptions to bridge the semantic gap in event detection. In the present invention, for each specific event type, a narrative event definition is systematically combined with representative trigger word examples. By integrating the entire pattern into a unified, semantically rich cue, the model can simultaneously identify trigger words for all event types in a single inference process. This design not only improves computational efficiency but also enables the model to utilize semantic relationships between categories for more accurate disambiguation.

[0084] (1) Narrative Event Description: To ground the model's reasoning process in established event definitions, this invention constructs comprehensive natural language descriptions based on the official guidelines for each dataset. Each definition is presented in a fluent narrative form, typically following the template: "An event...". For example, in the ACE05 dataset, "BusinessEvent" is defined as:

[0085] "A Business Event refers to actions related to organizations such as: creating, merging, declaring bankruptcy, or ending organizations."

[0086] This approach encourages the model to focus on the linguistic essence of each event, rather than treating labels as isolated or meaningless symbols. It is noteworthy that the number of definitions corresponds exactly to the total number of event types |E|, which allows for flexible variability across different datasets. Figure 2 In the text, "Personnel Event" is described as "an event that occurs when a person / entity changes roles between their job title and an organizational entity. This includes the start or termination of employment, intra-organizational transfers, nominations for organizational positions, or elections." "Conflict Event" is described as "an adversarial physical act that results in damage to property or minor physical contact, but excludes everyday verbal altercations (such as arguments, bickering, or disagreements) or peaceful expressions of demands (including collective petitions, public discussions, and spontaneous gatherings)." These descriptions are presented in a fluent narrative style, guiding the model to focus on the linguistic nature of the event rather than isolated labels.

[0087] (2) Examples of prototype trigger words: In order to provide a clear semantic basis for the narrative definition, this invention supplements each event type with a set of typical real-world trigger words. These candidate examples serve as concrete anchors, transforming abstract conceptual descriptions into identifiable textual clues. For example, representative trigger word examples for "BusinessEvent" are as follows: Trigger examples: started, open, create, etc.

[0088] This approach, which integrates narrative descriptions with trigger word examples, allows the model to simultaneously identify trigger words for all event types from a global perspective. This not only improves computational efficiency but also helps to resolve ambiguities by leveraging semantic relationships between categories.

[0089] Ultimately, this unified representation of event patterns ensures that the model maintains a global perspective, enabling it to extract all potential events simultaneously from a given text.

[0090] Input text (X). Input text X is the object of analysis, appearing after P and G in the prompts. This order ensures that the output is conditionalized entirely based on the task instructions and semantic definitions before analyzing the actual text, thus guiding the model to focus on the provided rules when recognizing events. Figure 2 In the example, the text to be analyzed is: "He lost the election to a deceased person." This section follows the task instructions and semantically aware narrative criteria, ensuring that the model is conditionalized based on the task rules before analyzing the actual text, guiding the model to focus on the provided criteria when recognizing events.

[0091] Output Labels (Y). The output sequence Y represents the structured result of the event detection task. Existing research indicates that the formatting of the output of large language models has a significant impact on extraction performance. Although symbol encoding strategies have proven effective in named entity recognition tasks, their application in event detection remains largely unexplored. This invention extends this paradigm to the event detection task, utilizing in-situ symbolic labeling to capture dynamic, action-oriented event trigger words within the original text structure. The model begins generating output after "result:\n". Each generated output Y begins with an event label, followed by the original sentence structure, where the identified trigger words are enclosed in special symbols "[" and "]". Specifically, the structure of the output sequence Y is defined as follows:

[0092] ;

[0093] in, This represents the type of event detected. This represents the identified trigger word marker. The event type and the character-level offset of the trigger word (i.e., its exact position in the text) can be easily extracted from Y. If no event is detected, the model will output "None". If the input text X contains multiple event instances, the model will enumerate them line by line (each instance on a separate line) to ensure clear structure and avoid overlapping markers.

[0094] This architecture ensures that the model can simultaneously complete classification and extraction tasks within a unified process. By directly anchoring event labels to the original sentence structure, the model can leverage its internal attention mechanism to resolve event ambiguity, while ensuring that the text position of the trigger word can be accurately restored through simple string alignment operations.

[0095] The effectiveness of the method of the present invention was verified through experiments:

[0096] This invention was tested on two widely used English datasets: ACE05 and CASIE. For the ACE05 dataset, this invention followed the standard data partitioning and preprocessing procedures used in the GoLLIE benchmark. For the CASIE dataset, this invention maintained the same preprocessing method, but since the former only evaluated the CASIE performance under a zero-sample setting, this invention adopted the "split1" partitioning configuration from the TextEE model to support supervised fine-tuning of the model. Finally, all data were automatically converted into narrative text format using the script of this invention to meet the requirements of the instruction fine-tuning task. This invention uses precision, recall, and F1 score as standard evaluation metrics, whose definitions are consistent with existing research. In the event detection task, a prediction is considered correct only if the text position offset of the predicted trigger word and its event type both perfectly match the standard annotation. The proposed framework is compared with GoLLIE as the main baseline, while incorporating other advanced methods based on large language models, including DeepStruct, InstructUIE, IEPILE, KnowCoder, AlignXIE, YAYI-UIE, LLM-PEE, RUIE, and KnowCoder-X.

[0097] This invention utilizes the LoRA method for supervised fine-tuning of three representative open-source large language models: Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, and DeepSeek-R1-Distill-Qwen-7B. The model implementation employs the QLoRA framework and applies 4-bit quantization. The LoRA adapter is applied to all linear layers of the Transformer module. Based on previous experiments, this invention sets the LoRA rank to 8 and the scaling factor to 16 to achieve a balance between efficiency and performance.

[0098] Training was performed on a single NVIDIA H800 (80GB VRAM) GPU with Bfloat16 precision for three epochs. The optimizer used was AdamW with a learning rate of 3e-4 and an epsilon parameter of 1e-7. To optimize the training process, a cosine learning rate scheduler was used, with a warm-up ratio of 0.03. The batch size for all experiments was fixed at 32. To handle long texts, the maximum sequence length for both input and output was set to 2048. During the inference phase, a greedy search strategy was used to generate the final prediction results.

[0099] Figure 3This paper presents a performance comparison between the proposed model and several strong baseline models. Experiments show that natural text-based instructions generally provide a better framework for event detection tasks compared to code-based instructions. Although code-driven models, such as GoLLIE and KnowCoder, exhibit stable performance, the narrative definition method of this invention achieves a significantly higher F1 score, especially in specialized domains. For example, Ours-LLaMA3 achieved a peak F1 score of 75.5% on the CASIE dataset, outperforming the best code baseline model (72.1%) by 3.4 percentage points. This success is attributed to the dual-driven design of the SENSE framework: semantically aware narrative criteria and symbolic encoding strategy. Unlike code models that rely on structured patterns, the narrative method of this invention better aligns with the pre-training characteristics of large language models by providing fluent descriptive text and lexical anchors. Simultaneously, the symbolic encoding strategy preserves the spatial contextual integrity of sentences by directly using symbols to mark trigger words in the original text. This avoids the information loss common in abstract code generation, allowing the model to directly map its semantic reasoning to the source text.

[0100] By comparing different large language model backbone networks, this invention reveals that Qwen2.5 and LLaMA3 exhibit their respective advantages in different task domains. Ours-Qwen2.5 demonstrates high performance in general event detection, achieving an F1 score of 76.4% on the ACE2005 dataset, ranking second overall, and performing very close to the current state-of-the-art model, InstructUIE. On the other hand, Ours-LLaMA3 demonstrates superior performance on the more technically demanding CASIE dataset, setting a new performance record. Although the DeepSeek model's performance is relatively moderate, the method proposed in this invention achieves consistently excellent performance across various baseline models, demonstrating the robustness and adaptability of the SENSE framework across different large language models.

[0101] Overall, the experimental results demonstrate superior performance on both the ACE05 and CASIE benchmark datasets. The LLaMA3-based model achieves the highest F1 score of 75.5% on the CASIE dataset, while the Qwen2.5-based model achieves a competitive score of 76.4% on the ACE05 dataset, significantly outperforming mainstream code-driven baseline models such as GoLLIE and KnowCoder. This achievement is primarily attributed to the dual innovative design of the SENSE framework: by integrating narrative event descriptions with typical trigger word examples to construct semantic awareness guidelines, it breaks the grammatical constraints of traditional code-based hints, fully capturing the rich narrative information required for complex events and clearly defining the semantic boundaries of events; simultaneously, it introduces a symbolic encoding mechanism to in-situ label event trigger words in the original text, ensuring that the extracted results are always anchored to the original semantic context, effectively connecting narrative understanding with structured extraction. The field of event detection is increasingly shifting towards a generative research paradigm based on large language models. This invention reveals that, compared to prompts limited by programming structures, narrative-driven guidelines with semantic awareness can more effectively stimulate the potential reasoning ability of large language models in event detection tasks, providing a practical and feasible technical path to overcome the limitations of existing methods and improve the efficiency of structured event extraction.

[0102] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this invention, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Furthermore, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0103] In one embodiment, a generative event detection apparatus for large language models is provided, comprising:

[0104] The sample acquisition module is used to construct training samples based on the input text and the event type and trigger words corresponding to the input text.

[0105] The narrative criteria construction module is used to obtain semantically aware narrative criteria based on the narrative description and typical trigger word examples of each event type;

[0106] The output sequence construction module is used to mark trigger words in situ in the input text using preset symbols and construct the output sequence by associating the text sequence with the corresponding event type.

[0107] The model training module is used to embed task instructions, semantically aware narrative rules, input text, and output sequences into a pre-built supervised prompt template. After performing noise-based regularization on the semantically aware narrative rules, the input is fed into the large language model, and the preset loss function is iteratively optimized to obtain the trained large language model. The task instructions clarify the role and execution requirements of the large language model. The execution requirements include recognizing event trigger words in the input text and associating them with corresponding predefined event types.

[0108] The model inference module is used to acquire the text to be detected, embed the task instructions, semantically aware narrative criteria and the text to be detected into the inference prompt template, and then input it into the trained large language model to generate the target output sequence.

[0109] The event detection module is used to extract the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence, so as to complete the event detection.

[0110] Specific limitations regarding generative event detection devices for large language models can be found in the limitations of generative event detection methods for large language models described above, and will not be repeated here. Each module in the aforementioned generative event detection device for large language models can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0111] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a generative event detection method for large language models. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0112] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0113] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0114] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A generative event detection method for large language models, characterized by, The method includes: Training samples are constructed based on the input text and the event type and trigger words corresponding to the input text. Based on the narrative description and typical trigger word examples of each event type, semantically aware narrative criteria are obtained; The system uses preset symbols to mark trigger words in situ in the input text and constructs the output sequence by associating them with the text sequence following the corresponding event type. The task instructions, the semantically aware narrative criteria, the input text, and the output sequence are embedded into a pre-built supervision prompt template. After performing noise-based regularization on the semantically aware narrative criteria, the data is input into a large language model. The preset loss function is iteratively optimized to obtain a trained large language model. The task instructions clarify the role and execution requirements of the large language model. The execution requirements include identifying event trigger words in the input text and associating them with corresponding predefined event types. The task instruction, the semantic-aware narrative criteria, and the text to be detected are embedded into the reasoning prompt template and then input into the trained large language model to generate the target output sequence. Event detection is completed by extracting the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence.

2. The method of claim 1, wherein, The input text is a sequence containing multiple lexical units, each of which is a basic semantic unit constituting the input text. The set of lexical units covers the complete semantic information of the input text.

3. The method of claim 1, wherein, The step of using preset symbols to mark trigger words in situ in the input text and associating them with the corresponding event types to construct the output sequence includes: Get the event type and corresponding trigger word of the input text; Preset symbols are used to mark the lexical units corresponding to the trigger words in the input text in situ. The output sequence is constructed based on the label corresponding to the event type and the input text following the in-situ marker trigger word.

4. The method of claim 3, wherein, The method further includes: If the input text contains multiple event types, then for each event type, the in-situ marker trigger word operation is performed to obtain multiple output subsequences corresponding to different event types; The output sequence is formed by combining all output subsequences in a line-by-line enumeration manner, and the trigger word labels of different output subsequences do not overlap.

5. The method of claim 1, wherein, Based on the narrative description and typical trigger word examples for each event type, the semantically aware narrative criteria include: For each event type, obtain a narrative description and corresponding typical trigger word examples, wherein the narrative description defines the event in natural language form; By fusing narrative descriptions with typical trigger word examples in a preset splicing method, a semantic perception sub-criteria for event types is obtained; By summarizing the semantic-aware sub-rules of all event types, the semantic-aware narrative rule is obtained.

6. The method of claim 1, wherein, The supervision prompt template is {P,G,X,Y}, and the reasoning prompt template is {P,G,X}, where P is the task instruction, G is the semantically aware narrative criterion, X is the input text, and Y is the output sequence.

7. The method of claim 1, wherein, Performing noise-based regularization on the semantically aware narrative criteria includes: One or more regularization noise-adding operations are performed on the semantically aware narrative criteria to complete noise-based regularization processing; the regularization noise-adding operations include: A random sorting algorithm is used to rearrange the semantic perception sub-criteria of each event type in the semantic perception narrative criteria. Based on a preset probability threshold, a subset of semantic awareness sub-criteria for certain event types are randomly selected and discarded, and the corresponding output sequence no longer contains instances of the discarded event types. Obtain the original narrative descriptions of each event type, use a preset generation model to paraphrase the original narrative descriptions to obtain paraphrased versions with diverse expressions, and replace the original narrative descriptions; For each event type, a candidate pool of trigger words is predefined. In each training cycle, a subset of trigger words is randomly selected from the candidate pool of trigger words corresponding to the event type as a typical example of trigger words for the event type.

8. A large language model-oriented generative event detection apparatus, characterized by, The device includes: The sample acquisition module is used to construct training samples based on the input text and the event type and trigger words corresponding to the input text; The narrative criteria construction module is used to obtain semantically aware narrative criteria based on the narrative description and typical trigger word examples of each event type; The output sequence construction module is used to mark trigger words in situ in the input text using preset symbols and construct the output sequence by associating the text sequence with the corresponding event type. The model training module is used to embed the task instructions, the semantically aware narrative criteria, the input text, and the output sequence into a pre-built supervision prompt template. After performing noise-based regularization on the semantically aware narrative criteria, the module is input into the large language model and iteratively optimized using a preset loss function to obtain a trained large language model. The task instructions clarify the role and execution requirements of the large language model. The execution requirements include identifying event trigger words in the input text and associating them with corresponding predefined event types. The model inference module is used to acquire the text to be detected, embed the task instructions, the semantic-aware narrative criteria and the text to be detected into the inference prompt template, and then input it into the trained large language model to generate the target output sequence. The event detection module is used to extract the event type, trigger word, and the position information of the trigger word in the text to be detected from the generated target output sequence, so as to complete the event detection.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images