A Communication Failure Event Argument Extraction Method Based on Large Model Event Focusing and Terminology Enhancement

Abstract

This invention discloses a method for extracting event arguments from communication failure documents based on large-scale model event focusing and terminology enhancement. The method includes: first, fine-tuning a large language model to extract event information and technical terminology from communication failure documents; second, constructing an event boundary-aware encoding module, which encodes the embedded event information prompt template using a non-autoregressive encoding method to enhance the ability to distinguish event boundaries; third, constructing a semantically enhanced technical terminology recognition module, which uses a two-layer attention mechanism to hierarchically interact technical terminology information with text features to improve the accuracy of terminology recognition; and finally, extracting event arguments based on the encoded text features and terminology recognition results using an attention focusing mechanism. This method achieves significant performance improvements on both communication failure datasets and telecommunications failure datasets, effectively solving the problems of difficulty in distinguishing text boundaries for multiple events and the difficulty in recognizing technical terms.

Description

Technical Field

[0001] This invention belongs to the field of communication technology, and in particular relates to a method for extracting argument of communication failure events based on large model event focusing and term enhancement. Background Technology

[0002] With the rapid development of information technology, communication networks are becoming increasingly large-scale, leading to a corresponding increase in the frequency and scope of failures. Timely and accurate identification and handling of failure events are crucial for ensuring the stable operation of communication networks. However, traditional manual fault analysis methods are inefficient and unable to meet the needs of modern communication networks when faced with massive amounts of text data such as network logs and alarm messages. Event argument extraction, an important research direction in natural language processing, aims to identify and extract events and their related participants from text, providing a new approach for the automated analysis of communication faults.

[0003] Event extraction tasks mainly consist of event detection and event argument extraction. Event detection aims to obtain the position of trigger words for events in a sentence and classify the event types corresponding to the trigger words; event argument extraction needs to identify the position of arguments in the sentence and classify the roles of the arguments. In recent years, research on event argument extraction has made significant progress driven by deep learning, but existing methods have the following shortcomings when processing communication fault data:

[0004] Event co-occurrence problem: Most existing methods ignore the co-occurrence problem of multiple events in the text, resulting in weak model ability to distinguish event boundaries. In communication failure data, different events are relatively independent, the relationships are relatively simple, and there are a lot of technical terms. The text is semantically ambiguous, making it difficult to effectively distinguish events by using trigger words or relationships between events.

[0005] Technical terminology identification problem: There are a large number of technical terms in the field of communications. Existing methods lack specific strategies and solutions when processing data containing technical terms, which makes it difficult to accurately identify the argument content corresponding to the technical terms and affects the effect of argument extraction. Summary of the Invention

[0006] This invention proposes a communication fault event argument extraction method based on large model event focusing and term enhancement to solve the problems existing in the prior art.

[0007] To achieve the above objectives, this invention provides a method for extracting argument of communication failure events based on large model event focusing and term enhancement, comprising the following steps:

[0008] Fine-tune the large language model, and extract event information and technical terminology information from communication fault documents using the fine-tuned large language model;

[0009] An event boundary-aware encoding module is constructed, which embeds the extracted event information into the prompt template and encodes the template using a non-autoregressive encoding method to obtain the encoded text features.

[0010] A semantically enhanced terminology recognition module is constructed. Through a two-layer attention mechanism, the extracted terminology information and the encoded text features are hierarchically interacted to output the terminology recognition results.

[0011] Based on the encoded text features and term recognition results, an attention-focusing mechanism is used to extract event arguments.

[0012] Optionally, the fine-tuning of the large language model includes:

[0013] The Qwen2.5-instruct-14B model was selected as the large language model;

[0014] The LoRA method is used to fine-tune the large model;

[0015] Construct prompt templates based on thought chains to enable large models to extract both event information and technical terminology information simultaneously.

[0016] Optionally, the extraction of event information and technical terminology information includes:

[0017] In the event extraction task, the maximum and minimum indices of the range of all arguments in the event are located as the event boundaries, and the event strings are extracted as training labels.

[0018] In the technical term extraction task, argument roles are abstracted into technical term types and used as labels for the technical term extraction task.

[0019] Optionally, the obtained encoded text features include:

[0020] Embed event information generated by the large model into the prompt template, and use tags to explicitly mark event boundaries;

[0021] Design corresponding prompt templates for different event types, and embed technical terms into the templates;

[0022] A non-autoregressive encoder is used to encode the concatenated prompt template to obtain the encoded text features.

[0023] Optionally, the semantically enhanced terminology recognition module includes:

[0024] Construct an embedding matrix of technical terms;

[0025] A multi-head attention mechanism is used to interactively fuse the embedding matrix with the encoded text features;

[0026] By further integrating the specialized terms extracted from the large model through a two-layer attention mechanism, the semantic representation of the terms is enhanced.

[0027] Optionally, the terminology recognition module uses conditional random fields for sequence labeling and log-likelihood functions as loss functions.

[0028] Optionally, the extraction of event arguments using an attention-focusing mechanism includes:

[0029] Extract features of different argument roles from the encoded event type template features;

[0030] Construct a span selector for each argument role to identify its start and end positions in the text;

[0031] The encoded features of the event text to be extracted are linearly transformed with the features of the terminology prediction module to calculate the probability distribution of the starting position of the argument role.

[0032] Optionally, the extraction of event arguments using the attention-focusing mechanism further includes:

[0033] Obtain the probability distribution of the starting positions of technical terms;

[0034] The probability distribution of the obtained technical terms is multiplied by the probability distribution of the argument roles to obtain the final probability distribution of argument extraction.

[0035] Optionally, a joint loss function is used during training, which includes argument extraction loss and term prediction loss, and the model parameters are updated through backpropagation.

[0036] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0037] Compared with the prior art, the present invention has the following advantages and technical effects:

[0038] This invention proposes a communication fault event argument extraction method based on large-model event focusing and terminology enhancement, effectively solving the problems of event co-occurrence and difficulty in identifying technical terms in communication fault documents. Through an event boundary-aware encoding module, the model can accurately focus on the text content of specific events, significantly enhancing its ability to distinguish event boundaries, thereby improving the accuracy of event argument extraction. Simultaneously, the semantically enhanced technical terminology recognition module utilizes a two-layer attention mechanism to improve the model's perception of technical terms, enhancing the accuracy of technical terminology recognition. Furthermore, the design of the large-model fine-tuning and event boundary-aware encoding module improves the model's adaptability and generalization ability, reduces dependence on prompt template design, and enables the model to better adapt to different types of events and complex semantic expressions. These improvements give this invention significant application value and broad prospects in the field of intelligent communication fault diagnosis. Attached Figure Description

[0039] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 This is a schematic diagram of a method according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of event extraction according to an embodiment of the present invention;

[0042] Figure 3 This is a diagram of multi-event sample argument extraction according to an embodiment of the present invention;

[0043] Figure 4 This is a diagram showing the label settings for the technical term extraction task in an embodiment of the present invention;

[0044] Figure 5 This is a statistical chart showing the argument prediction errors in multi-event samples according to an embodiment of the present invention.

[0045] Figure 6 This is a statistical chart showing the multi-event sample argument extraction effect of an embodiment of the present invention;

[0046] Figure 7 This is a comparison of the attention distribution of the LMET and Scented models in processing multi-event text according to an embodiment of the present invention.

[0047] Figure 8 This is a diagram illustrating multi-event data samples and results from different models in an embodiment of the present invention. Detailed Implementation

[0048] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0050] The present invention is described as follows:

[0051] With the rapid development of information technology, communication networks are becoming increasingly large-scale, leading to a corresponding increase in the frequency and scope of faults. Timely and accurate identification and handling of fault events are crucial for ensuring the stable operation of communication networks. However, traditional manual fault analysis methods are inefficient and unable to meet the needs of modern communication networks when faced with massive amounts of text data such as network logs and alarm information. Event argument extraction, as an important research direction in natural language processing, aims to identify and extract events and their related participants from text, providing a new approach for the automatic analysis of communication faults. The event extraction task mainly consists of event detection and event argument extraction. Event detection aims to obtain the position of trigger words for events in a sentence and classify the event types corresponding to the trigger words; event argument extraction requires identifying the position of arguments in the sentence and classifying the roles of the arguments. Figure 2 The given event sentence contains one trigger word and three arguments. The event detection task requires identifying the trigger word "adjustment" and classifying it as the event type: "Adjustment Machine". The argument extraction task requires identifying content such as "inter-frequency handover strategy of all cells of HLF1HZ base station on the east side of HC Middle School", "A2+A4", and "A2+A3", and classifying them as the correct argument roles: "Object", "InitialState", and "FinalState" respectively.

[0052] As cue-based learning has gained widespread acceptance, cue-template-based event argument extraction methods have emerged, becoming one of the mainstream technologies in the field due to their superior performance and generalization capabilities. Based on different argument extraction methods, cue-template-based event argument extraction methods are currently mainly divided into two types: cue-based classification learning methods and cue-based generative learning methods. The former primarily identifies arguments in events by labeling specific text fragments for classification, while generative argument extraction methods generate text content containing arguments and extract them according to certain rules. These methods address numerous problems by designing appropriate methodological architectures based on cue-based learning.

[0053] However, in communication failure data, different events are relatively independent, the relationships are relatively simple, and there are many technical terms, making the text semantically ambiguous, such as... Figure 3 As shown, the two events share the same trigger words, have no complex relationship, and exhibit no overlap in arguments. Therefore, it is difficult to effectively distinguish events using trigger words or relationships between them. Furthermore, existing methods lack specific strategies and solutions for handling data containing technical terms, limiting the effectiveness of event extraction in the communications domain. Therefore, how to better handle technical terms and enhance the model's ability to identify event boundaries is a major challenge in communications event extraction tasks.

[0054] like Figure 1 As shown, this embodiment provides a method for extracting communication failure event arguments based on large model event focusing and terminology enhancement, including:

[0055] The large model fine-tuning module is used to fine-tune the large language model, and extract event information and technical terminology information from communication fault documents through the fine-tuned large language model;

[0056] The event boundary-aware encoding module embeds the extracted event information into the prompt template and encodes the template using a non-autoregressive encoding method to obtain the encoded text features.

[0057] The semantically enhanced terminology recognition module uses a two-layer attention mechanism to hierarchically interact the extracted terminology information with the encoded text features, and outputs the terminology recognition results.

[0058] The attention-focused argument extraction module extracts event arguments based on encoded text features and term recognition results using an attention-focusing mechanism.

[0059] The large model fine-tuning module is as follows:

[0060] Directly applying a large, pre-trained language model to a specific domain often yields unsatisfactory results. Fine-tuning can enhance the adaptability of the large model to specific domains and improve task performance. For fine-tuning, the LoRA (Low-Rank Adaptation) method was used. In the design of the prompt templates, the Chain-of-Thought (CoT) method was employed, enabling the large model to extract relevant technical terms from events simultaneously. The Qwen2.5-instruct-14B model was selected as the large model.

[0061] Fine-tuning tasks are mainly divided into two categories: event extraction and terminology extraction. To perform fine-tuning for these two tasks, corresponding training labels need to be set for each before fine-tuning, thus requiring some expansion of the existing dataset. In the event extraction task, the maximum and minimum indices of the range of all arguments in each event are located and used as the boundaries of the event. The string representing this event is then extracted and used as the training label for the event extraction task. In the terminology extraction task, the argument content and role types in the dataset are analyzed and statistically analyzed. Finally, the different argument roles in the original dataset are abstracted into three major categories of terminology types, which are then used as the labels for the terminology extraction task. For example... Figure 4 As shown, “Owner” and “Subject” are the role types of different arguments in the original dataset, while “fault name” and “fault status” are the abstracted professional term types.

[0062] After completing the training labels required for the fine-tuning task, a prompt template based on the thought chain method was designed for fine-tuning large models. This prompt template can extract technical terms contained in events while extracting events. The prompt template is: "You are an expert in the field of event extraction. Below are some examples of event extraction. You need to extract the events contained in the text according to the following event extraction process.\nFirst, you need to determine whether there are one or more events in the event text (e.g., unreasonable 3G focus selection threshold configuration leads to increased L2U coverage redirection)? Then, for the event type and trigger word pair I provided: ['Event type: configuration fault type - trigger word: configuration', 'Event type: hardware and software anomaly type - trigger word: redirection'], extract the corresponding events?\nFor example, based on the above text and the two pairs of trigger words and event types provided, two events can be obtained: [{'Event content': 'unreasonable 3G focus selection threshold configuration', 'Event type': 'configuration fault type'}, {'Event content': 'increased L2U coverage redirection', 'Event type': 'hardware and software anomaly type'}] (Note that the extracted events should be a continuous text segment from the original text, and cannot be arbitrarily generated.)\n

[0063] Next, you need to extract the fault entities from each event based on the extracted event text. There are three types of fault entities: fault name, fault status, and fault indicator rate. For example, the fault entities that can be extracted from the two events above are: [[{'word':'3G priority selection threshold','type':'fault name'},{'word':'unreasonable','type':'fault status'}],[{'fault name':'L2U coverage','fault status':'increase'}]]\nBased on the example of extracting events above, extract the event from the following text: The license application failed because the low version license item was directly pushed to the high version. The event type and trigger word pairs involved in the text are: ['Event type: software and hardware anomaly - trigger word: application','Event type: machine adjustment - trigger word: push']".

[0064] The prompt template performs fine-tuning of event extraction by combining event types and trigger words. After successfully extracting events related to the trigger words, the prompt template further introduces prompt statements to extract technical terms from the events. Based on the above fine-tuning process, the final results include the events and technical terms contained in each training sample.

[0065] After completing the training labels required for fine-tuning, a cue template is constructed using the thought chain method to fine-tune the large model. This cue template first enables the large model to extract the events contained in the text, and then extracts the technical terms contained in the events. Based on the above fine-tuning process, the events and technical terms contained in each training sample can finally be obtained.

[0066] The event boundary awareness encoding module is as follows:

[0067] Unlike existing methods that focus on leveraging the correlation between events to address the event co-occurrence problem, the LMETE model, taking into account the characteristics of communication domain datasets, emphasizes ensuring the integrity and independence of individual events, and to this end, it designs an innovative event boundary-aware encoding module. Specifically, this method first embeds event information generated by a fine-tuned large language model into the prompt template, and then encodes it using a non-autoregressive encoder. The core of this module lies in the unique integration of non-autoregressive encoding and event embedding techniques in this embodiment, guiding the model to deeply analyze the special event boundary markers in the template. <s>During the coding phase, the model utilizes... <s>The marker accurately locates the target event for the current argument to be extracted through a self-attention mechanism. Subsequently, in the cross-attention calculation during the decoding phase, the model uses... <s>Using "" as the core anchor point, attention from features (argument roles such as Owner and Setting) used to locate argument boundaries is dynamically focused on the target event pointed to by the tag (see details). Figure 1 (CrossAttention component). This design ultimately achieves efficient aggregation of information about specific events, thereby significantly enhancing the model's ability to perceive event boundaries.

[0068] In building prompt templates focused on specific events, the first step is to precisely locate the events generated by the large model within the text and then use specific tags. <s>< / s> Perform explicit annotation (annotation effect as shown) Figure 1 (As shown in the prompt template module of the Zhongda model guiding event focus). Based on this, the argument entity type is embedded in the prompt template, and corresponding prompt templates are designed for the seven categories of fault events in the dataset (as shown in Table 1), and professional term types are embedded in them. Finally, the event text after explicit event labeling is concatenated with the template text of the corresponding event type to form the prompt template content of the final input encoder, as shown in formula (1):

[0069] X=[<s>E1< / s>E2,...P; J; W] (1)

[0070] Where E i Let be the i-th event contained in the text, P be the remaining string information in the event text, and W be the template for the specific event type in Table 1. J is the event type prompt prefix constructed for different training samples, mainly composed of the event type to which the data belongs and the trigger word.

[0071] Table 1

[0072]

[0073] With the continuous development of pre-trained models and large models, most current event argument extraction methods use pre-trained models as text encoders. Considering that professional terms in communication datasets are usually in English, this embodiment selects a multilingual (Multilingual T5, MT5) model as the encoder to enable the encoder to fully understand the semantics of English and Chinese words. LMETE strengthens the interaction between event text and template by encoding the constructed prompt template text using MT5 in a non-autoregressive form. As shown in formula (2):

[0074] C,T=CompositeMT5Encoder(X),

[0075]

[0076] CompositeMT5Encoder is a composite encoder where X is simultaneously input into both an MT5 encoder and a non-autoregressive decoder, ultimately yielding encoded text features C and T. C represents the encoded features of the original text, and T represents the features encoded from the template. N and M are the lengths of the event text and the template, respectively, and d... t This represents the encoding dimension of the vector.

[0077] The semantically enhanced terminology prediction module is as follows:

[0078] To address the large number of technical terms present in communication fault datasets, LMETE incorporates a semantically enhanced technical terminology module by integrating a large model. Term prediction is essentially a sequence labeling task. To gain a deeper understanding of argument semantics, the entity type to which the argument belongs is integrated into the event argument extraction process. Drawing inspiration from this method, an embedding matrix of technical terminology types is constructed based on event types in the communication domain data, as shown in formula (3):

[0079]

[0080] Where L is the number of event types in the dataset, and 3 represents the token type, which can be B, I, or O.

[0081] After creating the BIO Embedding layer, the embedding matrix B and the features [C,T] are fused using a multi-head attention mechanism to align the term label features in the embedding matrix B with the term-related features in [C,T], as shown in formulas (4)-(7):

[0082]

[0083] Where B' is the feature vector of B after expanding its shape to ensure it has the same batch size, preparing it for subsequent fusion, and W q W k W v The weight matrix is ​​a linear transformation, and the CMA(·) function represents a multi-head cross-attention network layer.

[0084] After the first layer of attention mechanism, the BIO Embedding layer has completed the initial perception and alignment of terms in the [C,T] vector. In order to enhance the perception of professional terms, a dual-layer attention fusion mechanism is designed. The second layer of attention mechanism enhances the semantic representation of professional terms in the fused feature vector by fusing the professional terms generated by the large model, as shown in formulas (8)-(11):

[0085]

[0086]

[0087] Where Z represents the technical terms generated by the large model, after shape expansion. Finally, the features fused by the two-layer attention mechanism are concatenated with [C,T] to obtain the final semantic vector used for technical term prediction, as shown in formulas (12)-(14):

[0088]

[0089] O = Linear2(O), O ∈ R Nx3xL (14)

[0090] Finally, Conditional Random Fields (CRF) were used for sequence labeling, and the log-likelihood function was used as the loss function for the prediction module. The loss is denoted as Loss. ner .

[0091] The attention-focused argument extraction module is as follows:

[0092] Existing argument extraction methods typically embed argument roles (such as Owner, Setting, etc.) into cue templates and decode argument type and scope by fusing their features. While this strategy is effective in distinguishing arguments of different event types, it struggles to precisely focus attention on the target event when processing texts containing multiple events, leading to decreased extraction performance. To address this challenge, this embodiment proposes an attention-focused argument extraction scheme. By deeply coupling the argument extraction module with the event boundary-aware encoding module, it ensures that the semantic information of argument roles is concentrated on the specific target event during the encoding stage, thereby achieving attention-focused processing of the argument extraction process. Figure 1 As shown, in extracting the event "Inappropriate RRU working mode setting leads to cell establishment failure when CPRI compression is changed to enhanced compression", the argument roles (such as Owner, Setting, etc.) focus on "inappropriate RRU working mode setting" during the encoding stage, which enhances the model's ability to distinguish the boundaries of different events. This is mainly due to the deep integration of the event boundary perception module and the argument extraction module.

[0093] Furthermore, to further improve the accuracy of argument selection, this embodiment introduces a dual-view optimization mechanism, utilizing domain knowledge (technical terminology) to assist in the judgment, thereby obtaining more accurate decoding results.

[0094] The encoded event type template features, i.e., the T vector in [C,T], contain all argument roles in a specific event type, as shown in Table 1. Features of different roles in T are extracted, and span selectors are constructed for each role, S. rolei and E rolei The beginning and end positions of the argument in the text are indicated by formulas (15)-(16):

[0095]

[0096] EmbeddingT(role i ) is the encoded feature in T corresponding to the i-th argument role, W start1 and W end1 It is a learnable weight matrix shared by all argument roles.

[0097] The encoded features C of the event text to be extracted and the final features O of the terminology prediction module are subjected to linear dimensional transformation to obtain the final hidden features F, as shown in formulas (17)-(18):

[0098]

[0099] Based on the above features, the probability distribution of the starting position of each argument role is calculated, as shown in formulas (19)-(20):

[0100]

[0101] After obtaining the probability distributions of different argument roles, in order to use a semantic understanding mechanism with dual perspectives of argument roles and technical terms to make joint decisions on argument selection, it is necessary to obtain the probability distribution of the starting position of technical terms. The value of the term is obtained by multiplying the probability distribution of the terminology with the probability distribution of the argument character, as described above, to obtain the final probability distribution of the argument extraction, as shown in formulas (21) and (22).

[0102]

[0103] The final argument extraction loss is calculated using the cross-entropy loss function. Probability distribution and standard distribution The loss is shown in formulas (23)-(24):

[0104]

[0105] L bce (x,y)=-(ylogx+(1-y)log(1-x)) (24)

[0106] Argument extraction loss and term prediction loss are used as a joint loss, and the loss from each training step is backpropagated and the parameters are updated, as shown in Equation (25).

[0107] L losss =L eae +L ner (25)

[0108] To verify the feasibility of this method, the following experiments were conducted in this embodiment:

[0109] Dataset selection:

[0110] To verify the practical effectiveness of the LMETE method in the field of communication fault diagnosis, experiments were conducted on publicly available Huawei communication fault datasets and China Telecom communication fault datasets. Table 2 shows the partitioning of the two datasets on the training and test sets. The datasets are described below:

[0111] Huawei Communication Fault Dataset: This dataset comes from "CCKS-2021" Huawei's process-type event extraction for the communication field. It is a publicly available event extraction dataset in the field of communication faults. Huawei experts defined fault events based on various process-type knowledge in the communication field, which is of great significance in analyzing key errors that occur in the operation and maintenance process.

[0112] Telecommunications Fault Dataset: This dataset is derived from real fault data generated by internal telecommunications operations and maintenance. Based on this, this embodiment constructs a high-quality dataset specifically for event argument extraction tasks.

[0113] Table 2

[0114]

[0115] The data parameter settings and evaluation indicators are as follows:

[0116] The encoder input length proposed in this embodiment is set to a maximum of 220 characters. For sentences exceeding the maximum length, only the first 220 characters are used; otherwise, the remaining characters are used... <pad>Completed. Because the experiment involves fine-tuning the large model and training the event argument extraction model, it involves two sets of experimental environments.

[0117] When fine-tuning the large model, the batch size during fine-tuning training was set to 4, and a total of 5 epochs were trained. The fine-tuning experiment was conducted on a server with Ubuntu 20.04 system version. The specific experimental environment information is shown in Table 3.

[0118] Table 3

[0119]

[0120] In the experiment of training the event argument extraction model, a total of 50 epochs were trained. At each epoch, the test and validation sets were saved, and the model with the best decoding result was selected as the final result of the experiment.

[0121] The event argument extraction experiment was conducted on a server running Ubuntu 20.04. Specific experimental environment information is shown in Table 4.

[0122] Table 4

[0123]

[0124] The model performance was evaluated using two common metrics: event argument extraction. For each metric, three indicators were used to measure the model's performance: precision, recall, and F1 score.

[0125] (1) Argument recognition (AI): If the predicted argument span matches the golden span, the argument is correctly recognized.

[0126] (2) Argument Classification (AC): If the predicted argument span matches the golden span and the predicted argument role is also correct, then the argument is correctly identified.

[0127] The experimental results are analyzed as follows:

[0128] In the experimental section, this embodiment compares and analyzes the proposed method with event argument extraction methods from the past five years on the China Telecom and Huawei communication fault datasets. Specifically, to highlight the superiority of the proposed method, experiments were conducted on classification-based and generative methods. The model performance on the Huawei communication fault public dataset and the China Telecom communication fault dataset is shown in Tables 5 and 6, where bold text indicates the best value in the same column of the experimental data.

[0129] The comparison method is as follows:

[0130] The EEQA method proposes a way to transform the event extraction task into a question-answering task. By designing question templates, it transforms event trigger word detection and event argument extraction into asking questions about the input sentence and returning the results of event extraction as the answer.

[0131] BartGen is a conditionally generated document-level event argument extraction model. This method first encodes the event template and context text without argument parameters, and then fills in the argument parameters in the template through conditional generation, thus realizing the parsing and extraction of event arguments.

[0132] The XGear method proposes a multilingual pre-trained model-based approach by designing language-independent argument parameter filling event templates. By transforming the event argument extraction task into a language generation task, it achieves zero-shot cross-lingual event argument parameter extraction.

[0133] The PAIE method adds argument roles to the cue template, captures the interaction between parameters through multi-role cues, and performs joint optimization through bipartite graph matching loss to achieve optimal argument span allocation, enabling the model to achieve better learning and generalization with limited training data.

[0134] DEGREE transforms the event argument extraction task into a conditional generation problem. Given contextual text and designed prompt templates, the model summarizes the events in the text into natural sentences that conform to predefined patterns, and finally extracts the argument parameters from the sentences through parsing. By utilizing label semantics and weak supervision signals, the model's learning efficiency is improved under low-resource conditions.

[0135] TagPrime provides a unified processing framework for tasks such as event extraction and relation extraction. In the event argument extraction task, this method adds event types and trigger words to the input text, utilizes the self-attention mechanism of the pre-trained model, enables the model to better capture relational structures under specific conditions, and accelerates the event argument reasoning process through parallel encoding.

[0136] TabEAE proposes an attention-based method for event argument extraction to strengthen the connections between different events. Unlike previous studies that only considered individual events, this method extracts all argument parameters for each event simultaneously during training. This definition considers all co-occurring events in the same context, enabling the model to better capture the semantic boundaries of events and improving its ability to handle multiple events.

[0137] Similar to TabEAE, the DEEIA method is also an event argument extraction method that considers the event co-occurrence problem. This method first strengthens the connection between events within and between events through an attention mechanism, and enhances the extraction capability of specific events through a specific event information aggregation module.

[0138] The Scented method effectively addresses the problems of weak semantic relevance, compromised semantic integrity, and one-sided semantic understanding in existing methods by customizing the inclusion of argument roles, argument entity types, and their correspondence in the prompt template and integrating argument entity type information into event argument extraction.

[0139] Table 5

[0140]

[0141] Table 6

[0142]

[0143]

[0144] Among the methods described above, the EEQA model performed relatively poorly. This is mainly attributed to its argument extraction mechanism through questioning, which places high demands on the model's semantic understanding capabilities. However, data in the field of communication faults is highly specialized and semantically ambiguous, significantly increasing the difficulty of understanding compared to datasets from other domains, directly limiting the effectiveness of event argument extraction. In contrast, the LMETE model proposed in this paper achieves superior results in both recall and F1 score. Specifically, on Huawei's publicly available communication fault dataset, compared to the best generative method, BartGen, LMETE improved recall by 9.76 and 8.64 percentage points on AI and AC metrics, respectively, and improved F1 scores by 11.51 and 10.30 percentage points, respectively. On the telecommunications dataset, LMETE improved recall by 4.90 and 4.97 percentage points on AI and AC metrics, respectively, and improved F1 scores by 7.64 and 7.58 percentage points, respectively. Compared to Scented, the best-performing model in classification learning methods, LMETE achieved a 6.64% and 6.29% improvement in recall on AI and AC respectively, and a 2.92% and 2.71% improvement in F1 score on the Huawei public dataset. On the telecommunications dataset, LMETE achieved a 4.88% and 5.56% improvement in recall on AI and AC respectively, and a 2.01% and 2.80% improvement in F1 score. This demonstrates that the LMETE method achieves excellent performance in event argument extraction. This significant performance improvement is mainly attributed to two key design features: first, the event focusing module effectively enhances the model's boundary discrimination ability when handling multiple event samples; second, the terminology prediction module improves the model's accuracy in terminology recognition by accurately predicting terminology within events. These innovative designs collectively contributed to the significant improvement in the model's performance metrics.

[0145] Meanwhile, it can be observed that the methods TabEAE and DEEIA, which specifically address the event co-occurrence problem, perform poorly on the communication fault dataset. This is mainly because these two methods enhance the model's ability to distinguish event boundaries by using attention mechanisms to explore the correlation between events and extracting multiple events simultaneously. However, given the relative independence of events in communication fault data, these methods are ineffective in handling the event co-occurrence problem. This phenomenon is analyzed and discussed in detail in the subsequent effectiveness analysis section of this embodiment.

[0146] This embodiment also included the following ablation experiment:

[0147] The core architecture of the LMETE method consists of two main modules: an event boundary-aware encoding module and a semantically enhanced terminology recognition module. To verify the effectiveness of each module and its contribution to overall performance, they were evaluated and analyzed in ablation experiments. The results of the ablation experiments on Huawei's publicly available communication fault dataset and telecom communication fault dataset are shown in Tables 7 and 8. The ablation experiments are described below:

[0148] (1) w / o-Terminology-LLM-Fusion: The model effect after removing the terminology module from LMETE.

[0149] (2) w / o-Event-Aware-Encoder: The model effect after removing the event boundary-aware encoding module from LMETE. (To illustrate the effectiveness of the event boundary-aware encoder, this embodiment systematically eliminates the model's ability to perceive specific event boundaries by replacing the specially designed event focus prompt template in the encoding module with a regular prompt template.)

[0150] Table 7

[0151]

[0152] Table 8

[0153]

[0154] Experiment 1 shows that after removing the terminology prediction module, LMETE scores on the Huawei and Telecom datasets decreased by 0.66% and 0.6% in AI-F1 and AC-F1, respectively, and by 0.48% and 0.59% in AC-F1. This indicates that the proposed dual-layer attention mechanism terminology module in the fusion model plays a positive role in improving the event argument extraction effect.

[0155] Experiment 2 removed the event-focusing cue template module from the event boundary-aware encoding module. Experimental results showed that the AI-F1 and AC-F1 scores decreased by 1.57% and 1.4% on the Huawei dataset, and by 1.35% and 1.32% on the China Telecom dataset. This significant performance drop validates the importance of the event boundary-aware module. Therefore, with the help of the event boundary-aware encoding module, the model can accurately focus on the text content of the event to be extracted, enhancing its ability to distinguish the boundaries of multiple event texts, thereby improving the performance of event argument extraction.

[0156] The validity analysis is as follows:

[0157] To verify the superiority of the LMETE method in the field of communication fault diagnosis, the LMETE method is compared and analyzed with recent methods in terms of effectiveness. (Since telecommunications datasets are confidential, publicly available datasets from Huawei are used as the analysis object in the effectiveness analysis).

[0158] Effectiveness analysis of boundary discrimination capability:

[0159] To verify the effectiveness of the LMETE model in distinguishing event boundaries when dealing with the co-occurrence of events, this study first conducted a statistical analysis of the text distribution exhibiting multiple events in the dataset, as shown in Table 9. Subsequently, a more in-depth analysis of the multiple-event samples was carried out from the following two dimensions:

[0160] (1) Analysis of the effect of argument extraction;

[0161] The aim is to demonstrate that the LMET model can effectively distinguish the boundaries between different events by examining its superior argument extraction performance in multi-event samples. Specifically, this embodiment compares and analyzes the LMETE method with the well-performing TagPrime and Scented methods, as well as the TabEAE and DEEIA methods used to address the event co-occurrence problem. The results are as follows: Figure 5 , Figure 6 As shown.

[0162] Table 9

[0163]

[0164] Table 9 shows the number of multi-event data samples in the dataset. It can be seen that the number of multi-event samples in the Huawei communication dataset accounts for 77.43%, which indicates that the handling of multi-event co-occurrence problems in the communication dataset is of great significance and value. Figure 5 This section illustrates the distribution of argument prediction errors for different methods on multi-event samples in the test set. Sum-Count represents the total number of argument prediction errors, while Bound-Count represents the number of arguments predicted to belong to another event within the ER-Count, indicating that the predicted arguments did not accurately grasp the event boundary and crossed it. Su-Bo-Percent refers to the proportion of Bo-Count in Sum-Count. Figure 6 The corresponding statistics show the effectiveness of event argument extraction on different methods for multiple event samples in the test set.

[0165] Through the Figure 5 Analysis of the results reveals that the TabEAE and DEEIA methods generate significantly more argument prediction errors than LMETE on multi-event samples. In terms of the Su-Bo-Percent error rate, TabEAE and DEEIA methods show increases of 35.31% and 42.78% respectively compared to LMETE. Their error rates are also significantly higher than the TagPrime and Scented baseline methods. This indicates that while TabEAE and DEEIA provide targeted solutions to the event co-occurrence problem and achieve good results on relevant domain datasets, they still have shortcomings in the communication domain, particularly in their ability to distinguish event boundaries. This is because their approach to event co-occurrence primarily involves associating cue templates and textual information for different events using an attention mechanism, leveraging the correlation between events to highlight event boundary information. This attention-based event association method places high demands on cue template design; a high-quality cue template may even require consideration of combinations of different synonyms. Furthermore, given the relative independence and technical terminology of events in the communication domain, overemphasizing the correlation between events can blur the boundaries between them. The method in this embodiment primarily relies on a large model to locate specific events in solving the multi-event problem. It improves event extraction while maintaining event independence, and has lower requirements for prompt template construction, exhibiting transferability and reliability. By comparing the model's performance on multiple event samples, it can be seen that the LMETE method has a strong advantage in distinguishing event boundaries, solving the event co-occurrence problem and achieving good performance in the event argument extraction task.

[0166] (2) Analysis of the internal coding mechanism of the model;

[0167] To illustrate how the boundary-aware coding module can focus attention on specific events during the encoding phase, thereby enhancing event boundary discrimination and improving argument extraction, this embodiment provides a visualization mechanism analysis of its internal boundary-aware module. This embodiment selects the current best baseline model, Scented, for comparison, and uses heatmap technology to visualize and compare the cross-attention distribution generated by the two models during the argument extraction phase, such as... Figure 7 As shown.

[0168] This figure clearly reveals the LMET model's precise attentional focus on different events, while the Scented model's attention is more diffuse. When processing this text, the Scented model exhibits highly similar cross-attention distributions among different argument roles (Owner, Index, State) when extracting different events, indicating its failure to effectively distinguish the contextual information of two independent events, leading to confusion in attention focus. This is mainly because both events in this complex text belong to the "indicator deterioration" type and share the same trigger words. Therefore, Scented cannot accurately locate the event boundaries simply by relying on event type and trigger words, making it difficult to accurately focus on the event to be extracted. In stark contrast, the LMET model demonstrates precise boundary awareness. When extracting the "ERAB success rate decline" event, the argument role adaptively focuses attention on related words such as "ERAB," "establish," "success rate," and "decline." Similarly, when processing the "RRC decline" event, attention precisely shifts and concentrates on the words "RRC" and "decline." This result strongly demonstrates that LMET's boundary-aware module can dynamically allocate and isolate attention for different events, significantly improving the model's argument extraction performance in complex scenes. This finding provides a theoretical explanation and empirical support for LMET's superior performance in handling multi-event samples from the perspective of attention mechanisms.

[0169] The final experimental results show that LMET, through its unique event boundary awareness mechanism, can more accurately capture and distinguish the boundary features of different events, thereby significantly improving the model's ability to process complex multi-event samples.

[0170] Effectiveness analysis of technical terminology recognition capability:

[0171] To verify the accuracy of the LMETE method in technical terminology recognition, we selected TagPrime and Scented methods, which performed well in the comparative experiment, and analyzed them together with LMETE in technical terminology recognition. The results of technical terminology recognition are shown in Table 10.

[0172] Table 10

[0173]

[0174] As shown in Table 10, the LMETE method outperforms the TagPrime and Scented methods by 5.25% and 5.27% in EI-P, 1.90% and 2.01% in EI-R, and EC-P and EC-R, respectively. The LMETE method achieves the best performance across all performance metrics for terminology prediction, validating the effectiveness of the terminology module in improving terminology recognition. With the help of the terminology recognition module, the LMETE method enhances the model's ability to recognize terms in the communication field and, with the help of other modules, improves the overall performance of the model.

[0175] The case analysis is as follows:

[0176] To more intuitively demonstrate the LMETE method's ability to distinguish event boundaries and the accuracy of its terminology prediction, LMETE is compared with more advanced methods such as TagPrime and Scented on typical multi-event data examples through experiments and analysis. Figure 8 As shown.

[0177] Figure 8 The results of Scented, TagPrime, and LMETE methods for event extraction from text containing multiple events are shown. The yellow and blue sections represent the results of argument extraction for events with different colored fonts in the text. Analysis reveals that the Scented method failed to extract the argument "Configuration information in the RRUCHAIN ​​chain number" from the events described in blue font. While the TagPrime method successfully extracted this argument, it had an incomplete extraction error for the argument "LTE-FDD Lampsite site BBU CPRI line rate BBU CPRI line rate" from the yellow events. This is mainly because these two methods do not focus on the model's ability to distinguish events and extract technical terms in multi-event examples, leading to errors in argument extraction, or even its failure to extract certain arguments. In contrast, LMETE, with its focus on specific event prompt templates and technical terminology prediction module, successfully and accurately extracted event arguments, improving the overall event argument extraction performance.

[0178] The above analysis demonstrates that LMETE achieves good performance in processing datasets containing multiple events and complex communication terminology. Specifically, LMETE, through its event boundary-aware encoding module, fully understands the boundary information between multiple event texts, thereby eliminating redundant information from other events during the extraction of specific events and accurately focusing on the semantic information of those specific events. Secondly, after obtaining the semantic expression focused on specific events, LMETE enhances its ability to extract technical terms from events through its terminology recognition module, accurately extracting arguments containing technical terms within events, achieving good overall performance in event argument extraction.

[0179] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.< / pad> < / s> < / s> < / s>

Claims

1. A method for extracting argument of communication failure events based on large model event focusing and term enhancement, characterized in that, Includes the following steps: Fine-tune the large language model, and extract event information and technical terminology information from communication fault documents using the fine-tuned large language model; An event boundary-aware encoding module is constructed, which embeds the extracted event information into the prompt template and encodes the template using a non-autoregressive encoding method to obtain the encoded text features. A semantically enhanced terminology recognition module is constructed. Through a two-layer attention mechanism, the extracted terminology information and the encoded text features are hierarchically interacted to output the terminology recognition results. Based on the encoded text features and term recognition results, an attention-focusing mechanism is used to extract event arguments. The extraction of event arguments using the attention-focusing mechanism includes: Extract features of different argument roles from the encoded event type template features; Construct a span selector for each argument role to identify its start and end positions in the text; The encoded features of the event text to be extracted are linearly transformed with the features of the terminology prediction module to calculate the probability distribution of the starting position of the argument role. Obtain the probability distribution of the starting positions of technical terms; The probability distribution of the obtained technical terms is multiplied by the probability distribution of the argument roles to obtain the final probability distribution of argument extraction.

2. The method according to claim 1, characterized in that, The fine-tuning of the large language model includes: The Qwen2.5-instruct-14B model was selected as the large language model; The LoRA method was used to fine-tune the large model; Construct prompt templates based on thought chains to enable large models to extract both event information and technical terminology information simultaneously.

3. The method according to claim 2, characterized in that, The extracted event information and technical terminology information include: In the event extraction task, the maximum and minimum indices of the range of all arguments in the event are located as the event boundaries, and the event strings are extracted as training labels. In the technical term extraction task, argument roles are abstracted into technical term types and used as labels for the technical term extraction task.

4. The method according to claim 1, characterized in that, The encoded text features include: Embed event information generated by the large model into the prompt template, and use tags to explicitly mark event boundaries; Design corresponding prompt templates for different event types, and embed technical terms into the templates; A non-autoregressive encoder is used to encode the concatenated prompt template to obtain the encoded text features.

5. The method according to claim 1, characterized in that, The semantically enhanced terminology recognition module includes: Construct an embedding matrix of technical terms; A multi-head attention mechanism is used to interactively fuse the embedding matrix with the encoded text features; By further integrating the specialized terms extracted from the large model through a two-layer attention mechanism, the semantic representation of the terms is enhanced.

6. The method according to claim 5, characterized in that, The terminology recognition module uses conditional random fields for sequence labeling and log-likelihood function as the loss function.

7. The method according to claim 1, characterized in that, During training, a joint loss function is used, which includes argument extraction loss and term prediction loss, and the model parameters are updated through backpropagation.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

Images