Communication fault event argument extraction method based on large model event focusing and term enhancement

Abstract

The invention discloses a communication fault event argument extraction method based on large model event focusing and term enhancement, which comprises the following steps of: firstly, finely adjusting a large language model to extract event information and professional term information in a communication fault document; secondly, an event boundary sensing coding module is constructed, the prompt template embedded with the event information is coded in a non-autoregressive coding mode, and the capability of distinguishing event boundaries is enhanced; thirdly, constructing a semantic enhanced terminology recognition module, and performing hierarchical interaction on terminology information and text features by using a double-layer attention mechanism to improve the accuracy of terminology recognition; and finally, based on the encoded text features and term recognition results, an attention focusing mechanism is adopted to extract event arguments. According to the method, the performance of a communication fault data set and the performance of a telecommunication communication fault data set are remarkably improved, and the problems that multi-event text boundaries are difficult to distinguish and technical terms are difficult to recognize are effectively solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of communication, and particularly relates to a communication fault event argument extraction method based on large model event focusing and term enhancement. BACKGROUND

[0002] With the rapid development of information technology, the scale of communication networks is increasingly large, and the frequency and scope of faults are also increasing. Timely and accurate identification and processing of fault events are crucial to ensuring the stable operation of communication networks. However, in the face of massive network logs, alarm information and other text data, traditional manual fault analysis methods are inefficient and cannot meet the needs of modern communication networks. Event argument extraction, as an important research direction in the field of natural language processing, aims to identify and extract events and their related participants from text, providing a new approach for the automatic analysis of communication faults.

[0003] The event extraction task mainly consists of event detection and event argument extraction. Event detection aims to obtain the position of the trigger word in the sentence and classify the event type corresponding to the trigger word; event argument extraction needs to identify the position of the argument in the sentence and classify the argument role. In recent years, event argument extraction research has made important progress driven by deep learning, but existing methods have the following shortcomings when dealing with communication fault data:

[0004] Event co-occurrence problem: existing methods mostly ignore the event co-occurrence problem of multiple events contained in the text, resulting in weak ability of the model to distinguish event boundaries. In communication fault data, different events have relative independence, the relationship is relatively simple, and there are a large number of professional terms, making the text relatively vague in semantic expression, so it is difficult to effectively distinguish events by using trigger words or the relationship between events.

[0005] Professional term identification problem: there are a large number of professional terms in the field of communication, and existing methods lack special coping strategies and solutions when dealing with data containing professional terms, making it difficult to accurately identify the argument content corresponding to the professional term, affecting the effect of argument extraction. SUMMARY

[0006] The present application proposes a communication fault event argument extraction method based on large model event focusing and term enhancement to solve the above-mentioned problems existing in the prior art.

[0007] To achieve the above-mentioned purpose, the present application provides a communication fault event argument extraction method based on large model event focusing and term enhancement, comprising the following steps:

[0008] Fine-tuning the large language model, and extracting event information and professional term information in the communication fault document through the fine-tuned large language model.

[0009] An event boundary-aware encoding module is constructed to embed the extracted event information into a prompt template and encode the template through a non-autoregressive encoding manner to obtain encoded text features;

[0010] A semantic-enhanced professional term recognition module is constructed to perform hierarchical interaction between the extracted professional term information and the encoded text features through a double-layer attention mechanism to output a term recognition result;

[0011] Based on the encoded text features and the term recognition result, an attention focusing mechanism is adopted to extract event arguments.

[0012] Optionally, the fine-tuning of the large language model comprises:

[0013] The Qwen2.5-instruct-14B model is selected as the large language model;

[0014] The LoRA method is adopted to fine-tune the large model;

[0015] A prompt template based on a thinking chain is constructed to enable the large model to simultaneously extract event information and professional term information.

[0016] Optionally, the extraction of the event information and the professional term information comprises:

[0017] In the event extraction task, the maximum index and the minimum index of the range of all arguments in the event are located as the event boundary, and an event string is extracted as a training label;

[0018] In the professional term extraction task, the argument role is abstracted as a professional term type, and is used as a label of the professional term extraction task.

[0019] Optionally, the obtaining of the encoded text features comprises:

[0020] The event information generated by the large model is embedded into the prompt template, and the event boundary is explicitly labeled using the label;

[0021] Corresponding prompt templates are designed for different event types, and the professional term types are embedded into the templates;

[0022] A non-autoregressive encoder is used to encode the spliced prompt template to obtain the encoded text features.

[0023] Optionally, the semantic-enhanced professional term recognition module comprises:

[0024] An embedding matrix of the professional term type is constructed;

[0025] The multi-head attention mechanism is used to interactively fuse the embedding matrix and the coded text features.

[0026] The double-layer attention mechanism is used to further fuse the professional terms extracted by the large model, and enhance the semantic representation of the terms.

[0027] Optionally, the professional term recognition module uses conditional random field for sequence labeling, and takes a log likelihood function as a loss function.

[0028] Optionally, the event argument extraction using the attention focusing mechanism comprises:

[0029] The features of different argument roles are extracted from the coded event type template features;

[0030] A span selector is constructed for each argument role to identify the starting and ending positions of the argument role in the text;

[0031] The coded features of the event text to be extracted are linearly transformed with the features of the professional term prediction module to calculate the probability distribution of the starting position of the argument role.

[0032] Optionally, the event argument extraction using the attention focusing mechanism further comprises:

[0033] The probability distribution of the starting position of the professional term is obtained;

[0034] The obtained probability distribution of the professional term is multiplied with the argument role probability distribution to serve as the final probability distribution of the argument extraction.

[0035] Optionally, a joint loss function is used in the training process, the joint loss function comprises an argument extraction loss and a professional term prediction loss, and the model parameters are updated through back propagation.

[0036] The application further provides a computer device comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to realize the steps of the method.

[0037] Compared with the prior art, the application has the following advantages and technical effects:

[0038] The application provides a communication fault event argument extraction method based on a large model event focus and term enhancement, which effectively solves the problems of event co-occurrence and professional term difficulty to identify in a communication fault document. Through the event boundary perception coding module, the model can accurately focus on the text content of a specific event, significantly enhance the ability to distinguish the event boundary, and thus improve the accuracy of event argument extraction. Meanwhile, the semantic enhancement professional term identification module uses a double-layer attention mechanism to improve the model's perception ability of professional terms and enhance the accuracy of professional term identification. In addition, based on the large model fine-tuning and the event boundary perception coding module design, the adaptability and generalization ability of the model are improved, the dependence on the prompt template design is reduced, and the model can better adapt to different types of events and complex semantic expressions. These improvements make the application have important application value and wide popularization prospect in the field of communication fault intelligent diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The illustrations, together with the description, serve to explain the application, but do not limit the application. In the drawings:

[0040] Figure 1 The method schematic diagram of the embodiment of the application is shown in the figure;

[0041] Figure 2 The event extraction schematic diagram of the embodiment of the application is shown in the figure;

[0042] Figure 3 The multi-event sample argument extraction diagram of the embodiment of the application is shown in the figure;

[0043] Figure 4 The label setting diagram of the professional term extraction task of the embodiment of the application is shown in the figure;

[0044] Figure 5 The multi-event sample argument prediction error condition statistics diagram of the embodiment of the application is shown in the figure;

[0045] Figure 6 The multi-event sample argument extraction effect statistics diagram of the embodiment of the application is shown in the figure;

[0046] Figure 7 The attention distribution comparison diagram of the LMET and Scented model when processing multi-event text of the embodiment of the application is shown in the figure;

[0047] Figure 8 The multi-event data sample and the result display diagram of different models of the embodiment of the application are shown in the figure. DETAILED DESCRIPTION

[0048] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other in the case of no conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0049] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0050] The present application is introduced as follows:

[0051] With the rapid development of information technology, the scale of communication network is increasingly large, and the frequency and influence range of faults also increase. Timely and accurately identifying and handling fault events is crucial to ensure the stable operation of the communication network. However, in the face of massive network logs, alarm information and other text data, the traditional manual fault analysis method is inefficient and difficult to meet the needs of modern communication networks. Event argument extraction, as an important research direction in the field of natural language processing, aims to identify and extract events and their related participants from text, providing a new idea for the automatic analysis of communication faults. The event extraction task mainly consists of event detection and event argument extraction tasks. Event detection aims to obtain the position of the trigger word in the sentence and classify the event type corresponding to the trigger word; event argument extraction needs to identify the position of the argument in the sentence and classify the argument role. For example Figure 2 An event sentence containing 1 trigger word and 3 arguments is given. The event detection task needs to identify the trigger word "adjust" and classify it as the event type "adjust machine class". The argument extraction task needs to identify "HC Middle School East Green HLF1HZ Base Station All Cell Inter-frequency Handover Strategy", "A2+A4", "A2+A3", etc. and classify them as the correct argument roles "Object", "InitialState" and "FinalState" respectively.

[0052] With the prompt learning officially entering the public's view, the event argument extraction method based on the prompt template also emerges as the times require. This method has become one of the mainstream technologies in this field due to its excellent performance and generalization ability. According to the different ways of argument extraction, the current event argument extraction method based on the prompt template mainly divides into two types: the classification learning method based on prompt learning and the generative learning method based on prompt learning. The former mainly identifies the arguments in the event through the classification of specific text fragments, and the generative argument extraction method generates text content containing arguments and extracts them according to certain rules. These methods solve many problems by designing corresponding method architectures on the basis of using prompt learning.

[0053] However, in the communication failure data, there is relative independence between different events, the relationship is relatively simple, and there are a large number of professional terms, and the text is relatively fuzzy in semantic expression, as shown in Figure 3 The trigger words of the two events are the same, there is no complex relationship between the events, and there is no overlap between the arguments, so it is difficult to effectively distinguish events by using trigger words or the relationship between events. At the same time, the existing method lacks special coping strategies and solutions when dealing with data containing professional terms, which limits the event extraction effect in the communication field. Therefore, how to better handle professional terms and enhance the recognition ability of the model to the event boundary is a big problem in the event extraction task in the communication field.

[0054] As shown in Figure 1 The embodiment provides a communication failure event argument extraction method based on a large model event focus and term enhancement, which includes the following steps:

[0055] A large model fine-tuning module is used to fine-tune a large language model, and event information and professional term information in the communication failure document are extracted by the fine-tuned large language model;

[0056] An event boundary-aware encoding module is used to embed the extracted event information into a prompt template, and the template is encoded by a non-autoregressive encoding method to obtain encoded text features;

[0057] A semantic-enhanced professional term recognition module is used to perform hierarchical interaction between the extracted professional term information and the encoded text features through a double-layer attention mechanism, and outputs a term recognition result;

[0058] An attention-focused argument extraction module is used to extract event arguments based on the encoded text features and the term recognition result by using an attention focus mechanism.

[0059] The large model fine-tuning module is as follows:

[0060] If the large language model obtained by original training is directly used in a specific field, the effect is often not satisfactory. Fine-tuning can enhance the adaptability of the large model in a specific field and improve the task performance. In the fine-tuning method, the LoRA (Low-Rank Adaptation, LoRA) method is used to fine-tune the large model. In the design of the prompt template, the Chain-of-Thought (CoT) method is used, so that the large model can extract events and corresponding professional terms in the events at the same time. In the selection of the large model, the Qwen2.5-instruct-14B model is used.

[0061] The fine-tuning task is mainly divided into event extraction and professional term extraction. In order to complete these two tasks, it is necessary to set up corresponding training labels before fine-tuning, so it is necessary to expand the existing data set to a certain extent. In the event extraction task, the maximum index and minimum index of all arguments in each event are located as the boundaries of the event, and the string of the event is extracted as the training label of the event extraction task. In the professional term extraction task, the argument content and role type in the data set are analyzed and counted, and finally the different argument roles in the original data set are abstracted into three major types of professional terms, which are used as the label of the professional term extraction task. As shown in FIG. 8, wherein "Owner", "Subject" and the like are the role types of different arguments in the original data set, and "Fault name", "Fault state" and the like are the abstracted professional term types. Figure 4

[0062] After the training labels required for fine-tuning are completed, a prompt template based on the thinking chain method is designed to fine-tune the large model. The prompt template can extract events while extracting professional terms contained in the events. The prompt template is "You are an expert in the field of event extraction. The following are some examples of event extraction. You need to extract the events contained in the text according to the following event extraction process. First, determine whether there is one or more events in the event text (3G focus selection threshold configuration is unreasonable, leading to an increase in L2U coverage redirection)? Then, according to the event type and trigger word pair I provide: ['Event type: configuration class fault class - trigger word: configuration', 'Event type: software and hardware exception class - trigger word: redirection'] extract the corresponding event?

[0063] ​Then you also need to extract the fault entity in each event from the extracted event text. There are three types of fault entities: fault name, fault state, and fault indicator rate. The fault entities extracted from the above two events are: [[{'word': '3G side selection threshold', 'type': 'fault name'}, {'word': 'unreasonable', 'type': 'fault state'}], [{'fault name': 'L2U coverage', 'fault state': 'increased'}]]\nBased on the above example of extracting events, extract the events in the following text: Due to the direct push of low version license items to high version, the license application fails. The event type and trigger word pair involved in the text are: ['event type: hardware and software exception class - trigger word: application', 'event type: adjust machine class - trigger word: push'].

[0064] The prompt template combines event type and trigger word to complete the fine-tuning task of event extraction. After successfully extracting the event related to the trigger word, the prompt template further introduces a prompt sentence to continue extracting professional terms from the event. Based on the above fine-tuning process, the events and professional terms contained in each training sample can be finally obtained.

[0065] After completing the training labels required for fine-tuning, a prompt template is constructed using the method of thinking chain to fine-tune the large model. The prompt template first enables the large model to extract the events contained in the text, and then extracts the professional terms contained in the events. Based on the above fine-tuning process, the events and professional terms contained in each training sample can be finally obtained.

[0066] The encoding module for event boundary perception is as follows:

[0067] Unlike existing methods that focus on using inter-event correlations to solve event co-occurrence problems, the LMETE model focuses on ensuring the integrity and independence of individual events by combining the characteristics of communication domain datasets. An innovative event boundary perception encoding module is designed for this purpose. Specifically, this method first embeds the event information generated by the fine-tuned large language model into the prompt template, and then encodes it using a non-autoregressive encoder. The core of this module is the innovative integration of non-autoregressive encoding and event embedding technology to guide the model to deeply analyze the special event boundary markers "" in the template. <s>". In the encoding phase, the model makes use of <s>"marker, which precisely locates the target event of the current argument to be extracted by a self-attention mechanism. Subsequently, in the cross-attention computation in the decoding stage, the model then takes the <s>” as the core anchor point, the attention of the features (Owner, Setting, etc. argument roles) used to locate the argument boundary is dynamically focused on the target event pointed to by the marker (see Figure 1 CrossAttention section for details). This design ultimately achieves efficient aggregation of specific event information, significantly enhancing the model's ability to perceive event boundaries.

[0068] In constructing the prompt template focusing on specific events, first, the event generated by the large model is accurately positioned in the text, and the specific label <s>< / s> ” is used for explicit annotation (annotation effect as shown in Figure 1 in the prompt template module of the large model guiding event focus). On this basis, the argument entity type is embedded in the prompt template, and corresponding prompt templates are designed for the seven categories of fault events in the data set (as shown in Table 1), and the professional term type is embedded in it. Finally, the event text after explicit annotation of the event and the template text corresponding to the event type are spliced to form the final input encoder prompt template content, as shown in equation (1):

[0069] X=[<s>E1< / s>E2,...P;J;W] (1)

[0070] Where E i is the i-th event contained in the text, P is the remaining string information in the event text, W is the template of the specific event type in Table 1. J is the event type prompt prefix constructed for different training samples, mainly composed of the event type to which the data belongs and the trigger word.

[0071] Table 1

[0072]

[0073] With the continuous development of pre-training models and large models, the current event argument extraction method mostly uses pre-training models as text encoders. Considering that professional terms in communication data are usually in English form, this embodiment selects a multilingual version of the (Multilingual T5, MT5) model as the encoder to enable the encoder to fully understand the semantics of English and Chinese words. LMETE uses MT5 to encode the constructed prompt template text in a non-autoregressive manner, strengthening the interaction between event text and templates. As shown in equation (2):

[0074] C,T=CompositeMT5Encoder(X),

[0075]

[0076] CompositeMT5Encoder is a composite encoder, X is input into the MT5 encoder and the non-autoregressive decoder at the same time, and finally the encoded text features C, T are obtained, where C is the encoded feature of the original text, and T is the encoded feature of the template. N and M are the lengths of the event text and the template, respectively, d t is the encoding dimension of the vector.

[0077] The semantic enhanced professional term prediction module is as follows:

[0078] In view of the large amount of professional terms in the communication fault data set, the LMETE designs a semantic enhanced professional term module by integrating a large model. The prediction of the term is essentially a sequence labeling task, in order to deeply understand the semantics of the argument, the entity type to which the argument belongs is integrated into the process of event argument extraction. According to this method, an embedding matrix of professional term type is constructed according to the event type on the communication field data, as shown in formula (3):

[0079]

[0080] Where L is the number of event types in the data set, and 3 represents the token label type, which can take B, I and O labels.

[0081] After creating the BIO Embedding layer, the embedding matrix B and the features [C, T] are interactively fused by using the multi-head attention mechanism, so that the term label features in the embedding matrix B are aligned with the features related to the term in [C, T], as shown in formula (4)-formula (7):

[0082]

[0083] Where B' is the feature vector after the B extension shape, so as to make it have the same batch size, and W q , W k , W v are linear transformation weight matrices, and CMA(·) function represents a multi-head cross attention network layer.

[0084] After the first layer of attention mechanism, the BIO Embedding layer has completed the preliminary perception and alignment of the term in the [C, T] vector. In order to enhance the perception ability of the professional term, a double-layer attention fusion mechanism is designed, and the second layer of attention mechanism enhances the semantic representation of the professional term in the fused feature vector by fusing the professional term generated by the large model, as shown in formula (8)-formula (11):

[0085]

[0086]

[0087] where Z is the professional term generated by the large model, and the feature after shape expansion. Finally, the fused feature of the double-layer attention mechanism is spliced with [C, T] to obtain the final semantic vector for professional term prediction, as shown in equations (12)-(14):

[0088]

[0089] O = Linear2(O), O e R Nx3xL (14)

[0090] Finally, a conditional random field (CRF) is used for sequence labeling, and a log-likelihood function is used as the loss function of the professional term prediction module, and the loss is denoted as Loss ner .

[0091] The argument extraction module with attention focus is as follows:

[0092] Existing argument extraction methods usually embed argument roles (such as Owner, Setting, etc.) into prompt templates, and decode argument types and ranges by fusing their features. Although this strategy is effective for distinguishing different event types of arguments, it is difficult to accurately focus attention on the target event when processing text containing multiple events, resulting in a decline in extraction performance. To address this challenge, the present embodiment proposes an argument extraction scheme with attention focus. By deeply coupling the argument extraction module with the event boundary perception encoding module, it is ensured that the semantic information of the argument role can be concentrated on a specific target event during the encoding stage, thereby achieving attention focus on the argument extraction process. As Figure 1 shown, in extracting the "RRU working mode setting is improper, which leads to the failure of cell establishment when the cell CPRI compression is enhanced compression," event, the argument roles (such as Owner, Setting, etc.) focus attention on "RRU working mode setting is improper" during the encoding stage, enhancing the model's ability to distinguish different event boundaries, which is mainly due to the deep integration of the event boundary perception module and the argument extraction module.

[0093] In addition, to further improve the accuracy of argument selection, the present embodiment introduces a dual-view optimization mechanism, which uses domain knowledge (professional terms) to assist in judgment to obtain more accurate decoding results.

[0094] The encoded event type template feature, i.e., the T vector in [C, T], contains all argument roles in the specific event type, as shown in Table 1. The features of different roles of T are taken out, and span selectors S rolei and E rolei , where and are the start and end positions of the i-th argument role in the text, as shown in formula (15) and formula (16):

[0095]

[0096] EmbeddingT(role i ) is the encoding feature of the i-th argument role in T, W start1 and W end1 are the weight matrices learned by all argument roles.

[0097] The encoding feature C of the text to be extracted and the last feature O of the professional term prediction module are linearly transformed to obtain the final hidden feature F, as shown in formula (17) and formula (18):

[0098]

[0099] Based on the above features, the probability distribution of the starting position of each argument role is calculated, as shown in formula (19) and formula (20):

[0100]

[0101] After obtaining the probability distribution of different argument roles, in order to use the semantic understanding mechanism of argument role and professional term dual perspective to make joint decision on argument selection, the value of the probability distribution of the starting position of the professional term needs to be obtained . As the method of obtaining the probability distribution of the argument role, the obtained probability distribution of the professional term is multiplied by the probability distribution of the argument role, as the final probability distribution of the argument extraction, as shown in formula (21) and formula (22):

[0102]

[0103] The final argument extraction loss adopts cross-entropy loss function The loss of the probability distribution and the standard distribution , as shown in formula (23) and formula (24):

[0104]

[0105] L bce (x,y)=-(ylogx+(1-y)log(1-x)) (24)

[0106] The argument extraction loss and the professional term prediction loss are taken as the joint loss, and the loss of each step of training is back propagated and the parameters are updated, as shown in formula (25).

[0107] L losss = L eae + L ner (25)

[0108] To verify the feasibility of the method, the following experiments were conducted in this embodiment:

[0109] Data set selection:

[0110] To verify the actual effectiveness of the LMETE method in the field of communication failure, experiments were conducted on the publicly disclosed Huawei communication failure data set and the telecom communication failure data set respectively. Table 2 shows the division of the two types of data sets on the training set and the test set. The data sets are introduced as follows:

[0111] Huawei communication failure data set: This data set comes from "CCKS-2021" Huawei Process-oriented Event Extraction in Communication Field, which is a publicly disclosed event extraction data set in the field of communication failure. It is defined by Huawei experts according to various process knowledge in the field of communication, and it is of great significance in analyzing key errors in the operation and maintenance process.

[0112] Telecom communication failure data set: This data set comes from real failure data generated by internal operation and maintenance of telecom. On this basis, this embodiment constructs a high-quality data set specially for event argument extraction task.

[0113] Table 2

[0114]

[0115] The data parameter settings and evaluation indexes are as follows:

[0116] The maximum length of the encoder input of the method in this embodiment is set to 220 words. If the sentence exceeds the maximum length, the first 220 words are taken. If it is insufficient, it is filled with <pad>Supplement. Since the experiment involves fine-tuning of large models and training of event argument extraction models, it involves two sets of experimental environments.

[0117] When fine-tuning the large model, the batch size during fine-tuning training is set to 4, and a total of 5 Epochs are trained. The fine-tuning experiment is performed on a server with a system version of Ubuntu 20.04, and the specific experimental environment information is shown in Table 3.

[0118] Table 3

[0119]

[0120] During the experiment of training the event argument extraction model, a total of 50 Epochs are trained. The test and validation set effects and the model are saved after each Epoch is trained, and the best decoding result is selected as the final result of the experiment.

[0121] The event argument extraction experiment is performed on a server with a system version of Ubuntu 20.04, and the specific experimental environment information is shown in Table 4.

[0122] Table 4

[0123]

[0124] The evaluation indicators of the model effect adopt two common evaluation indicators of event argument extraction. For each indicator, there are three indicators of precision (Precision), recall (Recall), and F1 score to measure the performance of the model:

[0125] (1) Argument Identification (AI): If the predicted argument span matches the gold span, the argument is correctly identified.

[0126] (2) Argument Classification (AC): If the predicted argument span matches the gold span and the predicted argument role is also correct, the argument is correctly identified.

[0127] The experimental results are analyzed as follows:

[0128] In the experimental part, this embodiment respectively compares and analyzes the event argument extraction methods in the past five years on the telecom and Huawei communication fault data sets. Specifically, in order to highlight the superiority of the proposed method, experiments are conducted on classification-based and generative-based methods. The model performance on the Huawei communication fault public data set and the telecom communication fault data set is shown in Tables 5 and 6, and the bold values in the table represent the optimal value in the same column in the experimental data.

[0129] The comparison methods are as follows:

[0130] EEQA method proposes a method to convert event extraction task into question and answer task, by designing question templates, event trigger detection and event argument extraction are converted into asking questions to input sentences and returning the results of event extraction as answers.

[0131] BartGen method is a document-level event argument extraction model based on conditional generation. This method first encodes the event template without filling the argument parameters and the context text, and finally fills the argument parameters in the template through conditional generation. Finally, the event argument analysis and extraction are realized.

[0132] XGear method designs a language-independent argument parameter filling event template, and proposes a method based on multilingual pre-training model. By converting the event argument extraction task into a language generation task, it realizes the extraction of zero-shot cross-language event argument parameters.

[0133] PAIE method adds argument roles to prompt templates, captures the interaction between parameters through multi-role prompts, and optimizes through bipartite graph matching loss to achieve optimal argument span allocation. It realizes better learning and generalization of the model under limited training data.

[0134] DEGREE converts the event argument extraction task into a conditional generation problem. Given the context text and the designed prompt template, the model summarizes the events in the text into natural sentences that conform to the predefined pattern, and finally extracts the argument parameters in the sentence. By utilizing label semantics and weak supervision signals, the learning efficiency of the model under low resource conditions is improved.

[0135] TagPrime provides a unified processing framework for event extraction, relation extraction and other tasks. In the event argument extraction task, the method adds event types and triggers to the input text, and uses the self-attention mechanism of the pre-training model to make the model better capture the relationship structure under specific conditions, and through parallel coding, it speeds up the process of event argument reasoning.

[0136] TabEAE proposes an event argument extraction method based on attention mechanism to strengthen the connection between different events. Unlike previous studies that only consider single events, the model extracts all argument parameters of each event during training. This definition considers all co-occurring events in the same context, allowing the model to better capture the semantic boundaries of events and improve the model's ability to handle multiple events.

[0137] DEEIA method is similar to TabEAE, which is an event argument extraction method considering the co-occurrence of events. First, the method strengthens the intra-event and inter-event connection through attention mechanism, and then strengthens the extraction ability of specific events through specific event information aggregation module.

[0138] Scented method effectively solves the problems of weak semantic correlation, damaged semantic integrity and one-sided semantic understanding in existing methods by customizing the event argument entity type information in event argument extraction by adding argument role, argument entity type and their corresponding relationship in the prompt template.

[0139] Table 5

[0140]

[0141] Table 6

[0142]

[0143]

[0144] In the above method, the performance of the EEQA model is relatively poor. This phenomenon is mainly due to the design mechanism of extracting arguments by questioning, which puts higher requirements on the semantic understanding ability of the model. However, the data in the communication failure field has the characteristics of strong professionalism and fuzzy semantic expression, and its understanding difficulty is significantly increased compared with other field data sets, which directly restricts the effect of event argument extraction. In contrast, the LMETE model proposed in this paper achieves more excellent results in recall rate and F1 score. Specifically, compared with the optimal model BartGen in the generative method, the recall rates of LMETE in AI and AC are increased by 9.76 and 8.64 percentage points respectively, and the F1 values are increased by 11.51 and 10.30 percentage points respectively on the Huawei public communication failure data set. On the Telecom data set, the recall rates of LMETE in AI and AC are increased by 4.90 and 4.97 percentage points respectively, and the F1 values are increased by 7.64 and 7.58 percentage points respectively. Compared with the optimal model Scented in the classification learning method, the recall rates of LMETE in AI and AC are increased by 6.64 and 6.29 respectively, and the F1 values are increased by 2.92 and 2.71 percentage points respectively on the Huawei public data set. On the Telecom data set, the recall rates of LMETE in AI and AC are increased by 4.88 and 5.56 percentage points respectively, and the F1 values are increased by 2.01 and 2.80 percentage points respectively. This shows that the LMETE method has achieved good performance in event argument extraction. This significant performance improvement is mainly due to two key designs of the model: first, the event focusing module effectively enhances the boundary distinguishing ability of the model when processing multiple event samples; second, the professional term prediction module improves the accuracy of the model in professional term identification by accurately predicting professional terms in events. These innovative designs together contribute to the significant improvement of the evaluation indicators.

[0145] It can also be found that the methods TabEAE and DEEIA, which specially handle event co-occurrence problems, perform poorly on the communication failure data set. This is mainly because these two methods use attention mechanisms to explore the correlation between events and extract multiple events at the same time, thereby enhancing the model's ability to distinguish event boundaries. However, for the phenomenon of relative independence between events on the communication failure data, this method does not perform well in handling event co-occurrence problems. In view of this phenomenon, the present embodiment focuses on analysis and discussion in the subsequent effectiveness analysis section.

[0146] The present embodiment also conducts ablation experiments as follows:

[0147] The core architecture of the LMETE method consists of two modules: an event boundary-aware encoding module and a semantic-enhanced specialized term identification module. To verify the effectiveness of each module and its contribution to the overall performance, the modules were evaluated and analyzed in ablation experiments. The results of the ablation experiments on the Huawei public communication fault dataset and the telecommunications communication fault dataset are shown in Tables 7 and 8. The ablation experiments are described as follows:

[0148] (1) w / o-Terminology-LLM-Fusion: the model effect after deleting the specialized term module based on LMETE.

[0149] (2) w / o-Event-Aware-Encoder: the model effect after deleting the event boundary-aware encoding module based on LMETE. (To illustrate the effectiveness of the event boundary-aware encoder, this embodiment systematically eliminates the model's ability to perceive specific event boundaries by replacing the specially designed event-focused prompt template in the encoding module with a general prompt template).

[0150] Table 7

[0151]

[0152] Table 8

[0153]

[0154] Through Experiment 1, it can be found that after deleting the specialized term prediction module, the LMETE on the Huawei and Telecom datasets decreased by 0.66%, 0.6% and 0.48%, 0.59% in AI-F1 and AC-F1 evaluation indicators, respectively. This indicates that the double-attention mechanism specialized term module of the proposed fusion large model plays a positive role in improving event argument extraction.

[0155] After removing the event-focused prompt template module in the event boundary-aware encoding module in Experiment 2, the experimental results show that the AI-F1 and AC-F1 indicators on the Huawei dataset and the Telecom dataset decreased by 1.57%, 1.4% and 1.35%, 1.32%, respectively. This significant performance decline verifies the importance of the event boundary-aware module. Therefore, under the action of the event boundary-aware encoding module, the model can accurately focus on the text content of the event to be extracted, enhance the ability to distinguish between multiple event boundaries, and thus improve the performance of event argument extraction.

[0156] The effectiveness analysis is as follows:

[0157] To validate the LMETE method's effectiveness in addressing communication failures, we compared and analyzed its effectiveness with recent methods. (Because telecommunications datasets are confidential, Huawei's publicly available dataset was used in the effectiveness analysis.)

[0158] Analysis of the effectiveness of boundary differentiation ability:

[0159] To verify the effectiveness of the LMETE model in distinguishing event boundaries when dealing with event co-occurrence, this study first conducted a statistical analysis of the distribution of texts with multiple events in the dataset. The specific results are shown in Table 9. Subsequently, an in-depth analysis of the multi-event samples was conducted from the following two dimensions:

[0160] (1) Analysis of argument extraction effect;

[0161] The purpose is to examine the excellent argument extraction effect of the LMET model in multi-event samples, thereby proving that the model can effectively distinguish the boundaries of different events. Specifically, this example compares the LMETE method with the TagPrime and Scented methods, which have better effects, and the TabEAE and DEEIA methods used to solve the event co-occurrence problem. The results are as follows: Figure 5 、 Figure 6 shown.

[0162] Table 9

[0163]

[0164] Table 9 counts the number of multi-event data samples in the dataset. It can be found that the number of multi-event samples in the Huawei communication dataset accounts for 77.43%, which shows that the processing of the multi-event co-occurrence problem in the communication dataset is of great significance and value. Figure 5 This figure shows the distribution of incorrect argument predictions for different methods across multiple event samples in the test set. Sum-Count represents the total number of incorrect argument predictions, while Bound-Count represents the number of incorrect argument predictions in ER-Count that fall within another event. This means that the predicted argument failed to grasp the event boundary and crossed it. Su-Bo-Percent refers to the ratio of Bo-Count to Sum-Count. Figure 6 Correspondingly, the event argument extraction effects of different methods on multi-event samples in the test set were statistically analyzed.

[0165] Through Figure 5 The results in Table 6 show that the number of element prediction errors of TabEAE and DEEIA methods on multi-event samples is much larger than the number of LMETE. The error ratio of Su-Bo-Percent of TabEAE and DEEIA methods is increased by 35.31% and 42.78% compared with LMETE, respectively. Compared with the error rates of TagPrime and Scented baseline methods, the error rates are also significantly higher, which shows that although TabEAE and DEEIA methods have made targeted solutions to the event co-occurrence problem and have achieved good results on related field datasets, there are still deficiencies in the field of communication, especially in the event boundary distinguishing ability. This is because they mainly solve the event co-occurrence problem by associating the text information and the prompt template of different events in the text through attention mechanism, and highlight the boundary information of the event by using the correlation between events. This way of associating events through attention has a high requirement for the design of the prompt template, and a high-quality prompt template even needs to consider the combination of different synonyms in design. At the same time, due to the relative independence of events in the field of communication and the existence of professional terms, over-emphasizing the correlation between events will blur the boundaries between events. The method of the embodiment mainly relies on a large model to locate a specific event to improve the event extraction effect while maintaining the independence of events, and has a lower requirement for the construction of the prompt template, and has the characteristics of portability and reliability. Through the comparison of the model on the multi-event sample, it can be found that the LMETE method has a strong advantage in distinguishing the boundaries of events, solves the problem of event co-occurrence, and achieves good performance in the event element extraction task.

[0166] (2) Analysis of the internal coding mechanism of the model;

[0167] In order to illustrate that the boundary perception coding module can focus attention on a specific event in the coding stage, and improve the element extraction effect while strengthening the event boundary distinguishing ability, the embodiment performs visual mechanism analysis on the internal boundary perception module. The embodiment selects the current optimal baseline model Scented for comparison, and visualizes and compares the cross-attention distribution generated by the two models in the element extraction stage through the heat map technology, as shown in FIG. 6. Figure 7

[0168] ​This figure clearly reveals the precise attention focusing ability of the LMET model on different events, while the attention of the Scented model is more divergent. When processing this text, the Scented model produces a high degree of similarity in the cross-attention distribution generated by different argument roles (Owner, Index, State) when extracting different events, indicating that it fails to effectively distinguish the context information of two independent events, resulting in confusion of the attention focus. This is mainly due to the fact that the two events contained in this complex text both belong to the "index deterioration class" type, and the trigger words are also the same, so Scented cannot accurately locate the boundaries of the event by relying on event type and trigger word, making it difficult to accurately focus on the event to be extracted. In sharp contrast, the LMET model exhibits precise boundary perception ability. When extracting the "ERAB success rate decline" event, the argument role can adaptively focus attention on "ERAB", "establish", "success rate", "decline" and other related word tokens. Similarly, when processing the "RRC decline" event, the attention is accurately transferred and focused on "RRC" and "decline" word tokens. This result powerfully demonstrates that the boundary perception module of LMET can dynamically allocate and isolate attention for different events, significantly improving the performance of the model in argument extraction in complex scenarios. This finding provides theoretical explanation and empirical support for the superior performance of LMET in processing multi-event samples from the attention mechanism level.

[0169] The final experimental results show that LMET, through its unique event boundary perception mechanism, can more accurately capture and distinguish the boundary features of different events, significantly improving the model's ability to process complex multi-event samples.

[0170] Professional term recognition ability effectiveness analysis:

[0171] To verify the accuracy of the LMETE method in professional term recognition, the TagPrime, Scented methods with better effects in the comparative experiment and LMETE were selected for analysis in professional term recognition. The effect of professional term recognition is shown in Table 10.

[0172] Table 10

[0173]

[0174] The experimental results in Table 10 show that the LMETE method improves the EI-P, EI-R, EC-P, and EC-R metrics by 5.25% and 5.27%, respectively, compared to the TagPrime and Scented methods, and by 1.90% and 2.01%. The LMETE method achieves optimal results across all performance metrics for terminology prediction, validating the effectiveness of the terminology module in improving terminology recognition. LMETE, with its terminology recognition module, enhances the model's ability to recognize communications terminology, and the other modules contribute to improved overall model performance.

[0175] The case analysis is as follows:

[0176] In order to more intuitively see the ability of the LMETE method to distinguish event boundaries and the accuracy of professional term prediction, LMETE is compared with the more advanced TagPrime and Scented methods to conduct experiments and analyses on typical multi-event data examples, such as Figure 8 shown.

[0177] Figure 8 The following figure shows the event extraction results of the Scented, TagPrime, and LMETE methods for text containing multiple events. The yellow and blue panels represent the argument extraction results for events in different colored fonts within the text. Analysis reveals that the Scented method fails to extract the argument "configuration information in the RRUCHAIN ​​chain number" for the event described in blue font. While the TagPrime method successfully extracts this argument, it incompletely extracts the argument "LTE-FDD Lampsite BBU CPRI line rate" for the yellow event. This is primarily due to the fact that these two methods fail to focus on the model's ability to distinguish events and extract specialized terminology in the multi-event sample, resulting in incorrect or even non-extraction of some arguments. In contrast, LMETE, with its focus on specific event prompt templates and specialized terminology prediction module, successfully and accurately extracts event arguments, improving event argument extraction performance.

[0178] The above analysis demonstrates that LMETE achieves excellent performance in processing datasets containing multiple events and complex communication terminology. Specifically, LMETE, through its event boundary-aware encoding module, is able to fully understand the boundary information between multiple event texts, thereby eliminating redundant information from other events during the extraction of a specific event and accurately focusing on the semantic information of that specific event. Secondly, after obtaining semantic expressions focused on a specific event, LMETE enhances its ability to extract terminology from that event through its terminology recognition module, more accurately extracting arguments containing terminology from the event, achieving excellent results in overall event argument extraction performance.

[0179] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.< / pad> < / s> < / s> < / s>

Claims

1. A communication fault event argument extraction method based on large model event focusing and term enhancement, characterized by: The following steps are involved: Fine-tune the large language model and use it to extract event information and professional terminology from communication failure documents. Construct an event boundary-aware encoding module, embed the extracted event information into the prompt template, and encode the template through non-autoregressive encoding to obtain the encoded text features; Build a semantically enhanced terminology recognition module that uses a two-layer attention mechanism to hierarchically interact the extracted terminology information with the encoded text features and outputs the terminology recognition results. Based on the encoded text features and term recognition results, an attention focusing mechanism is used to extract event arguments.

2. The method according to claim 1, characterized in that Fine-tuning the large language model includes: Select the Qwen2.5-instruct-14B model as the large language model; Use LoRA method to fine-tune the large model; Construct a prompt template based on thought chain to enable the large model to extract event information and professional terminology information at the same time.

3. The method according to claim 2, characterized in that The extracted event information and professional terminology information includes: In the event extraction task, the maximum index and minimum index of all argument ranges in the event are located as event boundaries, and the event strings are extracted as training labels; In the professional terminology extraction task, the argument role is abstracted into the professional terminology type and used as the label of the professional terminology extraction task.

4. The method according to claim 1, wherein The encoded text features include: Embed the event information generated by the large model into the prompt template and use labels to explicitly mark the event boundaries; Design corresponding prompt templates for different event types and embed professional terminology types into the templates; A non-autoregressive encoder is used to encode the concatenated prompt template to obtain the encoded text features.

5. The method according to claim 1, wherein The semantically enhanced professional terminology recognition module includes: Construct an embedding matrix of professional term types; A multi-head attention mechanism is used to interactively fuse the embedding matrix with the encoded text features; The professional terms extracted by the large model are further integrated through the double-layer attention mechanism to enhance the semantic representation of the terms.

6. The method according to claim 5, characterized in that The professional terminology recognition module uses conditional random fields for sequence labeling and uses the log-likelihood function as the loss function.

7. The method according to claim 1, characterized in that The method of extracting event arguments by using the attention focusing mechanism includes: Extract the features of different argument roles from the encoded event type template features; Construct a span selector for each argument role to identify its starting and ending positions in the text; The encoding features of the event text to be extracted are linearly transformed with the features of the professional terminology prediction module to calculate the probability distribution of the starting position of the argument role.

8. The method according to claim 7, characterized in that The method of extracting event arguments by using the attention focusing mechanism also includes: Get the probability distribution of the starting position of professional terms; The obtained probability distribution of professional terms is multiplied by the probability distribution of argument roles to obtain the final probability distribution of argument extraction.

9. The method according to claim 1, characterized in that A joint loss function is adopted in the training process, which includes argument extraction loss and term prediction loss, and the model parameters are updated through back propagation.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

Images