Metalearning-based few-sample Chinese network security event detection method

Abstract

The invention discloses a few-sample Chinese network security event detection method based on meta-learning. According to the method, firstly, a dynamic dimension transformation mechanism is designed, and geometric feature extraction of a multi-word composite trigger word is optimized; secondly, through a multi-stage knowledge distillation technology, in combination with intra-class features and inter-class features, the loss is compared, and the event detection precision and generalization ability are improved; and finally, a meta-learning training framework in few-sample learning is adopted, and efficient learning is carried out under a limited data condition. By combining meta-learning and knowledge distillation, the problems of complex trigger words, scarce data and the like in Chinese network security event detection are effectively solved, and the event detection performance in a few-sample scene is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and network security, in particular to event detection technology, and specifically to a few-sample Chinese network security event detection method based on meta-learning. The method is applicable to zero-sample and few-sample event detection scenarios, especially to the challenges of complex trigger words and scarce sample data in the Chinese environment. By combining meta-learning with knowledge distillation technology, the adaptability and generalization ability of event detection are improved. Background Art

[0002] In the field of network security, event detection (ED) is an important task that is responsible for real-time identification and classification of various security threats, including unauthorized intrusions, malware propagation, data leakage, and system vulnerability exploitation. Existing event detection methods are mainly divided into the following categories: (1) rule-based event detection methods; (2) machine learning-based event detection methods; (3) supervised learning-based event detection methods; (4) few-sample-based event detection methods.

[0003] Although existing methods have made great progress in the task of event detection, these methods are mostly concentrated in the English field. Cybersecurity event detection in the Chinese environment still faces new challenges due to the complexity of the Chinese language. The main challenges are: (a) Unlike English, the Chinese language lacks clear word separators, which means that the model must rely on word segmentation technology to identify vocabulary units when processing Chinese text. However, due to the precision limitations of word segmentation tools, inaccurate word segmentation will seriously affect event detection, especially when dealing with nested trigger word structures. For example, in English, the trigger word "ransomware" can be directly identified as a word, but in Chinese, "ransomware" may be incorrectly split into "ransom" and "software", destroying the integrity of the event semantics. This situation is particularly prominent under the complex technical vocabulary in the field of cybersecurity, making it difficult for the model to correctly classify events; (b) The trigger words in Chinese event detection often present a complex nested structure in the sentence and have strong context dependence. The model must not only accurately identify the trigger words, but also understand the relationship between the trigger words and other words. If the model cannot effectively capture the long-distance dependencies in the sentence, it is easy to cause incorrect event classification. For example, the trigger word "attack" may refer to different types of events in different contexts, such as "DDoS attack" or "supply chain attack", which places higher requirements on the model; (c) The existing Chinese cybersecurity event datasets are not only small in number, but also lack the quality and coverage of data annotation. Compared with the large-scale, high-quality annotated datasets in the English field, the data resources in the Chinese environment are significantly lacking, especially when it comes to emerging attack types, making it difficult to provide sufficient learning samples for the model. In addition, the sub-type annotations of many Chinese cybersecurity events are not detailed enough, resulting in the model's inability to accurately classify complex events and poor generalization ability.

[0004] The above problems have significantly hindered the development and application of Chinese network security incident detection methods, revealing the shortcomings of existing technologies in dealing with the unique language complexity of Chinese, data scarcity, and responding to new threats. Therefore, developing new and adaptable Chinese network security incident detection methods, especially solutions that can solve problems such as word segmentation errors, nested trigger words, data scarcity, and new threat identification, has become an important direction of current research.

[0005] In recent years, deep learning technology has achieved great success in the field of natural language processing. Inspired by this, some researchers have applied deep learning technology to the Chinese event detection task and have made certain research progress. Related research papers include: [1] "Event detection: Gate diversity and syntactic importance scores for graph convolution neural networks", [2] "Learning target-dependent sentence representations for Chinese event detection", [3] "Zero-and Few-Shot Event Detection via Prompt-Based Meta Learning", [4] "Span-based few-shot event detection via aligning external knowledge". The focus of the above Chinese event detection methods is how to capture the overall semantic and syntactic features of the trigger word boundary and improve the accuracy of trigger word recognition. Moreover, the processing targets all rely on a large and accurately annotated dataset, which is not suitable for Chinese network security event detection with a small number of samples.

[0006] That is to say, when applied to small sample data, existing technologies still face the following challenges: (1) Insufficient capture of long-distance dependencies and complex trigger word boundaries. In the case of small samples, it is difficult for the model to accurately capture the long-distance dependencies and complex trigger word boundaries in the sentence, especially the nested trigger word structure in Chinese, which leads to a decrease in the accuracy of trigger word recognition and event classification; (2) Data scarcity leads to poor model generalization ability. Chinese event detection methods rely on a large amount of labeled data, but in the field of network security, it is costly and difficult to obtain high-quality data sets, resulting in the prevalence of small sample scenarios. Models are difficult to train effectively when data is insufficient and cannot respond to new or unseen events. Summary of the invention

[0007] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and to provide a method for detecting Chinese network security incidents with a small number of samples based on meta-learning. The present invention constructs a meta-learning framework, combines multi-level knowledge distillation and a dynamic dimensionality transformation mechanism, and can effectively identify and classify Chinese network security incidents, especially complex nested trigger words and new types of threats, in the case of a small number of samples or zero samples. This method significantly improves the generalization ability and adaptability of the model, overcomes the reliance of existing methods on a large amount of annotated data, and solves the problem of low accuracy in trigger word recognition and event classification in the Chinese environment.

[0008] Technical solution: A method for detecting Chinese network security incidents with a small number of samples based on meta-learning of the present invention comprises the following steps:

[0009] Step S1: Input text sample sentence: and a sample prompt: "This text describes a <mask>Event. "Among them, Represents an input sequence c containing n characters i and trigger words and y i represents the label of the sample, N represents N event types, and K represents K labeled samples. Usually K is set to {5,10}. The input sequence c in i and trigger words and As well as the prompt word, use the pre-trained model RoBERTa as the text encoder to obtain the word unit embedding and position embedding results At the same time, in order to further enrich the semantic expression ability of sentences and capture more fine-grained language features, we use Stanford Core NLP to obtain the part-of-speech feature-level embedding of sentences. Finally, we get the i-th token representation h of the last hidden layer of the RoBERTa encoder i = Dropout([z1,z2,…,z n ]).

[0010] Step S2: Construct a meta-learning training framework, φ S As the base learner, θ is the initial meta-learner, and meta-training is used to optimize the parameter θ. Specifically, in each training step, given a task and learning rate λ, update the parameters in the inner loop In the formula, λ is the internal learning rate, For the task The model is trained using the Adma optimizer.

[0011] Step S3: The last hidden layer representation h obtained in step S1 is i , execute the trigger word weighting method to generate the candidate trigger word set S trigger =[S1,S2,…,S n ], and their boundary set S span =[(s1,e1),(s2,e2)…(s n ,e n )],(s i ,e i ) represents the starting and ending positions of the sentence where the i-th candidate trigger word is located.

[0012] Step S4: based on the candidate trigger word set S in step S3 trigger =[S1,S2,…,S n ], and each element S i = (start, end), calculate the geometric features of complex trigger words, center position and width c i and w i .

[0013] Step S5: Based on the geometric embedding features obtained in step S4, the interaction relationship between candidate trigger words is calculated to obtain the span representation of the trigger word with enhanced interaction relationship. In order to better estimate the importance of the predicted trigger words, the candidate trigger word weighting strategy is also used to calculate the attention weight of each candidate word, considering more the influence from other spans:

[0014]

[0015] Among them, α ij Candidate trigger word S i and the trigger word S j The attention weight between them. Using the weighted sum of span representations, we can get the trigger word span representation with enhanced interaction relationship:

[0016]

[0017] Among them, S i It is the representation of candidate trigger words, which contains the specific features of the trigger words, such as word boundaries, positions, etc.

[0018] Step S6: Based on the trigger word span representation span enhanced by the interaction relationship obtained in step S5, the classification probability and event category loss of the event category are calculated.

[0019] Step S7: After the data preprocessing phase in step S1, construct positive sample pairs and negative sample pairs according to the category label information. Randomly select a class with the same label {X i |i∈N} as positive samples. For each positive sample category, we generate a certain number of negative sample sets {X j |j∈N-1}. And calculate the supervised contrast loss

[0020] Step S8: To solve the catastrophic forgetting problem in the meta-learning method, calculate the multi-level knowledge distillation loss, including the feature distillation loss and predict distillation losses Optimizing the meta-learning base learner φ S

[0021] Furthermore, the detailed steps of step S3 are:

[0022] Step S3.1: Calculate the probability p of each word being classified as a trigger word through the fully connected layer i In addition, the method also combines its contextual information through the contextual attention mechanism of the RoBERTa model and uses the trigger word probability p i and attention score attention Calculate the last layer h of the encoder i The trigger word weight α in i .

[0023] α i =softmax(p i score attention (h i ))

[0024] Step S3.2: Select weight α i The top n words are used as possible trigger words. The index of these ranked high-weight words is marked as a set I = {i1, i2, ..., i n To further expand the scope of these trigger words, we applied a sliding window mechanism with a window width of 3 to generate a candidate trigger word set S trigger =[S1,S2,…,S n ]:

[0025]

[0026] Finally, the records form their boundary set S span =[(s1,e1),(s2,e2)…(s n ,e n )],(s i ,e i ) represents the starting and ending positions of the sentence where the i-th candidate trigger word is located.

[0027] 4. Further, the detailed steps of step S4 are:

[0028] Step S4.1: Perform dynamic dimensional transformation between all candidate trigger words to generate two set geometric association matrices:

[0029] Given a set of trigger words S generated in the trigger word recognition task trigger =[S1,S2,…,S n ], each element S i =(start,end), the width of each trigger word is w i = end-strart, and the center position of the trigger word c i :

[0030]

[0031] Step S4.2: Combine the two obtained association matrices, namely the width information and the center position information, to obtain the geometric position embedding information of the trigger word. ij and center ij After that, these two pieces of information will be transformed into the dimension matrix DM of the element. ij , generated by the following concatenation operation:

[0032]

[0033] here, It is represented as a concatenation operation, which combines the width information and the center position information. Next, we transform DM ij Embed into a high-dimensional space for subsequent processing. This embedding process is done using sine and cosine functions. For each dimension d of the embedding vector, the operation is as follows:

[0034]

[0035] Where d represents the dimension of the final embedding vector. In this way, we can incorporate the scale transformation information DM ij Encoded into a high-dimensional embedding vector, this embedding vector takes into account both the width and center position information of the element, and can effectively capture position-related features through the periodicity of sine and cosine. Finally, the embedding features of these trigger words are generated through the nonlinear GELU function:

[0036]

[0037] Furthermore, the event classification loss is calculated The detailed steps of step S6 are:

[0038] Step S6.1: By using soft hint technology, the traditional classification problem is transformed into a masked language model (MLM) task. <mask>Placeholder prompt: "This text describes a <mask>event.". <mask>The placeholder is used to mark the location of the prediction. The resulting prediction is an output e that represents the probability distribution over each possible word, where each word represents a possible answer.

[0039] Step S6.2, map the predicted candidate trigger word probability distribution to a specific event type. Different from the conventional predefined translator, this method builds a learnable translator based on the MLM prediction result e and the triggered span representation span. The process of calculating the event category prediction is defined as follows:

[0040]

[0041] based on We define the event category classification loss as

[0042]

[0043] Among them, p x,i ∈{0,1} indicates whether the true label of x is i. Represents the event category prediction result calculated by the classifier.

[0044] Furthermore, the detailed steps of step S7 are:

[0045] In step S7.1, in the comparison method, we first use two-layer MLP to reduce the dimension of sample features to obtain z i , to enhance the model's ability to characterize features:

[0046]

[0047] in, Represents sample X i The last hidden layer state of . Next, Norm() is a normalization operation to obtain the final feature representation z i .

[0048] Step S7.2: The feature representation of each positive sample is recorded as {z1,z2,…,z N }. Similarly, we calculate the features of negative samples In order to measure the similarity between two inputs, we designed a distance function S() as, which calculates the distance between sample pair representations through cosine similarity, where τ is a temperature hyperparameter used to adjust the sensitivity of similarity. The supervised contrast loss function can be defined as:

[0049]

[0050] The temperature hyperparameter τ in the formula also plays an important role in InfoNCEloss. InfoNCEloss can automatically discover hard negative samples, and τ can adjust the degree of attention to hard negatives. The smaller the τ value, the greater the attention to the separation of hard negative samples.

[0051] Finally, the detailed steps of step S8 are:

[0052] Step S8.1: In each training step, given the task and learning rate λ, update the parameter φ during the inner loop S :

[0053]

[0054] In the formula, λ is the internal learning rate, For the task The model is trained using the Adma optimizer. When i=1, Depend on and When i≠1, Additional Content That is, the multi-level knowledge distillation loss. This loss will be introduced in detail below.

[0055] Step S8.2: Calculate feature distillation. In this step, it is expected that the current base learner The encoder can extract the previous base learner Similar features enable it to learn useful information from the previous base learner. S ,make and Respectively and Extracted X S Secondly, in order to force When extracting features, we do not favor new event types and keep consistency with the previous ones. We use the method of calculating the cosine distance between two feature vectors. The feature distillation loss function is defined as follows:

[0056]

[0057] Among them, |X S | represents the number of samples used for feature distillation. By calculating the dot product, i.e., cosine similarity, of the feature vectors between the two base learners, if the dot product is closer to 1, it means that the two feature vectors are more similar. As an indicator to measure the difference between two features, it promotes the consistency of feature representation by minimizing the difference.

[0058] Step S8.3: In this step, in order to make the current base learner The predictions of the MLP layer on the previous dataset are similar to The MLP layers of have similar probability distributions. To this end, KL divergence is introduced as a tool to measure the similarities and differences between two probability distributions. Through this design, While learning new knowledge, it is possible to retain the knowledge that has been mastered to the greatest extent, thus avoiding the common catastrophic forgetting problem. Specifically, a loss function is defined to measure the difference between the predicted probability distribution of the current base learner and the predicted probability distribution of the previous base learner. The designed prediction loss function is as follows:

[0059]

[0060] In the above formula, N is the total number of samples in the data set. is the i-th sample, and Respectively and For samples The goal is to adjust the current base learner to minimize the loss. In this way, the prediction of the current base learner will statistically tend to the prediction of the previous base learner, achieving effective knowledge transfer.

[0061] Step S8.3, use L1 loss to measure the absolute distance between the estimated depth and the actual depth to supervise the depth estimation in the depth perception iteration stage.

[0062]

[0063] Among them, k and n k Represent the number of stages and the number of iterations in each stage, represents L1 loss;

[0064] Step S8.4: The entire multi-level knowledge distillation loss consists of the above two losses:

[0065]

[0066] Where α and β are adjustment coefficients.

[0067] The final loss function is the weighted sum of event category classification loss, supervised contrast loss and multi-level knowledge distillation loss, denoted as:

[0068]

[0069] Then optimize the model to achieve event detection.

[0070]

[0071] In the parameter update process of the meta-learner θ, the inner loop learner φ is considered S Here θ is the parameter of the meta-learner, μ is the learning rate of the meta-learner, is the loss function, D S is the support set, D Q is a queryset.

[0072] Beneficial effects: The present invention adopts a method combining a meta-learning framework with knowledge distillation technology to improve the performance of few-sample Chinese network security incident detection. The geometric feature extraction of multi-word compound trigger words is optimized through a dynamic dimensionality transformation mechanism. Then, multi-level knowledge distillation technology is used to combine the intra-class feature and inter-class feature contrast loss to enhance the generalization ability of the model. At the same time, a few-sample learning training framework is designed to enable efficient learning under limited data conditions to improve the accuracy and robustness of event detection. Finally, by integrating the geometric embedding features of trigger words and the trigger word span representation enhanced by the interaction relationship, efficient detection of complex Chinese network security incidents is achieved.

[0073] Compared with the prior art, the advantages of the present invention are as follows:

[0074] (1) Through the dynamic dimension transformation mechanism and meta-learning knowledge distillation technology, the present invention effectively integrates the complex trigger words and context information in Chinese network security incident detection, providing a solid foundation for the accurate identification and classification of Chinese network security incidents.

[0075] (2) The present invention adopts a few-sample learning training framework and a hybrid loss strategy to iteratively optimize the event detection model and calculate highly accurate classification results, thereby significantly improving the performance of network security event detection.

[0076] (3) The present invention can not only solve the challenges faced by existing network security incident detection methods in processing complex Chinese language structures, but also overcome the difficulties in model training under data-scarce environments, thus providing an important foundation for the development and application of Chinese network security incident detection technology.

[0077] (4) The present invention can provide efficient and accurate event detection support for network security monitoring, threat intelligence analysis, risk assessment and response strategy formulation, enhance network security protection capabilities, and make important contributions to building a more secure network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0079] Figure 2 is the event detection result in the embodiment.

[0080] Figure 3 It is a flow chart of the model algorithm of the present invention.

[0081] Figure 4 This is a comparison chart of event detection effects between the present invention and the existing method. DETAILED DESCRIPTION

[0082] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0083] Existing network security incident detection methods often ignore the complex language structure and contextual information of Chinese when processing text data, which leads to insufficient detection accuracy, especially in the face of few or zero-sample Chinese network security incident detection scenarios. To address this challenge, this paper proposes a few-sample Chinese network security incident detection method based on meta-learning, which effectively integrates the semantic information and contextual features in the text through the meta-learning framework, thereby significantly improving the accuracy and generalization ability of event detection.

[0084] like Figure 1 As shown, the main idea of ​​the present invention is: first, RoBERTa and Stanford Core NLP are used to perform in-depth preprocessing and feature extraction on text samples; then, by constructing a meta-learning training framework, the learning process of the model is further refined; further, by calculating candidate trigger words and their geometric features, the model's ability to capture key information in the text is enhanced; then, by analyzing the interactive relationship between trigger words, a richer relationship-enhanced trigger word representation is obtained; then, by calculating the event classification loss, the classification accuracy of the model is optimized; in addition, the comparison loss of intra-class features and inter-class features is introduced to further enhance the model's ability to distinguish; finally, through the calculation of multi-level knowledge distillation loss, the meta-learning parameters are effectively updated, and the overall performance of the model is improved. The present invention cleverly adopts a few-sample processing strategy based on meta-learning and knowledge distillation, efficiently integrates complex semantic information and trigger word features in the text, calculates accurate event classification through fine feature extraction and deep learning framework, and then realizes efficient detection of Chinese network security events, and finally improves the generalization ability and detection accuracy of the model.

[0085] The few-sample Chinese network security incident detection method based on meta-learning in this embodiment includes the following steps:

[0086] Step S1: Input text sample sentence: and the sample prompt: "This text describes a <mask>Event. "Among them, Represents an input sequence c containing n characters i and trigger words and y i represents the label of the sample, N represents N event types, and K represents K labeled samples. Usually K is set to {5,10}. The input sequence c in i and trigger words and As well as the prompt word, use the pre-trained model RoBERTa as the text encoder to obtain the word unit embedding and position embedding results At the same time, in order to further enrich the semantic expression ability of sentences and capture more fine-grained language features, we use Stanford CoreNLP to obtain the part-of-speech feature-level embedding of sentences. Finally, we get the i-th token representation h of the last hidden layer of the RoBERTa encoder i = Dropout([z1,z2,…,z n ]).

[0087] Step S2: Construct a meta-learning training framework, φ S As the base learner, θ is the initial meta-learner, and meta-training is used to optimize the parameter θ. Specifically, in each training step, given a task and learning rate λ, update the parameters in the inner loop In the formula, λ is the internal learning rate, For the task The model is trained using the Adma optimizer.

[0088] Step S3: The last hidden layer representation h obtained in step S1 is i , execute the trigger word weighting method to generate the candidate trigger word set S trigger =[S1,S2,…,S n ], and their boundary set S span =[(s1,e1),(s2,e2)…(s n ,e n )],(s i ,e i ) represents the starting and ending positions of the sentence where the i-th candidate trigger word is located. The detailed steps of step S3 are:

[0089] Step S3.1: Calculate the probability p of each word being classified as a trigger word through the fully connected layer i In addition, the method also combines its contextual information through the contextual attention mechanism of the RoBERTa model and uses the trigger word probability p i and attention score attention Calculate the last layer h of the encoder i The trigger word weight α in i .

[0090] α i =softmax(p i score attention (h i ))

[0091] Step S3.2: Select weight α i The top n words are used as possible trigger words. The index of these ranked high-weight words is marked as a set I = {i1, i2, ..., i n To further expand the scope of these trigger words, we applied a sliding window mechanism with a window width of 3 to generate a candidate trigger word set S trigger =[S1,S2,…,S n ]:

[0092]

[0093] Assume that the word at position 5 is identified as a candidate trigger word based on the weight sorting. After applying the sliding window, the following candidate trigger word boundary ranges are obtained: For the word at position 5, the sliding window range is (3,7), so the specific boundary range is expanded to [(3,5), (4,6), (5,7)]. This method helps us identify compound trigger words that may be missed in text analysis and record the boundary set S that forms them. span =[(s1,e1),(s2,e2)…(s n ,e n )],(s i ,e i ) represents the starting and ending positions of the sentence where the i-th candidate trigger word is located.

[0094] Step S4: based on the candidate trigger word set S in step S3 trigger =[S1,S2,…,S n ], and each element S i = (start, end), calculate the geometric features of complex trigger words, center position and width c i and w i Furthermore, the detailed steps of step S4 are:

[0095] Step S4.1: Perform dynamic dimensional transformation between all candidate trigger words to generate two set geometric association matrices:

[0096] Given a set of trigger words S generated in the trigger word recognition task trigger =[S1,S2,…,S n ], each element S i =(start,end), the width of each trigger word is w i =end-start, and the center position of the trigger word c i :

[0097]

[0098] Step S4.2: Combine the two obtained association matrices, namely the width information and the center position information, to obtain the geometric position embedding information of the trigger word. ij and center ij After that, these two pieces of information will be transformed into the dimension matrix DM of the element. ij , generated by the following concatenation operation:

[0099]

[0100] here, It is represented as a concatenation operation, which combines the width information and the center position information. Next, we transform DM ij Embed into a high-dimensional space for subsequent processing. This embedding process is done using sine and cosine functions. For each dimension d of the embedding vector, the operation is as follows:

[0101]

[0102] Where d represents the dimension of the final embedding vector. In this way, we can incorporate the scale transformation information DM ij Encoded into a high-dimensional embedding vector, this embedding vector takes into account both the width and center position information of the element, and can effectively capture position-related features through the periodicity of sine and cosine. Finally, the embedding features of these trigger words are generated through the nonlinear GELU function:

[0103]

[0104] Step S5: Based on the geometric embedding features obtained in step S4, the interaction relationship between candidate trigger words is calculated to obtain the span representation of the trigger word with enhanced interaction relationship. In order to better estimate the importance of the predicted trigger words, the candidate trigger word weighting strategy is also used to calculate the attention weight of each candidate word, considering more the influence from other spans:

[0105]

[0106] Among them, α ij Candidate trigger words i and the trigger word S j The attention weight between them. Using the weighted sum of span representations, we can get the trigger word span representation with enhanced interaction relationship:

[0107]

[0108] Among them, S i It is the representation of candidate trigger words, which contains the specific features of the trigger words, such as word boundaries, positions, etc.

[0109] Step S6: Based on the trigger word span representation span enhanced by the interaction relationship obtained in step S5, the classification probability and event category loss of the event category are calculated. The detailed steps of step S6 are:

[0110] Step S6.1: By using soft hint technology, the traditional classification problem is transformed into a masked language model (MLM) task. <mask>Placeholder prompt: "This text describes a <mask>event.". <mask>Placeholders are used to mark the location of the prediction. The generated prediction result is an output e representing the probability distribution of each possible word, where each word represents a possible answer. Step S6.2, map the predicted probability distribution of candidate trigger words to specific event types. Unlike conventional predefined translators, this method builds a learnable translator based on the MLM prediction result e and the triggered span representation span. The process of calculating event category prediction is defined as follows:

[0111]

[0112] here, is the event category predicted by the model. MLP() stands for Multi-Layer Perceptron, which is a neural network model used to learn complex function mappings from input data. Span refers to the enhanced trigger word span representation in the text, which may be part of a phrase or sentence. Symbol Indicates feature fusion, combining the representation of span and the representation of e as the input of MLP(). The arg max() function is used to select the event category with the highest probability from the output of MLP() as the prediction result. We define the event category classification loss as

[0113]

[0114] Among them, p x,i ∈{0,1} indicates whether the true label of x is i. If the true label of x is i, then p x,i =1, otherwise p x,i =0, represents the event category prediction result calculated by the classifier, |D S |Dataset D S The total number of samples in .

[0115] Step S7: After the data preprocessing phase in step S1, construct positive sample pairs and negative sample pairs according to the category label information. Randomly select a class with the same label {X i |i∈N} as positive samples. For each positive sample category, we generate a certain number of negative sample sets {X j |j∈N-1}. And calculate the supervised contrast loss The detailed steps of step S7 are:

[0116] In step S7.1, in the comparison method, we first use two-layer MLP to reduce the dimension of sample features to obtain z i , to enhance the model's ability to characterize features:

[0117]

[0118] in, Represents sample X i The last hidden layer state of . Next, Norm() is a normalization operation to obtain the final feature representation z i .

[0119] Step S7.2: The feature representation of each positive sample is recorded as {z1,z2,…,z N }. Similarly, we calculate the features of negative samples In order to measure the similarity between two inputs, we designed a distance function S() as, which calculates the distance between sample pair representations through cosine similarity, where τ is a temperature hyperparameter used to adjust the sensitivity of similarity. The supervised contrast loss function can be defined as:

[0120]

[0121] In the formula, Used to calculate z for two data points i and The similarity between them is measured using the cosine similarity. N represents the total number of data points in the dataset. j≠i means that z is excluded from the summation. i itself, only other data points are considered. The temperature hyperparameter τ also plays an important role in InfoNCEloss. InfoNCEloss can automatically discover hard negative samples, and τ can adjust the degree of attention to hard negatives. The smaller the τ value, the greater the attention to the separation of hard negative samples.

[0122] Step S8: To solve the catastrophic forgetting problem in the meta-learning method, calculate the multi-level knowledge distillation loss, including the feature distillation loss and predict distillation losses Optimizing the meta-learning base learner φ S For the task Using the previous base learner The knowledge of The detailed steps of step S8 are as follows:

[0123] Step S8.1: In each training step, given the task and learning rates λ, φ S As the base learner, θ is the initial meta-learner, and meta-training is used to optimize the parameters θ. In the inner loop, the current parameters of the meta-learner θ are used as the starting point. Perform several iterations of training to update the parameter φ S :Use the updated φ S In the query set In the outer loop, through multiple different tasks The collected training data updates the meta-learner θ.

[0124]

[0125] In the formula, λ is the internal learning rate, For the task The model is trained using the Adma optimizer. When i=1, Depend on and When i≠1, Additional Content That is, the multi-level knowledge distillation loss. This loss will be introduced in detail below.

[0126] Step S8.2: Calculate feature distillation. In this step, it is expected that the current base learner The encoder can extract the previous base learner Similar features enable it to learn useful information from the previous base learner. S ,make and Respectively and Extracted X S Secondly, in order to force When extracting features, we do not favor new event types and keep consistency with the previous ones. We use the method of calculating the cosine distance between two feature vectors. The feature distillation loss function is defined as follows:

[0127]

[0128] Among them, |X S | represents the number of samples used for feature distillation. By calculating the dot product, i.e., cosine similarity, of the feature vectors between the two base learners, if the dot product is closer to 1, it means that the two feature vectors are more similar. As an indicator to measure the difference between two features, it promotes the consistency of feature representation by minimizing the difference.

[0129] Step S8.3: In this step, in order to make the current base learner The predictions of the MLP layer on the previous dataset are similar to The MLP layers of have similar probability distributions. To this end, KL divergence is introduced as a tool to measure the similarities and differences between two probability distributions. Through this design, While learning new knowledge, it is possible to retain the knowledge that has been mastered to the greatest extent, thus avoiding the common catastrophic forgetting problem. Specifically, a loss function is defined to measure the difference between the predicted probability distribution of the current base learner and the predicted probability distribution of the previous base learner. The designed prediction loss function is as follows:

[0130]

[0131] In the above formula, N is the total number of samples in the data set. is the i-th sample, and Respectively and For samples The goal is to adjust the current base learner to minimize the loss. In this way, the prediction of the current base learner will statistically tend to the prediction of the previous base learner, achieving effective knowledge transfer.

[0132] Step S8.3, use L1 loss to measure the absolute distance between the estimated depth and the actual depth to supervise the depth estimation in the depth perception iteration stage.

[0133]

[0134] Among them, k and n k Represent the number of stages and the number of iterations in each stage, represents L1 loss;

[0135] Step S8.4: The entire multi-level knowledge distillation loss consists of the above two losses:

[0136]

[0137] Where α and β are adjustment coefficients.

[0138] The final loss function is the weighted sum of event category classification loss, supervised contrast loss and multi-level knowledge distillation loss, denoted as:

[0139]

[0140] Then optimize the model to achieve event detection.

[0141]

[0142] In the parameter update process of the meta-learner θ, the inner loop learner φ is considered S Here θ is the parameter of the meta-learner, μ is the learning rate of the meta-learner, is the loss function, D S is the support set, D Q is a queryset.

[0143] Embodiment 1:

[0144] The sample of text data input in this embodiment is as follows Figure 2 As shown, the present invention can identify trigger words that can represent events from the specified text and classify them into the correct event type, for example, S1: Cyber ​​criminals used phishing attack emails to steal a large amount of personal information. The "phishing attack" in S1 is identified as the trigger word of the event "phishing attack". In S2: "Anonymous" announced that its affiliated organization Black Free World leaked 28gb of data stolen from the Russian Central Bank.", the "leak" in S2 is identified as the trigger word of the event "data leakage".

[0145] Figure 3 The model training process of the present invention is shown. It can be seen that the present invention first uses RoBERTa and StanfordCore NLP to perform in-depth preprocessing and feature extraction on text samples; then, by constructing a meta-learning training framework, the learning process of the model is further refined; further, by calculating candidate trigger words and their geometric features, the model's ability to capture key information in the text is enhanced; then, by analyzing the interactive relationship between trigger words, a richer relationship-enhanced trigger word representation is obtained; then, by calculating the event classification loss, the classification accuracy of the model is optimized; in addition, the comparison loss of intra-class features and inter-class features is introduced to further improve the model's ability to distinguish; finally, through the calculation of multi-level knowledge distillation loss, the meta-learning parameters are effectively updated, and the overall improvement of model performance is achieved. The present invention cleverly adopts a few-sample processing strategy based on meta-learning and knowledge distillation, efficiently integrates complex semantic information and trigger word features in the text, and calculates accurate event classification through fine feature extraction and deep learning framework, thereby achieving efficient detection of Chinese network security events, and ultimately improving the generalization ability and detection accuracy of the model.

[0146] Figure 4 The event detection capabilities of the present invention are compared with those of the models in references [3] and [4]. Under the same few-sample situation, the present invention successfully identifies trigger words and achieves correct classification of events. MetaEvent does not fully recognize trigger words, but can also correctly classify event types. Thanks to its span extractor, SpanFSED is also suitable for solving the problem of trigger word matching without natural separators in the Chinese environment. However, SpanFSED performs poorly in dealing with the problem of multiple trigger word noise, so it will also identify "theft" as a trigger word as a "data leakage" event. Therefore, the present invention successfully achieves the event detection task under the Chinese few-sample situation.< / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask>

Claims

1. A Chinese network security incident detection method based on meta-learning, characterized in that Through dynamic dimension transformation and meta-learning knowledge distillation technology, efficient detection and generalization of complex Chinese trigger words and new event types can be achieved in the case of few or zero samples, including the following steps: Step S1: Input text sample sentence: and the sample prompt: "This text describes a <mask>Event. "Among them, Represents an input sequence c containing n characters i and trigger words and y i represents the label of the sample, N represents N event types, and K represents K labeled samples. Usually K is set to {5,10}. The input sequence c in i and trigger words and As well as the prompt word, use the pre-trained model RoBERTa as the text encoder to obtain the word unit embedding and position embedding results At the same time, in order to further enrich the semantic expression ability of sentences and capture more fine-grained language features, we use Stanford Core NLP to obtain the part-of-speech feature-level embedding of sentences. Finally, we get the i-th token representation h of the last hidden layer of the RoBERTa encoder i = Dropout([z1,z2,…,z n ]).< / mask> Step S2: Construct a meta-learning training framework, φ S As the base learner, θ is the initial meta-learner, and meta-training is used to optimize the parameter θ. Specifically, in each training step, given a task and learning rate λ, update the parameters in the inner loop In the formula, λ is the internal learning rate, For the task The model is trained using the Adma optimizer. Step S3: The last hidden layer representation h obtained in step S1 is i , execute the trigger word weighting method to generate the candidate trigger word set S trigger =[S1,S2,…,S n ], and their boundary set S span =[(s1,e1),(s2,e2)…(s n ,e n )],(s i ,e i ) represents the starting and ending positions of the sentence where the i-th candidate trigger word is located. Step S4: based on the candidate trigger word set S in step S3 trigger =[S1,S2,…,S n ], and each element S i = (start, end), calculate the geometric features of complex trigger words, center position and width c i and w i . Step S5: Based on the geometric embedding features obtained in step S4, the interaction relationship between candidate trigger words is calculated to obtain the span representation of the trigger word with enhanced interaction relationship. In order to better estimate the importance of the predicted trigger words, the candidate trigger word weighting strategy is also used to calculate the attention weight of each candidate word, considering more the influence from other spans: Among them, α ij Candidate trigger word S i and the trigger word S j The attention weight between them. Using the weighted sum of span representations, we can get the trigger word span representation with enhanced interaction relationship: Among them, S i It is the representation of candidate trigger words, which contains the specific features of the trigger words, such as word boundaries, positions, etc. Step S6: Based on the trigger word span representation span enhanced by the interaction relationship obtained in step S5, the classification probability and event category loss of the event category are calculated. Step S7: After the data preprocessing phase in step S1, construct positive sample pairs and negative sample pairs according to the category label information. Randomly select a class with the same label {X i |i∈N} as positive samples. For each positive sample category, we generate a certain number of negative sample sets {X j |j∈N-1}. And calculate the supervised contrast loss Step S8: To solve the catastrophic forgetting problem in the meta-learning method, calculate the multi-level knowledge distillation loss, including the feature distillation loss and predict distillation losses Optimizing the meta-learning base learner φ S .

2. The method for detecting Chinese network security incidents with a small number of samples based on meta-learning according to claim 1 is characterized in that: In step S2, a meta-learning training framework is constructed to learn common features from multiple network security event detection tasks and use the training method of support set and query set to enable the model to quickly adapt to new tasks under few or zero sample conditions. The framework includes the design of meta-loss function and task-level parameter optimization to ensure that the model can maintain good generalization ability when processing new event types.

3. According to the meta-learning-based few-sample Chinese network security incident detection method of claim 1, a trigger word weighting method is executed to generate a candidate trigger word set, and the detailed steps of step S3 are: Step S3.1: Calculate the probability p of each word being classified as a trigger word through the fully connected layer i In addition, the method also combines its contextual information through the contextual attention mechanism of the RoBERTa model and uses the trigger word probability p i and attention score attention Calculate the last layer h of the encoder i The trigger word weight α in i . α i =softmax(p i ·score attention (h i ) Step S3.2: Select weight α i The top n words are used as possible trigger words. The index of these ranked high-weight words is marked as a set I = {i1, i2, ..., i n To further expand the scope of these trigger words, we applied a sliding window mechanism with a window width of 3 to generate a candidate trigger word set S trigger =[S1,S2,…,S n ]: And record the boundary set S that forms them span =[(s1,e1),(s2,e2)…(s n ,e n )],(s i ,e i ) represents the starting and ending positions of the sentence where the i-th candidate trigger word is located.

4. The method for detecting Chinese network security incidents with a small number of samples based on meta-learning according to claim 1 is characterized in that: Calculate the geometric features of the compound trigger words, that is, the geometric relationship between the relative position offset and scale size change between the boundaries of different trigger words. The detailed steps of step S4 are: Step S4.1: Perform dynamic dimensional transformation between all candidate trigger words to generate two set geometric association matrices: Given a set of trigger words S generated in the trigger word recognition task trigger =[S1,S2,…,S n ], each element S i =(start,end), the width of each trigger word is w i =end-start, and the center position of the trigger word c i : Step S4.2: Combine the two obtained association matrices, namely the width information and the center position information, to obtain the geometric position embedding information of the trigger word. ij and center ij After that, these two pieces of information will be transformed into the dimension matrix DM of the element. ij , generated by the following concatenation operation: here, It is represented as a concatenation operation, which combines the width information and the center position information. Next, we transform DM ij Embed into a high-dimensional space for subsequent processing. This embedding process is done using sine and cosine functions. For each dimension d of the embedding vector, the operation is as follows: Where d represents the dimension of the final embedding vector. In this way, we can transfer the scale transformation information DM ij Encoded into a high-dimensional embedding vector, this embedding vector takes into account both the width and center position information of the element, and can effectively capture position-related features through the periodicity of sine and cosine. Finally, the embedding features of these trigger words are generated through the nonlinear GELU function:

5. The method for detecting Chinese network security incidents with a small number of samples based on meta-learning according to claim 1 is characterized in that: In step S5, the interaction relationship between candidate trigger words is calculated based on the geometric embedding features.

6. According to the meta-learning-based few-sample Chinese network security incident detection method of claim 1, calculate the event classification loss The detailed steps of step S6 are: Step S6.1: By using soft hint technology, the traditional classification problem is transformed into a masked language model (MLM) task. <mask>Placeholder prompt: "This text describes a <mask>event.". <mask> The placeholder is used to mark the location of the prediction. The resulting prediction is an output e that represents the probability distribution over each possible word, where each word represents a possible answer.< / mask> < / mask> < / mask> Step S6.2, map the predicted candidate trigger word probability distribution to a specific event type. Different from the conventional predefined translator, this method builds a learnable translator based on the MLM prediction result e and the triggered span representation span. The process of calculating the event category prediction is defined as follows: based on We define the event category classification loss as in, p x,i ∈{0,1} indicates whether the true label of x is i. Represents the event category prediction result calculated by the classifier.

7. According to the meta-learning-based few-sample Chinese network security incident detection method of claim 1, calculate the supervised contrast loss The detailed steps of step S7 are: In step S7.1, in the comparison method, we first use two-layer MLP to reduce the dimension of sample features to obtain z i , to enhance the model's ability to characterize features: in, Represents sample X i The last hidden layer state of . Next, Norm() is a normalization operation to obtain the final feature representation z i . Step S7.2: The feature representation of each positive sample is recorded as {z1,z2,…,z N }. Similarly, we calculate the features of negative samples In order to measure the similarity between two inputs, we designed a distance function S() as, which calculates the distance between sample pair representations through cosine similarity, where τ is a temperature hyperparameter used to adjust the sensitivity of similarity. The supervised contrast loss function can be defined as: The temperature hyperparameter τ in the formula also plays an important role in InfoNCEloss. InfoNCEloss can automatically discover hard negative samples, and τ can adjust the degree of attention to hard negatives. The smaller the τ value, the greater the attention to the separation of hard negative samples.

8. The method for detecting Chinese network security incidents with a small number of samples based on meta-learning according to claim 1 is characterized in that For the task Using the previous base learner The knowledge of The detailed steps of step S8 are as follows: Step S8.1: In each training step, given the task and learning rate λ, update the parameter φ during the inner loop S : In the formula, λ is the internal learning rate, For the task The model is trained using the Adma optimizer. When i=1, Depend on and When i≠1, Additional Content That is, the multi-level knowledge distillation loss. This loss will be introduced in detail below. Step S8.2: Calculate feature distillation. In this step, it is expected that the current base learner The encoder can extract the previous base learner Similar features enable it to learn useful information from the previous base learner. S ,make and Respectively and Extracted X S Secondly, in order to force When extracting features, we do not favor new event types and keep consistency with the previous ones. We use the method of calculating the cosine distance between two feature vectors. The feature distillation loss function is defined as follows: Among them, |X S | represents the number of samples used for feature distillation. By calculating the dot product, i.e., cosine similarity, of the feature vectors between the two base learners, if the dot product is closer to 1, it means that the two feature vectors are more similar. As an indicator to measure the difference between two features, it promotes the consistency of feature representation by minimizing the difference. Step S8.3: In this step, in order to make the current base learner The predictions of the MLP layer on the previous dataset are similar to The MLP layers of have similar probability distributions. To this end, KL divergence is introduced as a tool to measure the similarities and differences between two probability distributions. Through this design, While learning new knowledge, it is possible to retain the knowledge that has been mastered to the greatest extent, thus avoiding the common catastrophic forgetting problem. Specifically, a loss function is defined to measure the difference between the predicted probability distribution of the current base learner and the predicted probability distribution of the previous base learner. The designed prediction loss function is as follows: In the above formula, N is the total number of samples in the data set. is the i-th sample, and Respectively and For samples The goal is to adjust the current base learner to minimize the loss. In this way, the prediction of the current base learner will statistically tend to the prediction of the previous base learner, achieving effective knowledge transfer. Step S8.3, use L1 loss to measure the absolute distance between the estimated depth and the actual depth to supervise the depth estimation in the depth perception iteration stage. Among them, k and n k Respectively represent the number of stages and the number of iterations in each stage, represents L1 loss; Step S8.4: The entire multi-level knowledge distillation loss consists of the above two losses: Where α and β are adjustment coefficients. The final loss function is the weighted sum of event category classification loss, supervised contrast loss and multi-level knowledge distillation loss, denoted as: Then optimize the model to achieve event detection. In the parameter update process of the meta-learner θ, the inner loop learner φ is considered S Here θ is the parameter of the meta-learner, μ is the learning rate of the meta-learner, is the loss function, D S is the support set, D Q is a queryset.

Images