A network security entity identification method and system based on multi-layer channel attention

Abstract

The application discloses a network security entity identification method and system based on a multi-layer channel attention, and belongs to the technical field of network security entity identification. The specific method comprises the following steps: performing data preprocessing on a real network security entity text data set; constructing a named entity identification model based on a multi-layer channel attention; inputting the preprocessed network security entity text data set into the named entity identification model for model training, so as to obtain a trained named entity identification model; inputting an actual network security entity text data set into the trained named entity identification model, and outputting a label sequence; identifying the entity category of each word in the actual network security entity text data set according to the output label sequence, and realizing network security entity identification based on a multi-layer channel attention. The application can accurately identify various entity types in network security text, and improve the monitoring and analysis capability of network security events.

Description

Technical Field

[0001] This invention belongs to the field of network security entity recognition technology, and more specifically, relates to a network security entity recognition method and system based on multi-layer channel attention. Background Technology

[0002] Currently, the number and complexity of cybersecurity incidents are increasing daily. Security threats such as abnormal activity, malware, and intrusion behaviors in the network environment are becoming increasingly covert, leading to a continuous rise in the difficulty of threat detection and security incident analysis. Network Security Entity Recognition (NSER), as a key technology for addressing security threats, aims to automatically extract key information from large amounts of unstructured text data such as security logs, attack reports, and event alerts, identifying the types of entities involved, such as IP addresses, domain names, malware names, and attack methods. To improve the accuracy and adaptability of entity recognition, researchers have gradually explored combining BERT (Bidirectional Encoder Representations from Transformers, a pre-trained language model) with other network modules. However, existing methods often struggle to achieve lightweight design when integrating these modules, and the synergistic effect between different modules is limited, failing to achieve a balance between network complexity, recognition accuracy, and computational efficiency. Traditional named entity recognition methods may ignore the correlation between labels, leading to illogical entity label outputs.

[0003] Existing technical document 1 (CN118982025A) discloses a network security entity recognition method based on BERT-BiLSTM-CCA-CRF. Its shortcomings are that the CCA module only uses single-scale convolution operations, which fails to effectively capture multi-scale features, the collaboration between modules is limited, feature information is lost during transmission, and the label dependency of the CRF layer is not flexible enough in dynamic scenarios. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a network security entity recognition method and system based on multi-layer channel attention. This method employs a BERT model at the embedding layer to comprehensively capture contextual information, introduces a BiLSTM (Bi-directional Long Short-Term Memory) model to enhance feature representation, and incorporates an MSCA (Multi-Scale Channel Attention) mechanism to fully capture important features. Furthermore, a CRF (Conditional Random Field) model is added to the output layer to capture the dependencies between entity labels. Ultimately, this method can accurately identify multiple entity types in network security texts, thereby improving the monitoring and analysis capabilities of network security incidents.

[0005] The present invention adopts the following technical solution.

[0006] A first aspect of the present invention provides a network security entity recognition method based on multi-layer channel attention, comprising:

[0007] Preprocessing of real-world cybersecurity entity text datasets;

[0008] A named entity recognition model based on multi-layer channel attention is constructed. The preprocessed network security entity text dataset is input into the named entity recognition model for model training to obtain a trained named entity recognition model.

[0009] Input the actual network security entity text dataset into the trained named entity recognition model, and output a label sequence;

[0010] Based on the output label sequence, the entity category of each word in the actual cybersecurity entity text dataset is identified, realizing cybersecurity entity recognition based on multi-layer channel attention.

[0011] Preferably, data preprocessing is performed on the real cybersecurity entity text dataset, specifically including:

[0012] We used word segmentation tools to segment the original text in a real cybersecurity entity text dataset and labeled each word with its corresponding tag.

[0013] Remove stop words from the segmented and tagged text;

[0014] Standardize the text after removing stop words;

[0015] Enhancement of standardized text data;

[0016] By using association rule algorithms, potential dangerous events in data-enhanced text are identified, resulting in a preprocessed cybersecurity entity text dataset.

[0017] Preferably, a named entity recognition model based on multi-layer channel attention is constructed. The preprocessed network security entity text dataset is input into the named entity recognition model for training, resulting in a trained named entity recognition model. Specifically, this includes:

[0018] A named entity recognition model based on multi-layer channel attention is constructed based on BERT, BiLSTM, MSCA and CRF models.

[0019] The preprocessed network text dataset is divided into a test set and a training dataset.

[0020] The named entity recognition model is trained using 5-fold cross-validation. The dataset to be trained is divided into 5 parts and 5 rounds of cross-validation are performed. In each round of cross-validation, 1 part of the data is selected as the validation set and the remaining 4 parts of the data are used as the training set. Each round of cross-validation yields an optimal named entity recognition model.

[0021] After 5-fold cross-validation, the test set is used to verify the 5 best named entity recognition models obtained from each round of cross-validation, and the best named entity recognition model obtained from 5-fold cross-validation is obtained.

[0022] Preferably, the named entity recognition model is trained using 5-fold cross-validation. The training dataset is divided into 5 parts, and 5 rounds of cross-validation are performed. In each round of cross-validation, one part of the data is selected as the validation set, and the remaining 4 parts of the data are used as the training set. Each round of cross-validation yields an optimal named entity recognition model, specifically including:

[0023] Input a single training set into the BERT model of the named entity recognition model to generate context-sensitive embedding vectors;

[0024] The generated embedding vectors are input into the BiLSTM model of the named entity recognition model to extract bidirectional features from the text sequence, thereby obtaining bidirectional integrated features.

[0025] The bidirectional integrated features extracted by the BiLSTM model are fed into the MSCA of the named entity recognition model for multi-scale feature extraction to obtain multi-scale features.

[0026] The generated multi-scale features are input into the CRF model of the named entity recognition model, the label at each position in the multi-scale features is decoded, and the current optimal label sequence is output.

[0027] The remaining single training set in the training set is repeatedly input into the named entity recognition model. After four training iterations, four named entity recognition models with optimal label sequences of different outputs are obtained.

[0028] A learning rate decay strategy is adopted to dynamically adjust the learning rate of the named entity recognition model based on the performance of the model with the best label sequence of different outputs on the validation set. The named entity recognition model with the best performance among the four models is determined as the best named entity recognition model in this round of cross-validation.

[0029] Preferably, a single training set is input into the BERT model of the named entity recognition model to generate context-sensitive embedding vectors. The generated embedding vectors are then input into the BiLSTM model of the named entity recognition model for bidirectional feature extraction of the text sequence, specifically including:

[0030] The BERT model captures the semantic information of text through a multi-layer Transformer structure, generates context-sensitive word embeddings, and converts the word embeddings into context-sensitive embedding vectors.

[0031] The BiLSTM model processes both the forward and backward sequences of text in the embedded vectors, captures the dependencies between sentences within the text, extracts bidirectional features, and integrates the extracted bidirectional features to obtain bidirectional integrated features.

[0032] Preferably, the bidirectional integrated features extracted by the BiLSTM model are input into the MSCA of the named entity recognition model for multi-scale feature extraction to obtain multi-scale features, specifically including:

[0033] The bidirectional integrated features extracted by the BiLSTM model are fed into the MSCA of the named entity recognition model. The features are extracted and fused by multi-scale convolution with ordinary convolution and dilated convolution to obtain preliminary fused features. Then, a channel attention mechanism is used to assign weights to the preliminary fused features of different channels to obtain channel attention-weighted features. Finally, the preliminary fused features and the channel attention-weighted features are fused to obtain the final multi-scale features.

[0034] Preferably, the bidirectional features extracted by the BiLSTM model are fed into the MSCA of the named entity recognition model, and multi-scale convolution extraction using ordinary convolution and dilated convolution is combined to obtain preliminary fused features, specifically including:

[0035] MSCA sets the kernel of ordinary convolution to 1×1 and 3×3, and extracts features from bidirectional features according to different kernels of ordinary convolution.

[0036] By inserting fixed dilated intervals into the convolution kernel of dilated convolution, features with different receptive fields than those of ordinary convolution are extracted from bidirectional features based on the convolution kernel of dilated convolution.

[0037] The features extracted by the ordinary convolution and dilated convolution are standardized by batch normalization, and the features extracted by the ordinary convolution and dilated convolution are fused by addition to obtain preliminary fused features.

[0038] Preferably, a channel attention mechanism is used to assign weights to the context-dependent information features of different channels to obtain channel attention-weighted features, specifically including:

[0039] The initial fusion features are extracted by average pooling to obtain the overall mean of all features in each channel and by max pooling to extract the strongest response value in each channel.

[0040] The features obtained from average pooling and max pooling are fused and then linearly transformed through a shared convolutional layer.

[0041] The features that have undergone linear transformation are then nonlinearly mapped using the ReLU activation function.

[0042] The features after nonlinear mapping are used to generate a channel weight matrix through the Sigmoid function;

[0043] Next, perform element-wise multiplication between the channel weight matrix and the preliminary fused features, and apply the weights to the feature channels of the preliminary fused features to obtain channel attention-weighted features.

[0044] Preferably, the generated multi-scale features are input into the CRF model of the named entity recognition model, the label at each position in the multi-scale features is decoded, and the current optimal label sequence is output, specifically including:

[0045] The CRF model establishes a dependency model between labels by using contextual dependencies in multi-scale features. It then uses dynamic programming to decode and calculate the optimal label sequence for each position label in the dependency model and its preceding and following words, and outputs the current optimal label sequence.

[0046] A second aspect of this invention proposes a network security entity recognition system based on multi-layer channel attention, comprising the following steps:

[0047] The data preprocessing module is used to preprocess real-world cybersecurity entity text datasets.

[0048] Model building module: used to build a named entity recognition model based on multi-layer channel attention. The preprocessed network security entity text dataset is input into the named entity recognition model for model training to obtain a trained named entity recognition model.

[0049] The label sequence acquisition module is used to input the actual network security entity text dataset into the trained named entity recognition model and output a label sequence.

[0050] Category recognition module: used to identify the entity category of each word in the actual cybersecurity entity text dataset based on the output label sequence, realizing cybersecurity entity recognition based on multi-layer channel attention.

[0051] A third aspect of the present invention discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the computer program, when loaded onto the processor, implements the aforementioned network security entity recognition method based on multi-layer channel attention.

[0052] The fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the aforementioned network security entity recognition method based on multi-layer channel attention.

[0053] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0054] (1) In the data preprocessing of this invention, the word segmentation tool is specially customized in order to better identify terms and abbreviations unique to the field of network security and reduce word segmentation errors; stop words that have no practical significance for the task are removed to reduce the computational burden of the model and improve processing efficiency; standardized processing ensures the feasibility and consistency of batch input.

[0055] (2) The named entity recognition model based on multi-scale convolutional attention mechanism constructed in this invention has the advantages of high-precision recognition, efficient computation, and deep integration of multiple modules. Among them, the named entity recognition model based on multi-scale convolutional attention mechanism is a BERT-BiLSTM-MSCA-CRF structure. By combining contextual semantics, serialization dependency relations, multi-scale attention and label association decoding, it significantly improves the recognition accuracy of network security entities and effectively solves the shortcomings of existing methods in recognition accuracy. The model optimizes the combination design of BERT embedding, channel attention and CRF layer, so that the model has low computational resource consumption while maintaining high recognition accuracy. It is suitable for resource-constrained real-world environments, such as embedded systems or mobile devices. The multi-scale channel attention mechanism enables the model to adapt to different information features in network security texts and adapt to dynamic and ever-changing entity representation methods and context changes. It is suitable for various security threat detection scenarios. The model achieves deep integration of multiple modules in structure. It innovatively combines pre-trained language models with multi-channel attention mechanism, which enhances the accurate recognition and effective extraction of network security entities. It provides more efficient and stable technical support for threat detection and intelligence analysis in the field of network security, and can greatly improve the response speed and monitoring capabilities of network security systems.

[0056] (3) This invention proposes a multi-scale channel attention mechanism that combines ordinary convolution and dilated convolution, while dynamically adjusting channel weights to improve the model's ability to capture multi-scale features. In addition, the deep fusion between modules optimizes the integrity of feature transmission, significantly reduces computational complexity, significantly improves the accuracy and efficiency of network security entity recognition, enhances the model's ability to adapt to dynamic scenarios, and is suitable for real-time threat analysis scenarios. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the process of a high-precision network security entity recognition method based on multi-layer channel attention provided in accordance with an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the overall network structure of the named entity recognition model based on a multi-scale convolutional attention mechanism provided in accordance with an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram illustrating the specific process of inputting a preprocessed network text dataset into a named entity recognition model based on a multi-scale convolutional attention mechanism and outputting a label sequence, according to an embodiment of the present invention.

[0060] Figure 4 This is a schematic diagram of the structure of the BERT model provided according to an embodiment of the present invention;

[0061] Figure 5This is a schematic diagram of the structure of the multi-scale convolutional attention mechanism provided in accordance with an embodiment of the present invention;

[0062] Figure 6 This is a schematic diagram of a high-precision network security entity recognition system based on multi-channel attention, provided according to an embodiment of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0064] like Figure 1 As shown, Embodiment 1 of the present invention provides a high-precision network security entity recognition method based on multi-layer channel attention, comprising the following steps:

[0065] Step 1: Perform data preprocessing on the real cybersecurity entity text dataset.

[0066] In a preferred but non-limiting embodiment of the present invention, the text data in the real network security entity text dataset includes network security entities such as intrusion detection logs, malware descriptions, IP addresses, and domain names.

[0067] More preferably, to ensure that the model can accurately handle technical terms and abbreviations in the field of cybersecurity, the dataset was first cleaned and preprocessed. The data preprocessing included the following steps:

[0068] Step 1.1: Word segmentation and labeling. Use a word segmentation tool to segment the original text into words and label each word with a corresponding tag so that the named entity recognition model can perform named entity recognition in the future.

[0069] Step 1.2: Remove stop words, removing stop words that have no practical significance for the task.

[0070] Step 1.3: Standardization processing, unifying all sentences into a fixed-length format. Sentences that are too short are padded, while excessively long sentences are truncated to the length suitable for the model.

[0071] Step 1.4: Data augmentation. Use data augmentation techniques such as random deletion and replacement to enhance the diversity of data and the generalization ability of the model, thereby better adapting to the dynamic data in real cybersecurity scenarios.

[0072] Step 1.5: The association rule algorithm processes the data to obtain potential dangerous events.

[0073] After data augmentation, association rule algorithms are used to further process the data and identify potential dangerous events. Association rule algorithms help uncover patterns and events that may indicate potential dangers by analyzing the relationships between items in a cybersecurity dataset. For example, by analyzing the relationships between data items such as intrusion detection logs, malware descriptions, and domain / IP addresses, the algorithm can discover connections between specific behavioral patterns and potential attack events. The key to this step is identifying potential dangerous events through rule mining and providing the foundational data for subsequent processing.

[0074] The potential dangerous events identified by the association rule algorithm constitute the preprocessed cybersecurity entity text dataset, which will be fed as input to the subsequent Named Entity Recognition (NER) model. The task of the NER model is to further extract and classify key information from these potential dangerous events, such as the attack source IP, the target system, and the attack time. By identifying specific entities from the text data, the NER model helps to better understand the specific details of potential dangerous events and provides information support for subsequent prevention and response.

[0075] Step 2: Construct a named entity recognition model based on multi-layer channel attention.

[0076] like Figure 2 As shown, a named entity recognition model based on multi-layer channel attention is constructed based on the BERT model, BiLSTM model, MSCA, and CRF model. The structure is BERT-BiLSTM-MSCA-CRF, specifically including:

[0077] Step 2.1, the named entity recognition model uses the BERT model at the embedding layer to capture the contextual information of the input data;

[0078] The BERT model is used for word vector pre-training in the embedding layer of the named entity recognition model. By capturing contextual information and handling global dependencies in the text through a self-attention mechanism, the model can comprehensively understand the semantic relationships between words. This provides richer and more dynamic contextual semantic representations, especially for technical terms and abbreviations appearing in complex cybersecurity texts. The pre-training of the BERT model can effectively improve the processing ability for these words.

[0079] Step 2.2: The named entity recognition model uses a BiLSTM model in the feature extraction layer to extract features from the output data of the embedding layer;

[0080] In the feature extraction layer, a BiLSTM model is incorporated to learn the temporal features of the input sequence. BiLSTM can extract information from both forward and backward directions, thus providing a comprehensive model of the syntax and context of the input text.

[0081] Step 2.3: The named entity recognition model uses MSCA in the attention layer to focus on the discriminative features of different dimensions in the output data of the feature extraction layer through different channels;

[0082] Furthermore, a multi-channel attention mechanism is introduced at the attention layer. This mechanism can focus on key features of different dimensions in the input data through different channels. MSCA not only helps the model focus on the most discriminative features, but also effectively improves the extraction accuracy of important information and avoids interference from redundant data through multi-level attention processing.

[0083] Step 2.4: The named entity recognition model uses a CRF model at the output layer to capture the serialization dependency relationship between entity tags based on the output data of MSCA.

[0084] In the output layer, a CRF model is incorporated to capture the serialization dependencies between entity tags. CRF learns the dependency structure between tags, ensuring the coherence and consistency of the output entity tags. The introduction of the CRF layer optimizes these dependencies, making the final entity recognition results more accurate and consistent with actual grammatical structures. Through the collaborative work of these layers, the model of this invention can not only effectively identify multiple entity types in cybersecurity text but also maintain high performance in diverse cybersecurity scenarios. Ultimately, this model can identify various entities, including intrusion detection logs, malware descriptions, IP addresses, and domain names, providing strong technical support for cybersecurity analysis and defense.

[0085] Step 3: Input the preprocessed network text dataset into the named entity recognition model based on the multi-scale convolutional attention mechanism for model training, and obtain the trained named entity recognition model.

[0086] In a preferred but non-limiting embodiment of the invention, such as Figure 3 As shown, the preprocessed network text dataset is input into a named entity recognition model based on a multi-scale convolutional attention mechanism, and the model outputs a label sequence, specifically including:

[0087] Step 3.1: Divide the preprocessed network text dataset into a test set and a training set. Divide the training set into five parts, using one part as the validation set and the other four parts as the training set in each round of cross-validation. Specifically, to ensure the model's generalization ability, 5-fold cross-validation is used to prevent overfitting. Therefore, the dataset is divided into five parts, and a different part is selected as the validation set in each round of cross-validation, with the remaining four parts used as the training set. Each training iteration refers to training the model using different validation and training sets in different rounds of cross-validation. In this way, the model undergoes five rounds of training and validation during cross-validation, using a different validation set in each round, ensuring the stability and reliability of the training results.

[0088] Step 3.2: Train the named entity recognition model using the training set. Each round of cross-validation training includes:

[0089] Step 3.2.1: Input a single training set into the BERT embedding layer to generate context-sensitive embedding vectors.

[0090] Specifically, the embedding layer of the named entity recognition model is based on the BERT pre-trained model. The BERT model can generate semantic representations of each word in its context, possessing bidirectional contextual understanding capabilities, making it particularly suitable for complex entity relationships in the cybersecurity field. Its specific implementation steps are as follows:

[0091] Step 3.2.1.1: Semantic Information Capture. The BERT model generates context-sensitive word embeddings through a multi-layered Transformer structure, enabling it to deeply capture the semantic information of the text. The BERT model not only obtains the meaning of each word but also captures its contextual relationships, thus ensuring the model's accurate understanding of technical terms and entity relationships, such as... Figure 4 As shown.

[0092] Step 3.2.1.2: Embedding Vector Generation. Word embeddings are converted into context-sensitive embedding vectors. Each word embedding is represented as a high-dimensional vector containing rich semantic information and contextual dependencies. This embedding vector serves as input to subsequent layers, providing the model with an accurate semantic foundation.

[0093] Step 3.2.2: Input the embedding vector generated by each BERT model into the BiLSTM model to extract bidirectional features from the text sequence;

[0094] Specifically, to further enhance the understanding of sequence features, this invention incorporates a BiLSTM model after the BERT model embedding. The BiLSTM model can better model the dependencies between contextual information, providing support for subsequent entity recognition. The specific implementation steps are as follows:

[0095] Step 3.2.2.1: Bidirectional Feature Extraction. The BiLSTM model can simultaneously process the forward and backward sequences of text in the embedded vectors, capturing the contextual dependencies within sentences and extracting bidirectional features. Through bidirectional feature extraction, the BiLSTM model generates rich contextual representations, enhancing its ability to understand complex sequences.

[0096] Step 3.2.2.2: Integrate the extracted bidirectional features to obtain bidirectional integrated features. The features output by BiLSTM will contain integrated information from the context, which can more accurately represent the semantics and positional relationships of entities and improve recognition accuracy.

[0097] Step 3.2.3: Input the output features of the BiLSTM layer into the MSCA, perform attention allocation in the channel dimension, and generate the final multi-scale feature representation;

[0098] Specifically, such as Figure 5 As shown. Based on the BiLSTM output, the MSCA proposed in this application embodiment can effectively improve the model's ability to identify cybersecurity entities. The specific implementation steps of the multi-scale convolutional attention module are as follows:

[0099] Step 3.2.3.1: Multi-scale convolution extraction. MSCA combines ordinary convolution (Conv1, Conv3) and dilated convolution to achieve comprehensive capture of features at different scales. Ordinary convolution extracts local features using 1×1 and 3×3 kernels. The 1×1 kernel focuses on pointwise feature transformation, while the 3×3 kernel can capture a wider range of local patterns. These convolution operations provide basic local feature information for subsequent processing. Meanwhile, dilated convolution expands the receptive field by inserting fixed dilation intervals into the kernel, enhancing the ability to capture long-range contextual information without increasing computational complexity, thus compensating for the shortcomings of ordinary convolution in capturing global features.

[0100] The features extracted after convolution are standardized using Batch Normalization (BN) to ensure the stability of the data distribution and accelerate the model's convergence. Subsequently, the features extracted by different convolution kernels are integrated using an addition operation. This fusion process not only preserves the detailed information of local features but also introduces global contextual features, giving the final generated preliminary multi-scale integrated features stronger expressive power.

[0101] Through this multi-scale convolution design, MSCA can simultaneously focus on local details and global structure, making the model more adaptable and robust when processing complex cybersecurity text data, and providing diverse feature inputs for subsequent attention weighting.

[0102] Step 3.2.3.2: Channel attention weighting. The fused multi-scale features are then subjected to average pooling and max pooling operations. Average pooling extracts the overall mean of all features in each channel, reflecting the global distribution information within the channel; max pooling extracts the strongest response value from each channel to highlight salient features. These two pooling operations generate two global feature representations, summarizing channel information from different perspectives. Next, the features obtained from average pooling and max pooling are fused and linearly transformed through a shared convolutional layer (Conv1) to refine a more discriminative feature representation. To introduce non-linearity, the features after convolution are further non-linearly mapped using the ReLU activation function to capture the complex relationships between channels. This process makes the features more adaptive and effectively describes important task-related information. The convolutional and activated features are then used to generate a channel weight matrix using the Sigmoid function. Each value in the weight matrix ranges from 0 to 1 and is used to measure the importance of each feature channel. A higher weight indicates that the channel is more critical in the current task; a lower weight indicates that the channel is less important. Finally, the channel weight matrix is ​​multiplied element-wise with the preliminary fused features, applying the weights to the feature channels of the preliminary fused features. Features of high-weight channels are amplified, thereby strengthening their ability to express key information; features of low-weight channels are suppressed, reducing interference from irrelevant information. This dynamic weighting process allows the model to focus more on task-relevant important features, significantly improving the effectiveness and discriminative power of feature representation.

[0103] The introduction of channel attention weighting enables the MSCA module to dynamically adjust its focus based on the features of the current input, ensuring that the model can make full use of the most valuable information while suppressing the interference of noise and redundant features, thus providing optimized input for subsequent feature fusion and decoding steps.

[0104] Step 3.2.4: Input the multi-scale features generated by MSCA into the CRF model, decode the label at each position, and output the label sequence;

[0105] Specifically, to ensure the logical consistency of sequence labeling, a CRF decoding layer is added to the output layer of the model. CRF captures the dependencies between labels, ensuring that the generated label sequence conforms to the logical rules of named entity recognition. The specific steps are as follows:

[0106] Step 3.2.4.1: Tag Dependency Modeling. CRF ensures the rationality of the tag sequence by modeling the dependencies between tags (e.g., the initial tag "B" of a named entity should be followed by the internal tag "I"). Through this modeling, the CRF layer can effectively avoid tag conflicts that may occur when tagging independently.

[0107] Step 3.2.4.2: Dynamic Programming Decoding. CRF decoding uses a dynamic programming algorithm to compute the optimal label sequence. Dynamic programming ensures the efficiency of the decoding process, enabling the output of the optimal label path within a reasonable time, thereby improving the model's practicality.

[0108] Step 3.2.4.3: Tag output. The CRF layer finally outputs a tag sequence to determine the entity category of each word, thereby achieving accurate entity recognition of cybersecurity text.

[0109] Step 3.2.5: Repeat steps 3.2.1-3.2.5, and obtain 4 different models after 4 training sessions;

[0110] Step 3.2.6: Adopt a learning rate decay strategy to dynamically adjust the learning rate of the model based on the performance of different models on the validation set, determine the model with the best performance among the four models, and set it as the best performing model for this cross-validation.

[0111] Specifically, a learning rate decay strategy is employed during training, dynamically adjusting the learning rate based on performance on the validation set to ensure smooth training and optimal results. Simultaneously, evaluation metrics are used: precision, recall, and F1 score are calculated on the validation set to assess model performance, ensuring optimal accuracy and efficiency. Experiments show that a learning rate of 5e-5 achieves the best model performance.

[0112] Step 3.3: After 5-fold cross-validation, the test set is used to validate the 5 optimal named entity recognition models obtained from each round of cross-validation, and the optimal named entity recognition model of 5-fold cross-validation is obtained.

[0113] Step 4: Input the actual network security entity text dataset into the trained named entity recognition model, and output a label sequence;

[0114] After training, the model was applied to real-world cybersecurity text data to test its performance in practical scenarios. The accuracy of the model in complex cybersecurity texts was further verified by comparing it with manually labeled data. The specific application testing steps are as follows:

[0115] Step 4.1: Actual data testing. Input actual cybersecurity text data into the model and compare and analyze the entity labels output by the model.

[0116] Step 4.2: Performance Validation. The model's performance in a real-world environment is evaluated by calculating metrics such as precision, recall, and F1 score.

[0117] Step 4.3: Error Analysis. Analyze the cases with identification errors to find the reasons for the model's misjudgment and provide a basis for subsequent model optimization.

[0118] Step 5: Identify the entity category of each word in the actual cybersecurity entity text dataset based on the output label sequence, thereby achieving entity recognition of cybersecurity text.

[0119] The network security entity recognition method based on multi-scale convolutional attention mechanism proposed in this invention has successfully achieved efficient and accurate entity recognition, verifying the practicality and effectiveness of the model in complex domain tasks.

[0120] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0121] (1) In the data preprocessing of this invention, the word segmentation tool is specially customized in order to better identify terms and abbreviations unique to the field of network security and reduce word segmentation errors; stop words that have no practical significance for the task are removed to reduce the computational burden of the model and improve processing efficiency; standardized processing ensures the feasibility and consistency of batch input.

[0122] (2) The named entity recognition model based on multi-scale convolutional attention mechanism constructed in this invention has the advantages of high-precision recognition, efficient computation, and deep integration of multiple modules. Among them, the named entity recognition model based on multi-scale convolutional attention mechanism is a BERT-BiLSTM-MSCA-CRF structure. By combining contextual semantics, serialization dependency relations, multi-scale attention and label association decoding, it significantly improves the recognition accuracy of network security entities and effectively solves the shortcomings of existing methods in recognition accuracy. The model optimizes the combination design of BERT embedding, channel attention and CRF layer, so that the model has low computational resource consumption while maintaining high recognition accuracy. It is suitable for resource-constrained real-world environments, such as embedded systems or mobile devices. The multi-scale channel attention mechanism enables the model to adapt to different information features in network security texts and adapt to dynamic and ever-changing entity representation methods and context changes. It is suitable for various security threat detection scenarios. The model achieves deep integration of multiple modules in structure. It innovatively combines pre-trained language models with multi-channel attention mechanism, which enhances the accurate recognition and effective extraction of network security entities. It provides more efficient and stable technical support for threat detection and intelligence analysis in the field of network security, and can greatly improve the response speed and monitoring capabilities of network security systems.

[0123] (3) This invention proposes a multi-scale channel attention mechanism that combines ordinary convolution and dilated convolution, while dynamically adjusting channel weights to improve the model's ability to capture multi-scale features. In addition, the deep fusion between modules optimizes the integrity of feature transmission, significantly reduces computational complexity, significantly improves the accuracy and efficiency of network security entity recognition, enhances the model's ability to adapt to dynamic scenarios, and is suitable for real-time threat analysis scenarios.

[0124] like Figure 6 As shown, Embodiment 2 of the present invention provides a network security entity recognition system based on multi-layer channel attention, which runs the network security entity recognition method based on multi-layer channel attention described in Embodiment 1, including:

[0125] The data preprocessing module is used to preprocess real-world cybersecurity entity text datasets.

[0126] Model building module: used to build a named entity recognition model based on multi-layer channel attention. The preprocessed network security entity text dataset is input into the named entity recognition model for model training to obtain a trained named entity recognition model.

[0127] The label sequence acquisition module is used to input the actual network security entity text dataset into the trained named entity recognition model and output a label sequence.

[0128] Category recognition module: used to identify the entity category of each word based on the output label sequence, realizing network security entity recognition based on multi-layer channel attention.

[0129] Embodiment 3 of the present invention further illustrates the network security entity recognition method based on multi-scale convolutional attention mechanism provided by the present invention through experiments.

[0130] The experiments used a self-built dataset to evaluate model performance. This dataset contains a large number of cybersecurity terms and abbreviations, making it suitable for validation of named entity recognition tasks. The experiments focused on evaluating the model's performance in terms of precision, recall, and F1 score.

[0131] (1) Experimental setup

[0132] The experiment was conducted on a computer equipped with an NVIDIA RTX 3090 graphics card and 32GB of RAM. The software environment included Python 3.8, the PyTorch deep learning framework, and the BERT pre-trained model was loaded through the transformers library.

[0133] (2) Dataset characteristics

[0134] The experimental dataset contains text data from real-world cybersecurity incidents, covering cybersecurity entities such as intrusion detection logs, malware descriptions, IP addresses, and domain names. To enhance the model's generalization ability, the dataset underwent data cleaning and augmentation, and was divided into training and test sets.

[0135] (3) Experimental comparison

[0136] Multiple comparative experiments were conducted, including different model configurations, to verify the performance of the proposed model on the NER task. The comparative models included: a combined model without BERT integration, a combined model with BERT integration, and the proposed multi-scale convolutional attention model. Specific experimental results are shown in Table 1.

[0137] Table 1 Comparative Experiment

[0138]

[0139] (4) Experiments with different learning rates

[0140] The experiment also examined the impact of different learning rates on the proposed model, including learning rates of 1e-4, 5e-5, 3e-5, 1e-5, and 5e-6. The experimental results are shown in Table 2, revealing the differences in model performance under different learning rates.

[0141] Table 2 Comparison of different learning rates

[0142]

[0143]

[0144] (5) Analysis of experimental results

[0145] Our model outperforms other comparative models in F1 score, particularly achieving significant improvements in precision and recall. The introduction of a multi-scale convolutional attention mechanism enables the model to simultaneously focus on local and global features and adaptively adjust channel weights. A learning rate of 5e-5 is the best among all experimental groups, demonstrating that a moderate learning rate can achieve a balance between training speed and stability.

[0146] Embodiment 4 of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements a network security entity recognition method based on multi-layer channel attention as described in Embodiment 1.

[0147] Embodiment 5 of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a network security entity recognition method based on multi-layer channel attention as described in Embodiment 1.

[0148] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A network security entity recognition method based on multi-layer channel attention, characterized in that, include: Preprocessing of real-world cybersecurity entity text datasets; A named entity recognition model based on multi-layer channel attention is constructed. A preprocessed network security entity text dataset is input into the named entity recognition model for training, resulting in a trained named entity recognition model, including: A named entity recognition model based on multi-layer channel attention is constructed using BERT, BiLSTM, MSCA, and CRF models. The preprocessed cybersecurity entity text dataset is divided into a test set and a training set. The named entity recognition model is trained using 5-fold cross-validation. The training dataset is divided into 5 parts, and 5 rounds of cross-validation are performed. In each round of cross-validation, one part of the data is selected as the validation set, and the remaining 4 parts are used as the training set. Each round of cross-validation yields an optimal named entity recognition model, including: A single training set is input into the BERT model of the named entity recognition model to generate context-sensitive embedding vectors; the generated embedding vectors are then input into the BiLSTM model of the named entity recognition model to extract bidirectional features from the text sequence, resulting in bidirectional integrated features. The bidirectional integrated features extracted by the BiLSTM model are fed into the MSCA of the named entity recognition model for multi-scale feature extraction, resulting in multi-scale features, including: The bidirectional integrated features extracted by the BiLSTM model are fed into the MSCA of the named entity recognition model. These features are then extracted and fused using multi-scale convolutions, including ordinary convolution and dilated convolution, to obtain preliminary fused features, including: MSCA sets the kernels of ordinary convolution to 1×1 and 3×3, and extracts features from bidirectional features according to the different kernels of ordinary convolution. By inserting fixed dilated intervals into the kernels of dilated convolution, features with different receptive fields from ordinary convolution are extracted from the bidirectional features according to the kernels of dilated convolution. The features extracted by ordinary convolution and dilated convolution are standardized by batch normalization, and the features extracted by ordinary convolution and dilated convolution are fused using addition operation to obtain preliminary fused features. Next, a channel attention mechanism is used to assign weights to the preliminary fusion features of different channels, obtaining channel attention-weighted features. Then, the preliminary fusion features and the channel attention-weighted features are fused to obtain the final multi-scale features. The generated multi-scale features are input into the CRF model of the named entity recognition model, and the label at each position in the multi-scale features is decoded to output the current optimal label sequence. The remaining single training sets in the training set are repeatedly input into the named entity recognition model. After four training iterations, four named entity recognition models with optimal label sequences of different outputs are obtained. A learning rate decay strategy is adopted to dynamically adjust the learning rate of the model according to the performance of the named entity recognition models with optimal label sequences of different outputs on the validation set. The named entity recognition model with the best performance among the four models is determined as the optimal named entity recognition model for this round of cross-validation. After 5-fold cross-validation, the test set is used to verify the five optimal named entity recognition models obtained from each round of cross-validation to obtain the optimal named entity recognition model of 5-fold cross-validation. Input the actual network security entity text dataset into the trained named entity recognition model, and output a label sequence; Based on the output label sequence, the entity category of each word in the actual cybersecurity entity text dataset is identified, realizing cybersecurity entity recognition based on multi-layer channel attention.

2. The network security entity recognition method based on multi-layer channel attention according to claim 1, characterized in that: Data preprocessing is performed on real-world cybersecurity entity text datasets, specifically including: We used word segmentation tools to segment the original text in a real cybersecurity entity text dataset and labeled each word with its corresponding tag. Remove stop words from the segmented and tagged text; Standardize the text after removing stop words; Enhancement of standardized text data; By using association rule algorithms, potential dangerous events in data-enhanced text are identified, resulting in a preprocessed cybersecurity entity text dataset.

3. The network security entity recognition method based on multi-layer channel attention according to claim 1, characterized in that: A single training set is input into the BERT model of the named entity recognition model to generate context-sensitive embedding vectors. The generated embedding vectors are then input into the BiLSTM model of the named entity recognition model for bidirectional feature extraction of the text sequence, specifically including: The BERT model captures the semantic information of text through a multi-layer Transformer structure, generates context-sensitive word embeddings, and converts the word embeddings into context-sensitive embedding vectors. The BiLSTM model processes both the forward and backward sequences of text in the embedded vectors, captures the dependencies between sentences within the text, extracts bidirectional features, and integrates the extracted bidirectional features to obtain bidirectional integrated features.

4. The network security entity recognition method based on multi-layer channel attention according to claim 1, characterized in that: A channel attention mechanism is used to assign weights to the preliminary fusion features of different channels, obtaining channel attention-weighted features, specifically including: The initial fusion features are extracted by average pooling to obtain the overall mean of all features in each channel and by max pooling to extract the strongest response value in each channel. The features obtained from average pooling and max pooling are fused and then linearly transformed through a shared convolutional layer. The features that have undergone linear transformation are then nonlinearly mapped using the ReLU activation function. The features after nonlinear mapping are used to generate a channel weight matrix through the Sigmoid function; Next, perform element-wise multiplication between the channel weight matrix and the preliminary fused features, and apply the weights to the feature channels of the preliminary fused features to obtain channel attention-weighted features.

5. The network security entity recognition method based on multi-layer channel attention according to claim 1, characterized in that: The generated multi-scale features are input into the CRF model of the named entity recognition model. The label at each position in the multi-scale features is decoded, and the current optimal label sequence is output, specifically including: The CRF model establishes a dependency model between labels by using contextual dependencies in multi-scale features. It then uses dynamic programming to decode and calculate the optimal label sequence for each position label in the dependency model and its preceding and following words, and outputs the current optimal label sequence.

6. A network security entity recognition system based on multi-layer channel attention, running the network security entity recognition method based on multi-layer channel attention as described in any one of claims 1 to 5, characterized in that: The data preprocessing module is used to preprocess real-world cybersecurity entity text datasets. Model building module: used to build a named entity recognition model based on multi-layer channel attention. The preprocessed network security entity text dataset is input into the named entity recognition model for model training to obtain a trained named entity recognition model. The label sequence acquisition module is used to input the actual network security entity text dataset into the trained named entity recognition model and output a label sequence. Category recognition module: used to identify the entity category of each word in the actual cybersecurity entity text dataset based on the output label sequence, realizing cybersecurity entity recognition based on multi-layer channel attention.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements a network security entity recognition method based on multi-layer channel attention as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a network security entity recognition method based on multi-layer channel attention as described in any one of claims 1 to 5.

Images