A multi-perspective collaborative network security named entity recognition method integrating multi-source gating and hybrid coding
Patent Information
- Application Number
- CN202611227752.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-13
- Publication Date
- 2026-11-10
AI Technical Summary
Existing methods struggle to effectively identify complex named entities in cybersecurity texts, especially strongly formatted entities such as hash values, URLs, and vulnerability numbers. Furthermore, they fail to balance long-range dependency modeling with local boundary awareness, resulting in insufficient recognition accuracy and recall.
By employing a method that integrates multi-source gating and hybrid coding, this approach utilizes network security multi-source information representation construction, hybrid sequence coding, and an improved span prediction mechanism. It combines the Mamba state-space model and neighborhood-gated attention mechanism, explicitly introducing type prior constraints and keyword bias to achieve adaptive fusion of multi-source heterogeneous information and span-level entity prediction.
It improves the accuracy and robustness of identifying complex network security entities, especially the ability to identify boundary-sensitive entities such as IP addresses, URLs, and hash values, and is suitable for automated processing of network security threat intelligence.
Smart Images

Figure CN123070104A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and network security information processing technology, and in particular to a multi-perspective collaborative network security named entity recognition method that integrates multi-source gating and hybrid coding. Background Technology
[0002] As cyberattacks become increasingly complex, organized, and intelligent, the cybersecurity field generates a massive amount of unstructured or semi-structured text data, such as threat intelligence reports, security bulletins, vulnerability descriptions, attack analysis reports, and malware analysis documents. These texts typically contain key security entities such as vulnerability IDs, malware families, attack groups, attack tools, IP addresses, URLs, hash values, file paths, and CVE numbers. The ability to automatically and accurately extract these entities from massive amounts of cybersecurity text is a crucial foundation for automated threat intelligence processing, cybersecurity knowledge graph construction, attack attribution analysis, and security operations decision-making.
[0003] However, cybersecurity named entity recognition (NAME) tasks differ significantly from general domain named entity recognition tasks. Cybersecurity texts are characterized by dense terminology, numerous abbreviations, the continuous emergence of new entities, irregular entity boundaries, and highly formatted entity surface morphology. For example, entities such as IP addresses, hash values, CVE numbers, domain names, and file paths often rely on character-level patterns and domain rules for accurate identification; while entities such as attack organizations, malware, and attack tools require a combination of contextual semantics and external domain knowledge for judgment. Existing methods that rely solely on general pre-trained language models or traditional sequence labeling models struggle to fully characterize the semantic, structural, and rule-based features of these complex entities, easily leading to problems such as inaccurate entity boundary identification, incorrect category classification, and insufficient generalization ability for novel entities.
[0004] To address these issues, existing research primarily employs rule matching, traditional machine learning, deep neural networks, or pre-trained language models for network security named entity recognition. However, these methods still have certain gaps with the actual engineering application requirements: (1) Some methods rely only on contextual semantic representation and fail to make full use of the character morphological features, explicit rule patterns and external knowledge information that are common in network security entities, resulting in insufficient recognition of strongly formatted entities such as hash values, URLs and vulnerability numbers; (2) Although some methods introduce multi-source features, they usually use simple splicing or weighted summation to fuse them, and cannot dynamically select the most effective information source according to different lexical units and different entity types, which easily introduces redundant features and noise interference; (3) Existing sequence encoders mostly use single structures such as recurrent networks, convolutional networks or global self-attention mechanisms, which are difficult to simultaneously take into account long-range dependency modeling and local entity boundary perception, especially when dealing with cross-sentence related entities and boundary sensitive entities; (4) Traditional word-by-word annotation methods have limited ability to model irregular boundaries and long-span entities, while existing span prediction methods often lack the structured use of entity type priors and domain keyword clues, making it difficult to improve the recognition accuracy and recall rate of complex entities at the same time.
[0005] Therefore, it is necessary to propose a named entity recognition method for cybersecurity text, which can integrate multi-source heterogeneous information such as domain semantics, retrieval knowledge, character morphology and rule patterns in the input representation stage, model global dependencies and local boundary structures simultaneously in the sequence encoding stage, and explicitly introduce type prior constraints and keyword bias guidance in the output prediction stage, thereby improving the boundary recognition capability, category discrimination capability and engineering applicability of complex entities in cybersecurity. Summary of the Invention
[0006] The purpose of this invention is to propose a multi-perspective collaborative network security named entity recognition method that integrates multi-source gating and hybrid encoding. This method targets network security texts such as threat intelligence, security announcements, and vulnerability descriptions. It comprehensively extracts domain-pre-trained semantic features, retrieval-enhanced knowledge features, character-level morphological features, and domain pattern rule features. It also achieves adaptive fusion of multi-source heterogeneous information through a lexical-level gating mechanism. Simultaneously, it utilizes the Mamba state-space model and a neighborhood-gated attention mechanism to collaboratively model long-range dependencies and local entity boundary information, and combines type prior constraints and keyword bias guidance to perform span-level entity prediction. This improves the accuracy and robustness of recognizing complex network security entities such as vulnerability numbers, malware, attack organizations, IP addresses, hash values, and CVE numbers.
[0007] To achieve the above objectives, the present invention provides the following solution: A multi-perspective collaborative network security named entity recognition method integrating multi-source gating and hybrid coding includes: A network security multi-source information representation construction method is adopted to generate a fusion representation from the input word sequence; wherein, the fusion representation includes: basic semantic representation, retrieval enhancement representation, character-level morphological representation and domain pattern feature representation; A hybrid sequence coding method is used to simultaneously model long-range dependencies and local boundary structures in cybersecurity texts in the fused representation, and to obtain the hybrid sequence coding output. Based on the hybrid sequence encoding output, an improved span prediction mechanism is used to predict entity output and obtain entity recognition results.
[0008] Optionally, a network security multi-source information representation construction method is used to generate a fusion representation from the input lexical sequence, including: Inputting the sequence of words into a pre-trained language model in the cybersecurity domain captures the contextual semantic relationships in cybersecurity texts and obtains the basic semantic representation of each word; By introducing a cybersecurity knowledge base and supplementing lexical units with background knowledge and implicit association information through a cross-attention mechanism, enhanced retrieval representations can be obtained. Multi-scale convolution is used to extract local morphological features of words to obtain character-level morphological representations; For entity patterns identified through explicit rules, the explicit rule knowledge is transformed into a learnable neural representation to obtain domain pattern feature representations; The basic semantic representation, retrieval enhancement representation, character-level morphological representation, and domain pattern feature representation are dynamically fused to obtain the fused representation.
[0009] Optionally, a cybersecurity knowledge base can be introduced to enhance the retrieval of the input sentence, including: The input sentence is encoded into a query vector, and the most relevant query vectors are retrieved from a pre-built cybersecurity knowledge base. K Each document is then encoded into a vector and stacked into a document representation matrix. The document representation matrix is then mapped to a unified semantic space through linear projection. Using the aforementioned basic semantic representation as the query and the mapped document representation matrix as the key and value, a cross-attention mechanism is constructed to obtain cross-attention output; The cross-attention output is further transformed and normalized through a two-layer feedforward adapter to obtain the retrieval enhancement representation.
[0010] Optionally, extracting local morphological features of words using multi-scale convolution includes: For the i each word element t iThe characters are mapped to an embedding vector sequence, and then one-dimensional convolution with multiple different kernel sizes is used to extract local morphological features; Concatenate the outputs of multiple convolutional kernels to obtain a character-level representation; The character-level morphological representation is obtained by mapping the character-level representation to the same dimension as the basic semantic representation through linear projection and layer normalization.
[0011] Optionally, a hybrid sequence encoding method is employed to simultaneously model long-range dependencies and local boundary structures in the fused representation, including: A selective state-space model is used to perform sequence modeling on the fused representation to obtain the state-space branch output. The selective state-space model selectively retains and propagates key information in the text sequence by inputting relevant state transition parameters, input mapping parameters, and output mapping parameters. Attention is calculated within the local neighborhood of the current word to enhance the sequence encoding's ability to perceive local boundary structures and obtain the output of the neighborhood-gated attention branch; The output of the state space branch and the output of the neighborhood gated attention branch are fused together in the form of residuals to obtain the hybrid sequence encoding output.
[0012] Optionally, using a selective state-space model to perform sequence modeling on the fused representation includes: For the fusion representation of the input The selective state-space model uses hidden states η i Establish a sequence recursion relationship: in, For position i The output representation of the state-space branch. A i Here is the state transition matrix. B i For the input mapping matrix, C i This is the output mapping matrix.
[0013] Optionally, calculating attention within the local neighborhood of the word at the current position includes: For the i Fusion representation of input at each position Its neighborhood range is defined as ,in, j Indicates the location index within the neighborhood. w For the neighborhood radius, Query vectors are generated through learnable linear transformations; for N ( i Input at any position within ) Generate key vectors and value vectors; Based on the query vector and key vector, obtain the basic attention score; Introduce a gating function to construct the gating signal: in, W g This indicates that the parameter is learnable. g ij Indicates position j Position i The intensity of information contribution; the final attention weight is: in, Indicates the final attention weight. Indicates the basic attention score. l Indicates the neighborhood range N ( i Any position in ); Based on the value vector and the final attention weights, the neighborhood-gated attention branch output is obtained: in, Indicates position i The output representation of the neighborhood-gated attention branch. v j Represents a value vector.
[0014] Optionally, based on the hybrid sequence encoding output, the entity output prediction using an improved span prediction mechanism includes: Based on the hybrid sequence encoding output, the span score matrix is obtained; Apply an upper triangular mask to the span score matrix to retain only candidate spans whose start position is no later than the end position; at the same time, use a valid bit mask to remove invalid candidates generated at the fill positions. During the decoding stage, candidate entity spans with scores higher than a preset threshold are selected, sorted from highest to lowest score, and a greedy strategy is used to remove conflicting or overlapping candidate spans. Finally, the entity recognition result is output, which includes: entity text, entity start position, entity end position, entity category, and confidence score.
[0015] Optionally, obtaining the span score matrix based on the hybrid sequence encoding output includes: The hybrid sequence is encoded and output, mapped to the query vector and key vector space corresponding to each entity category, and a query representation is obtained. and key representation ,in Indicates the first c Class entity in position i The starting point is indicated at the location. Indicates the first c Class entity in position j The endpoint is indicated at the location. d h For entity head dimension; the first c Entities in span ( i , j The basic span score is: Calculate the position separately i As the first c Prior score of the starting point of the entity class, and its position. j As the first c Prior score of entity endpoint: in, Indicates position i As the first c Prior score of the starting point of the class entity, Indicates position j As the first c Prior score of entity endpoint, and These are the linear mapping parameters for the start and end points, respectively. and For bias terms; Then, construct the type prior span score: Let the keyword tag sequence corresponding to the input word be . Z ={ z 1, z 2,…, z n}, n Given the length of the lexical sequence, map the keyword tags to the keyword semantic representation: Where Emb(∙) represents the keyword tag embedding function, The projection matrix; Calculate the bias scores of keywords for different entity categories at the start and end points respectively: in, and These are the keyword starting point projection matrix and the ending point projection matrix, respectively. and For bias terms, and The bias scores are for the start and end points, respectively. This yields the keyword bias span score: The base span score, type prior span score, and keyword bias span score are weighted and fused to obtain the final span score: in, λ 1 and λ 2 is a learnable scalar parameter.
[0016] The beneficial effects of this invention are as follows: First, this invention adaptively fuses domain pre-trained semantics, retrieval enhancement knowledge, character-level morphological features, and domain pattern rules through multi-source input representation and word-level gating fusion mechanism, avoiding information redundancy and noise introduction problems caused by traditional simple splicing or fixed weighting methods.
[0017] Second, this invention models long-range dependencies with linear complexity through Mamba state space branching, enabling the model to capture cross-sentence associations, long-span entities, and long-distance semantic dependencies in cybersecurity texts at a lower computational cost.
[0018] Third, this invention enhances the local context modeling capability through neighborhood-gated attention branches, enabling the model to more accurately perceive keywords, special symbols, number fragments, and formatted structures near entity boundaries, thereby improving the ability to identify boundary-sensitive entities such as IP addresses, URLs, hash values, file paths, and CVE numbers.
[0019] Fourth, this invention coordinates global semantic representation and local boundary representation through a dual-branch residual fusion mechanism, enabling the model to take into account both long-range dependencies and local fine-grained features, thereby improving the stability of complex network security entity recognition.
[0020] Fifth, this invention introduces type prior constraints and keyword bias guidance in span prediction, so that span prediction not only relies on start and end position matching, but also can explicitly utilize entity type boundary priors and keyword clues in the field of network security, thereby improving the unified discrimination capability of complex entity boundaries and categories.
[0021] Sixth, this invention is applicable to automated and structured processing scenarios of cybersecurity threat intelligence, and can provide entity extraction capabilities for security operations, attack tracing, vulnerability analysis, intelligence knowledge graph construction, and automated defense systems. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a schematic diagram of a multi-perspective collaborative network security named entity recognition method that integrates multi-source gating and hybrid coding according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the network security multi-source information representation construction method according to an embodiment of the present invention; Figure 3 This is a diagram of the neighborhood-gated attention branch architecture according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the span prediction method based on the fusion type prior and keyword bias in an embodiment of the present invention. Detailed Implementation
[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0026] like Figure 1 As shown, this embodiment proposes a multi-perspective collaborative network security named entity recognition method that integrates multi-source gating and hybrid coding, including: A network security multi-source information representation construction method is adopted to generate a fusion representation from the input word sequence; wherein, the fusion representation includes: basic semantic representation, retrieval enhancement representation, character-level morphological representation and domain pattern feature representation; A hybrid sequence coding method is used to simultaneously model long-range dependencies and local boundary structures in cybersecurity texts in the fused representation, and to obtain the hybrid sequence coding output. Based on the hybrid sequence encoding output, an improved span prediction mechanism is used to predict entity output and obtain entity recognition results.
[0027] Furthermore, by employing a network security multi-source information representation construction method, the fusion representation generated from the input lexical sequence includes: Inputting the sequence of words into a pre-trained language model in the cybersecurity domain captures the contextual semantic relationships in cybersecurity texts and obtains the basic semantic representation of each word; By introducing a cybersecurity knowledge base and supplementing lexical units with background knowledge and implicit association information through a cross-attention mechanism, enhanced retrieval representations can be obtained. Multi-scale convolution is used to extract local morphological features of words to obtain character-level morphological representations; For entity patterns identified through explicit rules, the explicit rule knowledge is transformed into a learnable neural representation to obtain domain pattern feature representations; The basic semantic representation, retrieval enhancement representation, character-level morphological representation, and domain pattern feature representation are dynamically fused to obtain the fused representation.
[0028] Furthermore, a hybrid sequence encoding method is employed to simultaneously model long-range dependencies and local boundary structures in cybersecurity text for the fused representation, including: A selective state-space model is used to perform sequence modeling on the fused representation to obtain the state-space branch output. The selective state-space model selectively retains and propagates key information in the text sequence by inputting relevant state transition parameters, input mapping parameters, and output mapping parameters. Attention is calculated within the local neighborhood of the current word to enhance the sequence encoding's ability to perceive local boundary structures and obtain the output of the neighborhood-gated attention branch; The output of the state space branch and the output of the neighborhood gated attention branch are fused together in the form of residuals to obtain the hybrid sequence encoding output.
[0029] Furthermore, based on the hybrid sequence encoding output, the entity output prediction using an improved span prediction mechanism includes: Based on the hybrid sequence encoding output, the span score matrix is obtained; Apply an upper triangular mask to the span score matrix to retain only candidate spans whose start position is no later than the end position; at the same time, use a valid bit mask to remove invalid candidates generated at the fill positions. During the decoding stage, candidate entity spans with scores higher than a preset threshold are selected, sorted from highest to lowest score, and a greedy strategy is used to remove conflicting or overlapping candidate spans. Finally, the entity recognition result is output, which includes: entity text, entity start position, entity end position, entity category, and confidence score.
[0030] Specifically, this embodiment adopts the following technical solution: 1. Multi-source heterogeneous feature extraction and lexical-level semantic space expansion: This embodiment uses pre-trained language models in the field of cybersecurity, retrieval enhancement generation technology, character-level convolutional coding technology, and domain rule pattern coding technology to automatically extract multi-source heterogeneous features from cybersecurity texts. It also dynamically allocates the contribution weights of various feature sources through a lexical-level gating fusion mechanism to generate fused lexical representations for cybersecurity entity recognition tasks. This effectively expands the semantic space and structural feature space of cybersecurity texts, and initially solves the problem that a single semantic representation is insufficient to fully characterize complex security entities.
[0031] 2. Hybrid Encoding of Global Long-Range Dependencies and Local Boundary Information: This embodiment introduces the Mamba state space model and a neighborhood-gated attention mechanism, using them as parallel encoding branches to perform multi-view collaborative encoding of the fused lexical representation. The Mamba state space branch is used to capture long-range contextual dependencies, while the neighborhood-gated attention branch is used to capture local fine-grained features near entity boundaries. A dual-branch residual fusion mechanism is used to achieve complementary enhancement of global semantic information and local structural information, thereby solving the problem that existing encoders struggle to simultaneously handle long-range dependency modeling and local entity boundary awareness.
[0032] 3. Type Prior and Keyword Bias Jointly Guided Span Prediction: This embodiment transforms the network security named entity recognition task into a span-level entity boundary and category joint discrimination task. Based on the basic span scoring mechanism, type prior constraints and keyword bias guidance are introduced, enabling the model to simultaneously judge the start and end position matching relationship of candidate spans, the rationality of entity type boundaries, and the strength of domain keyword prompts, thereby improving the recognition accuracy of complex entities, irregular boundary entities, and long span entities.
[0033] More specifically, the multi-perspective collaborative network security named entity recognition method that integrates multi-source gating and hybrid coding proposed in this embodiment includes the following steps: (1) Data collection This embodiment uses two publicly available datasets in the field of cybersecurity, DNRTI and APTNER, for model training.
[0034] (2) Construct a cybersecurity knowledge base To enhance the model's domain adaptability in entity recognition within the cybersecurity field, this embodiment constructs a lightweight Retrieval-Augmented Generation (RAG) knowledge base. This knowledge base uses the MITRE ATT&CK framework and NIST CVE vulnerability data as its primary data sources, parsing the data information into structured text fragments: for ATT&CK, the name, description, and examples of each technique are extracted; for CVE, the CVE-ID, vulnerability description, and affected product information are extracted. Subsequently, all fragments are vectorized and encoded, and a precise index file is constructed using the FAISS library, ultimately resulting in a vector knowledge base containing professional cybersecurity knowledge.
[0035] (3) Construction of multi-source information representation for network security This embodiment proposes a method for constructing multi-source information representations for network security, used to generate fused representations for input lexical units. For example... Figure 2 As shown, the method includes five core components: basic semantic representation, retrieval enhancement representation, character-level morphological representation, domain pattern feature representation, and multi-branch gating fusion.
[0036] 1) Basic semantic representation components In the basic semantic representation component, this embodiment inputs the input lexical sequence into a pre-trained language model in the cybersecurity domain to obtain the basic semantic representation of each lexical. This component can capture the contextual semantic relationships in cybersecurity text, such as the relationship between attack organizations and attack tools, the relationship between vulnerability numbers and exploitation behaviors, and the relationship between malware and command and control addresses. Specifically, given a lexical sequence... T ={ t 1, t 2,…, t n},in n Indicates the length of the word sequence. t i Indicates the first i Each word element. This embodiment will... T Input the domain-specific pre-trained model SecureBERT 2.0 to obtain the basic semantic representation matrix. H b . H b Each row in the model represents the contextual semantic vector of the corresponding word. Because this pre-trained model has been trained on a cybersecurity corpus, it can more accurately express the semantic features of terms in fields such as vulnerabilities, attack organizations, malware, attack tools, hash values, and domain names.
[0037] 2) Retrieval Enhancement Presentation Component Relying solely on the contextual information of the input sentence itself is often insufficient to fully express the background knowledge and implicit relationships behind cybersecurity entities. Therefore, this embodiment sets up a retrieval enhancement representation component to incorporate external cybersecurity knowledge.
[0038] First, in this embodiment, the input sentence is encoded into a query vector. It also retrieves the most relevant entries from a pre-built cybersecurity knowledge base. K Each document is then encoded into a vector and stacked into a document representation matrix. D ={ d 1, d 2,…, d K}, Then, it is mapped to a unified semantic space through linear projection: in W d Let be the projection matrix. b d For bias terms, This is the projected representation of the document. Based on this, this embodiment uses basic semantic representation. H b For querying, the document to be retrieved is represented. Construct a cross-attention mechanism for keys and values: Where MHA(∙) represents multi-head attention operation, This represents the cross-attention output. Subsequently, it is further transformed and normalized through a two-layer feedforward adapter to obtain the retrieval enhancement representation: Where MLP(∙) represents a nonlinear mapping, and LN(∙) represents a layer normalization operation. This represents the final enhanced retrieval feature.
[0039] 3) Character-level morphological representation components For the i each word element t i Map its characters to an embedded vector sequence c i Then, multiple one-dimensional convolutions with different kernel sizes are used to extract local morphological features. For the first... s Let there be n convolutional kernels, and let their output be: Where Conv(∙) represents the kernel size as... sIn a one-dimensional convolution, ReLU(∙) represents the activation function, and MaxPool(∙) represents max pooling. Concatenating the outputs of multiple convolution kernels yields a character-level representation: in, S This indicates the number of convolutional kernels. Subsequently, linear projection and layer normalization are used to map the character-level representation to the same dimension as the underlying semantic representation. in, W c and b c These represent the character feature projection parameters.
[0040] 4) Domain Pattern Feature Representation Component Besides character-level morphological features, there are also numerous entity patterns in the cybersecurity field that can be identified through explicit rules. Therefore, this embodiment sets up a domain pattern feature representation component to transform cybersecurity rule knowledge into learnable neural representations. For the first... i each word element t i Its pattern feature vector is defined as , m This represents the pattern feature dimension. The pattern feature representation is then obtained through linear mapping and layer normalization: in, W p and b p These represent the pattern feature projection parameters.
[0041] 5) Multi-branch gating fusion component After obtaining the domain pre-trained semantic representation, retrieval enhancement representation, character-level morphological representation, and domain pattern feature representation, this embodiment dynamically fuses the above heterogeneous information through a multi-branch gating fusion component.
[0042] For the i each word element t i The branches are concatenated as follows: Where [∙;∙] denotes vector concatenation, and then three gating coefficients are generated using a two-layer feedforward network and a sigmoid activation function: in , , These represent the search enhancement branch, character-level branch, and pattern feature branch, respectively, for the first... i The gating weight of each word element, W 1, W 2, b 1, b 2 represents the learnable parameters, GELU(∙) represents the Gaussian error linear unit, and σ(∙) represents the Sigmoid activation function.
[0043] Finally, the first i The fusion representation of 1 word element is defined as: Furthermore, this embodiment performs a mask clearing operation on invalid positions and combines Dropout and LayerNorm to improve representation robustness.
[0044] This gating mechanism can automatically adjust the contribution of various information sources based on the characteristics of different terms. For example, for attack organization names such as "APT29", the model can increase the weight of domain semantic representation and retrieval enhancement representation; for IP addresses such as "192.168.1.1", the model can increase the weight of character-level morphological representation and domain pattern feature representation; for filenames such as "powershell.exe", the model can simultaneously utilize character morphology, pattern rules, and contextual semantic information.
[0045] (4) Mixed sequence coding To simultaneously model long-range dependencies and local boundary structures in cybersecurity texts, this embodiment proposes a hybrid sequence coding method, which includes Mamba state space branching, neighborhood-gated attention branching, and a dual-branch residual fusion component.
[0046] 1) Mamba state space branches In the Mamba state space branch, this embodiment employs a selective state space model to perform sequence modeling on the fused lexical representation. This branch selectively preserves and propagates key information in the text sequence by inputting relevant state transition parameters, input mapping parameters, and output mapping parameters.
[0047] For the input sequence representation Mamba state space branches through hidden states η i Establish a sequence recursion relationship: in, For position i The output representation of the state-space branch. Ai Here is the state transition matrix. B i For the input mapping matrix, C i This is the output mapping matrix. Unlike the classic state-space model, the above parameters... A i , B i , C i It is no longer a constant, but determined by the input. x i Dynamically generated: Where A is a fixed globally shared parameter, s B (∙), s C (∙), s Δ (∙) is a linear projection function, Δ i The step size parameter is used for discretization. τ Δ (∙) represents a nonlinear transformation. This design transforms the model from a time-invariant system to a time-varying system, thereby enabling content-awareness capabilities and selectively propagating key information and suppressing noise within the sequence.
[0048] In cybersecurity named entity recognition tasks, this branch can capture cross-sentence or long-distance entity associations. For example, if the name of an attacking organization appears at the beginning of the text, and the name of the attack tools or malware used by it appears at the end of the text, the Mamba state space branch can retain relevant contextual information during sequence propagation, thereby assisting in entity boundary and category determination.
[0049] 2) Neighborhood-gated attention branch In the neighborhood-gated attention branch, this embodiment only calculates attention within the local neighborhood of the current position word, and its architecture is as follows: Figure 3 As shown.
[0050] For the input sequence representation For the first i The input at each position represents Its neighborhood range is defined as ,in j Indicates the location index within the neighborhood. w Let be the neighborhood radius. For Query vectors are generated through learnable linear transformations; for N ( iInput at any position within ) Generate key vectors and value vectors: in W q , W k , W v These are learnable parameters. The basic attention score is defined as: Building upon this, a gating function is introduced to dynamically filter neighborhood information. Specifically, the gating signal is constructed as follows: That These are learnable parameters. g ij ∈(0,1) represents the position j Position i The intensity of information contribution. The final attention weight is defined as: And obtain the output of the neighborhood-gated attention branch: in Indicates position i The output representation of the neighborhood-gated attention branch.
[0051] This branch enhances the model's ability to perceive local boundary structures. For example, when identifying "CVE-2021-44228", the model can focus on the local combination relationships between "CVE", hyphens, year, and number; when identifying URLs, the model can focus on protocol headers, domain names, path separators, and parameter symbols; and when identifying file paths, the model can focus on local patterns such as drive letters, directory separators, and file extensions.
[0052] 3) Dual-branch residual fusion component After obtaining the Mamba state space branch output and the neighborhood-gated attention branch output, this embodiment fuses them using a dual-branch residual fusion component. For the input sequence representation... The outputs of the Mamba state space branch and the neighborhood-gated attention branch are as follows: The fusion result of each layer is defined as follows: in The parameters are learnable and are used to control the contribution of the Mamba state space branch and the neighborhood-gated attention branch to the final representation, respectively. Finally, the outputs of the two branches are fused using residuals to achieve a balance between global dependency modeling and local structure modeling. Let the output of the hybrid sequence encoding be denoted as... Y ={ y 1, y 2,…, y n} .
[0053] (5) Span prediction of fusion type prior and keyword bias This embodiment employs an improved span prediction mechanism for entity output prediction, the specific framework of which is as follows: Figure 4 As shown. Unlike traditional word-by-word meta-sequence annotation methods, this embodiment uses an improved Efficient GlobalPointer to uniformly score all possible start and end position pairs, transforming the entity recognition task into a span-level binary classification problem, thus making it more suitable for recognizing long-span entities, irregular boundary entities, and complex format entities. For common... C This module aims to identify entity categories for any entity category. c ∈{1, 2, … , C} and any position pair ( i , j Generate the corresponding span score, where 1 ≤ i ≤ j ≤ n .
[0054] 1) Basic span scoring Encode the mixed sequence output Y Mapping to the query vector and key vector space corresponding to each entity category yields the query representation. and key representation ,in Indicates the first c Class entity in position i The starting point is indicated at the location. Indicates the first c Class entity in position j The endpoint is indicated at the location. d h For the entity head dimension. Therefore, the first... c Entities in span ( i , j The basic span score is: 2) Type Prior Constraints To enhance boundary adaptability for different entity categories, the position is calculated separately. i As the first c Prior score of the starting point of the entity class, and its position. j As the first c Prior score of entity endpoint: in Indicates position i As the first c Prior score of the starting point of the class entity, Indicates position j As the first c Prior score of entity endpoint, and These are the linear mapping parameters for the start and end points, respectively. and This is a bias term.
[0055] Then, construct the type prior span score: 3) Keyword bias guidance Let the keyword tag sequence corresponding to the input word be . Z ={ z 1, z 2,…, z n}, mapping keyword tags to keyword semantic representations: Where Emb(∙) represents the keyword tag embedding function, This is the projection matrix. Then, the bias scores of the keywords for the start and end points of different entity categories are calculated respectively: in, and These are the keyword starting point projection matrix and the ending point projection matrix, respectively. and This is the bias term. Therefore, the keyword bias span score is obtained: 4) Final span score fusion and decoding This embodiment weights and fuses the basic span score, the type prior span score, and the keyword bias span score to obtain the final span score: in, λ 1 and λ 2 is a learnable scalar parameter. Subsequently, an upper triangular mask is applied to the span score matrix to retain only candidate spans whose start position is no later than the end position; at the same time, an effective bit mask is used to remove invalid candidates generated at the fill positions.
[0056] In the decoding stage, this embodiment first filters candidate entity spans with scores higher than a preset threshold, then sorts them from highest to lowest score, and uses a greedy strategy to remove conflicting or overlapping candidate spans, finally outputting the entity recognition result. The output result includes entity text, entity start position, entity end position, entity category, and confidence score.
[0057] (6) In this embodiment, precision, recall, and F1 score are selected as model performance evaluation metrics. Precision represents the proportion of correct entities among the results predicted by the model; recall represents the proportion of all real entities correctly identified by the model; and the F1 score is the harmonic mean of precision and recall, used to comprehensively evaluate the model's recognition performance. The calculation methods for the above three metrics are as follows: .
[0058] (7) Experiments show that this embodiment can achieve better recognition results than existing mainstream methods in the field of named entity recognition of network security threat intelligence, and can provide effective technical support for automated and structured processing of network security threat intelligence.
[0059] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A multi-perspective collaborative network security named entity recognition method integrating multi-source gating and hybrid coding, characterized in that, include: A network security multi-source information representation construction method is adopted to generate a fusion representation from the input word sequence; wherein, the fusion representation includes: basic semantic representation, retrieval enhancement representation, character-level morphological representation and domain pattern feature representation; A hybrid sequence coding method is used to simultaneously model long-range dependencies and local boundary structures in cybersecurity texts in the fused representation, and to obtain the hybrid sequence coding output. Based on the hybrid sequence encoding output, an improved span prediction mechanism is used to predict entity output and obtain entity recognition results.
2. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding as described in claim 1, characterized in that, The method of constructing multi-source information representations for network security is adopted to generate a fusion representation from the input lexical sequence, including: Inputting the sequence of words into a pre-trained language model in the cybersecurity domain captures the contextual semantic relationships in cybersecurity texts and obtains the basic semantic representation of each word; By introducing a cybersecurity knowledge base and supplementing lexical units with background knowledge and implicit association information through a cross-attention mechanism, enhanced retrieval representations can be obtained. Multi-scale convolution is used to extract local morphological features of words to obtain character-level morphological representations; For entity patterns identified through explicit rules, the explicit rule knowledge is transformed into a learnable neural representation to obtain domain pattern feature representations; The basic semantic representation, retrieval enhancement representation, character-level morphological representation, and domain pattern feature representation are dynamically fused to obtain the fused representation.
3. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding as described in claim 2, characterized in that, Introducing a cybersecurity knowledge base and performing enhanced retrieval processing on the input sentences includes: The input sentence is encoded into a query vector, and the most relevant query vectors are retrieved from a pre-built cybersecurity knowledge base. K Each document is then encoded into a vector and stacked into a document representation matrix. The document representation matrix is then mapped to a unified semantic space through linear projection. Using the aforementioned basic semantic representation as the query and the mapped document representation matrix as the key and value, a cross-attention mechanism is constructed to obtain cross-attention output; The cross-attention output is further transformed and normalized through a two-layer feedforward adapter to obtain the retrieval enhancement representation.
4. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding as described in claim 2, characterized in that, Extracting local morphological features of words using multi-scale convolution includes: For the i each word element t i The characters are mapped to an embedding vector sequence, and then one-dimensional convolution with multiple different kernel sizes is used to extract local morphological features; Concatenate the outputs of multiple convolutional kernels to obtain a character-level representation; The character-level morphological representation is obtained by mapping the character-level representation to the same dimension as the basic semantic representation through linear projection and layer normalization.
5. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding as described in claim 1, characterized in that, The hybrid sequence encoding method is used to simultaneously model long-range dependencies and local boundary structures in cybersecurity text for the fused representation, including: A selective state-space model is used to perform sequence modeling on the fused representation to obtain the state-space branch output. The selective state-space model selectively retains and propagates key information in the text sequence by inputting relevant state transition parameters, input mapping parameters, and output mapping parameters. Attention is calculated within the local neighborhood of the current word to enhance the sequence encoding's ability to perceive local boundary structures and obtain the output of the neighborhood-gated attention branch; The output of the state space branch and the output of the neighborhood gated attention branch are fused together in the form of residuals to obtain the hybrid sequence encoding output.
6. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding as described in claim 5, characterized in that, Sequence modeling of the fused representation using a selective state-space model includes: For the fusion representation of the input The selective state-space model uses hidden states η i Establish a sequence recursion relationship: in, For position i The output representation of the state-space branch. A i Here is the state transition matrix. B i For the input mapping matrix, C i This is the output mapping matrix.
7. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding as described in claim 5, characterized in that, Calculating attention within the local neighborhood of the word at the current position includes: For the i Fusion representation of input at each position Its neighborhood range is defined as ,in, j Indicates the location index within the neighborhood. w For the neighborhood radius, Query vectors are generated through learnable linear transformations; for N ( i Input at any position within ) Generate key vectors and value vectors; Based on the query vector and key vector, obtain the basic attention score; Introduce a gating function to construct the gating signal: in, W g This indicates that the parameter is learnable. g ij Indicates position j Position i The intensity of information contribution; the final attention weight is: in, α ij Indicates the final attention weight. e ij Indicates the basic attention score. l Indicates the neighborhood range N ( i Any position in ); Based on the value vector and the final attention weights, the neighborhood-gated attention branch output is obtained: in, Indicates position i The output representation of the neighborhood-gated attention branch. v j Represents a value vector.
8. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding according to claim 1, characterized in that, Based on the hybrid sequence encoding output, entity output prediction using an improved span prediction mechanism includes: Based on the hybrid sequence encoding output, the span score matrix is obtained; Apply an upper triangular mask to the span score matrix to retain only candidate spans whose start position is no later than the end position; at the same time, use a valid bit mask to remove invalid candidates generated at the fill positions. During the decoding stage, candidate entity spans with scores higher than a preset threshold are selected, sorted from highest to lowest score, and a greedy strategy is used to remove conflicting or overlapping candidate spans. Finally, the entity recognition result is output, which includes: entity text, entity start position, entity end position, entity category, and confidence score.
9. The multi-view collaborative network security named entity recognition method integrating multi-source gating and hybrid coding as described in claim 8, characterized in that, Based on the hybrid sequence encoding output, obtaining the span score matrix includes: The hybrid sequence is encoded and output, mapped to the query vector and key vector space corresponding to each entity category, and a query representation is obtained. and key representation ,in Indicates the first c Class entity in position i The starting point is indicated at the location. Indicates the first c Class entity in position j The endpoint is indicated at the location. d h For entity head dimension; the first c Entities in span ( i , j The basic span score is: Calculate the position separately i As the first c Prior score of the starting point of the entity class, and its position. j As the first c Prior score of entity endpoint: in Indicates position i As the first c Prior score of the starting point of the class entity, Indicates position j As the first c Prior score of entity endpoint, and These are the linear mapping parameters for the start and end points, respectively. and For bias terms; Then, construct the type prior span score: Let the keyword tag sequence corresponding to the input word be . Z ={ z 1, z 2,…, z n }, n Given the length of the lexical sequence, map the keyword tags to the keyword semantic representation: Where Emb(∙) represents the keyword tag embedding function, The projection matrix; Calculate the bias scores of keywords for different entity categories at the start and end points respectively: in, and These are the keyword starting point projection matrix and the ending point projection matrix, respectively. and For bias terms, and The bias scores are for the start and end points, respectively. This yields the keyword bias span score: The base span score, type prior span score, and keyword bias span score are weighted and fused to obtain the final span score: in, λ 1 and λ 2 is a learnable scalar parameter.
