Protocol type adaptive clustering method and system for unknown message format

Abstract

The invention discloses an unknown message format-oriented protocol type adaptive clustering method, which comprises the following steps of: vectorizing a message segment sequence to obtain a two-dimensional vector corresponding to each segment of message; a message feature enhancement module is constructed, message vectors of different dimensions are unified through a message vector alignment technology and a hybrid expert self-attention mechanism, and the category specialization ability of the model during feature extraction is enhanced; based on self-adaptive clustering of comparative learning, generating an adjacent matrix by calculating message vector similarity and message format distance, performing Laplacian filtering by using the adjacent matrix to generate a comparative view, and obtaining a message vector for clustering through comparative learning; and performing clustering quality evaluation according to the message vectors to generate an optimal clustering number, and clustering the message vectors in combination with the optimal clustering number to obtain a clustering result. According to the method, the semantic features of the messages can be effectively extracted, and the problem that the message clustering effect is poor under the condition that the types of the messages are unknown is solved.

Description

Technical Field

[0001] The present invention relates to the fields of software engineering and artificial intelligence, and in particular to a protocol type adaptive clustering method and system for unknown message formats. Background Art

[0002] The cloud-edge-end architecture has become the core model of the power Internet of Things system. Its characteristics are that the end device resources are limited, while the edge devices and cloud servers undertake the main data processing and computing tasks. This distributed architecture significantly improves the efficiency and scalability of the power Internet of Things. However, the wide variety of communication protocols used by the traditional power Internet of Things mid-end devices has created a major obstacle to the migration and integration of end devices to the new system. Therefore, how to enable the new system to effectively identify and parse the messages of various protocols has become a key issue that needs to be solved urgently.

[0003] Traditional protocol analysis methods are highly dependent on protocol documents or manually designed parsing rules, which not only requires in-depth analysis of protocol details, but also faces the challenges of the surge in the number of IoT devices and the diversification of protocols, making manual parsing both impractical and inefficient. Against this background, protocol reverse engineering (PRE) came into being. Its basis lies in the effective clustering of unknown protocol messages. Accurate protocol clustering results can greatly facilitate the subsequent parsing of messages.

[0004] In recent years, many studies on the clustering of unknown protocol messages have made great progress. Among them, len-Kmeans uses the maxBIDE algorithm to mine the maximum frequent sequence in the protocol message set, and then calculates the fuzzy membership of each protocol message to each maximum frequent sequence according to the fuzzy set theory to construct the fuzzy membership vector. Finally, the k-means algorithm is used to divide different types of protocol messages into multiple clusters. NKW uses the PSS algorithm to perform coarse clustering on the test data set based on the continuity of the protocol format field and extracts the initial clustering center of the K-means algorithm. Then, the data is iteratively processed using the message distance and the improved convergence function to achieve further clustering of the messages. However, the message features extracted by these protocol clustering methods are relatively coarse-grained and cannot fully capture the semantic information in the message. At the same time, these methods usually rely on a predetermined number of clusters, which is not always available in real scenarios.

[0005] To address the above challenges, researchers proposed a network protocol identification method based on convolutional neural networks, which uses the powerful capabilities of neural networks to mine deep features in messages. However, this method faces the limitation of obtaining labeled sample data when training the message feature extraction module, and the number of unknown protocol types still needs to be pre-set when performing clustering analysis. FEAC analyzes the general structure of the protocol specification and then characterizes the unknown traffic into a protocol specification fusion vector (PSFV). After that, representation learning is used to refine the information of PSFV to compress the dimension and reduce the computational complexity. Finally, the refined PSFV and DBSCAN algorithm are combined to realize the protocol clustering of unknown traffic. However, the training process of the FEAC method is not end-to-end, resulting in inconsistent optimization between feature extraction and clustering tasks, which limits the improvement of overall clustering performance.

[0006] In summary, although the protocol clustering method has made significant progress in recent years, the existing solutions still have deficiencies in the deep extraction of message features and the clustering of protocol types of unknown types, which limits the improvement of clustering performance. To this end, the present invention proposes a protocol type adaptive clustering method and system for unknown message formats to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to propose a protocol type adaptive clustering method and system for unknown message formats, which can effectively deal with the problems of feature depth extraction and unknown number of types in protocol clustering tasks.

[0008] In order to achieve the above technical objectives, the technical solution adopted by the present invention is: In a first aspect, the present invention discloses a protocol type adaptive clustering method for unknown message formats, the method comprising the following steps: Vectorize the message byte sequence to obtain a two-dimensional vector corresponding to each message segment; Construct a message feature enhancement module, unify message vectors of different dimensions through message vector alignment technology and hybrid expert self-attention mechanism, and enhance the model's category specialization ability when extracting features; Adaptive clustering based on contrastive learning generates an adjacency matrix by calculating the message vector similarity and message format distance, and uses the adjacency matrix to perform Laplace filtering to generate a contrast view. The message vector used for clustering is obtained through contrastive learning. The optimal number of clusters is generated according to the clustering quality evaluation of the message vectors, and the message vectors are clustered based on the optimal number of clusters to obtain the clustering result.

[0009] Furthermore, the process of vectorizing the message byte sequence to obtain a two-dimensional vector corresponding to each message segment includes the following steps: For messages ,in is the number of bytes in the message, in bytes As units, by embedding the function Map each byte into a one-dimensional vector : ; In the formula Indicates that the size is The mapping relationship between bytes and vectors in the randomly initialized embedding matrix; represents the size of the input space, Indicates the dimension size of one-dimensional vector embedding; Stack the multiple one-dimensional vectors obtained to obtain the two-dimensional vector corresponding to each message segment : ; In the formula Represents a stacking operation, which concatenates multiple one-dimensional vectors into a two-dimensional vector by row.

[0010] Furthermore, a message feature enhancement module is constructed. The process of unifying message vectors of different dimensions through message vector alignment technology and hybrid expert self-attention mechanism includes the following steps: Construct the first hybrid expert model, which consists of the first gating layer and An independent multi-layer perceptron model is composed of two-dimensional vectors corresponding to each message segment. After filling to a uniform size, the filled message vector is divided multiple times through multiple preset segmentation schemes; after each division, the part that does not contain valid information is discarded, and the results obtained by each segmentation scheme are input into the corresponding multi-layer perceptron model for processing: ; ; in Represents the operation function of the multilayer perceptron model, represents the filling operation, Indicates The split operation of the split scheme, Indicates that according to After processing multiple segments by segmentation schemes, all All fragments in are of dimension A two-dimensional vector, represents the hidden layer size, Indicates the dimension size of one-dimensional vector embedding; Each filled two-dimensional vector Calculated through the first gating layer, outputting a Probability distribution: ; in, represents the gating layer, is a trainable weight matrix; Each element in corresponds to a multi-layer perceptron model The weight of the processing result, ; Based on the scores of the gating layer, the processing results of the multi-layer perceptron model are weighted and combined, and the aligned message vector is calculated as: ; in, is the operating function of the first hybrid expert model.

[0011] Furthermore, the process of enhancing the model's ability to specialize in categories when extracting features includes the following steps: For the aligned message vector , respectively, through three different linear transformation layers , , To generate the query matrix , key-value matrix and the value matrix : ; in , and The size of , and Performing matrix multiplication yields a matrix of size The feature weight matrix of After activation, with the value matrix Perform matrix multiplication to get the attention matrix: ; in represents the activation function, is a hyperparameter used to measure the attention weight, Represents a transpose operation; Construct a second hybrid expert model, which consists of the second gating layer and The attention matrix of each input is calculated through the second gating layer, and the feedforward layer that best matches the current input characteristics is selected. The gating mechanism calculates the activation score of each feedforward layer based on the input. : ; in and is the learnable parameter of the gating layer, and the elements in the activation score Corresponding to The scores of the feed-forward layers; Let the set of feed-forward layers be represented as , each feed-forward layer specializes the input attention matrix to generate intermediate outputs for specific categories: ; The feed-forward layer generates After the intermediate outputs, we get the intermediate output set , and then the activation scores calculated by the second gating layer are weighted and aggregated to generate the final feature representation: ; in Is the size of A one-dimensional vector of .

[0012] Furthermore, based on the adaptive clustering of contrastive learning, an adjacency matrix is ​​generated by calculating the message vector similarity and the message format distance, and the adjacency matrix is ​​used to perform Laplace filtering to generate a contrast view. The process of obtaining the message vector for clustering through contrastive learning includes the following steps: The message pair similarity matrix is ​​calculated by cosine similarity. The formula is as follows: ; Among them represents the inner product operation, represents the modulo operation, Indicates The characteristics of a message are represented by Indicates The characteristics of a message are represented by After calculating the cosine similarity between each pair of message feature representations, a shape of The message similarity matrix, Indicates the number of messages; Through adjustable threshold parameters Control the ratio of similarity matrix mapping to 1, and map the message pair similarity matrix to the adjacency matrix Specifically, the values ​​in the message pair correlation matrix are compared with the adjustable threshold parameter Compare and if the correlation value is greater than or equal to the adjustable threshold parameter , the element at the corresponding position is set to 1, indicating that there is an edge connection; if the correlation value is less than the adjustable threshold parameter , then the element at the corresponding position is 0, indicating no edge connection; By calculating the minimum operation cost required for conversion between all message pairs, a distance matrix is ​​constructed, and the threshold parameter is adjusted Control the ratio of the distance matrix mapping to 1 and map the distance matrix to the adjacency matrix Specifically, the values ​​in the distance matrix are compared with the adjustable threshold parameter Compare and if the distance value is greater than or equal to the adjustable threshold parameter , the element at the corresponding position is set to 0, indicating no edge connection; if the distance value is less than the adjustable threshold parameter , then the element at the corresponding position is 0, indicating that there is an edge connection; The embedding vectors of all messages are stacked to form a feature matrix: ; In the formula Indicates stacking operation; Using the adjacency matrix and the adjacency matrix , and their corresponding degree matrices and , respectively, for the feature matrix Perform Laplace filtering to generate two view feature matrices with different semantic association characteristics and : ; ; in Indicates the number of filtering times. represents the identity matrix; In both view feature matrices and Perform comparative learning between them, and then use linear combination to calculate the final feature matrix of the message: .

[0013] Furthermore, the process of generating an optimal clustering number according to the clustering quality evaluation of the message vector and clustering the message vector in combination with the optimal clustering number to obtain a clustering result includes the following steps: The current state Represented as a feature matrix and cluster status The combination of features matrix is the feature matrix generated after contrastive learning exist Status at the moment, cluster status The feature matrix The mean of Through the quality assessment module Output quality score vectors for different cluster numbers ; Based on the quality score vector , select the number of clusters at time t through a greedy strategy : ; in Indicates selection The one with the largest median value, Represents random probability. As training progresses, Gradually increase; Combined with the number of clusters , for the feature matrix Use k-means clustering to get the clustering results.

[0014] Furthermore, the quality assessment module is used to Output quality score vectors for different cluster numbers The process includes the following steps: Through the linear transformation layer and For the feature matrix and cluster status Perform linear transformation to extract more representative features; normalize the transformed results and use the activation function Perform activation and connect the two activation results together: ; in Represents a splicing operation; Using the output linear layer and Function to calculate quality score vector : .

[0015] Furthermore, the method further comprises: Using Cluster Status Perform quality assessment to get the number of clusters , for the feature matrix Use k-means clustering to calculate the overall clustering optimization loss function To update the byte embedding function , the first hybrid expert model, the second hybrid expert model, and the parameters in the contrastive learning module; after the parameters are updated, a new feature matrix is ​​obtained ; Based on the number of clusters For the feature matrix Use k-means clustering to get the cluster center; according to the obtained cluster center and feature matrix , calculate the clustering quality evaluation index , through clustering quality evaluation index To optimize the quality assessment module: ; in and Respectively represent and Cluster centers, Indicates that it belongs to The cluster center message vectors; calculate the quality assessment optimization loss function : .

[0016] Furthermore, the overall clustering optimization loss function is The calculation process includes the following steps: make represents the number of preset segmentation schemes, then Represents the effective information ratio of different segmentation schemes, combined with the weights generated by the gating layer , calculate the byte sequence information redundancy loss function : ; Among them represents the two-norm; Calculate the contrastive learning loss function for: ; in and Respectively represent Messages in two different views and The vector representation in is, is a hyperparameter for similarity scaling, Indicates the number of messages. There are two views: A message vector represents, Refers to the division vector representation Other The message vector represents Vector representation; Combination The number of clusters at a time , for the feature matrix Use k-means clustering to get the cluster center ; Clustering loss function for: ; ; ; in is the feature matrix Middle The vector representation of a message, Indicates the calculation of KL divergence, represents the cluster distribution, Indicates a sharp clustered distribution; right , , Perform weighted averaging to obtain the overall clustering optimization loss function , based on the overall clustering optimization loss function To update the model: ; In the formula, , and are hyperparameters that control the contribution of each term in the loss.

[0017] In a second aspect, the present invention discloses a protocol type adaptive clustering system for unknown message formats, the system comprising a cluster number determination module and a clustering module; The clustering module vectorizes the message byte sequence to obtain a two-dimensional vector corresponding to each message segment; constructs a message feature enhancement module through message vector alignment technology and hybrid expert self-attention mechanism; adaptive clustering based on contrastive learning obtains message vectors for clustering through contrastive learning, and performs k-means clustering on the message vectors according to the optimal number of clusters generated by the cluster number module to obtain clustering results; the cluster number module performs clustering quality assessment on the message vectors generated in the clustering module, and generates optimal clusters according to the quality assessment results.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The method and system for adaptive clustering of protocol types for unknown message formats of the present invention effectively realizes deep extraction of message features through message sequence vectorization, message feature enhancement, and comparative learning, supports adaptive clustering of protocol types through a quality assessment module, improves the performance of protocol type clustering, and further solves the problem of poor message clustering effect when the message type and type are unknown. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a protocol type adaptive clustering method for unknown message formats; Figure 2The principle framework diagram of the protocol type adaptive clustering method for unknown message formats; Figure 3 A clustering visualization result diagram provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0021] like Figure 1 As shown, the present invention discloses a protocol type adaptive clustering method for unknown message formats, including message vectorization, message feature enhancement, and adaptive clustering based on contrastive learning. The present invention first vectorizes the message byte sequence to obtain a high-dimensional, dense vector representation to serve the subsequent message feature enhancement module; secondly, through the message vector alignment technology and the hybrid expert self-attention mechanism, the message vectors of different dimensions are unified, and the category specialization ability of the model when extracting features is enhanced; then, the adjacency matrix is ​​generated by calculating the message vector similarity and the message format distance, and these matrices are used to perform Laplace filtering to generate a contrast view, and the message vectors used for clustering are obtained through contrastive learning; finally, the optimal number of clusters is generated according to the clustering quality evaluation of the message vector, and the message vector is clustered in combination with the optimal number of clusters to obtain the clustering result. The present invention effectively extracts the semantic features contained in the message, can automatically determine the optimal number of clusters, and achieves the performance improvement of the protocol type adaptive clustering task.

[0022] like Figure 2 The figure shows the structure diagram of the protocol type adaptive clustering system for unknown message formats of the present invention, including message sequence vectorization, message vector low-redundancy alignment module, hybrid expert self-attention mechanism, comparative learning module for constructing message similarity view and distance view, and adaptive clustering module.

[0023] S1, message sequence vectorization.

[0024] The message byte sequence is vectorized to obtain a high-dimensional, dense vector. First, for the message ,in is the number of bytes in the message, in bytes As units, by embedding the function Map each byte into a one-dimensional vector , Indicates the dimension size of the vector embedding: ; In the formula Indicates that the size is The mapping relationship between bytes and vectors in the randomly initialized embedding matrix, Indicates the size of the input space. During the model training process, the embedding matrix will be continuously optimized to learn a better embedding strategy. Then, multiple one-dimensional vectors are stacked to obtain a two-dimensional vector corresponding to each message segment. .

[0025] ; In the formula Represents a stacking operation, which concatenates multiple one-dimensional vectors into a two-dimensional vector by row.

[0026] S2, message feature enhancement module.

[0027] Align message vectors of different lengths, and then use the hybrid expert self-attention module to enhance message features.

[0028] (1) Unify message vectors of different dimensions through message vector alignment technology. Figure 2 As shown in the figure, the message vector alignment technology includes multi-scheme segmentation and MLP mapping as well as segmentation scheme weight calculation. After filling the two-dimensional message vector to a uniform size, the message vector is divided multiple times according to the preset multiple segmentation schemes. After each division, the part that does not contain valid information is discarded, and the results of each segmentation scheme are input into an independent multi-layer perceptron (MLP) model for processing: ; ; in represents the filling operation, Indicates The split operation of the split scheme, Indicates that according to After processing multiple segments by segmentation schemes, all All fragments in are of dimension A two-dimensional vector, Represents the hidden layer size.

[0029] The weight calculation of the segmentation scheme is used to fuse the two-dimensional vectors of different segmentation schemes, and the mixture of experts (MoE) model is used to calculate the weights of the fusion for different processing results. MoE consists of a gating layer and multiple independent MLPs mentioned above, which are called "experts". Each filled two-dimensional vector Will be calculated through the gating layer and output a Probability distribution: ; in, represents the gating layer, is a trainable weight matrix, The number of experts represented. Each element in corresponds to an expert The weight of the processing results. Based on the scores of the gating layer, the processing results of the experts are weighted and combined, and the aligned vector is calculated as: .

[0030] (2) By hybridizing the expert self-attention mechanism, the model’s category-specific ability in feature extraction is enhanced.

[0031] Figure 2 Self-Attention in the message vector , respectively, through three different linear transformation layers , , To generate the query matrix , key-value matrix and the value matrix : ; in , and The size of . and Performing matrix multiplication yields a matrix of size The feature weight matrix. After activation, with the value matrix Perform matrix multiplication to get the attention matrix: ; in represents the activation function, is a hyperparameter used to measure the attention weight, Represents a transpose operation.

[0032] Figure 2 The Mixture of Experts (MoE) model in consists of a gating layer and a series of FFNs, which are called "experts". The attention matrix of each input is calculated through the gating layer to select the expert module that best matches the current input characteristics. The gating mechanism calculates the activation score of each expert based on the input : ; in and is the learnable parameter of the gating network. The elements in this activation score Corresponding to The scores of experts. Let the expert set be represented as , represents the number of experts, each of whom specializes the input attention matrix to generate intermediate outputs for specific categories: ; The feed-forward layer generates After the intermediate outputs, we get the intermediate output set , and then perform weighted aggregation on these intermediate outputs through the activation scores calculated by the gating network to generate the final feature representation: ; in Is the size of A one-dimensional vector of .

[0033] S3, Adaptive clustering based on contrastive learning.

[0034] Construct message similarity views and distance views, obtain message vectors for clustering through comparative learning between views, perform clustering quality assessment on message vectors to generate the optimal number of clusters, and cluster message vectors based on the optimal number of clusters to obtain clustering results.

[0035] (1) Contrastive learning. The adjacency matrix is ​​generated by calculating the message vector similarity and message format distance, and these matrices are used for Laplace filtering to generate contrast views. The message vectors used for clustering are obtained through contrastive learning between views.

[0036] Each message will get a one-dimensional vector after being processed by the message feature enhancement module , the one-dimensional vectors of multiple messages form a message feature matrix. The similarity between message pairs in the message feature matrix is ​​calculated by cosine similarity to form a similarity matrix. The formula is as follows: ; Among them represents the inner product operation, represents the modulo operation, . Through an adjustable threshold parameter Controls the proportion of similarity matrix mapping to 1. Compare the values ​​in the correlation matrix with the threshold. If the correlation value is greater than or equal to the threshold, the element at the corresponding position is set to 1; if the correlation value is less than the threshold, the element at that position is 0.

[0037] The similarity between bytes is measured by the degree of overlap of functions that the bytes of a message may have. and , and their function sets are and By calculating the intersection and union of these two sets, the Jaccard similarity coefficient can be used to express their similarity: ; in represents the difference set, represents the intersection, Indicates the number of elements in a set. On this basis, the Jaccard distance is used to represent their differences: ; After obtaining the differences between bytes, we can further use dynamic programming to calculate the minimum operation cost required to convert one byte sequence into another byte sequence. The formula is as follows: ; in Indicates that the first byte sequence is Bytes are converted to the first The minimum cost of bytes, Indicates that the first byte sequence Insert the second byte after the The cost of bytes, Indicates to delete the first By calculating the minimum operation cost required for conversion between all message pairs, a distance matrix is ​​constructed. Controls the proportion of distance matrix mapping to 1. Compare the value in the distance matrix with the threshold. If the distance value is greater than or equal to the threshold, the element at the corresponding position is set to 0; if the distance value is less than the threshold, the element at that position is 1.

[0038] The embedding vectors of all messages are stacked to form a feature matrix: ; Using two different adjacency matrices and , and their corresponding degree matrices and , respectively, for the feature matrix Laplacian filtering is performed to generate two contrasting views with different semantic association characteristics: ; ; in Indicates the number of filtering times. Represents the identity matrix.

[0039] After the above Laplace filtering process, we get two different view feature matrices and We calculate the final feature matrix of the message through the linear combination of the two views. The formula is: .

[0040] (2) Adaptive clustering. First, calculate the feature matrix The average value of the cluster status , current status is the feature matrix and cluster status Then the quality assessment module is used to Output quality estimates for different numbers of clusters .

[0041] ; ; in Represents a splicing operation, and is the input linear layer, is the output linear layer, is the activation function.

[0042] exist middle, The number of clusters is The clustering quality estimated at . Based on the quality score vector , select the number of clusters through a greedy strategy , the formula is: ; in It means to select the item with the largest value in the vector. Represents probability. In the early stages of training, a smaller random probability It can promote the model to explore more possible cluster numbers. As the training progresses, It will gradually increase, making the model more inclined to select the optimal number of clusters generated by the quality assessment module.

[0043] Combined with the number of clusters , for the feature matrix Use k-means clustering to get the clustering results.

[0044] (3) Joint training: Joint training is used to achieve overall model optimization, including clustering optimization and quality assessment optimization.

[0045] Clustering optimization includes three parts: byte sequence information redundancy loss function when aligning message vectors, contrastive learning loss function, and clustering loss function. The three loss functions are combined to obtain the joint loss, and the model is updated based on the joint loss.

[0046] After the segmentation operation in the message vector alignment, the ratio of the valid information in the segmented fragments is calculated to obtain Combined with the weights generated by the MoE gating layer , byte sequence information redundancy loss function as follows: ; Among them Denotes the bi-norm. Contrastive learning loss function for: ; in and Respectively represent Messages in two different views and The vector representation in is, is a hyperparameter for similarity scaling, Indicates the number of messages. There are two views: A message vector represents, Refers to the division vector representation Other The message vector represents Vector representation.

[0047] The number of clusters generated by the quality assessment module at time t , for the feature matrix Using k-means clustering, the cluster center can be obtained after clustering Clustering loss function as follows: ; ; ; in is the feature matrix Middle The vector representation of a message, Indicates the calculation of KL divergence, represents the cluster distribution, Represents a sharpened cluster distribution. The overall cluster optimization loss function is yes , , The weighted average of: ; In the formula, , and are hyperparameters that control the contribution of each term in the loss.

[0048] A clustering-oriented quality evaluation index To optimize the quality assessment module: ; in and Respectively represent and Cluster centers, Indicates that it belongs to The cluster center message vector. First, use the state Perform quality assessment to get the number of clusters , for the feature matrix Using k-means clustering, we then calculate Update the model. After the model is updated, we get The new feature matrix at each moment . Then continue with is the number of clusters, and the feature matrix Use k-means clustering to get the cluster centers. Based on these cluster centers and feature matrices , calculate the clustering quality evaluation Finally, the quality assessment optimizes the loss function as follows: ; At this point, the training optimization process of the invention has been calculated. All experiments were carried out on a server running Windows 11 (64-bit), equipped with an NVIDIA GeForce GTX 2080 Ti graphics processing unit (GPU) and 64gb of memory. PyTorch and Python were used to implement it, and the Adam optimizer was used during the training process. In order to evaluate the present invention, six protocol messages were captured and preprocessed from a real network environment, including DNS, HTTP, Modbus, OICP, s7, and smtp. A part of the message from each protocol was selected to form a data set for testing. It was compared with six benchmark methods, including: UPGMA, K-means, DBSCAN, IDFD, DSIMVC, and DML. The present invention uses the Adam optimizer with a learning rate of 0.0001. The vector dimension mapped to the byte is 50, the preset segmentation scheme is [50, 100, 200, 300, 500], the number of rows after the message vector is aligned is 400, and the number of experts is 5. The number of Laplace filters is 2. and Set to 0.025, Set to 1, and Set to 0.5, It is set to 1. Homogeneity, completeness, and v-measure scores are used as evaluation indicators to compare the performance of different methods.

[0049] (1) Performance comparison experiment. The values ​​of homogeneity, completeness, and v-measure of the clustering results of the present invention and various baselines on the same data set are shown in Table 1. The method proposed in the present invention (UPACM) performs best in the three evaluation indicators of homogeneity, completeness, and v-measure, reaching 94.58%, 92.17%, and 93.36%, respectively. Further analysis shows that the performance of deep learning-based methods (IDFD and DML) is better than that of traditional machine learning methods (UPGMA, K-means, and DBSCAN), which shows that deep learning has more advantages in feature extraction and can effectively mine the potential information of messages, highlighting the effectiveness of deep learning methods. Among deep learning methods, DSIMVC focuses on filling in missing data in views, so the effect is slightly inferior; while DML and the present method, which focus on optimizing feature representation, perform better, highlighting the importance of optimizing feature representation. Compared with DML, this method eliminates redundant information when vectorizing messages. The superiority of the results proves that after removing redundant information, the model can more accurately capture the core features of the message, thereby improving the learning effect.

[0050] (2) Ablation experiment. As shown in Table 2, the role of the main modules in the present invention is analyzed through ablation studies. In the table, w / o-contrastive means that the model with the contrastive learning module removed, w / o-vector alignment means that after the message is vectorized, all two-dimensional message vectors are padded to a uniform size, and w / o-feature enhancement means that the self-attention hybrid expert model is not used to enhance the features. It can be seen from Table 2 that after losing the contrastive learning module, the overall effect of the model decreases, which shows that the contrastive learning module can capture the intrinsic structure and characteristics of the data, and these features can reflect a more comprehensive data pattern. If the removal of redundant information is not considered and the message vectors are aligned using a simple padding method, the overall effect of the model will also decrease. It can be seen that the redundant information contained in the padded vector has an impact on feature extraction. Finally, the removal of the feature enhancement module also leads to a decrease in the overall effect of the model, which can prove that the feature enhancement module can reduce the feature interference between different categories and significantly improve the distinguishing ability of feature representation. In general, compared with UPACM, the other three models are slightly insufficient in the ability to model message features, and their performance has declined.

[0051] (3) Visualization analysis experiment. In order to further demonstrate the effectiveness of the adaptive clustering of protocol types proposed in the present invention and visualize it, the clustering results are mapped to a two-dimensional space through the t-SNE dimensionality reduction algorithm (t-SNE Visualization), and each type of protocol (class0 to class5) is displayed in a different color, as shown in Figure 2. Figure 3 As shown, the horizontal and vertical coordinates 1 and 2 represent the mapping position of the protocol in the two-dimensional space. Figure 3 It can be seen that the same type of protocol messages are closely clustered, and there are obvious boundaries between clusters, which shows the distinguishability between different protocol types, thus fully verifying the effectiveness of the proposed method.

[0052] In summary, the present invention proposes a new protocol type adaptive clustering method for unknown message formats. First, the message byte sequence is vectorized to obtain a high-dimensional, dense vector representation to serve the subsequent message feature enhancement module; secondly, through the message vector alignment technology and the hybrid expert self-attention mechanism, the message vectors of different dimensions are unified, and the category specialization ability of the model in extracting features is enhanced; then, the adjacency matrix is ​​generated by calculating the message vector similarity and the message format distance, and these matrices are used for Laplace filtering to generate a comparison view, and the message vectors for clustering are obtained through comparative learning; finally, the optimal cluster number is generated according to the clustering quality evaluation of the message vector, and the message vector is clustered in combination with the optimal cluster number to obtain the clustering result. Finally, a large number of experiments of the SOTA method were carried out on a data set consisting of 6 protocol messages to verify the effectiveness of the present invention.

[0053] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0054] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0055] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0056] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0057] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0058] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A protocol type adaptive clustering method for unknown message formats, characterized in that: The method comprises the following steps: Vectorize the message byte sequence to obtain a two-dimensional vector corresponding to each message segment; Build a message feature enhancement module that unifies message vectors of different dimensions through message vector alignment technology and a hybrid expert self-attention mechanism, and enhances the model's category-specific capabilities when extracting features. Adaptive clustering based on contrastive learning generates an adjacency matrix by calculating message vector similarity and message format distance. This adjacency matrix is ​​then used to perform Laplace filtering to generate a contrast view. Contrastive learning is then used to obtain the message vectors used for clustering. The optimal number of clusters is generated based on the clustering quality evaluation of the message vectors, and the message vectors are clustered based on the optimal number of clusters to obtain the clustering result.

2. The protocol type adaptive clustering method for unknown message formats according to claim 1 is characterized in that: The process of vectorizing the message byte sequence to obtain the two-dimensional vector corresponding to each message segment includes the following steps: For messages ,in is the number of bytes in the message, in bytes As a unit, by embedding the function Map each byte into a one-dimensional vector : ； In the formula Indicates that the size is The mapping relationship between bytes and vectors in the randomly initialized embedding matrix; represents the size of the input space, Indicates the dimension size of one-dimensional vector embedding; Stack the multiple one-dimensional vectors obtained to obtain the two-dimensional vector corresponding to each message segment : ； In the formula Represents a stacking operation, which concatenates multiple one-dimensional vectors into a two-dimensional vector by row.

3. The protocol type adaptive clustering method for unknown message formats according to claim 1, characterized in that: The process of building a message feature enhancement module and unifying message vectors of different dimensions through message vector alignment technology and hybrid expert self-attention mechanism includes the following steps: Construct the first hybrid expert model, which consists of the first gating layer and An independent multi-layer perceptron model is composed of two-dimensional vectors corresponding to each message segment. After filling to a uniform size, the padded message vector is divided multiple times using multiple preset segmentation schemes. After each division, the part that does not contain valid information is discarded, and the results obtained by each segmentation scheme are input into the corresponding multi-layer perceptron model for processing: ； ； in Represents the operation function of the multilayer perceptron model, Represents a filling operation, Indicates the The split operation of the split scheme, Indicates that according to After processing multiple segments by the segmentation scheme, all All fragments in are of dimension A two-dimensional vector, represents the hidden layer size, Indicates the dimension size of one-dimensional vector embedding; Each filled two-dimensional vector Calculate through the first gating layer and output a Probability distribution: ； in, represents the gating layer, Is a trainable weight matrix; Each element in corresponds to a multilayer perceptron model The weight of the processing results, ; Based on the scores of the gating layer, the processing results of the multi-layer perceptron model are weighted and combined, and the aligned message vector is calculated as: ； in, is the operating function of the first hybrid expert model.

4. The protocol type adaptive clustering method for unknown message formats according to claim 1, characterized in that: The process of increasing the model's ability to specialize in categories when extracting features involves the following steps: For the aligned message vector , respectively, through three different linear transformation layers , , To generate the query matrix , key-value matrix and the value matrix : ； in 、 and The size of , and Performing matrix multiplication yields a matrix of size The feature weight matrix of After activation, with the value matrix Perform matrix multiplication to get the attention matrix: ； in represents the activation function, is a hyperparameter used to measure the attention weight, Represents a transpose operation; Construct the second hybrid expert model, which consists of the second gating layer and The attention matrix of each input is calculated through the second gating layer, and the feedforward layer that best matches the current input characteristics is selected. The gating mechanism calculates the activation score of each feedforward layer based on the input. : ； in and is the learnable parameter of the gating layer, and the elements in the activation score Corresponding to The score of the feed-forward layer; Let the set of feedforward layers be represented as , each feed-forward layer specializes the input attention matrix and generates intermediate output for a specific category: ； Feed-forward layers generate After the intermediate output, we get the intermediate output set , and then perform weighted aggregation on the intermediate outputs through the activation scores calculated by the second gating layer to generate the final feature representation: ； in Is the size of A one-dimensional vector of .

5. The protocol type adaptive clustering method for unknown message formats according to claim 1, characterized in that: Adaptive clustering based on contrastive learning generates an adjacency matrix by calculating message vector similarity and message format distance. Laplace filtering is then performed on the adjacency matrix to generate a contrast view. The process of obtaining message vectors for clustering through contrastive learning includes the following steps: The message pair similarity matrix is ​​calculated using cosine similarity. The formula is as follows: ； Among them represents the inner product operation, represents the modular operation, Indicates the The characteristics of a message are represented by Indicates the The characteristics of a message are represented by After calculating the cosine similarity between each pair of message feature representations, a shape is formed. The message similarity matrix, Indicates the number of messages; Through adjustable threshold parameters Control the ratio of similarity matrix mapping to 1, and map the message pair similarity matrix to the adjacency matrix Specifically, the values ​​in the message pair correlation matrix are compared with the adjustable threshold parameters Compare and if the correlation value is greater than or equal to the adjustable threshold parameter , then the element at the corresponding position is set to 1, indicating that there is an edge connection; if the correlation value is less than the adjustable threshold parameter , then the element at the corresponding position is 0, indicating no edge connection; By calculating the minimum operation cost required for conversion between all message pairs, a distance matrix is ​​constructed, and the threshold parameter is adjusted Control the ratio of the distance matrix mapping to 1 and map the distance matrix to the adjacency matrix Specifically, the values ​​in the distance matrix are compared with the adjustable threshold parameter Compare, if the distance value is greater than or equal to the adjustable threshold parameter , then the element at the corresponding position is set to 0, indicating no edge connection; if the distance value is less than the adjustable threshold parameter , then the element at the corresponding position is 0, indicating that there is an edge connection; The embedding vectors of all messages are stacked to form a feature matrix: ； In the formula Indicates stacking operation; Using the adjacency matrix and the adjacency matrix , and their corresponding degree matrices and , respectively for the feature matrix Perform Laplace filtering to generate two view feature matrices with different semantic association characteristics and : ； ； in Indicates the number of filtering times, represents the identity matrix; In both view feature matrices and Perform comparative learning between them, and then use linear combination to calculate the final feature matrix of the message: 。 6. The protocol type adaptive clustering method for unknown message formats according to claim 1, characterized in that: The process of performing clustering quality assessment based on message vectors to generate the optimal number of clusters and clustering the message vectors based on the optimal number of clusters to obtain the clustering result includes the following steps: The current state Represented as a feature matrix and cluster status Combination, feature matrix is the feature matrix generated after contrastive learning exist Status at the moment, cluster status The characteristic matrix The mean of Through the quality assessment module Output quality score vectors for different cluster numbers ; Based on the quality score vector , select the number of clusters at time t through the greedy strategy : ； in Indicates selection The one with the largest median value, Represents random probability. As training progresses, Gradually increase; Combined cluster number , for the feature matrix Use k-means clustering to obtain clustering results.

7. The protocol type adaptive clustering method for unknown message formats according to claim 6, characterized in that: Through the quality assessment module Output quality score vectors for different cluster numbers The process includes the following steps: Through the linear transformation layer and For the feature matrix and cluster status Perform linear transformation to extract more representative features; normalize the transformed results and use the activation function Perform activation and concatenate the two activation results together: ； in Represents a splicing operation; Use output linear layer and Function to calculate quality score vector : 。 8. The protocol type adaptive clustering method for unknown message formats according to claim 7, characterized in that: The method further comprises: Using Cluster Status Perform quality assessment to get the number of clusters , for the feature matrix Use k-means clustering to calculate the overall clustering optimization loss function To update the byte embedding function , the first hybrid expert model, the second hybrid expert model, and the parameters in the contrastive learning module; after the parameters are updated, a new feature matrix is ​​obtained ; Based on the number of clusters For the feature matrix Use k-means clustering to get the cluster center; according to the obtained cluster center and feature matrix , calculate the clustering quality evaluation index , through clustering quality evaluation index To optimize the quality assessment module: ； in and Respectively represent and cluster centers, Indicates that it belongs to The first cluster center message vectors; calculate the quality assessment optimization loss function : 。 9. The protocol type adaptive clustering method for unknown message formats according to claim 8, characterized in that: The overall clustering optimization loss function The calculation process includes the following steps: make Represents the number of preset segmentation schemes, then Represents the effective information ratio of different segmentation schemes, combined with the weight generated by the gating layer , calculate the byte sequence information redundancy loss function : ； Among them represents the two-norm; Calculate the contrastive learning loss function for: ； in and Respectively represent Messages in two different views and The vector representation in , is a hyperparameter for similarity scaling, Indicates the number of messages, which can be found in two views. The message vector is represented by Refers to the division vector representation Others The message vector represents Vector representation; Combine Number of clusters at a time , for the feature matrix Use k-means clustering to get the cluster center ; Clustering loss function for: ； ； ； in is the feature matrix Middle The vector representation of a message, Indicates the calculation of KL divergence, represents the cluster distribution, Indicates a sharp cluster distribution; right 、 、 Perform weighted averaging to obtain the overall clustering optimization loss function , based on the overall clustering optimization loss function Perform model update: ； Where, 、 and are hyperparameters that control the contribution of each term in the loss.

10. A protocol type adaptive clustering system for unknown message formats based on the method according to any one of claims 1 to 9, characterized in that: The system includes a module for determining the number of clusters and a clustering module; The clustering module vectorizes the message byte sequence to obtain a two-dimensional vector corresponding to each message segment; through the message vector alignment technology and the hybrid expert self-attention mechanism, a message feature enhancement module is constructed; Adaptive clustering based on contrastive learning obtains message vectors for clustering through contrastive learning, and performs k-means clustering on the message vectors according to the optimal number of clusters generated by the cluster number module to obtain clustering results; the cluster number module performs clustering quality assessment on the message vectors generated in the clustering module and generates the optimal cluster based on the quality assessment results.

Images