Unknown protocol message clustering method and device based on multi-factor feature fusion

Abstract

The embodiment of the invention discloses an unknown protocol message clustering method and device based on multi-factor feature fusion. A specific embodiment of the method comprises the following steps: generating a message grayscale image according to a collected original message flow data set; splicing the environment information factor, the global feature factor and the local feature factor to obtain a spliced feature vector; generating a data point distance set according to the splicing feature vector; determining the data point distance with the maximum value in each data point distance group in the data point distance group set to generate a core distance, and obtaining a core distance set; according to the core distance set, a hierarchical clustering tree is constructed, nodes of the hierarchical clustering tree represent clusters, and edges of the hierarchical clustering tree represent the merging relation between the clusters; and generating a clustering distribution result according to the hierarchical clustering tree. According to the embodiment, the problem of dimension disasters is avoided, and the stability of the clustering result is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a method and device for clustering unknown protocol packets based on multi-factor feature fusion. Background Art

[0002] With the increasing diversity and complexity of network protocols, reverse engineering of unknown protocols has become an important research area in network security. Protocol reverse engineering is a technique that analyzes the multi-dimensional features of protocol messages to restore the protocol's grammatical structure and reconstruct the finite state machine. Currently, clustering unknown protocol messages typically involves using binary protocols to achieve clustering.

[0003] However, the inventors have discovered that when clustering unknown protocol messages using the above method, the following technical problems often occur: Traditional protocol clustering methods typically rely on a single feature (such as packet length or field value), making them difficult to handle when packet lengths are variable or when similarities between sequences are complex. This makes it difficult to fully capture the essential differences in protocol behavior. Due to insufficient processing capabilities for high-dimensional features and a lack of effective dimensionality reduction methods, they are prone to the curse of dimensionality. Furthermore, their poor robustness to noisy data and outliers affects the stability of clustering results.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] Some embodiments of the present disclosure propose a method and device for clustering unknown protocol packets based on multi-factor feature fusion to solve the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a method for clustering unknown protocol packets based on multi-factor feature fusion, the method comprising: generating a packet grayscale image based on a collected original packet traffic data set; performing packet feature extraction on the above-mentioned packet grayscale image to generate a packet feature factor set, wherein the above-mentioned packet feature factor set comprises: an environmental information factor, a global feature factor, and a local feature factor; splicing the above-mentioned environmental information factor, the above-mentioned global feature factor, and the above-mentioned local feature factor to obtain a spliced ​​feature vector; generating a data point distance group set based on the above-mentioned spliced ​​feature vector; determining the data point distance of the maximum value in each data point distance group in the above-mentioned data point distance group set to generate a core distance and obtain a core distance set; constructing a hierarchical clustering tree based on the above-mentioned core distance set, wherein the nodes of the above-mentioned hierarchical clustering tree represent clusters, and the edges of the above-mentioned hierarchical clustering tree represent the merge relationship between clusters; generating a clustering allocation result based on the above-mentioned hierarchical clustering tree.

[0008] In a second aspect, some embodiments of the present disclosure provide an unknown protocol message clustering device based on multi-factor feature fusion, the device comprising: a first generating unit, configured to generate a message grayscale image based on a collected original message traffic data set; an extracting unit, configured to perform message feature extraction on the message grayscale image to generate a message feature factor set, wherein the message feature factor set comprises: an environmental information factor, a global feature factor, and a local feature factor; a splicing unit, configured to splice the environmental information factor, the global feature factor, and the local feature factor. A splicing feature vector is obtained; a second generating unit is configured to generate a data point distance group set based on the splicing feature vector; a determining unit is configured to determine the data point distance of the maximum value in each data point distance group in the data point distance group set to generate a core distance and obtain a core distance set; a constructing unit is configured to construct a hierarchical clustering tree based on the core distance set, wherein the nodes of the hierarchical clustering tree represent clusters, and the edges of the hierarchical clustering tree represent the merging relationship between clusters; a third generating unit is configured to generate a clustering assignment result based on the hierarchical clustering tree.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0011] The above-described various embodiments of the present disclosure have the following beneficial effects: The unknown protocol message clustering method based on multi-factor feature fusion in some embodiments of the present disclosure avoids the curse of dimensionality and improves the stability of clustering results. Specifically, the curse of dimensionality and the reduced stability of clustering results are caused by the fact that traditional protocol clustering methods typically rely on a single feature (such as message length or field value), which is difficult to handle when message length is not fixed and the similarity between sequences is complex, making it difficult to fully capture the essential differences in protocol behavior. Due to the insufficient processing power of high-dimensional features and the lack of effective dimensionality reduction methods, the curse of dimensionality is easily caused. Furthermore, the poor robustness to noisy data and outliers affects the stability of clustering results. Based on this, the original three-dimensional exhibition page of the unknown protocol message clustering method based on multi-factor feature fusion in some embodiments of the present disclosure first generates a message grayscale image based on the collected raw message traffic dataset. This can transform network traffic data that is originally difficult to intuitively understand into easy-to-process image data. Message features are extracted from the grayscale image of the message to generate a set of message feature factors. The set of message feature factors includes an environmental information factor, a global feature factor, and a local feature factor. Efficient integration of multi-level information is achieved through parallel feature extraction and a structured dimensionality reduction strategy. The environmental information factor, the global feature factor, and the local feature factor are concatenated to generate a concatenated feature vector. Density connectivity relationships in the feature space are automatically analyzed to eliminate low-density noise points and identify potential cluster structures. Based on the concatenated feature vector, a set of data point distance groups is generated. For each data point distance group in the set of data point distance groups, the maximum data point distance in the group is determined to generate a core distance, resulting in a core distance set. A hierarchical clustering tree is constructed based on the core distance set, where nodes represent clusters and edges represent cluster-to-cluster merging relationships. This avoids the curse of dimensionality and improves the stability of the clustering results. Cluster assignment results are generated based on the hierarchical clustering tree. This avoids the curse of dimensionality and improves the stability of the clustering results. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flow chart of some embodiments of the unknown protocol message clustering method based on multi-factor feature fusion according to the present disclosure; Figure 21 is a schematic structural diagram of some embodiments of the unknown protocol message clustering device based on multi-factor feature fusion according to the present disclosure; Figure 3 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure; Figure 4 This is a framework diagram of a multi-factor feature fusion message clustering method suitable for implementing some embodiments of the present disclosure; Figure 5 is a message formatting flow chart of some embodiments of the unknown protocol message clustering method based on multi-factor feature fusion according to the present disclosure; Figure 6 1 is a schematic diagram of protocol message image format conversion according to some embodiments of the unknown protocol message clustering method based on multi-factor feature fusion disclosed herein; Figure 7 is a diagram of a Dense Net model architecture suitable for implementing some embodiments of the present disclosure; Figure 8 is a diagram of a Vision Transformer model architecture suitable for implementing some embodiments of the present disclosure; Figure 9 is a schematic diagram of message length suitable for implementing some embodiments of the present disclosure; Figure 10 is a schematic diagram of message directions suitable for implementing some embodiments of the present disclosure; Figure 11 Schematic diagram of a UMAP dimensionality reduction algorithm suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0015] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0020] Figure 1 This is a process 100 of some embodiments of the unknown protocol message clustering method based on multi-factor feature fusion of some embodiments of the present disclosure. The unknown protocol message clustering method based on multi-factor feature fusion includes the following steps: Step 101: Generate a message grayscale image based on the collected original message traffic data set.

[0021] In some embodiments, an execution subject (eg, a computing device) of the unknown protocol message clustering method based on multi-factor feature fusion may generate a message grayscale image based on a collected original message traffic data set.

[0022] Here, the protocol message image format conversion diagram is as follows Figure 6 shown.

[0023] Optionally, the execution entity may generate a packet grayscale image based on the collected original packet traffic data set by performing the following steps: The first step is to clean the original packet traffic dataset to generate a cleaned packet traffic dataset.

[0024] As an example, the execution entity can remove packets with invalid payloads and abnormal frame structures from the original packet traffic dataset to generate a cleaned packet traffic dataset. For example, consider a network traffic dataset containing multiple protocols, some of which may be invalid due to validation failures or abnormal retransmissions. Through data cleansing, these invalid packets can be removed, retaining valid protocol packets.

[0025] In the second step, the packet lengths of the cleaned packet traffic dataset are aligned to generate an aligned packet traffic dataset.

[0026] As an example, the execution subject can align the packet lengths of the cleaned packet traffic dataset by zero-filling and truncation operations to generate an aligned packet traffic dataset. For example, suppose the length of one packet is 100 bytes and the other is 150 bytes. By zero-filling, the 100-byte packet is filled to 150 bytes. By truncation, the 150-byte packet is truncated to 100 bytes, and finally the length of all packets is unified to 100 bytes. Figure 5 shown.

[0027] Optionally, the execution entity may perform packet length alignment on the cleaned packet flow dataset through the following steps to generate an aligned packet flow dataset: In a first sub-step, in response to determining that there is cleaned message traffic data less than a preset message length in the cleaned message traffic data set, at least one cleaned message traffic data less than the preset message length is padded with zero values ​​to obtain a padded message traffic data set.

[0028] Here, the preset message length may refer to 28 bytes. The zero-value padding may refer to filling the portion less than 28 bytes with 0.

[0029] In the second sub-step, in response to determining that there is cleaned message flow data greater than the preset message length in the cleaned message flow data set, at least one cleaned message flow data greater than the preset message length is truncated to obtain a truncated message flow data set.

[0030] Here, the truncation may refer to removing a portion exceeding 28 bytes.

[0031] In a third sub-step, at least one cleaned message flow data set equal to the preset message length in the cleaned message flow data set, the padded message flow data set and the truncated message flow data set are determined as an aligned message flow data set.

[0032] The third step is to perform pixel mapping on the aligned packet traffic data set to generate a packet grayscale value set and obtain a packet grayscale image.

[0033] As an example, the execution entity can use data visualization technology to map a one-dimensional byte stream into a two-dimensional texture feature space and, through a pixel mapping mechanism, convert a discrete byte sequence into a grayscale image. For example, assuming a data packet contains the byte sequence [0x01, 0x02, 0x03, ..., 0xFF], each byte is mapped to a pixel's grayscale value, forming a grayscale image of the packet.

[0034] Step 102 : extracting message features from the message grayscale image to generate a message feature factor set, wherein the message feature factor set includes: an environmental information factor, a global feature factor, and a local feature factor.

[0035] In some embodiments, the execution entity may perform message feature extraction on the message grayscale image to generate a message feature factor set.

[0036] Optionally, the execution entity may perform message feature extraction on the message grayscale image through the following steps to generate a message feature factor set: In the first step, the grayscale image of the message is input into the deep network model to obtain the local feature vector.

[0037] Here, the above-mentioned deep network model can be used to capture local features of the message. The above-mentioned deep network model can refer to a Dense Net model architecture composed of dense feature extraction layers, dense connection modules and transition structures. Figure 7 As shown in the figure, the dense feature extraction layer uses convolution operations to extract primary features. The dense connection module, as the core of the network, achieves cross-layer feature reuse through a dense interconnection mechanism. The transition structure optimizes the spatial resolution and parameter scale of the feature map through pooling operations and 1×1 convolutions. From input to output, data undergoes initialization processing, alternating stacking of multiple dense blocks and transition layers, and ultimately completes the classification task. First, using a 7×7 convolution kernel, a preprocessing convolution operation is performed to achieve an initial transformation of the feature space, capturing the contour features of the packet protocol fields and the local pattern features of the encrypted content. Next, feature dimensionality reduction is performed through a maximum pooling operation, compressing the spatial dimensions of the feature map and improving the model's robustness to positional information. Data flows through a cascade structure consisting of multiple dense blocks and transition layers. The convolutional layers within each dense block use a dense connection mechanism to stack and integrate the multidimensional feature tensors output by the previous layers along the channel dimension, improving feature reuse efficiency and optimizing the gradient propagation path. The convolutional layer adopts a bottleneck architecture, first compressing the channel dimension using a 1×1 convolution kernel, and then modeling spatial features using a 3×3 convolution kernel. The transition layer reduces the number of channels using 1×1 convolutions, and further reduces the spatial dimension using a 2×2 average pooling layer to extract higher-level global features. Through this multi-layered structure, deep semantic information is fully extracted. Finally, a global average pooling operation is used to reduce and compress the high-dimensional feature tensor, converting it into a single feature vector.

[0038] The second step is to perform feature dimensionality reduction on the above local feature vectors to generate local feature factors.

[0039] As an example, the execution subject may perform feature dimensionality reduction on the local feature vector by using the UMAP dimensionality reduction algorithm to generate a local feature factor. Figure 11 As shown in Figure 2, UMAP, as a manifold learning method, can effectively capture these nonlinear structures and preserve the local details and global distribution characteristics of the data compared to other dimensionality reduction algorithms.

[0040] The third step is to input the above message grayscale image into the feature extraction model to obtain the global feature vector.

[0041] Here, the feature extraction model mentioned above can refer to the ViT model. The above feature extraction model can divide the grayscale image of the message into multiple sub-regions of equal size according to a regular grid, namely image blocks. Each sub-region is regarded as a "word". After these spatially discrete vectors are serialized, they are combined with spatial position encoding to form a feature sequence and input into the multi-layer Transformer architecture for feature learning. The architecture of the ViT model (Vision Transformer model) is as follows: Figure 8 As shown in Figure 2, the core architectural unit of the ViT model consists of multiple stacked encoder layers, each of which performs the crucial functions of feature extraction and information integration. This hierarchical structure primarily comprises three interrelated components: a multi-head attention (MHA) module, a feed-forward network (FFN) module, and a layer normalization (LN) module. The multi-head attention mechanism processes multiple attention heads in parallel, capturing diverse feature patterns from different subspaces, dynamically aggregating feature vectors and establishing long-range dependencies between elements. The outputs of each attention head are concatenated and linearly projected, then fused with the original input via a residual path and normalized to form the final output. This approach preserves the original information while enhancing feature representation. The feed-forward fully connected network consists of two linear transformation layers combined with a nonlinear activation function. The first linear projection layer expands features to a high-dimensional space, and after activation with a ReLU function, the second projection layer restores the feature dimensions. This "expand-contract" architectural design enables deep processing of the features output by the attention mechanism. Layer normalization applies normalization along feature dimensions, eliminating the impact of dimensional differences on training. This approach is particularly useful in scenarios where sequence lengths vary dynamically. Specifically, it performs mean-variance normalization on all feature dimensions of a single sample, then restores the representational power of the features through a learnable affine transformation. This approach, combined with residual connections, effectively suppresses gradient anomalies during deep network training and ensures the stability of model parameter updates.

[0042] The fourth step is to perform feature dimensionality reduction on the above global feature vector to generate a global feature factor.

[0043] As an example, the execution subject may perform feature dimensionality reduction on the global feature vector by using the UMAP dimensionality reduction algorithm to generate a global feature factor. Figure 11 As shown in Figure 2, UMAP, as a manifold learning method, can effectively capture these nonlinear structures and preserve the local details and global distribution characteristics of the data compared to other dimensionality reduction algorithms.

[0044] The fifth step is to extract the message length features from the above message grayscale image to generate a message length feature set.

[0045] like Figure 9 As shown in the figure, the blue part is the field information representing the message length.

[0046] In the sixth step, the data packet directional features of the above message grayscale image are one-hot encoded to obtain the encoded directional features.

[0047] Here, the directional characteristics of the data packet include uplink and downlink. The uplink and downlink relationship of the data packet is also an important factor affecting the clustering of data packet message types. The direction of the data packet (uplink / downlink) describes the behavioral roles of the two communicating parties. For example, the request-response mode usually shows a direction-sensitive length distribution feature: the client request is usually a short uplink packet, and the server response may be a long downlink packet. The diagram of the message direction is as follows Figure 10 As shown, data packets of the same type are often used in the same transmission direction.

[0048] Step 7: Generate an environmental information vector based on the above-mentioned message length feature set and the above-mentioned encoded direction feature.

[0049] Here, the length feature divides the data packet categories from the perspective of payload size, and the direction feature supplements the context information from the perspective of communication role, jointly constructing a more complete environmental information vector.

[0050] In the eighth step, the above environmental information vector is subjected to feature dimensionality reduction to generate an environmental information factor.

[0051] As an example, the execution subject may perform feature dimensionality reduction on the environment information vector using the UMAP dimensionality reduction algorithm to generate an environment information factor. Figure 11 As shown in Figure 2, UMAP, as a manifold learning method, can effectively capture these nonlinear structures and preserve the local details and global distribution characteristics of the data compared to other dimensionality reduction algorithms.

[0052] In the ninth step, the environmental information factor, the global characteristic factor and the local characteristic factor are determined as a message characteristic factor set.

[0053] Step 103: Concatenate the environmental information factor, the global feature factor, and the local feature factor to obtain a concatenated feature vector.

[0054] In some embodiments, the execution entity may concatenate the environmental information factor, the global feature factor, and the local feature factor to obtain a concatenated feature vector.

[0055] Here, the above-mentioned splicing may refer to connection.

[0056] Step 104: Generate a data point distance group set based on the above concatenated feature vector.

[0057] In some embodiments, the execution entity may generate a data point distance group set based on the concatenated feature vector.

[0058] Optionally, the execution entity may generate a data point distance group set based on the concatenated feature vector by the following steps: In the first step, the above concatenated feature vector is normalized to generate a normalized concatenated feature vector.

[0059] As an example, the execution entity may normalize the concatenated feature vector so that the value of each dimension is between 0 and 1. For example, assuming that the concatenated feature vector is [0.1, 0.2, ..., 1.0, 0.5, 0.6, ..., 0.9, 1, 0, 0, 0, 1, 0], the normalized value is [0.05, 0.1, ..., 0.5, 0.25, 0.3, ..., 0.45, 0.5, 0, 0, 0, 0.5, 0].

[0060] The second step is to construct a matrix of the above normalized concatenated feature vectors to generate a feature matrix.

[0061] As an example, the above execution entity can form a feature matrix by combining the normalized feature vectors of all messages. For example, assuming there are 1000 messages and the normalized feature vector of each message is 26-dimensional, a 1000×26 feature matrix is ​​constructed.

[0062] The third step is to determine the distance between each data point and the adjacent data point in the feature matrix to generate a data point distance group, thereby obtaining a data point distance group set.

[0063] As an example, the above execution entity can calculate the core distance of each data point in the feature matrix, that is, the distance from the point to its nearest neighbor within a specified minimum sample number range. Assuming the minimum sample number range is set to 5, for the first data point in the feature matrix, the distances to its second, third, fourth, and fifth nearest neighbors are determined to obtain a set of data point distances.

[0064] Step 105 : determining the maximum data point distance in each data point distance group in the data point distance group set to generate a core distance, thereby obtaining a core distance set.

[0065] In some embodiments, the execution entity may determine the maximum data point distance in each data point distance group in the data point distance group set to generate a core distance and obtain a core distance set.

[0066] Step 106: construct a hierarchical clustering tree based on the core distance set.

[0067] In some embodiments, the execution entity may construct a hierarchical clustering tree based on the core distance set, wherein the nodes of the hierarchical clustering tree represent clusters, and the edges of the hierarchical clustering tree represent merging relationships between clusters.

[0068] As an example, the above execution entity can construct a hierarchical clustering tree using an agglomerative hierarchical clustering method. Each node in the tree represents a cluster, and the edges represent the merging relationships between clusters. For example, points with close distances are gradually merged to form small clusters, and then the small clusters are merged into large clusters, ultimately constructing a hierarchical clustering tree.

[0069] Step 107: Generate a clustering assignment result based on the above hierarchical clustering tree.

[0070] In some embodiments, the execution entity may generate a clustering assignment result based on the hierarchical clustering tree.

[0071] like Figure 4 shown.

[0072] Optionally, the execution entity may generate a clustering assignment result according to the hierarchical clustering tree by performing the following steps: In the first step, noise points are identified on the above hierarchical clustering tree to generate a noise point dataset.

[0073] As an example, the execution subject may be in a hierarchical clustering tree, if the core distance of a data point X is much larger than the core distances of other points in the cluster, then X is regarded as a noise point.

[0074] The second step is to generate a non-noise data point set based on the above noise point data set.

[0075] As an example, the execution entity may remove the noise point data set from the hierarchical clustering tree to obtain a non-noise data point set.

[0076] The third step is to assign cluster labels to the above non-noise data point set to generate clustering assignment results.

[0077] As an example, the above execution entity can assign a cluster label to each non-noise data point in the non-noise data point set, and mark the noise points as a special category (for example, -1). For example, suppose three clusters are ultimately determined, namely cluster 1, cluster 2, and cluster 3. Data points belonging to cluster 1 are assigned label 1, data points belonging to cluster 2 are assigned label 2, data points belonging to cluster 3 are assigned label 3, and noise points are assigned label -1.

[0078] To comprehensively evaluate the performance of the proposed method, we compared it with four mainstream protocol reverse engineering tools: Netzob, a traditional method based on sequence alignment; Netplier, which focuses on key field extraction; CNNPRE, a deep learning method based on DenseNet-169; and Preunn, a method based on self-organizing map neural networks. These tools are representative in their respective fields, but when processing complex and noisy network traffic data, they may suffer from issues such as insufficient local information extraction and a strong reliance on a predetermined number of clusters.

[0079] The comparative experiments illustrate the advantages of the present invention in terms of protocol recognition accuracy and robustness. Each tool uses the default or recommended parameters of its publicly available version to ensure fairness in the comparison. The minimum cluster size of the key parameter of HDBSCAN of the present invention is set to 20, which has been verified by preliminary experiments to be well adapted to the density distribution of the data set. This evaluation selected silhouette coefficient, homogeneity, completeness, V-measure and execution time as measurement criteria, aiming to comprehensively examine the effectiveness of the clustering algorithm and the computational efficiency of the tool. In the comparative experiment, except for the method proposed in this study, all other tools used the preset parameters of their publicly available versions to ensure the fairness and objectivity of the comparison process. The clustering tool comparison experiment table is as follows: In terms of internal clustering validity evaluation, the silhouette coefficient of the present invention reached 0.65. This value not only surpasses Netzob (0.48) based on traditional sequence alignment and Preunn (0.45) based on self-organizing map, but also outperforms Netplier (0.50) focused on keyword recognition and CNNPRE (0.60) using a single convolutional neural network model. This result strongly proves that by deeply fusing heterogeneous visual features and traffic metadata derived from DenseNet and ViT, the method of the present invention can effectively improve the intra-class cohesion and inter-class separation of clustering results, thereby obtaining a more structurally reasonable cluster division.

[0080] In the external clustering validity assessment, the homogeneity index (0.70) of our method indicates that the generated clusters are highly pure, meaning that the majority of samples within the cluster belong to the same real-world protocol message type. A high completeness index (0.68) indicates that our method successfully groups message samples belonging to the same real-world category into the same cluster as completely as possible. In comparison, Netzob, Netplier, and Preunn performed relatively poorly on these two metrics. CNNPRE, however, achieved higher scores (0.68 for homogeneity and 0.61 for completeness) thanks to its deep feature extraction capabilities, but failed to surpass our method. These findings reveal the effectiveness of multi-factor feature fusion strategies in capturing the complex internal structure of protocol data. As a comprehensive measure of homogeneity and completeness, our method's V-measure (0.69) scored higher than Netzob (0.53), Netplier (0.58), and Preunn (0.50), and outperformed CNNPRE (0.64). This demonstrates that our method exhibits greater stability and overall superior performance when considering both the internal consistency of clustering results and the completeness of data coverage.

[0081] Regarding computational efficiency, Netplier exhibits the lowest execution time (23s) due to its non-deep learning characteristics. The running time of the method of the present invention (75s) is slightly higher than that of CNNPRE (65s). This phenomenon is in line with theoretical expectations. It is due to the fact that the method of the present invention integrates two computationally intensive deep learning models, DenseNet and ViT, in the feature extraction stage. Compared with the architecture of the single model of CNNPRE, the computational overhead has increased. However, the execution time of this method is still significantly lower than that of Netzob (95s), which has a higher computational complexity, and is on the same order of magnitude as the time cost of Preunn (70s). Although running time is not the only evaluation metric, in actual protocol reverse engineering scenarios, efficient computing speed can improve analysis efficiency, and this method achieves better running efficiency while ensuring clustering accuracy.

[0082] Comparative experimental results demonstrate that the proposed method, based on multi-factor feature fusion and the HDBSCAN clustering strategy, outperforms mainstream protocol reverse engineering tools in key clustering metrics such as silhouette coefficient, homogeneity, completeness, and V-measure. Although computational efficiency is slightly compromised due to the use of more advanced feature extraction techniques, it remains within an acceptable range for practical applications, and this compromise results in a substantial improvement in clustering quality. These results fully demonstrate the practicality and superiority of the proposed method for analyzing complex network traffic data, providing solid theoretical and practical support for reverse engineering of unknown protocols.

[0083] Further references Figure 2As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an unknown protocol message clustering device based on multi-factor feature fusion. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the unknown protocol message clustering device based on multi-factor feature fusion can be specifically applied to various electronic devices.

[0084] like Figure 2 As shown, in some embodiments, the unknown protocol message clustering device 200 based on multi-factor feature fusion includes: a first generating unit 201, an extracting unit 202, a splicing unit 203, a second generating unit 204, a determining unit 205, a constructing unit 206, and a third generating unit 207. The first generating unit 201 is configured to generate a message grayscale image based on the collected original message traffic data set; the extracting unit 202 is configured to perform message feature extraction on the message grayscale image to generate a message feature factor set, wherein the message feature factor set includes: an environmental information factor, a global feature factor, and a local feature factor; the splicing unit 203 is configured to splice the environmental information factor, the global feature factor, and the local feature factor to obtain a spliced ​​feature vector; the second generating unit 204 is configured According to the above-mentioned splicing feature vector, a data point distance group set is generated; the determination unit 205 is configured to determine the data point distance of the maximum value in each data point distance group in the above-mentioned data point distance group set to generate a core distance and obtain a core distance set; the construction unit 206 is configured to construct a hierarchical clustering tree according to the above-mentioned core distance set, wherein the nodes of the above-mentioned hierarchical clustering tree represent clusters, and the edges of the above-mentioned hierarchical clustering tree represent the merging relationship between clusters; the third generation unit 207 is configured to generate a cluster allocation result according to the above-mentioned hierarchical clustering tree.

[0085] It is understandable that the units recorded in the unknown protocol message clustering device 200 based on multi-factor feature fusion are similar to the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the unknown protocol message clustering device 200 based on multi-factor feature fusion and the units contained therein, and will not be described in detail here.

[0086] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, a computing device) suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Figure 3As shown, the computer device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions, which, when executed, may enable the processor to execute any of the above methods. The processor is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium, which, when executed by the processor, may enable the processor to execute any of the above methods. The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present disclosure, and does not constitute a limitation on the computer device to which the solution of the present disclosure is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0087] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0088] In one embodiment, the processor is used to run a computer program stored in a memory to implement the following steps: generating a message grayscale image based on a collected original message traffic data set; performing message feature extraction on the message grayscale image to generate a message feature factor set, wherein the message feature factor set includes: an environmental information factor, a global feature factor, and a local feature factor; splicing the environmental information factor, the global feature factor, and the local feature factor to obtain a spliced ​​feature vector; generating a data point distance group set based on the spliced ​​feature vector; determining the data point distance of the maximum value in each data point distance group in the data point distance group set to generate a core distance and obtain a core distance set; constructing a hierarchical clustering tree based on the core distance set, wherein the nodes of the hierarchical clustering tree represent clusters, and the edges of the hierarchical clustering tree represent the merge relationship between clusters; generating a cluster allocation result based on the hierarchical clustering tree.

[0089] An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the unknown protocol message clustering method based on multi-factor feature fusion mentioned above in the present disclosure.

[0090] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a flash memory card, etc., provided on the computer device.

[0091] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0092] The above descriptions are merely some preferred embodiments of the present disclosure and illustrate the underlying technical principles. Those skilled in the art should understand that the scope of the invention encompassed by the embodiments of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for clustering unknown protocol packets based on multi-factor feature fusion, characterized in that: include: Generate a packet grayscale image based on the collected original packet traffic data set; Performing message feature extraction on the message grayscale image to generate a message feature factor set, wherein the message feature factor set includes: an environmental information factor, a global feature factor, and a local feature factor; Splicing the environmental information factor, the global feature factor and the local feature factor to obtain a spliced ​​feature vector; generating a data point distance group set according to the concatenated feature vector; Determining the maximum data point distance in each data point distance group in the data point distance group set to generate a core distance, thereby obtaining a core distance set; Constructing a hierarchical clustering tree based on the core distance set, wherein the nodes of the hierarchical clustering tree represent clusters, and the edges of the hierarchical clustering tree represent the merging relationship between clusters; A cluster assignment result is generated according to the hierarchical clustering tree.

2. The method according to claim 1, characterized in that Generating a cluster assignment result according to the hierarchical clustering tree includes: performing noise point identification on the hierarchical clustering tree to generate a noise point data set; Generating a non-noise data point set according to the noise point data set; Cluster labels are assigned to the non-noise data point set to generate a cluster assignment result.

3. The method according to claim 1, characterized in that Generating a data point distance group set according to the spliced ​​feature vector includes: Normalizing the concatenated feature vector to generate a normalized concatenated feature vector; Performing matrix construction on the normalized concatenated feature vectors to generate a feature matrix; The distance between each data point and an adjacent data point in the feature matrix is ​​determined to generate a data point distance group, thereby obtaining a data point distance group set.

4. The method according to claim 1, wherein Generating a message grayscale image based on the collected original message traffic data set includes: Performing data cleaning on the original packet traffic data set to generate a cleaned packet traffic data set; Performing message length alignment on the cleaned message flow data set to generate an aligned message flow data set; Pixel mapping is performed on the aligned message traffic data set to generate a message grayscale value set and obtain a message grayscale image.

5. The method according to claim 4, characterized in that The step of aligning the message lengths of the cleaned message flow data set to generate an aligned message flow data set includes: In response to determining that there is cleaned message flow data less than a preset message length in the cleaned message flow data set, zero-filling is performed on at least one cleaned message flow data less than the preset message length to obtain a filled message flow data set; In response to determining that there is cleaned message flow data greater than the preset message length in the cleaned message flow data set, truncating at least one cleaned message flow data greater than the preset message length to obtain a truncated message flow data set; At least one cleaned message flow data set equal to the preset message length in the cleaned message flow data set, the padded message flow data set, and the truncated message flow data set are determined as an aligned message flow data set.

6. The method according to claim 1, characterized in that The extracting message features from the message grayscale image to generate a message feature factor set includes: Inputting the message grayscale image into a deep network model to obtain a local feature vector; Performing feature dimensionality reduction on the local feature vector to generate a local feature factor; Inputting the message grayscale image into a feature extraction model to obtain a global feature vector; Performing feature dimensionality reduction on the global feature vector to generate a global feature factor; Extracting message length features from the message grayscale image to generate a message length feature set; Performing one-hot encoding on the data packet directional features of the message grayscale image to obtain encoded directional features; generating an environmental information vector according to the message length feature set and the encoded direction feature; Performing feature dimensionality reduction on the environmental information vector to generate an environmental information factor; The environmental information factor, the global characteristic factor and the local characteristic factor are determined as a message characteristic factor set.

7. A device for clustering unknown protocol packets based on multi-factor feature fusion, characterized in that: include: The first generating unit is configured to generate a packet grayscale image based on the collected original packet traffic data set; an extraction unit configured to perform message feature extraction on the message grayscale image to generate a message feature factor set, wherein the message feature factor set includes: an environmental information factor, a global feature factor, and a local feature factor; a splicing unit configured to splice the environmental information factor, the global feature factor and the local feature factor to obtain a spliced ​​feature vector; A second generating unit is configured to generate a data point distance group set according to the concatenated feature vector; a determining unit configured to determine the maximum data point distance in each data point distance group in the data point distance group set to generate a core distance and obtain a core distance set; a construction unit configured to construct a hierarchical clustering tree according to the core distance set, wherein the nodes of the hierarchical clustering tree represent clusters, and the edges of the hierarchical clustering tree represent merging relationships between clusters; The third generating unit is configured to generate a cluster assignment result according to the hierarchical clustering tree.

8. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable medium, characterized in that A computer program is stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images