Host asset surveying and mapping method based on protocol stack fingerprints

Abstract

The invention relates to the technical field of digital information transmission, in particular to a protocol stack fingerprint-based host asset surveying and mapping method, which comprises the following steps of: designing a host discovery algorithm of a combined multi-protocol data packet by referring to an open system interconnection seven-layer network model layering protocol; performing feature analysis and data preprocessing based on a protocol stack fingerprint rule base of a network mapper transmission control protocol, and constructing a protocol stack fingerprint data set of a host asset operating system; the host asset operating system fingerprint identification classification model is constructed based on the random forest algorithm, the coverage rate of survival host discovery is remarkably improved, and the limitation of a traditional rule-based matching method is effectively overcome. According to the method, the accuracy rate in the fingerprint identification task of the coarse-grained operating system reaches 97%, the weighted F1 value in the fingerprint identification task of the fine-grained operating system reaches 0.87, the technical level of host asset surveying and mapping is improved, and a more comprehensive and accurate technical path is provided for network asset security assessment.

Description

Technical Field

[0001] The present invention relates to the technical field of digital information transmission, and particularly to a method for mapping host assets based on protocol stack fingerprints. Background Art

[0002] In the field of network security offense and defense, host assets carry key business functions, such as user databases, confidential files, and production data, and the inherent kernel vulnerabilities of the host operating system are important factors for its compromise. To block potential vulnerability risks in advance, fingerprint recognition of the host asset operating system becomes crucial. However, in an environment where there are a large number of host assets in the target network and the security protection is complete, host asset mapping based on a single protocol probe packet faces significant challenges.

[0003] Currently, host asset fingerprint recognition based on active detection mainly relies on the response protocol stack fingerprint rule library of network scanners and is achieved by matching the identifiers in the protocol stack response data packet headers. However, the presence of target network security protection devices makes it crucial to efficiently discover live hosts rather than perform full-network segment detection. At the same time, a large number of detection packets may affect network services, resulting in high costs. Compared with the characteristics of application layer HTTP protocol response messages, it is difficult to understand the characteristics of transmission layer underlying protocol response data packets, and existing protocol stack fingerprint rule libraries have problems such as untimely updates, high maintenance costs, and insufficient generalization ability, making it difficult to cope with the rapid iteration of operating system versions. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for mapping host assets based on protocol stack fingerprints, aiming to solve the problems of incomplete coverage of live host discovery by single protocol detection and insufficient generalization ability of rule matching based on response message characteristics in host asset operating system fingerprint recognition during the host asset mapping process.

[0005] To achieve the above purpose, the present invention provides a method for mapping host assets based on protocol stack fingerprints, including the following steps; Design a host discovery algorithm for combining multi-protocol packets by referring to the hierarchical protocols of the Open Systems Interconnection seven-layer network model; Based on the Transmission Control Protocol stack fingerprint rule library of Network Mapper, perform feature analysis and data preprocessing to construct a protocol stack fingerprint dataset of the host asset operating system; Construct a classification model for host asset operating system fingerprint recognition based on the random forest algorithm.

[0006] Among them, the multi-protocol packet includes the Internet Control Message Protocol in the network layer, the Transmission Control Protocol and User Datagram Protocol in the transport layer, the Hypertext Transfer Protocol / Hypertext Transfer Security Protocol, Secure Shell Protocol, and File Transfer Protocol in the application layer.

[0007] Among them, the Internet Control Message Protocol (ICMP) can directly detect communication based on the Internet Protocol address of the host, and choose to use the raw payload of the ICMP to send detection data packets. In terms of the transport layer protocol and application layer protocol, the corresponding common ports of each protocol are respectively selected as the ports for sending data packets when detecting the target host.

[0008] Among them, the fingerprint rule library captures the combined differences of fingerprint rule samples among feature sets. By means of Cartesian product combination and exhaustive enumeration, it extracts the complete feature space of fingerprint rule records in the combination range of feature set values, and uses one-hot encoding to convert multi-class labels into binary matrices, and encodes the numerical type features.

[0009] Among them, the initial fingerprint recognition classification model consists of 100 decision trees, with no limit on the decision depth, to capture the fingerprint feature patterns of the Transmission Control Protocol (TCP) stack in the complex transport layer of the encoded dataset.

[0010] The method for mapping host assets based on protocol stack fingerprints of the present invention refers to the hierarchical protocol of the Open Systems Interconnection (OSI) seven-layer network model, and designs a host discovery algorithm for combining multi-protocol data packets; based on the Nmap TCP protocol stack fingerprint rule library, it conducts feature analysis and data preprocessing to construct a protocol stack fingerprint dataset of the host asset operating system; based on the random forest algorithm, it constructs a fingerprint recognition classification model for the host asset operating system. This method significantly improves the coverage rate of live host discovery through the host discovery algorithm for combining multi-protocol data packets; based on the Nmap TCP protocol stack fingerprint rule library, it conducts feature analysis and data preprocessing to construct a TCP protocol stack fingerprint dataset of the host asset operating system; and uses the random forest algorithm to construct a fingerprint recognition classification model, effectively overcoming the limitations of the traditional rule matching method. The accuracy rate of this method in the coarse-grained operating system fingerprint recognition task reaches 97%, and the weighted F1 value in the fine-grained operating system fingerprint recognition task reaches 0.87. It not only improves the technical level of host asset mapping, but also provides a more comprehensive and accurate technical path for network asset security assessment, and solves the problems of incomplete coverage of live host discovery by single-protocol detection and insufficient generalization ability of rule matching based on response message features in host asset operating system fingerprint recognition during the process of host asset mapping. Description of the Drawings

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic diagram of the host discovery algorithm for combining multi - protocol data packets.

[0013] Figure 2 It is a schematic diagram of the host asset operating system fingerprint classification based on random forest.

[0014] Figure 3 It is a schematic diagram of the training process of the operating system fingerprint recognition model with different granularities.

[0015] Figure 4 It is a schematic diagram of the original rule set structure.

[0016] Figure 5 It is a flowchart of the host asset mapping method based on protocol stack fingerprint provided by the present invention. Detailed implementation manners

[0017] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present invention and should not be construed as a limitation to the present invention.

[0018] Please refer to Figures 1 to 5 , the present invention provides a host asset mapping method based on protocol stack fingerprint, including the following steps; S1 Refer to the hierarchical protocols of the Open System Interconnection seven - layer network model to design a host discovery algorithm for combining multi - protocol data packets; In the embodiment of the present invention, by referring to the hierarchical protocols of the Open System Interconnection (OSI) seven - layer network model, the Internet Control Message Protocol (ICMP) in the network layer, the Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) in the transport layer, the Hypertext Transfer Protocol / Hypertext Transfer Security Protocol (HTTP / HTTPS), Secure Shell Protocol (SSH), and File Transfer Protocol (FTP) in the application layer are used as the request payload protocol set for detecting the survival of target host assets. According to the protocol characteristics of different levels, the original payload of the ICMP protocol is used for detection, and common ports are selected as the detection packet - sending ports in the transport layer and application layer protocols to ensure the comprehensiveness and effectiveness of detection.

[0019] In the host asset fingerprint recognition work of this method, the detection of host liveness is crucial for the subsequent fingerprint recognition and mapping of target host assets. In the stage of detecting the liveness of target hosts, request payloads are designed with reference to network protocols at different layers of the OSI seven-layer network model. In the networked target host assets, considering the differences in the functions of network protocols at different layers, such as the network layer ICMP protocol responsible for network diagnosis and error reporting, the transport layer TCP protocol responsible for end-to-end data transmission, and the application layer HTTP protocol responsible for web data transmission and communication, etc. To make full use of the protocol implementations of target hosts at different network layers, combined with active detection technology, first, this method selects the ICMP protocol in the network layer, the TCP protocol and UDP protocol in the transport layer, the HTTP / HTTPS protocol, SSH protocol, and FTP protocol in the application layer as the request payload protocols for detecting the liveness of target host assets. Secondly, since the ICMP protocol can directly detect and communicate based on the host IP, the original payload of the ICMP protocol is selected for packet sending detection. For the transport layer protocol and application layer protocol, the commonly used ports corresponding to each protocol are respectively selected as the detection packet sending ports of the target host. The commonly used target ports of each protocol are shown in Table 1.

[0020] Table 1 Commonly Used Host Ports of Each Network Protocol To sum up, the defined 7 protocols at 3 levels are the request payload protocol set: , and the detection set of target ports for each protocol is .

[0021] The host discovery algorithm that combines multi-protocol data packets solves the problem that traditional detection methods in the target network are prone to missing live hosts. First, due to the differences in the complexity of protocol implementations at different layers, the time-consuming for host-to-host interaction, and the request payloads, this method uses the bottom-up hierarchical method of the OSI network protocol model as the sending order of request data payloads at different layers. Secondly, considering the filtering rules of security device rules in the target network for frequent sending of detection data packets from the same source host, this algorithm adds a random detection packet sending interval with a time range of [0, 5] seconds during the detection process to alleviate rule filtering. Further, in the detection part of the TCP protocol in the transport layer, the half-open scanning method is adopted to reduce the time-consuming required for the TCP three-way handshake between communicating hosts and reduce the possibility of triggering rule filtering. The GET method is selected for the HTTP and HTTPS protocols in the application layer to simplify the complexity of the request payload of the detection data packet and the data transmission burden between the communication with the target host. Let the set of target hosts to be detected be , and the set of detected live hosts be . The host discovery algorithm that combines multi-protocol data packets is as shown in Figure 1 .

[0022] S2 performs feature analysis and data preprocessing based on the network mapper transmission control protocol stack fingerprint rule library, and constructs a protocol stack fingerprint data set for the operating system of the host assets. In the embodiment of the present invention, aiming at the semi-structured data characteristics of the original label set and fingerprint rule set, through regular matching and standardization processing, the unstructured fingerprint rules are converted into structured feature data, and a protocol stack fingerprint feature set for the operating system of the host assets is constructed. To capture the combined differences of fingerprint rule sample records among feature sets, the complete feature space of the combination of fingerprint rule records in the value range of feature sets is extracted by means of Cartesian product combination exhaustion. Finally, one-hot encoding is used to convert the multi-class labels into binary matrices, and the numerical type features are encoded for preparing data for subsequent model input.

[0023] This method uses the operating system fingerprint label set and the original fingerprint rule set of the network scanner Network Mapper (Nmap) as the original data sets of the operating system fingerprint labels and protocol stack fingerprint features respectively. Among them, since the original label set and fingerprint rule set are semi-structured data, in the data standardization stage, the unstructured fingerprint rules need to be converted into structured feature data. The original operating system label set DS OS_Label and the protocol stack fingerprint rule set DS OS_Finger are structured as Figure 4 shown.

[0024] In the standardization process of the original operating system label set, label information extraction is performed on the label set DS OS_Label through regular matching. Since the label set structure consists of the manufacturer set S Manufa , the operating system type set S OSType , the version set S OSVersion , and the device type set S Device , the standardized operating system label set is defined as shown in Formula 1.

[0025] (1) Since there are differences in the actual usage ratios of different types of operating systems, there is a situation of class imbalance in the sample data contained in the rule set DS OS_Finger . This method selects the top 5 labels in terms of the number of classes in the label set as the coarse-grained label set of the operating system fingerprint, and the classes containing version information under it as the fine-grained label set, which are defined as L OSMajor and L OSMinor respectively. Among them, L OSMajor = {Andorid, BSD, Linux, Solaris, Windows}. According to the coarse-grained label set LOSMajor Filter out category labels outside the category range filtering label range to obtain the target category operating system label set , where .

[0026] In the feature standardization process of the fingerprint rule set, since the content starting with "#", "Fingerprint", and "CPE" has nothing to do with the operating system fingerprint rule, it needs to be removed first before constructing the structured fingerprint rule set to obtain the fingerprint rule containing the fingerprint category and the original TCP / IP protocol stack identifier. Secondly, in the original TCP / IP protocol stack identifier part, this method selects seven types of TCP / IP protocol stack identifiers, namely Sequence Number Features (SEQ), TCP Options (OPS), TCP Window Size (WIN), Explicit Congestion Notification (ECN), TCP Test Responses (T1~T7), UDP Test Response (U1), and ICMP Echo (IE), to obtain the protocol stack fingerprint feature set F of the host asset operating system ProtoSta ={SEQ, OPS, WIN, ECN, T1~T7, U1, IE}. Finally, since the feature set F ProtoSta contains nested and refined protocol stack feature fields, the structured feature set F of the extended protocol stack fingerprint rule is constructed based on the refined feature fields ProtoStaExt ={SEQ Ext , OPS Ext , WIN Ext , ECN Ext , (T1~T7) Ext , U1 Ext , IE Ext}, and the corresponding feature value set is V ProtoStaExt . In the sample record standardization process of the fingerprint rule set, since the feature value set of the structured feature set F ProtoStaExt is V ProtoStaExt and there are multiple possible values in it, in order to capture the combined differences in the values of the fingerprint rule sample records among the feature set V ProtoStaExt , the value range of the refined feature values in the original protocol stack fingerprint rule set DS OS_Finger is exhausted in a Cartesian product combination manner to extract the complete feature space of the value range combination of the fingerprint rule records among the feature set V ProtoStaExt . Standardized protocol stack fingerprint rule set It can be defined as shown in Formula 2.

[0027] (2) The quality of the data cleaning result will directly affect the learning effect of the subsequent model on the protocol stack fingerprint data features. Therefore, after obtaining the standardized protocol stack fingerprint rule set in the redundant feature processing, features with zero variance in floating-point type and integer type are removed to reduce the complexity of the subsequent model input data and remove noise features. In the sample record filtering, fingerprint rule samples that do not belong to the target category operating system label set are removed to obtain the feature value set after label filtering . In the sample record cleaning, since the structure types of the refined protocol stack feature identifiers in the feature value set are respectively hexadecimal character range tuple type, character type, numerical type, and null type (Not a Number, NaN).

[0028] According to the specific meaning identification of the TCP / IP protocol stack feature identifier, in the fingerprint rule set the feature values of the tuple type need to be converted into decimal numerical tuples for representation, and a random number in this range is taken as the feature value; the character type and numerical type retain the original values; in the null type, for character and numerical features, "NO" is selected as a placeholder to fill and represent the information missing state, and duplicate items in the sample are removed to complete the data cleaning of the standardized protocol stack fingerprint rule set .

[0029] To improve the classification performance of the subsequent model on minority classes and prevent overfitting on majority classes, in the sample class balance processing, the fingerprint rule samples of each class are balanced for the majority class by combining a preset sampling ratio with a random undersampling strategy, and at the same time, a sample quantity threshold strategy is used to merge and balance the minority class. The field explanation of the fingerprint feature set F ProtoStaExt after the standardization of the protocol stack fingerprint data is shown in Table 2.

[0030] Table 2 TCP protocol stack feature set Field meaning The structured protocol stack fingerprint data set after the class balance of fingerprint rules is as shown in Formula 3.

[0031] (3) In the data set After cleaning, it needs to be encoded before it can be used as input for subsequent model data. In label encoding, first, the multi-class labels are converted into a binary matrix using one-hot encoding. Second, the numerical type features are rounded down and converted into character type, combined with one-hot encoding to complete the feature encoding. Structured protocol stack fingerprint dataset , and the encoded protocol stack fingerprint dataset is .

[0032] S3 constructs a host asset operating system fingerprint recognition classification model based on the random forest algorithm.

[0033] In the embodiment of the present invention, by constructing a classification model based on random forest, the complexity and generalization ability problems of traditional classification models in dealing with high-dimensional TCP protocol stack fingerprint features are effectively solved. The optimal features are selected based on the minimum Gini impurity criterion, and the node splitting threshold and the minimum number of samples in the leaf node are set to ensure the generalization ability of the model. The optimal hyperparameters are found through GridSearchCV grid search, the early stopping strategy is used to prevent overfitting, and 5-fold cross-validation is used to achieve efficient and accurate recognition of coarse-grained and fine-grained operating system fingerprints.

[0034] In the design part of the host asset operating system fingerprint recognition classification model of this method, an operating system protocol stack fingerprint recognition model based on random forest is constructed to handle the recognition and classification tasks of coarse-grained and fine-grained operating system protocol stack fingerprints. In the model structure design, the model structure initially consists of 100 decision trees, and there is no limit on the decision depth to capture the complex TCP protocol stack fingerprint feature patterns in the encoded dataset. The splitting of each decision tree is based on the minimum Gini Impurity criterion, and its calculation formula is shown in Formula 4, where S represents the sample set of the current node, c represents the number of categories, and represents the probability that the sample belongs to the i-th category. By minimizing the Gini impurity, the model can select the optimal features for splitting, so as to effectively capture the complex patterns in the data.

[0035] (4) To prevent the classification model from over-splitting on extremely small data subsets, the model sets the minimum sample threshold for controlling the splitting of decision tree nodes to 2. At the same time, the minimum number of samples in the leaf nodes of the tree is controlled to be 1, ensuring that each leaf node contains at least 1 sample. Through the above parameter settings and initializations of the model, a benchmark random forest classification model is constructed, as Figure 2 shown.

[0036] During the model training process, first, to find the optimal model parameters of the random forest classifier, the GridSearchCV grid search method is used to find the optimal hyperparameters. Second, an early stopping strategy is adopted to prevent the model from overfitting and reduce the training time of the classification model. Finally, 5-fold cross-validation is used in the stability verification test of the model to verify the stability of the model's performance on the TCP protocol stack fingerprint dataset. Through the above model structure design and training strategy, the identification and classification tasks of fine-grained and coarse-grained operating system fingerprints can be effectively achieved. Based on the design of the operating system fingerprint recognition model structure using random forest, for the identification and classification tasks of operating system fingerprints with different granularities, first, the encoded dataset is divided into a label set S L and a feature set S F . Second, the stratified sampling method is used to divide the dataset into a training set Set Train and a test set Set Test , which are used as the data input for the classifier. Finally, the random forest classifier is initialized and the model is fitted on the training set Set Train to generate multiple decision trees and learn the data features of the encoded protocol stack fingerprints. Combining the model training strategy designed in the previous section, the optimal classification model for fitting fingerprint data is found to achieve the identification and classification tasks of TCP protocol stack fingerprints of operating systems with different granularity categories. The training process of the operating system fingerprint recognition model with different granularities is as shown in Figure 3 .

[0037] The above-disclosed is only the preferred embodiment of the host asset mapping method based on protocol stack fingerprints of the present invention. Of course, the scope of the rights of the present invention cannot be limited by this. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. A host asset mapping method based on protocol stack fingerprint, characterized in that: The steps include: Referring to the open system interconnection seven-layer network model layered protocol, design a host discovery algorithm that combines multi-protocol data packets; Based on the protocol stack fingerprint rule base of the network mapper transmission control protocol, feature analysis and data preprocessing are performed to build the protocol stack fingerprint dataset of the host asset operating system; A host asset operating system fingerprint recognition and classification model is constructed based on the random forest algorithm.

2. The host asset mapping method based on protocol stack fingerprint as claimed in claim 1, It is characterized by: The multi-protocol data packet includes the Internet Control Message Protocol in the network layer, the Transmission Control Protocol and the User Datagram Protocol in the transport layer, the Hypertext Transfer Protocol / Hypertext Transfer Security Protocol, the Secure Shell Protocol and the File Transfer Protocol in the application layer.

3. The host asset mapping method based on protocol stack fingerprint as claimed in claim 2, characterized in that; The Internet Control Message Protocol can directly perform detection communication based on the Internet Protocol address of the host, and choose to use the original payload of the Internet Control Message Protocol to send detection data packets. In terms of transport layer protocol and application layer protocol, the commonly used ports corresponding to each protocol are selected as the ports for sending data packets when detecting the target host.

4. The host asset mapping method based on protocol stack fingerprint as claimed in claim 1, characterized in that ； The fingerprint rule library captures the combination differences of fingerprint rule sample records between feature sets, extracts the complete feature space of fingerprint rule records in the feature set value range combination through Cartesian product combination exhaustive enumeration, converts multi-classification labels into binary matrices using one-hot encoding, and encodes numerical type features.

5. The host asset mapping method based on protocol stack fingerprint as claimed in claim 1, characterized in that ； The fingerprint recognition classification model initially consists of 100 decision trees, with no restriction on decision depth, to capture the fingerprint feature patterns of the transport control protocol stack of the complex transport layer in the coded data set.

Images