A passive operating system identification method based on active learning

Abstract

The present invention discloses a passive operating system identification method based on active learning, comprising the following steps: using Wireshark to extract the fingerprint of network traffic to obtain a fingerprint data set; preprocessing the fingerprint data set, including separating the operating system type, IP and TCP signatures, and using an oversampling algorithm BorderLineSMOTE and an undersampling algorithm TomeLink to combine and sample the balanced fingerprint data set; inputting the preprocessed fingerprint data set into a fingerprint recognition model obtained by pre-training to obtain a recognition result output by the fingerprint recognition model, wherein the fingerprint recognition model is obtained by active learning training; the passive operating system identification method based on active learning provided by the present invention reduces the demand for the number of data sets, solves the problem that the classification results of machine learning tend to be biased towards the operating system category that accounts for a relatively large proportion, and thus improves the recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and in particular to a passive operating system identification method based on active learning. Background Art

[0002] As the scale of network infrastructure continues to expand, the complexity of the network is also increasing, network management has become more difficult, and network security issues have become increasingly prominent. Therefore, it is necessary to collect detailed information about the dynamic characteristics and complexity of large heterogeneous networks in order to discover potential security issues. Operating system identification technology is one of the key technologies for network security assessment and protection, and operating system fingerprinting is the process of remotely detecting the operating system of the target device. The operating systems of different network devices are diverse. The type and version of the operating system running on the remote system provides important information to the intruder. Once an unsafe version of the operating system is detected, the device is vulnerable to remote attacks. Therefore, it is of great significance to study the problem of remote network host operating system (OS) detection, that is, operating system fingerprinting technology.

[0003] There are two main ways to identify an operating system: active and passive. Active detection is to actively send test packets to the target device, analyze the TCP / IP protocol stack in the response packet, and thus detect OS information. The most popular active operating system identification tool is Nmap. Although the active detection method is faster, it injects additional traffic and is generally considered an attack behavior of the host, so the sent detection packets are easily intercepted by firewalls or intrusion detection systems. Passive detection is to check and analyze the packets transmitted between terminal hosts without injecting any traffic into the network. It sets a detector at the network node, captures the traffic data flowing through the node by sniffing, and then performs packet analysis on the captured traffic to obtain OS information. Passive sniffing only relies on information extracted from regular TCP / IP packet headers. The most popular passive tools currently include p0f, Satori, and NetworkMiner. Satori is a passive operating system fingerprinting tool rewritten in Python.

[0004] Operating system fingerprint recognition has gradually evolved from the traditional recognition method based on fingerprint library feature matching to the recognition method based on machine learning. With the development of machine learning technology, more and more researchers are combining machine learning methods with operating system fingerprint recognition to improve fingerprint recognition capabilities. However, machine learning has also exposed a series of problems. For example, when faced with an unbalanced data set, the prediction results of machine learning may be biased towards the operating system category with a larger proportion; in order to obtain a higher accuracy rate, a large amount of training data must be input; the performance on large data sets is relatively weak and has a high time complexity. Summary of the invention

[0005] The purpose of the present invention is to provide a passive operating system recognition method based on active learning, so as to achieve model training with less operating system labeled data and obtain higher recognition accuracy.

[0006] A first aspect of an embodiment of the present invention provides a passive operating system identification method based on active learning, comprising the following steps:

[0007] Use Wireshark to extract the fingerprint of network traffic to obtain the fingerprint dataset;

[0008] Preprocessing the fingerprint dataset includes separating the operating system type, IP and TCP signatures, and using an oversampling algorithm BorderLineSMOTE and an undersampling algorithm TomeLink to sample and balance the fingerprint dataset;

[0009] The preprocessed fingerprint data set is input into a pre-trained fingerprint recognition model to obtain a recognition result output by the fingerprint recognition model, wherein the fingerprint recognition model is obtained through active learning training.

[0010] Preferably, the method of extracting the fingerprint of network traffic using Wireshark to obtain the fingerprint data set is specifically as follows:

[0011] The TCP / IP data packets are filtered by the filter function of Wireshark, and the TCP / IP data packets are read and parsed by the Python scapy library, and the fingerprint data is extracted from the captured traffic information PCAP file to obtain a fingerprint data set.

[0012] Preferably, the preprocessing of the fingerprint data set includes separating the operating system type, IP and TCP signatures, and using the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink to sample the balanced fingerprint data set, including:

[0013] Separate the operating system type, IP and TCP signatures, wherein the operating system type includes at least one of Windows 10, Windows 7 or 8, Linux 2.2.x, Linux 3.x, Windows Server 2003, Windows XP, Windows Server 2008, and Windows NT kernel 5.x;

[0014] The separated operating system type is encoded as a label, and the TCP signature is discretized, wherein each discretized item is an independent feature, and the items obtained after the TCP signature is discretized include: at least one of the message lifetime ttl, the window size wsize, the option length olen, the payload classification pclass, the version ver, the maximum segment size mss, the window scale scale, the placeholder nop, the selective confirmation sack, the timestamp ts, the option list end eol, the df flag, the flow label, and the id number, wherein the placeholder nop, the maximum segment size mss, the selective confirmation sack, the timestamp ts, and the option list end eol belong to the TCP optional part, and the df flag, the flow label, and the id number belong to the abnormal feature item quirks part;

[0015] The labels contained in the fingerprint dataset are divided into majority and minority classes according to the sample proportions, the minority class is supplemented by the oversampling algorithm BorderLineSMOTE, and the samples in the majority class are eliminated by the undersampling algorithm TomeLink to ensure that overfitting and important data are not lost.

[0016] Preferably, it also includes:

[0017] According to the characteristics of each attribute, the four features of packet lifetime ttl, window size wsize, option length olen, and payload classification pclass are set to numeric type, and the features of version ver, maximum segment size mss, window scale scale, TCP optional part, and abnormal feature item quirks part are set to object type;

[0018] The numeric features are dimensionless, and the original data is converted into data with a mean of 0 and a variance of 1;

[0019] For the object classification features, the mode is used to fill the missing values ​​and the features are encoded as numbers. The features of the TCP optional part and the abnormal feature item quirks part use 1 to represent the presence of the item and 2 to represent the absence of the item. The version ver, maximum segment size mss and window scale are one category corresponding to one number.

[0020] Preferably, the training of the fingerprint recognition model includes:

[0021] Set the acquired fingerprint dataset and network traffic as the unlabeled sample pool U n , from the unlabeled sample pool U n Randomly select fingerprint samples to mark and obtain the initial fingerprint training set T i ;

[0022] Construct classifier C according to the classification algorithm m ;

[0023] The initial fingerprint training set T i Input classifier C m The initial classification model is trained to obtain the initial classification model, and the unlabeled sample pool U is classified by the initial classification model. n Classify unlabeled fingerprint samples in;

[0024] Find the unlabeled sample pool U through different query strategies and rules n The fingerprint sample U with the largest number of features a , and according to the manual a Marking

[0025] For the classifier C m Update the marked fingerprint sample U a Add to the initial fingerprint training set T i , update the initial fingerprint training set T i Make T n =T i U a , update the unlabeled sample pool U n make U n →U n / U a ; Update the initial fingerprint training set T n Input to classifier C m Train in to get a new classification model;

[0026] Repeat the above process and continuously update the classification model until the model convergence conditions are met.

[0027] The present invention has at least the following beneficial effects:

[0028] The distribution of operating systems in operating system fingerprint recognition is uneven, and there is a problem of unbalanced data sets, which will cause the classification results of machine learning to be biased towards operating system categories that account for a large proportion. The present invention provides a passive operating system recognition method based on active learning, which uses the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink combined sampling to balance the fingerprint data set, thereby solving the problem that the classification results of machine learning are biased towards operating system categories that account for a large proportion, thereby improving the recognition accuracy;

[0029] In the study of operating system fingerprint recognition, it is often very difficult to obtain large data sets, and when the data set is large, data labeling is also extremely difficult. The present invention provides a passive operating system recognition method based on active learning, which separates the operating system type, IP and TCP signatures to make the distinction of fingerprint data more detailed, reduce the demand for the number of data sets, and improve the recognition accuracy.

[0030] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of a passive operating system identification method based on active learning provided by an embodiment of the present invention;

[0032] Figure 2 Schematic diagrams of 6 common operating system fingerprints of the present invention;

[0033] Figure 3 It is a comparison chart of the recognition results of the first five classification algorithms of the data balance of the present invention;

[0034] Figure 4 It is a comparison diagram of the recognition results after the parameters of the five classification algorithms of the present invention are adjusted to the optimal value;

[0035] Figure 5 It is a comparison chart of the recognition results of five classification algorithms after data balancing of the present invention;

[0036] Figure 6 It is a comparison chart of the running time of five classification algorithms after data balancing of the present invention;

[0037] Figure 7 It is a schematic diagram of the accuracy of five classification algorithms of the present invention at different iteration times when the training set uses 1% of the original training set;

[0038] Figure 8 It is a comparison chart of the accuracy of five classification algorithms at different numbers of iterations after active learning of the present invention;

[0039] Fig. 9 It is a comparison chart of the running time of five classification algorithms at different numbers of iterations after active learning of the present invention. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0041] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0042] Combination Figure 1-Figure 9 To illustrate this embodiment,

[0043] This embodiment is a passive operating system identification method based on active learning, comprising the following steps:

[0044] Use Wireshark to extract the fingerprint of network traffic to obtain the fingerprint dataset;

[0045] Preprocess the fingerprint dataset, including separating the operating system type, IP and TCP signatures, and use the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink to sample and balance the fingerprint dataset;

[0046] The preprocessed fingerprint data set is input into the pre-trained fingerprint recognition model to obtain the recognition result output by the fingerprint recognition model, and the fingerprint recognition model is obtained through active learning training.

[0047] The distribution of operating systems in operating system fingerprint recognition is uneven, and there is a problem of unbalanced data sets, which will cause the classification results of machine learning to be biased towards operating system categories that account for a large proportion. The present invention provides a passive operating system recognition method based on active learning, which uses the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink combined sampling to balance the fingerprint data set, thereby solving the problem that the classification results of machine learning are biased towards operating system categories that account for a large proportion, thereby improving the recognition accuracy;

[0048] In the study of operating system fingerprint recognition, it is often very difficult to obtain large data sets, and when the data set is large, data labeling is also extremely difficult. The present invention provides a passive operating system recognition method based on active learning, which separates the operating system type, IP and TCP signatures to make the distinction of fingerprint data more detailed, reduce the demand for the number of data sets, and improve the recognition accuracy.

[0049] The update speed of the common Nmap fingerprint library P0f fingerprint library cannot keep up with the frequency of new operating system versions and updated network protocol implementations. Therefore, they may not be able to accurately identify the latest version of the operating system or new network stack features. Therefore, the present invention extracts the fingerprint of the network traffic passing through a node of Chengdu University of Technology between 14:30-18:15 on March 21, 2024 by using Wireshark, a total of 24352 samples, of which there are 8 categories of operating systems, namely Windows 10, Windows 7 or 8, Linux 2.2.x, Linux 3.x, Windows Server 2003, WindowsXP, Windows Server 2008, and Windows NT kernel 5.x.

[0050] Use Wireshark to extract the fingerprint of network traffic to obtain the fingerprint dataset, specifically:

[0051] The TCP / IP data packets are filtered through the filter function of Wireshark, and the TCP / IP data packets are read and parsed using the Python scapy library. The fingerprint data is extracted from the captured traffic information PCAP file to obtain the fingerprint data set.

[0052] The fingerprint dataset is preprocessed, including separation of operating system type, IP and TCP signatures, and the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink are combined to sample the balanced fingerprint dataset, including:

[0053] Separate the operating system type, IP and TCP signatures, the operating system type includes at least one of Windows 10, Windows 7 or 8, Linux 2.2.x, Linux 3.x, Windows Server 2003, Windows XP, Windows Server 2008, and Windows NT kernel 5.x;

[0054] The separated operating system type is encoded as a label, and the TCP signature is discretized, wherein each discretized item is an independent feature, and the items obtained after the TCP signature is discretized include: at least one of the following: message lifetime ttl, window size wsize, option length olen, payload classification pclass, version ver, maximum segment size mss, window scale scale, placeholder nop, selective confirmation sack, timestamp ts, option list end eol, df flag, flow label, and id number, wherein the placeholder nop, maximum segment size mss, selective confirmation sack, timestamp ts, and option list end eol belong to the TCP optional part, and the df flag, flow label, and id number belong to the abnormal feature item quirks part;

[0055] The labels contained in the fingerprint dataset are divided into majority and minority classes according to the sample proportions. The oversampling algorithm BorderLineSMOTE is used to supplement the minority class, and the undersampling algorithm TomeLink is used to eliminate samples in the majority class to ensure that overfitting will not occur and important data will not be lost.

[0056] Use the filter function of Wireshark to filter out TCP / IP data packets, use Python's scapy library to read and parse TCP / IP data packets, and extract fingerprint data from the captured traffic information PCAP file. Then process the extracted fingerprint data, including separating the operating system type, IP and TCP signature, and using the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink to combine sampling to balance the fingerprint data set; encode the separated operating system type as a label, and the operating system types include at least one of Windows 10, Windows 7 or 8, Linux 2.2.x, Linux 3.x, Windows Server 2003, Windows XP, Windows Server 2008, and Windows NT kernel 5.x; Since the TCP signature contains the key features required for fingerprint identification, it is necessary to discretize the TCP signature and treat each discrete item as an independent feature, so as to better demonstrate the role of each feature value. It contains at least one of 14 features, including message lifetime ttl, window size wsize, option length olen, payload classification pclass, version ver, maximum segment size mss, window scale scale, placeholder nop, selective confirmation sack, timestamp ts, option list end eol, df flag, flow label, and id number. Among them, placeholder nop, maximum segment size mss, selective confirmation sack, timestamp ts, and option list end eol belong to the TCP optional part, and df flag, flow label, and id number belong to the abnormal feature item quirks part.

[0057] Among them, ttl is the message lifetime, which indicates the maximum number of hops allowed for IP data packets to be transmitted in the network. It is mainly used to prevent IP data packets from looping infinitely in the network. The initial value is set by the operating system of the sender, so it can be used as a factor in operating system identification.

[0058] wsize is the window size, which is used to measure the host's receive window size. The TCP window size determines the maximum amount of unconfirmed data that the receiver can receive.

[0059] olen is the option length. For IPv4, it indicates the length of the IP option field. For IPv6, it indicates the length of the extension header.

[0060] pclass stands for payload classification, which means payload size classification, and can include different data packet length categories, such as small packets, medium packets, and large packets. The classification of payload size can be used to identify traffic patterns and indirectly infer the operating system or application type.

[0061] ver is the version, which is represented by a 4-bit binary number, usually converted to a decimal number of 4 for IPv4 or 6 for IPv6. The version field is not directly used for operating system identification, but it can be used to determine the version of the IP header, so as to correctly parse the IP data packet and select the appropriate operating system identification method.

[0062] Mss stands for Maximum Segment Size, which indicates the maximum segment size that can be received in a TCP connection. It depends on the transmitted data, and some operating systems may use a specific default MSS value, but these operating systems may also use different MSS values ​​depending on changes in the network environment.

[0063] scale is the window scaling factor, which is used to expand the TCP window size to allow more efficient data transmission. It is generally a fixed value between 0 and 14 or represented by *.

[0064] nop is a placeholder that does not contain any data and is only used as an alignment tool. It is used to occupy a byte position so that subsequent options can be arranged in a specific alignment.

[0065] Sack is a selective acknowledgment option that allows the receiver to tell the sender which data segments it has successfully received.

[0066] ts is a timestamp option, which is used to measure the round-trip time and prevent old data packets from being incorrectly accepted. The timestamp field contains a send timestamp and an echo timestamp. The sender adds a send timestamp to the TCP header, and the receiver returns an echo timestamp.

[0067] eol stands for "end of option list". eol is a 1-byte option used to mark the end of TCP options.

[0068] df is an IP layer flag that indicates whether the packet can be fragmented.

[0069] The flow label is a field in the IPv6 header that is used to mark the flow of a packet. It is used to provide flow-based routing and prioritization in the network.

[0070] In the IP protocol, the ID number is used to uniquely identify a fragment of a data packet. When an IP data packet is fragmented, each fragment will carry the same ID number, and the receiving end can reassemble all fragments into the original data packet based on this ID number.

[0071] TCP options are used to provide additional functions and optimize network performance. Since different operating systems and devices implement and use TCP differently, the types of operating systems and devices can be identified by analyzing the use of these options.

[0072] The quirks section is used to identify abnormal behavior and potential attack behaviors. These features are different from standard TCP implementations and may show abnormal characteristics in certain situations. For example, the df feature: If this flag is set in an unexpected situation, it indicates some kind of abnormality or attack.

[0073] Dividing the TCP options part into the TCP optional part and the abnormal characteristic quirks part helps to distinguish between standard and normal TCP configurations and abnormal and potentially malicious TCP behaviors.

[0074] Since Windows 10 and Windows Server 2008 account for 96.2% of the fingerprint data set of the present invention, it can be seen that the data set is highly unbalanced. The imbalance of the data will make it difficult for the model to accurately identify the OS with low traffic, and the recognition result will be biased towards the OS type with high traffic, thereby reducing the recognition accuracy. The present invention first uses the unbalanced data set in five models, such as Figure 3 As shown, the accuracy of the five models is relatively low. The relatively good one is the logistic regression model, with an accuracy of 70%, and the worst one is the SVM model, with only 42.7%. When the model parameters are all adjusted to the optimal level, the most likely cause of low-precision results is the data set quality problem. The present invention adopts a method of balancing data sets based on combined sampling of the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink. The simple use of the oversampling algorithm BorderLineSMOTE will produce useless noise data and widen the minority class boundary to achieve the opposite effect. Therefore, the labels contained in the data set are first divided into majority classes and minority classes according to the sample proportion, and then the oversampling algorithm BorderLineSMOTE is used to supplement the minority class, and the undersampling algorithm TomeLink is used to eliminate the samples in the majority class, so as to ensure that there will be no overfitting and no loss of important data.

[0075] Also includes:

[0076] According to the characteristics of each attribute, the four features of packet lifetime ttl, window size wsize, option length olen, and payload classification pclass are set to numeric type, and the features of version ver, maximum segment size mss, window scale scale, TCP optional part, and abnormal feature item quirks part are set to object type;

[0077] The numeric features are dimensionless, and the original data is converted into data with a mean of 0 and a variance of 1;

[0078] For the object classification features, the mode is used to fill the missing values ​​and the features are encoded as numbers. The features of the TCP optional part and the abnormal feature item quirks part use 1 to represent the presence of the item and 2 to represent the absence of the item. The version ver, maximum segment size mss and window scale are one category corresponding to one number.

[0079] like Figure 2 As shown in the figure, the fingerprint feature data of 6 common operating systems are Windows XP, Windows 7, Linux 2.2, Windows 10, Linux 3.0, and Windows Server 2003. The fingerprint feature data includes the operating process system OPS, message lifetime TTL, WIN, DF, and maximum segment size MSS. WIN is a field in the TCP protocol, which indicates the size of the receiver's buffer. The receiver uses it to tell the sender the number of bytes it can receive. Figure 2 It can be seen that the fingerprint feature data of each operating system are different, so the operating system can be accurately distinguished by the fingerprint feature data. According to the characteristics of each attribute, the present invention sets the four features of the message lifetime ttl, window size wsize, option length olen, and payload classification pclass to numeric type, and sets the features of the version ver, maximum segment size mss, window scaling scale, TCP optional part, and abnormal feature item quirks part to object type; for numeric numerical features, it is dimensionless, and the original data is converted into data with a mean of 0 and a variance of 1. For the object type feature, the mode is first used to fill the missing values, and then it is encoded into a number, wherein the features of the TCP optional part and the abnormal feature item quirks part use 1 to represent the appearance of the item, and 2 to represent the absence of the item. For the remaining version ver, maximum segment size mss, and window scaling scale parts, one category corresponds to one number.

[0080] The training of the fingerprint recognition model includes:

[0081] Set the acquired fingerprint dataset and network traffic as the unlabeled sample pool U n , from the unlabeled sample pool U n Randomly select fingerprint samples to mark and obtain the initial fingerprint training set T i ;

[0082] Construct classifier C according to the classification algorithm m;

[0083] Select 5 popular classification algorithms to construct 5 classifiers C respectively m , including KNN, Random Forest, Decision Tree, Support Vector Machine (SVM) and Logistic Regression classification algorithms;

[0084] The initial fingerprint training set T i Input classifier C m The initial classification model is trained in the unlabeled sample pool U n Classify unlabeled fingerprint samples in;

[0085] Find the unlabeled sample pool U through different query strategies and rules n The fingerprint sample U with the largest number of features a , and based on the manual fingerprint sample U a Marking

[0086] For classifier C m Update the marked fingerprint sample U a Add to the initial fingerprint training set T i , update the initial fingerprint training set T i Make T n =T i U a , update the unlabeled sample pool U n make U n →U n / U a ; Update the initial fingerprint training set T n Input to classifier C m Train in to get a new classification model;

[0087] Repeat the above process and continuously update the classification model until the model convergence conditions are met.

[0088] Since Windows 10 and Windows Server 2008 account for 96.2% of the fingerprint data set of the present invention, it can be seen that the data set is highly unbalanced. The imbalance of the data will make it difficult for the model to accurately identify the OS with low traffic, and the recognition result will be biased towards the OS type with high traffic, thereby reducing the recognition accuracy. The present invention first uses the unbalanced data set in five models, such as Figure 3As shown, the accuracy of the five models is relatively low. The relatively good one is the logistic regression model, with an accuracy of 70%, and the worst one is the SVM model, with only 42.7%. When the model parameters are all adjusted to the optimal level, the most likely cause of low-precision results is the data set quality problem. The present invention adopts a method of balancing data sets based on combined sampling of the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink. The simple use of the oversampling algorithm BorderLineSMOTE will produce useless noise data and widen the minority class boundary to achieve the opposite effect. Therefore, the labels contained in the data set are first divided into majority classes and minority classes according to the sample proportion, and then the oversampling algorithm BorderLineSMOTE is used to supplement the minority class, and the undersampling algorithm TomeLink is used to eliminate the samples in the majority class, so as to ensure that there will be no overfitting and no loss of important data.

[0089] After that, the balanced data set is input into five classification models to realize operating system identification, and the performance indicators of the five classification models before and after active learning are compared to find the optimal model. The present invention uses the python programming language and the scikit-learn library as the experimental environment to realize the identification of the operating system fingerprint data set, divides the fingerprint data set into a training set and a test set, uses 70% as the training set, and the remaining 30% as the test set to test the performance of the model. The comparative experimental verification process before and after active learning is as follows:

[0090] Before active learning, the balanced data set is input into the classification model to realize operating system identification. The present invention selects five classification algorithms: KNN, Random Forest, Decision Tree, Support Vector Machine (SVM) and Logistic Regression. The ratio of training set to test set is 7:3. Figure 4-Figure 6It can be seen that these five algorithms have achieved good results in various evaluation indicators. The highest accuracy is the Logistic Regression algorithm, with an accuracy of 96.64%; the poorest effect is the nearest neighbor algorithm (KNN), with an accuracy of only 93.69%. This is because the number of operating system fingerprint feature sets is large and the distinction is difficult. As the number of features increases, the amount of calculation and noise also increase, which affects the prediction ability of the KNN classification model and reduces the accuracy. The Logistic Regression algorithm is suitable for large-scale data sets. It can reduce overfitting through feature selection and regularization and effectively overcome the influence of noise. In terms of running time, the Decision Tree algorithm has the shortest running time of 0.162 seconds, the SVM algorithm has the longest running time of 39.423 seconds, and the Logistic Regression algorithm runs for 1.79 seconds.

[0091] Furthermore, the KNN, Random Forest, Decision Tree, SVM and Logistic Regression are used as the basic models of active learning, and 1%, 5%, 10% and 20% of the original training set are used as the training sample sets for active learning, that is, the number of samples is 170, 852, 1704 and 3409 respectively. The number of iterations is set to 0, 5, 10, 15, 20, 25 and 30, and ten-fold cross validation is performed at different numbers of iterations. Figure 7-Figure 9 It can be seen that the best classification accuracy is the Logistic Regression algorithm. When the number of iterations is 20, the Logistic Regression algorithm achieves the highest accuracy of 98.53%. Whether a model performs well depends not only on the classification accuracy, but also on the algorithm running time. Figure 8 It can be seen that the KNN algorithm has the shortest running time, followed by the Logistic Regression algorithm. However, the accuracy of the KNN algorithm is much lower than that of the Logistic Regression algorithm, so the classification model using the Logistic Regression algorithm is the optimal classification model.

[0092] It should also be noted that the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "includes one..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements. The words "first", "second" and the like are used to indicate names, but do not indicate any specific order. The above schematically describes the invention and its implementation methods, which is not restrictive. The invention can be implemented in other specific forms without departing from the spirit or basic features of the invention. What is shown in the accompanying drawings is only one of the implementation methods of the invention, and the actual structure is not limited thereto. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, and does not deviate from the purpose of the invention, and designs a structural mode and an embodiment similar to the technical solution without creativity, they should all belong to the scope of protection of this patent.

Claims

1. A passive operating system identification method based on active learning, characterized in that: The following steps are involved: Use Wireshark to extract the fingerprint of network traffic to obtain the fingerprint dataset, including: Filter TCP / IP data packets through the filter function of Wireshark, read and parse the TCP / IP data packets with Python's scapy library, and extract fingerprint data from the captured traffic information PCAP file to obtain a fingerprint data set; The fingerprint dataset is preprocessed, including separating the operating system type, IP and TCP signatures, and using the oversampling algorithm BorderLineSMOTE and the undersampling algorithm TomeLink to sample the balanced fingerprint dataset, including: Separate the operating system type, IP and TCP signatures, the operating system type includes: at least one of Windows 10, Windows 7 or 8, Linux 2.2.x, Linux 3.x, Windows Server 2003, Windows XP, Windows Server 2008, and Windows NT kernel 5.x; The separated operating system type is encoded as a label, and the TCP signature is discretized, wherein each discretized item is an independent feature, and the items obtained after the TCP signature is discretized include: at least one of the message lifetime ttl, the window size wsize, the option length olen, the payload classification pclass, the version ver, the maximum segment size mss, the window scale scale, the placeholder nop, the selective confirmation sack, the timestamp ts, the option list end eol, the df flag, the flow label, and the id number, wherein the placeholder nop, the maximum segment size mss, the selective confirmation sack, the timestamp ts, and the option list end eol belong to the TCP optional part, and the df flag, the flow label, and the id number belong to the abnormal feature item quirks part; The labels contained in the fingerprint dataset are divided into majority class and minority class according to the sample proportion, the minority class is supplemented by the oversampling algorithm BorderLineSMOTE, and the samples in the majority class are eliminated by the undersampling algorithm TomeLink to ensure that overfitting and important data are not lost; The preprocessed fingerprint data set is input into a pre-trained fingerprint recognition model to obtain a recognition result output by the fingerprint recognition model, wherein the fingerprint recognition model is obtained through active learning training.

2. The method for identifying a passive operating system based on active learning according to claim 1, characterized in that: Also includes: According to the characteristics of each attribute, the four features of packet lifetime ttl, window size wsize, option length olen, and payload classification pclass are set to numeric type, and the features of version ver, maximum segment size mss, window scale scale, TCP optional part, and abnormal feature item quirks part are set to object type; The numeric features are dimensionless, and the original data is converted into data with a mean of 0 and a variance of 1; For the object classification features, the mode is used to fill the missing values ​​and the features are encoded as numbers. The features of the TCP optional part and the abnormal feature item quirks part use 1 to represent the presence of the item and 2 to represent the absence of the item. The version ver, maximum segment size mss and window scale are one category corresponding to one number.

3. The method for identifying a passive operating system based on active learning according to claim 1, characterized in that: The training of the fingerprint recognition model includes: Set the acquired fingerprint dataset and network traffic as the unlabeled sample pool U n , from the unlabeled sample pool U n Randomly select fingerprint samples to mark and obtain the initial fingerprint training set T i ; Construct classifier C according to the classification algorithm m ; The initial fingerprint training set T i Input classifier C m The initial classification model is trained to obtain the initial classification model, and the unlabeled sample pool U is classified by the initial classification model. n Classify unlabeled fingerprint samples in; Find the unlabeled sample pool U through different query strategies and rules n The fingerprint sample U with the largest number of features a , and according to the manual a Marking For the classifier C m Update the marked fingerprint sample U a Add to the initial fingerprint training set T i , update the initial fingerprint training set T i Make T n =T i U a , update the unlabeled sample pool U n make U n →U n / U a ; Update the initial fingerprint training set T n Input to classifier C m Train in to get a new classification model; Repeat the above process and continuously update the classification model until the model convergence conditions are met.

Images