Internet of Things equipment identification method and system based on flag recessive characteristics

Abstract

The invention relates to the technical field of Internet of Things equipment identification, in particular to an Internet of Things equipment identification method and system based on flag recessive characteristics, and the method is divided into a fingerprint database construction stage and an equipment identification stage. In the fingerprint database construction stage, preprocessing, feature vector construction, representation vector extraction and feature weight calculation processing are performed on a group of collected HTTP / HTTPS response message heads with manufacturer and type labels, and then a fingerprint database is constructed; in the device identification stage, similarity calculation is carried out on the header of the HTTP / HTTPS response message of the target device and the items in the fingerprint database to obtain the manufacturer and the type of the target device. According to the method, the fingerprint database is automatically constructed by taking the combination, the value and the distinguishability of the header of the HTTP response message as the recessive characteristics of the equipment. According to the method, the fingerprint database can be automatically constructed for the equipment without obvious characteristics in the service flag, and the blank of a traditional method is filled. Compared with a method based on artificial intelligence classification, the method has better interpretability and expandability.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things device identification, and in particular to an Internet of Things device identification method and system based on flag implicit features. Background Art

[0002] With the vigorous development of IoT technology, the number of IoT devices in cyberspace has exploded, affecting all aspects of human society. According to Statista, by 2025, the number of IoT devices will reach about 75.44 billion, playing an important role in industrial production, home life, smart transportation and other fields. However, due to the limitations of computing resources and working environment, IoT devices generally lack security protection technology and management methods, resulting in frequent security incidents against IoT devices, and malware that specifically targets security vulnerabilities in IoT devices continues to appear, and even large quantities of IoT devices are controlled by attackers and used as a springboard for launching large-scale DDoS attacks. Accurate identification of IoT devices is the basis of security activities such as risk assessment, threat perception, and vulnerability protection, and is of great significance to improving the security level of IoT.

[0003] At present, there are two main methods for IoT device identification: precise pattern matching and artificial intelligence classification. Precise pattern matching methods, such as Nmap and SinFP, compare the information obtained through port scanning technology with the entries in the fingerprint library to identify the device. This method sends a request message that follows a specific application protocol (such as HTTP, FTP, POP3, etc.) to a specific IP and port, extracts a fragment with iconic information (called a flag) from the obtained response message, and performs precise pattern matching with the entries in the pre-established fingerprint library. Finally, the device attributes (such as manufacturer, type, model, etc.) carried in the matched fingerprint library entry are used as the result of device identification. The artificial intelligence classification method generally obtains a classification model by learning the features in the device network traffic to identify the device. This method can be further divided into two methods based on machine learning and deep learning. Among them, the method based on machine learning needs to manually select features in the traffic based on expert experience, and uses random forest algorithms, KNN, SVM and other algorithms to classify the device. The method based on deep learning uses algorithms such as neural networks (NN) and CNN to eliminate the manual feature extraction work in machine learning methods, automatically learn features from the original network traffic of the device and classify the device.

[0004] The IoT device recognition method based on exact pattern matching has a very high matching accuracy but lacks flexibility. On the one hand, this method requires that the network service flags of the target device contain explicit features so that the features in the flags can be defined in the form of regular expressions in the fingerprint. However, due to device feature reasons or for security protection needs, many devices do not contain information with obvious features in their service flags. Therefore, this method cannot be applied to the recognition of such devices. On the other hand, even if the flags contain obvious features, this method still requires experts familiar with the device characteristics to manually extract the feature patterns in the flags. Facing a large number of emerging IoT devices, this task is extremely laborious and difficult to sustain. The IoT device recognition method based on artificial intelligence classification has a relatively high matching accuracy but lacks interpretability and scalability. First of all, various machine learning and deep learning algorithms have the characteristics of black boxes. It is impossible to understand the causal relationship between the training data and the prediction results, and it is also impossible to predict when the model will make mistakes. Therefore, it is impossible to fully establish trust in the model. Secondly, once it is necessary to recognize a device of a manufacturer or type that has not appeared in the label dataset before, it is necessary to spend a lot of time retraining and validating the classification model, and the scalability is very poor.

[0005] The existing methods for identifying Internet of Things devices using network attributes as data sources mainly include two methods: based on exact pattern matching and based on artificial intelligence classification. The main defects of these two methods are summarized as follows:

[0006] Device recognition based on exact pattern matching relies on the strings with obvious features (also known as protocol flags or flags) existing in the protocol response messages of the device. Usually, the strings are captured in the form of regular expressions to form the explicit fingerprint of the device. The disadvantage of the device recognition method based on the explicit fingerprint is that it requires cumbersome manual work to design the matching rules, and it requires obvious device features in the flag data. There are still a large number of IoT devices in the real world that do not carry strings with obvious features in the response messages. When the flag data lacks explicit features, this method will fail. The explicit feature is the identification information directly exposed and easily obtained by the device.

[0007] The method based on artificial intelligence classification generally obtains a classification model by learning the implicit features in the device network traffic to identify the device. Most of these methods are based on the "closed world", that is, they can only classify the data in the closed dataset. In the "open world", these artificial intelligence-based classifiers need to update the dataset and retrain to support the recognition of new types of devices, and the scalability is relatively poor. The implicit feature is the feature indirectly shown by the device and can be obtained through analysis and calculation. Summary of the Invention

[0008] The present invention aims to solve the problems that the method based on exact pattern matching relies on the explicit features of device flags, has low analysis efficiency and high manual intervention, and the method based on artificial intelligence classification relies on artificial intelligence to mine the implicit features of devices, lacking interpretability and scalability. A method and system for identifying Internet of Things (IoT) devices based on the implicit features of flags are proposed. Using the combination, value and distinguishability of the response message headers of the HTTP protocol as the implicit features of the devices, a device fingerprint database can be constructed relying on the device response data detected remotely without manual intervention. The device identification criterion based on similarity calculation is adopted, and the fingerprint database can be flexibly expanded, so as to identify IoT devices even when there is no obvious feature information in the service flag.

[0009] To achieve the above object, the technical solution adopted is as follows:

[0010] The present invention provides a method for identifying IoT devices based on the implicit features of flags. This method is divided into two stages: the fingerprint database construction stage and the device identification stage;

[0011] In the fingerprint database construction stage, a fingerprint database is constructed after preprocessing, feature vector construction, characterization vector extraction and feature weight calculation of a group of HTTP / HTTPS response message headers collected with vendor and type tags;

[0012] In the device identification stage, the vendor and type of the target device are obtained by calculating the similarity between the headers of the HTTP / HTTPS response message of the target device and the entries in the fingerprint database.

[0013] According to the method for identifying IoT devices based on the implicit features of flags of the present invention, further, the steps of constructing the fingerprint database include:

[0014] Feature space construction, using the headers that are common in the HTTP response message and can reflect device differences to construct the feature space The i-th header in it is denoted as h i ;

[0015] Initial data acquisition and marking, sending an HTTP request to IoT devices with known vendors and types, generating a record R=(D, V, T) in the form of a triple for the response message from each device and forming a labeled data set Ω1, where D is the header information of the response message, V is the vendor of the device, and T is the type of the device;

[0016] Data preprocessing, splitting the header information D of the response message into a triple set, performing type conversion on the header values, and performing deduplication processing to form a labeled data set represented in the form of key-value pairs;

[0017] Feature vector construction, according to the feature space Generate the feature vector for each record;

[0018] Extract the representation vector and generate the representation vector according to the feature vector;

[0019] Calculate the feature weights, calculate the importance of each feature according to the representation vector and the feature vector, generate the weight vector, and construct the fingerprint entry.

[0020] According to the method for identifying Internet of Things devices based on flag implicit features of the present invention, further, split the header information D of the response message into a set S composed of triples (k, v, c), where k is a certain header string, v is the value of the header, and c is the type of the value of the header; replace D in the record R with the obtained triple set S to form a tag data set Ω2 represented in the form of key-value pairs, and the form of each record is R = (S, V, T).

[0021] According to the method for identifying Internet of Things devices based on flag implicit features of the present invention, further, according to the feature space The generation of the feature vector for each record includes: if the HTTP response message of the device contains the header h i , then the i-th component f i of the feature vector F takes the value v and type c corresponding to h in the response message, otherwise f i takes the value (Null, Null); replace S in the record R with the obtained feature vector F to form a tag data set Ω3 represented in the form of a feature vector, and the form of each record is R = (F, V, T). i ; Replace S in record R with the obtained feature vector F to form a tag data set Ω3 represented in the form of a feature vector, and the form of each record is R = (F, V, T).

[0022] According to the method for identifying Internet of Things devices based on flag implicit features of the present invention, further, the generation of the representation vector according to the feature vector includes: if the i-th component f i of the feature vector of the device is not equal to (Null, Null), then f repri = 1, otherwise f repri = 0, and thus construct the representation vector F repr of the device; then add F repr to the record R to form a tag data set Ω4 represented by the feature vector and the representation vector, and the form of each record is R = (F, F repr , V, T).

[0023] According to the method for identifying Internet of Things devices based on flag implicit features of the present invention, further, the process of generating the weight vector and constructing the fingerprint entry is as follows: for multiple devices with the same representation vector, for each feature vector F, calculate the importance θ i of its i-th component f i , and for the importance θ i of all featuresPerform normalization to generate the weight vector W; finally, use (F, F repr , V, T) and W to form a fingerprint entry FP = (F, F repr , W, V, T), and add it to the fingerprint database.

[0024] According to the method for identifying Internet of Things devices based on flag implicit features of the present invention, further, for the component f i of the feature vector, calculate its importance θ i which includes: among all feature vectors, the reciprocal of the number of feature vectors with the same value as the current feature vector on the i-th feature is the importance θ i of this feature.

[0025] According to the method for identifying Internet of Things devices based on flag implicit features of the present invention, further, the steps of device identification include:

[0026] Target data acquisition and preparation: Send an HTTP request to the device to be identified, obtain the header information of the response message, and generate a target feature vector and a target representation vector;

[0027] Candidate fingerprint screening: Screen out candidate fingerprints related to the target device according to the target representation vector;

[0028] Target device manufacturer and type identification: Determine the manufacturer and type of the target device by calculating the similarity between the target feature vector and the candidate fingerprint feature vector.

[0029] According to the method for identifying Internet of Things devices based on flag implicit features of the present invention, further, the process of calculating the similarity is as follows:

[0030] For each candidate fingerprint FP, calculate the similarity between the target feature vector F T and the fingerprint feature vector FP.F The formula for the similarity is:

[0031]

[0032] Among them, is the similarity of the feature component, and w i is the weight component;

[0033] For the data type of the feature component being string type, calculate the similarity using the ratio of the edit distance to the maximum string length;

[0034] For the data type of the feature component being integer type, calculate the similarity using the ratio of the absolute value of the difference to the maximum value of the feature value;

[0035] For the data type of the feature component being an enumeration type, the maximum value of the enumeration value is used to calculate the similarity.

[0036] Furthermore, the present invention also provides an Internet of Things device identification system based on flag implicit features for implementing the above-mentioned Internet of Things device identification method based on flag implicit features, including a fingerprint database construction module and a device identification module, where:

[0037] The fingerprint database construction module is used to construct a fingerprint database after performing preprocessing, feature vector construction, characterization vector extraction, and feature weight calculation on a set of HTTP / HTTPS response packet headers with manufacturer and type tags collected;

[0038] The device identification module is used to calculate the similarity between the headers of the HTTP / HTTPS response packets of the target device and the entries in the fingerprint database to obtain the manufacturer and type of the target device.

[0039] Adopting the above technical solution, the beneficial effects obtained are:

[0040] 1. It does not rely on expert experience and reduces manual intervention.

[0041] During the construction of the fingerprint database in the present invention, there is no need to rely on expert experience to extract features from the HTTP response headers, and only a small amount of manual processing is required in the data marking step. Although type and encoding conversions need to be performed on the values of some headers in the preprocessing step, the conversion and encoding rules for certain headers (such as Content-length, Connection, etc.) can be pre-implemented in the form of plugins, and the plugin code can be called when encountering the corresponding headers to complete the preprocessing. Other steps such as feature vector construction, characterization vector extraction, and feature weight calculation can all be completed in an automated manner, effectively reducing manual intervention, reducing the manual workload, and having good flexibility.

[0042] 2. It is applicable to devices without obvious features in the service flag.

[0043] Traditional methods based on exact pattern matching rely on obvious features in the device flag, while the present invention can identify devices that do not have obvious features in the service flag by analyzing the combination, value, and distinguishability of the HTTP response headers, making up for the identification gap of traditional methods.

[0044] 3. It has good scalability.

[0045] First of all, the feature space is extensible. If a new HTTP header that can reflect device differences is encountered, it can be added to without affecting the existing fingerprints.

[0046] Secondly, the fingerprint database is extensible. If the feature vector of a certain fingerprint FP' to be introduced is unique, that is FP.F repr ≠FP'.F repr , then this fingerprint can be directly added to the fingerprint database without affecting the existing fingerprints. If the feature vector of FP' already exists in the fingerprint database, after adding FP' to the fingerprint database, the feature weights of the fingerprints with the same feature vector as FP' can be recalculated with reference to the steps of calculating feature weights, and it will not affect other fingerprints in the fingerprint database. Generally speaking, the fingerprints constructed by the present invention have high independence and can be conveniently expanded, so they have good scalability.

[0047] 4. Improve device recognition efficiency

[0048] Since the HTTP headers of different types of devices are very likely to have different combinations, the present invention adds a candidate fingerprint screening step in the device recognition stage, which can greatly accelerate the efficiency of device recognition. Suppose the total number of fingerprint database entries is m. When ignoring the candidate fingerprint screening step, m similarity calculations are required to identify the manufacturer and type of the target device. After introducing the candidate screening step, if the number of candidate fingerprints obtained is n, only n similarity calculations are required to identify the manufacturer and type of the target device. Since the HTTP header combinations of different types of devices often have great differences, it is very likely that n << m. Therefore, candidate fingerprint screening is expected to greatly reduce the computational intensity and improve the recognition efficiency.

[0049] 5. Robustness and interpretability

[0050] The present invention measures the matching degree between the target device and the fingerprint based on similarity calculation, and has better robustness than the device recognition method based on explicit fingerprints and better interpretability than the device recognition method based on implicit fingerprints. For example: newly manufactured devices may modify the Web server version number given in the Server header of the HTTP response message, or make minor modifications to the content of the Web home page, resulting in a small change in the value of the Content-Length header. However, these changes will not cause a large change in the result of the similarity calculation, so the correct device recognition result may still be obtained. Brief description of the drawings

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments of the present invention will be briefly introduced below. Among them, the drawings are only used to show some embodiments of the present invention, rather than limiting all embodiments of the present invention thereto.

[0052] Figure 1It is the flowchart of the fingerprint database construction stage in the IoT device identification method based on the implicit features of flags in the embodiments of the present invention;

[0053] Figure 2 It is the flowchart of the device identification stage in the IoT device identification method based on the implicit features of flags in the embodiments of the present invention;

[0054] Figure 3 They are three examples of HTTP response headers of the IoT device in the embodiments of the present invention. Detailed implementation manners

[0055] In the following, the exemplary solutions of the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings of the specific embodiments of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meaning understood by those with ordinary skills in the relevant field.

[0056] HTTP / HTTPS is a service protocol commonly supported by IoT devices. The response information mainly includes three parts: a status line, response headers, and a response body. Among them, the response header part is represented in the form of key-value pairs and has a relatively simple format. In addition to the response headers (such as Server, WWW-Authenticate, etc.) stipulated in the HTTP / HTTPS specification, a web server can also customize response headers (usually prefixed with "X-", such as X-Powered-By, X-Frame-Options, etc.). Based on the observation that there are similarities in the combination and values of the HTTP protocol response message headers of the same type of IoT devices.

[0057] First, although a manufacturer can arbitrarily modify the combination and value-taking manner of the response headers returned by the web server, once the device is manufactured, this information is stored in the device firmware. Since the firmware of most IoT devices is rarely updated, this information remains almost unchanged during the device's life cycle and can thus be regarded as an inherent characteristic of the device. Second, when the web service of an IoT device enables an authentication mechanism, we generally cannot obtain meaningful response bodies, but we can still obtain relatively complete response headers at this time. Finally, and most importantly, the distinguishability. Based on the observation of the HTTP response headers of IoT devices, it is found that IoT devices of the same manufacturer and type generally have similar header combinations and values, while IoT devices of different manufacturers or types have significant differences in header combinations and values. For example, Figure 3The HTTP response headers of two routers from D-Link and one router from ASUS are given. The two D-Link routers have the same combination of HTTP headers, and the values of other headers are exactly the same except for a slight difference in the value of the Server header. However, the router from ASUS has a significantly different combination of HTTP headers from the D-Link routers. Based on the above analysis, this solution believes that the HTTP response header information of IoT devices has properties such as being long-term unchanged, easy to obtain, and clearly distinguishable, and is suitable as a feature for identifying IoT devices.

[0058] Based on the above findings, this embodiment discloses an IoT device identification method based on flag hidden features, including a fingerprint database construction step and a device identification step.

[0059] Step S1: Construct a fingerprint database by preprocessing, feature vector construction, feature vector extraction, and feature weight calculation on a set of HTTP / HTTPS response message headers with manufacturer and type tags collected. This step specifically includes sub-steps S101 - S106, as Figure 1 shown.

[0060] Step S101: Feature space construction

[0061] Construct a feature space using the headers commonly found in HTTP response messages that can reflect device differences These headers include Server, Connection, Content-type, Content-encoding, Transfer-encoding, Content-length, Last-modified, etc. Set the size n of the feature space to 64, The unspecified headers are marked with Null, and new headers that can reflect device differences discovered during the subsequent fingerprint database construction process can be added to it.

[0062] The symbols appearing in the text are explained below:

[0063] The feature space composed of n HTTP headers is denoted as The i-th header in it is denoted as h i , and in this solution, HTTP headers and HTTP features have the same meaning.

[0064] F: The feature vector of the device is denoted as F = [f1, f2,..., f n . If the HTTP response message of the device contains the header h i , then f iRetrieve the value and type corresponding to h in the response message, otherwise f i take the value as (Null, Null). i

[0065] F repr : The characterization vector of the device is denoted as the i-th component of the characterization vector if and only if the i-th component of the feature vector of the device f i ≠ (Null, Null), otherwise

[0066] FP: The fingerprint of the device is denoted as the five-tuple FP = (F, F repr , W, V, T), where F is the feature vector of the device, F repr is the characterization vector of the device, W is the weight vector, V is the manufacturer of the device, and T is the type of the device.

[0067] Let R be a certain record, then an element X in R is denoted as R.X.

[0068] Step S102, Initial data acquisition and marking

[0069] Select devices that support the HTTP / HTTPS protocol to build the initial data set. Send HTTP requests to the commonly used port numbers of HTTP / HTTPS services such as 80 / 8080 / 443 / 8443 of a batch of Internet of Things devices with known manufacturers and types. For the response messages with the status code "200 OK" from each device, generate records in the form of triples (D, V, T) and form the labeled data set Ω1, where D is the header information of the response message, V is the manufacturer of the device, and T is the type of the device.

[0070] Step S103, Data preprocessing

[0071] First, for each record R = (D, V, T) in the data set Ω1, split R.D into a set S composed of triples (k, v, c), where k is a certain header string, v is the value of the header, and c is the type of the value of the header. In the protocol message, the values of the HTTP headers exist in string form. For the convenience of subsequent similarity measurement, convert the value types to data types such as integer (int), enumeration (enum), string (str), etc. according to the semantics of each header. Then, remove duplicates from the encoded records. For multiple identical records, only keep one; finally, replace D in the record R with the obtained triple set S to form the labeled data set Ω2 represented in key-value pair form, where each record is in the form of a triple (S, V, T).

[0072] Step S104, Construct feature vector

[0073] Generate a feature vector for each record R = (S, V, T) in the data set Ω2, given The i-th head h in i , the i-th component f of the eigenvector F i The value selection rule of is shown in formula (1). The obtained feature vector F is used to replace S in the record R to form a label data set Ω3 represented by a feature vector, where each record is in the form of a triple (F, V, T). Note that due to the differences in firmware versions, there may be multiple header combinations in the HTTP responses of different instances of the same type of device, and the same header may also have multiple values, and thus multiple feature vectors. Therefore, a type of device may have multiple corresponding records in the data set.

[0074]

[0075] Step S105: Extracting the characterization vector

[0076] Extract the representation vector F for each record R = (F, V, T) in the dataset Ω3 repr , given the i-th component f of the eigenvector RF i , F repr The i-th component of The value selection rule of is shown in formula (2). Then, using R = (F, V, T) and F repr Construct a four-tuple record (F,F repr ,V,T), forming a label data set Ω4 represented by feature vectors and representation vectors.

[0077]

[0078] Step S106: Calculate feature weights

[0079] Multiple devices may have the same characterization vector. Let the set of feature vectors of all devices with characterization vector X in the data set Ω4 be if The value of a certain eigenvector F on the i-th eigencomponent is unique in If the values ​​of other feature vectors in the i-th feature component are taken, it can be considered that this feature component has greater importance in identifying the device manufacturer and type to which the feature vector F belongs.

[0080] For each Let its value on the i-th feature be f i ,remember for The value of the i-th feature is f i The set of eigenvectors of can be defined using formula (3) any header h in i the importance θ in device recognition i , which means that when the feature f is used alone i the probability of obtaining a correct recognition result in device recognition. In particular, when it means that f i has a unique value and θ i takes the maximum value of 1. When f i =(Null,Null), it means that f i has no effect on device recognition and θ i takes the maximum value of 0.

[0081]

[0082] By comprehensively considering each feature and performing normalization processing, the weight vector W = [w1, w2,..., w n can be obtained, where w i represents the weight of the header h in i device recognition and is calculated according to formula (4). The larger the value of a certain weight component, the better the discrimination of this feature for different devices.

[0083]

[0084] After that, use (F, F repr , V, T) and W to form a fingerprint entry FP = (F, F repr , W, V, T) and add it to the fingerprint database Ω5.

[0085] Step S2, By calculating the similarity between the headers of the HTTP / HTTPS response messages of the target device and the entries in the fingerprint database, the manufacturer and type of the target device are obtained. This step specifically includes sub-steps S201 - S203, as Figure 2 shown.

[0086] Step S201, Target data acquisition and preparation

[0087] Send an HTTP request to the common HTTP / HTTPS service port numbers such as 80 / 8080 / 443 / 8443 of the device to be recognized. For the response messages with the status code "200 OK" from each device, save the header information D T of the response message. Preprocess the values in D T , and split D T into a set S composed of triples (k, v, c) T, where k is a certain header string, v is the value of the header, and c is the type of the value of the header. Then, referring to formula (1), the target feature vector F is obtained T , and at the same time, referring to formula (2), the target representation vector is obtained

[0088] Step S202, Candidate fingerprint screening

[0089] The header combination in the HTTP response message can reflect the characteristics of the device. Therefore, this solution uses the representation vector as the basis for screening candidate fingerprints. Considering that the header combinations in the HTTP response messages of multiple devices may be the same, given a representation vector, multiple associated candidate fingerprints may be obtained. This step can filter out the fingerprint entries in the fingerprint database that are irrelevant to the target device and improve the efficiency of device recognition. Let the initial candidate fingerprint set For each fingerprint entry FP in the fingerprint database Ω5, if Then there is

[0090] Step S203, Target device manufacturer and type identification

[0091] For each FP in the candidate fingerprint set , the similarity between the target feature vector F T of the target device defined by formula (5) and the feature vector FP.F of the fingerprint entry is Among them, and f i are the i-th components of F T and FP.F respectively, is the and f i when the data types of the feature components are defined as string, integer, and enumeration by formulas (6), (7), and (8) respectively i between the similarities, and w is the minimum fingerprint in the fingerprint set Ω5 such that That is Then is the manufacturer of the target device, is the type of the target device.

[0092]

[0093] In formula (6), represents and f i the edit distance between, and |f i .v| represents the length of the eigenvalue string. In formula (7), represents and f i the absolute value of the difference between denotes and f i the maximum value. In formula (8), max(f i .v) represents the maximum value of the enumerated values.

[0094] Correspondingly, this embodiment also discloses an Internet of Things device identification system based on flag implicit features, which includes a fingerprint database construction module and a device identification module, where:

[0095] The fingerprint database construction module is used to construct a fingerprint database after preprocessing, feature vector construction, representation vector extraction, and feature weight calculation on a set of HTTP / HTTPS response message headers with manufacturer and type tags collected.

[0096] The device identification module is used to calculate the similarity between the header of the HTTP / HTTPS response message of the target device and the entries in the fingerprint database to obtain the manufacturer and type of the target device.

[0097] The present invention proposes an Internet of Things device identification method and system based on flag implicit features. The invention automatically constructs a fingerprint database with the combination, value, and distinguishability of the response message headers of the HTTP protocol as the overall features, and realizes device identification by calculating the similarity between the service flag and the fingerprint. Compared with the device identification method based on explicit fingerprints, the method proposed by the present invention can automatically construct a fingerprint database for devices without obvious features in the service flag, effectively making up for the blank of the identification ability of traditional fingerprint databases. Compared with the device identification method based on implicit fingerprints, it has better interpretability and scalability.

[0098] Finally, it should be noted that: the above embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacement on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An Internet of Things device identification method based on the implicit features of flags, characterized in that, This method is divided into two stages: the fingerprint database construction stage and the device identification stage; In the fingerprint database construction stage, a fingerprint database is constructed by preprocessing, feature vector construction, feature vector extraction, and feature weight calculation on a set of HTTP / HTTPS response message headers with vendor and type tags collected; In the device identification stage, the vendor and type of the target device are obtained by calculating the similarity between the headers of the HTTP / HTTPS response message of the target device and the entries in the fingerprint database.

2. The method for identifying Internet of Things devices based on the hidden features of flags according to claim 1, wherein The steps of fingerprint database construction include: Feature space construction: Construct a feature space using headers that are common in HTTP response messages and can reflect device differences The i-th header in [] is denoted as h i ; Initial data acquisition and marking: Send an HTTP request to Internet of Things devices with known vendors and types, generate a record R=(D, V, T) in the form of a triple for the response message from each device, and form a tag data set Ω1, where D is the header information of the response message, V is the vendor of the device, and T is the type of the device; Data preprocessing: Split the header information D of the response message into a triple set, perform type conversion on the header values, and perform deduplication processing to form a tag data set represented in the form of key-value pairs; Construct a feature vector according to the feature space Generate the feature vector of each record; Extract feature vectors: Generate feature vectors according to the feature vectors; Calculate feature weights: Calculate the importance of each feature according to the feature vectors and the feature vectors, generate a weight vector, and construct fingerprint entries.

3. The method for identifying an Internet of Things device based on the hidden features of a flag according to claim 2, wherein Split the header information D of the response message into a set S composed of triples (k, v, c), where k is a certain header string, v is the value of the header, and c is the type of the value of the header; Replace D in the record R with the obtained triple set S to form a tag data set Ω2 represented in the form of key-value pairs, and the form of each record is R=(S, V, T).

4. The method for identifying an Internet of Things device based on the implicit features of a flag according to claim 3, wherein According to the feature space The feature vector generated for each record includes: If the device's HTTP response message contains the header h i , then the i-th component f of the eigenvector F i Get h in the response message i The corresponding value v and type c, otherwise f i The value is (Null, Null); the obtained feature vector F is used to replace S in the record R to form a label data set Ω3 represented by a feature vector, and each record is in the form of R = (F, V, T).

5. The method for identifying Internet of Things devices based on the implicit features of flags according to claim 4, wherein Generating a representation vector based on the feature vector includes: if the i-th component f of the feature vector of the device i ≠(Null,Null), then Otherwise Construct the representation vector F of the device in this way repr ; then add F to the record R repr , forming a labeled data set Ω4 represented by the feature vector and the representation vector, and the form of each record is R=(F,F repr ,V,T).

6. The method for identifying Internet of Things devices based on the implicit features of flags according to claim 5, wherein The process of generating the weight vector and constructing the fingerprint entry is as follows: for multiple devices with the same representation vector, for each feature vector F, calculate the importance θ i of its i-th component f i . Normalize the importance θ i of all features to generate the weight vector W; finally, use (F, F repr , V, T) and W to form a fingerprint entry FP = (F, F repr , W, V, T), and add it to the fingerprint database.

7. The method for identifying Internet of Things devices based on the hidden features of flags according to claim 6, characterized in that For the component f of the eigenvector i , calculate its importance θ i including: among all eigenvectors, the reciprocal of the number of eigenvectors with the same value as the current eigenvector on the i-th feature is the importance θ of this feature i .

8. The method for identifying an Internet of Things device based on the implicit features of a flag according to claim 6, wherein The steps of device identification include: Target data acquisition and preparation: Send an HTTP request to the device to be identified, obtain the header information of the response message, and generate a target feature vector and a target feature vector; Candidate fingerprint screening: Screen out candidate fingerprints related to the target device according to the target feature vector; Target device vendor and type identification: Determine the vendor and type of the target device by calculating the similarity between the target feature vector and the candidate fingerprint feature vector.

9. The method for identifying an Internet of Things device based on the implicit features of a flag according to claim 8, wherein The process of calculating similarity is as follows: For each candidate fingerprint FP, calculate the target feature vector F T The similarity between the fingerprint feature vector FP.F Similarity The calculation formula is: Among them, d(f i T , f i ) is the similarity of the feature components, and w i is the weight component; For the data type of the feature component being a string type, calculate the similarity using the ratio of the edit distance to the maximum string length; For the data type of the feature component being an integer type, calculate the similarity using the ratio of the absolute value of the difference to the maximum value of the feature value; For the data type of the feature component being an enumerated type, calculate the similarity using the maximum value of the enumerated values.

10. An Internet of Things device identification system based on the implicit features of flags, characterized in that, An Internet of Things device identification method based on flag implicit features according to any one of claims 1-9, comprising a fingerprint database construction module and a device identification module, wherein: The fingerprint database construction module is used to construct a fingerprint database by preprocessing, feature vector construction, feature vector extraction, and feature weight calculation on a set of HTTP / HTTPS response message headers with vendor and type tags collected; The device identification module is used to obtain the vendor and type of the target device by calculating the similarity between the headers of the HTTP / HTTPS response message of the target device and the entries in the fingerprint database.

Images