Identification method and device of Internet of Things equipment, storage medium and electronic equipment

Abstract

The invention discloses an Internet of Things equipment identification method and device, a storage medium and electronic equipment. Relates to the field of Internet of Things, and the method comprises the following steps: obtaining device data of a to-be-identified Internet of Things device, the device data comprising at least one of device information of the Internet of Things device and device flow data of the Internet of Things device, and working audio data of the Internet of Things device; performing feature extraction on the equipment data to obtain a target feature set; and determining a target device type of the Internet of Things device according to the target feature set. According to the invention, the technical problem of low identification accuracy of the device type of the Internet of Things device in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things, and in particular to an identification method, device, storage medium and electronic device of Internet of Things devices. Background Art

[0002] With the rapid development of computers, smart homes, smart cities, industrial systems, and smart healthcare have become hot topics. At the same time, the security protection of the Internet of Things has not been significantly improved with the rapid development of the Internet of Things. The vulnerabilities of IoT devices pose a huge threat. IoT devices often lack "visibility", making their security issues more difficult.

[0003] The first step in IoT security is to identify IoT devices. Knowing the types of devices connected to the network helps to strengthen security. For example, by identifying IoT devices, IoT assets can be identified and behavior can be controlled, while targeted vulnerability management and security defense can be completed. Currently, related technologies identify the device types of IoT devices based on manually set rules, which results in low recognition accuracy.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method, apparatus, storage medium and electronic device for identifying an Internet of Things device, so as to at least solve the technical problem of low accuracy in identifying the device type of an Internet of Things device in the related art.

[0006] According to one aspect of an embodiment of the present invention, a method for identifying an Internet of Things device is provided, comprising: obtaining device data of an Internet of Things device to be identified, wherein the device data includes at least one of device information of the Internet of Things device and device traffic data of the Internet of Things device, and working audio data of the Internet of Things device; performing feature extraction on the device data to obtain a target feature set; and determining a target device type of the Internet of Things device according to the target feature set.

[0007] Furthermore, the identification method of the Internet of Things device also includes: when the device data includes device traffic data, feature extraction is performed on the device traffic data according to multiple preset feature dimensions to obtain feature values ​​corresponding to the multiple feature dimensions, and statistical feature extraction is performed on the feature values ​​corresponding to the multiple feature dimensions; when the device data includes working audio data, Mel-frequency cepstral coefficients in the working audio data are extracted, and feature extraction is performed on the extracted Mel-frequency cepstral coefficients through an audio feature extraction model; when the device data includes device information, feature extraction is performed on the device information through a target language model.

[0008] Furthermore, the identification method of the Internet of Things device also includes: for each feature in the target feature set, using at least one first device identification model to process the feature to obtain the identification result corresponding to the feature under each first device identification model, wherein the first device identification model refers to a device identification model that matches the feature, and the feature is one of the following: a first feature that matches device traffic data, a second feature that matches working audio data, and a third feature that matches device information; according to the identification result of each feature under each first device identification model, determine the target device type of the Internet of Things device.

[0009] Furthermore, the identification method of the Internet of Things device also includes: for each feature, determining a target identification result of the feature matching according to the identification result of the feature under each first device identification model; and determining a target device type of the Internet of Things device according to the target identification result of each feature matching.

[0010] Furthermore, the identification result includes at least a probability value that the IoT device belongs to different device types, wherein the method for identifying the IoT device also includes: for each device type, determining a target probability value corresponding to the IoT device under the device type according to the probability value of the device type in each identification result; and determining a target identification result according to the target probability value corresponding to the IoT device under each device type.

[0011] Furthermore, the target identification result includes the device type to which the identified Internet of Things device belongs and the target probability value corresponding to the device type, wherein the identification method of the Internet of Things device also includes: judging whether the same device type exists in the target identification results of each feature match; in the case that the same device type exists in the target identification result, determining the target device type of the Internet of Things device according to the number of occurrences of each device type in the target identification result; in the case that the same device type does not exist in the target identification result, determining the target device type of the Internet of Things device according to the target probability value in each target identification result.

[0012] Furthermore, the identification method of the Internet of Things device also includes: performing feature fusion on the features in the target feature set to obtain the target features; processing the target features using a second device identification model to obtain the identification results of the second device identification model, and determining the target device type of the Internet of Things device based on the identification results of the second device identification model.

[0013] According to another aspect of an embodiment of the present invention, there is also provided an identification device for an Internet of Things device, including: an acquisition module, used to acquire device data of an Internet of Things device to be identified, wherein the device data includes at least one of device information of the Internet of Things device and device traffic data of the Internet of Things device, and working audio data of the Internet of Things device; a feature extraction module, used to extract features from the device data to obtain a target feature set; and a determination module, used to determine a target device type of the Internet of Things device based on the target feature set.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for identifying an Internet of Things device when running.

[0015] According to another aspect of an embodiment of the present invention, an electronic device is further provided, the electronic device comprising one or more processors; a memory for storing one or more programs, so that when the one or more programs are executed by the one or more processors, the one or more processors are implemented to run the programs, wherein the programs are configured to execute the above-mentioned method for identifying Internet of Things devices when running.

[0016] In the embodiment of the present invention, the device type of the IoT device is identified based on the multi-dimensional data of the IoT device, by acquiring the device data of the IoT device to be identified, and then extracting features from the device data to obtain a target feature set, thereby determining the target device type of the IoT device based on the target feature set. The device data includes at least one of the device information of the IoT device and the device flow data of the IoT device, as well as the working audio data of the IoT device.

[0017] In the above process, by acquiring the device data of the IoT device to be identified, and setting the device data to include at least one of the device information of the IoT device and the device traffic data of the IoT device, as well as the working audio data of the IoT device, effective acquisition of multi-dimensional data of the IoT device to be identified is achieved. By performing feature extraction on the device data, a target feature set is obtained, and accurate determination of the multi-dimensional features of the IoT device is achieved, thereby effectively improving the accuracy of determining the target device type of the IoT device according to the target feature set.

[0018] It can be seen that the solution provided in the present application achieves the purpose of identifying the device type of the Internet of Things device based on the multi-dimensional data of the Internet of Things device, thereby achieving the technical effect of improving the recognition accuracy, and further solving the technical problem of low recognition accuracy of the device type of the Internet of Things device in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 Schematic diagram of an optional method for identifying an IoT device according to an embodiment of the present invention Figure 1 ;

[0021] Figure 2 is a schematic diagram of an optional target recognition result of determining a first feature match according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of an optional target recognition result of determining a second feature match according to an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of an optional target recognition result of determining a third feature match according to an embodiment of the present invention;

[0024] Figure 5 is an exemplary embodiment of an optional method for determining the target device type according to an embodiment of the present invention. Figure 1 ;

[0025] Figure 6 is an exemplary embodiment of an optional method for determining the target device type according to an embodiment of the present invention. Figure 2 ;

[0026] Figure 7 Schematic diagram of an optional method for identifying an IoT device according to an embodiment of the present invention Figure 2 ;

[0027] Figure 8 is a schematic diagram of an optional identification device for an Internet of Things device according to an embodiment of the present invention;

[0028] Fig. 9 is a schematic diagram of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] Example 1

[0033] According to an embodiment of the present invention, an embodiment of a method for identifying an Internet of Things device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] Figure 1 Schematic diagram of an optional method for identifying an IoT device according to an embodiment of the present invention Figure 1 ,like Figure 1 As shown, the method comprises the following steps:

[0035] Step S101, obtaining device data of the IoT device to be identified, wherein the device data includes at least one of device information of the IoT device and device traffic data of the IoT device, and working audio data of the IoT device.

[0036] Optionally, electronic devices, application systems, servers and other devices may be used as the execution subject of the present application. In this embodiment, the target identification system is used as the execution subject to execute the identification method of the Internet of Things device.

[0037] In some embodiments, before the IoT device accesses the intranet, that is, when the IoT device requests access to the intranet, the device type of the IoT device can be identified to determine whether to allow the IoT device to access the intranet based on the identification result (that is, the target device type of the IoT device).

[0038] In some embodiments, after the IoT device is connected to the intranet, the device type of the IoT device can also be identified to perform behavior control on the IoT device based on the identification results to enhance IoT security. For example, behavior control may mean not allowing the traffic of the IoT device to be sent to devices outside the intranet, or not allowing devices outside the intranet to access the IoT device.

[0039] Optionally, the device data includes at least one of device information of the IoT device and device traffic data of the IoT device, and working audio data of the IoT device.

[0040] In some embodiments, the device data includes device information, device traffic data, and working audio data.

[0041] Optionally, the device traffic data of the IoT device can be collected through a traffic collection device. The device traffic data is the traffic sent by the IoT device. For example, the device traffic data can refer to the traffic sent by the IoT device to the firewall or gateway associated with the intranet, or it can refer to the traffic sent by the IoT device to the terminal held by the user in the organization where the intranet is located. Optionally, the device traffic data can be collected by the target identification system, or by the user, and then the device traffic data is marked based on the unique device identifier of the IoT device (such as MAC (Media Access Control Address, media access control address) address, serial number, product ID, device name, device key), etc., and then the marked device traffic data is sent to the target identification system so that the target identification system can obtain the device traffic data corresponding to the IoT device to be identified.

[0042] Optionally, the working audio data of the IoT device refers to the audio data generated when the IoT device is running. The user can set up a corresponding audio acquisition device next to the IoT device to collect the working audio data of the IoT device from startup to operation, and mark the working audio data based on the device unique identifier of the IoT device, and then send the marked working audio data to the target recognition system, so that the target recognition system can obtain the working audio data corresponding to the IoT device to be identified.

[0043] In some embodiments, the collection time range of the working audio data is the same as the collection time range of the device traffic data.

[0044] Optionally, the device information of an IoT device refers to descriptive information for the IoT device, for example, the device information includes the flow information of the IoT device (such as: device flow data of IoT device A in time period B), content (such as the protocol name adopted by the device, device attributes, etc.), production year, manufacturer, etc. The device information of the IoT device can be determined by the user. The user can mark the collected device information of the IoT device based on the device unique identifier of the IoT device, and then send the marked device information to the target identification system so that the target identification system can obtain the device information corresponding to the IoT device to be identified.

[0045] Step S102, extracting features from device data to obtain a target feature set.

[0046] In some embodiments, the target recognition system may use the same feature extraction method to extract features from different types of data in the device data, or the target recognition system may use different feature extraction methods to extract features from different types of data in the device data.

[0047] For example, the above-mentioned feature extraction methods include but are not limited to: feature extraction using a feature extraction model, statistical methods, principal component analysis, linear discriminant methods, etc.

[0048] In some embodiments, the target feature set includes at least two features. For example, when the device data includes the above-mentioned device information, device flow data, and working audio data, the target feature set may include a first feature matched by the device flow data, a second feature matched by the working audio data, and a third feature matched by the device information.

[0049] Step S103: determining the target device type of the IoT device according to the target feature set.

[0050] In some embodiments, each device type may include information about the device in multiple target dimensions, including but not limited to: device type, device brand, device model, hardware (such as hard disk, memory, etc.) model in the device, etc.

[0051] Optionally, after determining the target feature set, the target recognition system may process the features in the target feature set through a device recognition model to determine the target device type of the IoT device.

[0052] In some embodiments, different features in the target feature set can be processed by different device recognition models to obtain recognition results corresponding to each feature, thereby determining the target device type of the IoT device based on the recognition results corresponding to each feature. For example, the recognition result of the traffic dimension is obtained by a device recognition model that matches the device traffic data, the recognition result of the audio dimension is obtained by a device recognition model that matches the working audio data, and the recognition result of the device information dimension is obtained by a device recognition model that matches the device information, thereby determining the target device type of the IoT device based on the recognition results of the aforementioned dimensions.

[0053] In some embodiments, the features in the target feature set may be fused, and the fused features may be processed through a device recognition model to obtain a unique recognition result, thereby determining the target device type of the IoT device based on the recognition result.

[0054] Optionally, after obtaining the target device type of the IoT device, it can be determined whether to allow the IoT device to access the intranet based on the target device type of the IoT device, or a behavior control method for the IoT device can be determined based on the target device type of the IoT device.

[0055] Based on the scheme defined in the above steps S101 to S103, it can be known that in the embodiment of the present invention, the device type of the IoT device is identified based on the multi-dimensional data of the IoT device, by obtaining the device data of the IoT device to be identified, and then extracting features from the device data to obtain a target feature set, thereby determining the target device type of the IoT device based on the target feature set. The device data includes at least one of the device information of the IoT device and the device flow data of the IoT device, as well as the working audio data of the IoT device.

[0056] It is easy to notice that in the above process, by acquiring the device data of the IoT device to be identified, and setting the device data to include at least one of the device information of the IoT device and the device traffic data of the IoT device, as well as the working audio data of the IoT device, the effective acquisition of multi-dimensional data of the IoT device to be identified is achieved. By performing feature extraction on the device data, a target feature set is obtained, and accurate determination of the multi-dimensional features of the IoT device is achieved, thereby effectively improving the accuracy of determining the target device type of the IoT device according to the target feature set.

[0057] It can be seen that the solution provided in the present application achieves the purpose of identifying the device type of the Internet of Things device based on the multi-dimensional data of the Internet of Things device, thereby achieving the technical effect of improving the recognition accuracy, and further solving the technical problem of low recognition accuracy of the device type of the Internet of Things device in the related technology.

[0058] In an optional embodiment, in the process of extracting features from device data, the target recognition system may, when the device data includes device traffic data, perform feature extraction on the device traffic data based on multiple preset feature dimensions to obtain feature values ​​corresponding to the multiple feature dimensions, and perform statistical feature extraction on the feature values ​​corresponding to the multiple feature dimensions; when the device data includes working audio data, extract Mel-frequency cepstral coefficients from the working audio data, and perform feature extraction on the extracted Mel-frequency cepstral coefficients through an audio feature extraction model; when the device data includes device information, perform feature extraction on the device information through a target language model.

[0059] In some embodiments, when the device data includes device flow data, the target identification system can perform feature extraction on the device flow data based on multiple preset feature dimensions, obtain feature values ​​corresponding to the multiple feature dimensions, and perform statistical feature extraction on the feature values ​​corresponding to the multiple feature dimensions.

[0060] Optionally, the aforementioned multiple feature dimensions include but are not limited to packet size, packet interval time, transport layer protocol, packet TCP (Transmission Control Protocol) source and destination ports, UDP (User Datagram Protocol) source and destination ports, etc., and the aforementioned statistical features include but are not limited to mean value, absolute energy value, skewness, kurtosis, etc.

[0061] For example, the target recognition system can perform data modeling and feature data extraction on the collected device traffic data, extracting 7 types of basic data such as packet size, packet interval time, transport layer protocol, packet TCP source and destination port, UDP source and destination port (that is, obtaining feature values ​​corresponding to multiple feature dimensions), and on this basis, using multiple consecutive packets sent from IoT devices, calculate the average value, absolute energy value, skewness, kurtosis and other 10 statistical features of these 7 types of basic data, thereby constructing a fingerprint feature data containing 70+ features for the IoT device. Optionally, in the aforementioned process, non-numeric data can also be converted into numerical features in advance.

[0062] Optionally, before extracting features from the device traffic data according to the preset multiple feature dimensions, the target recognition system may also pre-process the device traffic data, for example, deleting data packets that are too large or too small, deleting abnormal data, etc. After performing statistical feature extraction on the feature values ​​corresponding to the multiple feature dimensions, the target recognition system may perform feature pre-processing on the obtained fingerprint feature data, for example, deleting abnormal feature values, etc.

[0063] In some embodiments, when the device data includes working audio data, the target recognition system can extract Mel Frequency Cepstrum Coefficients (MFCC) from the working audio data, and perform feature extraction on the extracted Mel Frequency Cepstrum Coefficients through an audio feature extraction model.

[0064] Optionally, the aforementioned audio feature extraction model may be a VGGish model or other deep learning models with audio feature extraction function.

[0065] For example, the target recognition system can extract the Mel frequency cepstral coefficients from the collected working audio data, and then extract the Mel frequency cepstral coefficients through the VGGish model to obtain VGGish features, that is, to obtain audio feature data. Optionally, if the feature dimension is too high, the PCA (Principal Components Analysis) method can be used for dimensionality reduction.

[0066] Optionally, before extracting the Mel-frequency cepstral coefficients in the working audio data, the target recognition system may also perform audio preprocessing on the working audio data, for example, the audio preprocessing includes at least one of the following: segmentation, pre-emphasis, normalization, frame windowing, Mel filter bank, etc. After performing feature extraction on the extracted Mel-frequency cepstral coefficients through the audio feature extraction model, the target recognition system may perform feature preprocessing on the obtained audio feature data, for example, deleting abnormal feature values, etc.

[0067] In some embodiments, when the device data includes device information, the target recognition system may extract features from the device information using a target language model.

[0068] Optionally, the aforementioned target language model may be a Word2vec model or other deep learning model with text feature extraction capability.

[0069] For example, the target recognition system can first perform text preprocessing on the device information, and the text preprocessing includes at least one of the following: noise removal, word segmentation, and stop word removal. After that, the target recognition system can perform word embedding processing on the device information through the Word2vec model to obtain device information feature data.

[0070] It should be noted that for device traffic data, by constructing several statistical indicator data such as mean and kurtosis after extracting features, the representativeness of traffic data features can be effectively improved. For working audio data, by extracting Mel-frequency cepstral coefficients, effective extraction of features in audio is achieved. For device information, by using the target language model to extract features, effective extraction of semantic information in device information is achieved, thereby improving feature extraction accuracy.

[0071] In an optional embodiment, in the process of determining the target device type of the IoT device according to the target feature set, the target recognition system can process each feature in the target feature set using at least one first device recognition model to obtain the recognition result corresponding to the feature under each first device recognition model, thereby determining the target device type of the IoT device according to the recognition result of each feature under each first device recognition model. The first device recognition model refers to a device recognition model that matches the feature, and the feature is one of the following: a first feature that matches the device flow data, a second feature that matches the working audio data, and a third feature that matches the device information.

[0072] In some embodiments, after performing statistical feature extraction on the feature values ​​corresponding to multiple feature dimensions, a first feature of the device traffic data matching is obtained (equivalent to the above-mentioned fingerprint feature data), after performing feature extraction on the extracted Mel-frequency cepstral coefficients through an audio feature extraction model, a second feature of the working audio data matching is obtained (equivalent to the above-mentioned audio feature data), and after performing feature extraction on the device information through a target language model, a third feature of the device information matching is obtained (equivalent to the above-mentioned device information feature data).

[0073] In some embodiments, the first device identification model refers to a device identification model that matches the feature, and different features correspond to different first device identification models.

[0074] In some embodiments, for the first feature, at least one first device identification model is used to identify the device type based on the feature of the device traffic data. The at least one first device identification model can be trained based on the first training sample set, where the training samples in the first training sample set are sample first features corresponding to the sample device traffic data of the sample IoT device, and the real labels in the first training sample set represent the real device type of the sample IoT device. Among them, the first training sample set can be divided into a training set and a validation set, and the process of obtaining the sample first feature is the same as the process of obtaining the first feature, so it will not be repeated here.

[0075] In some embodiments, for the first feature, there may be multiple first device identification models. Model types of the first device identification model include but are not limited to SVM (Support Vector Machine), LR (Logistic Regression), RF (Random Forest), XGB (eXtreme Gradient Boosting), DT (Decision Tree), etc.

[0076] In some embodiments, for the second feature, at least one first device identification model is used to identify the device type based on the feature of the working audio data. The at least one first device identification model can be trained according to the second training sample set, and the training samples in the second training sample set are sample second features corresponding to the sample working audio data of the sample IoT device, and the real labels in the second training sample set represent the real device type of the sample IoT device. Among them, the second training sample set can be divided into a training set and a validation set, and the process of obtaining the sample second feature is the same as the process of obtaining the second feature, so it will not be repeated here.

[0077] In some embodiments, for the second feature, the first device identification model may be multiple. Model types of the first device identification model include but are not limited to SVM (Support Vector Machine), LR (Logistic Regression), RF (Random Forest), XGB (eXtreme Gradient Boosting), DT (Decision Tree), etc.

[0078] In some embodiments, for the third feature, at least one first device identification model is used to identify the device type based on the feature of the device information. The at least one first device identification model can be trained based on the third training sample set, and the training samples in the third training sample set are sample third features corresponding to the sample device information of the sample IoT device, and the real labels in the third training sample set represent the real device type of the sample IoT device. Among them, the third training sample set can be divided into a training set and a validation set, and the process of obtaining the sample third feature is the same as the process of obtaining the third feature, so it will not be repeated here.

[0079] In some embodiments, for the third feature, the first device identification model may be one. The model type of the first device identification model includes but is not limited to a TextCNN model, a recurrent neural network model, and the like.

[0080] Optionally, for each feature in the target feature set, the target recognition system may process the feature using at least one first device recognition model to obtain a recognition result corresponding to the feature under each first device recognition model.

[0081] In some embodiments, the identification result includes at least a probability value that the IoT device belongs to different device types.

[0082] Optionally, after obtaining the recognition result, the target recognition system may determine the target device type of the IoT device according to the recognition result of each feature under each first device recognition model.

[0083] For example, the device type corresponding to the maximum probability value in all recognition results is directly determined as the target device type.

[0084] For another example, for each device type, the total probability value of the device type is calculated according to the probability values ​​in all recognition results, so that the device type corresponding to the maximum total probability value is determined as the target device type.

[0085] For another example, for each feature, the target recognition result of the feature matching is first determined based on the recognition result of the feature under each first device recognition model, and then the target device type of the Internet of Things device is determined based on the target recognition results of each feature matching.

[0086] It should be noted that, through the above process, it is possible to use different device identification models to identify the IoT device in different dimensions, that is, to establish a multi-level classification task, which can effectively improve the accuracy of device identification. In addition, by using at least one device identification model to process the features for device identification, on the one hand, it can effectively avoid the local optimum caused by the deviation brought by a single experiment, thereby improving the accuracy of the identification result. On the other hand, it has strong scalability and is easy to mix with other algorithm technologies, thereby improving the application flexibility of this application.

[0087] In an optional embodiment, in the process of determining the target device type of the Internet of Things device based on the recognition results of each feature under each first device recognition model, the target recognition system can determine the target recognition result of the feature matching for each feature based on the recognition results of the feature under each first device recognition model, thereby determining the target device type of the Internet of Things device based on the target recognition result of each feature matching.

[0088] For example, Figure 2 is a schematic diagram of an optional target recognition result for determining the first feature matching according to an embodiment of the present invention, according to Figure 2 An optional process of obtaining a target recognition result by matching the first feature is described. Figure 2As shown, in the case where the device data includes device flow data, the target recognition system can first pre-process the device flow data, then extract features such as packet size and interval time, then perform statistical feature extraction, and perform feature pre-processing to obtain the first feature. Afterwards, the first feature is input into the corresponding multiple first device recognition models (i.e. Figure 2 The SVM, LR, RF, XGB, DT and other models in the first device recognition model are used to obtain the recognition results of each first device recognition model, and then the recognition results of each first device recognition model are fused to obtain the target recognition result of the first feature matching.

[0089] For example, Figure 3 is a schematic diagram of an optional target recognition result for determining the second feature matching according to an embodiment of the present invention, according to Figure 3 An optional process for obtaining a target recognition result by matching the second feature is described. Figure 3 As shown in FIG. 1 , when the device data includes working audio data, the target recognition system can first pre-process the working audio data, then perform MFCC feature extraction, then perform feature extraction on MFCC using the VGGish model, and perform feature pre-processing on the extracted features to obtain the second feature. Afterwards, the second feature is input into the corresponding multiple first device recognition models (i.e. Figure 3 The RF, XGB, DT and other models in the first device recognition model are used to obtain the recognition results of each first device recognition model, thereby fusing the recognition results of each first device recognition model to obtain the target recognition result of the second feature matching.

[0090] For example, Figure 4 is a schematic diagram of an optional target recognition result of determining the third feature matching according to an embodiment of the present invention, according to Figure 4 An optional process for obtaining a target recognition result by matching the third feature is described. Figure 4 As shown in FIG. 1 , when the device data includes device information, the target recognition system can first pre-process the device information, and then perform word embedding processing on the device information through the Word2vec model to obtain the third feature. Then, the third feature is input into the corresponding first device recognition model (i.e. Figure 4 The TextCNN model in the text is used to obtain the recognition result of the first device recognition model, thereby determining the recognition result of the first device recognition model as the target recognition result of the third feature matching.

[0091] It should be noted that by first integrating the recognition results of the dimension where each feature is located, the target recognition result of the dimension where the feature is located is obtained, which reduces the biased recognition of a single mode. By determining the target device type of the IoT device based on the target recognition results of different dimensions, effective reference to the recognition results of different dimensions is achieved, thereby further improving the recognition accuracy.

[0092] In an optional embodiment, the recognition result includes at least a probability value that the IoT device belongs to different device types. In the process of determining the target recognition result of feature matching based on the recognition result of the feature under each first device recognition model, the target recognition system can determine the target probability value corresponding to the IoT device under the device type for each device type based on the probability value of the device type in each recognition result, thereby determining the target recognition result based on the target probability value corresponding to the IoT device under each device type.

[0093] Optionally, when there is only one first device identification model corresponding to a certain feature, that is, when there is only one identification result corresponding to the feature, for each device type, the target identification system can directly determine the probability value of the device type in the identification result as the target probability value corresponding to the Internet of Things device under the device type.

[0094] Optionally, when there are multiple first device identification models corresponding to a certain feature, that is, when there are multiple identification results corresponding to the feature, for each device type, the target identification system can average the probability values ​​of the device type in each identification result or perform weighted summation and then average processing, so as to determine the processed probability value as the target probability value corresponding to the IoT device under the device type.

[0095] For example, if there are three recognition results, the three recognition results are "A device type: 30%, B device type: 70%", "A device type: 20%, B device type: 80%", "A device type: 40%, B device type: 60%", and the processing method is average processing, then the target probability value corresponding to the IoT device under device type A is 30%, and the target probability value corresponding to the IoT device under device type B is 70%.

[0096] Optionally, after obtaining the target probability value corresponding to the IoT device under each device type, the target recognition system may determine the target recognition result according to the target probability value corresponding to the IoT device under each device type.

[0097] In some embodiments, the target recognition result includes the device type to which the identified IoT device belongs and the target probability value corresponding to the device type. For example, the device type with the largest target probability value is determined as the device type to which the identified IoT device belongs.

[0098] It should be noted that, through the above process, by determining the target recognition result of feature matching according to the probability value in the recognition result of feature matching, the recognition accuracy can be further improved.

[0099] In an optional embodiment, the target identification result includes the device type to which the identified Internet of Things device belongs and the target probability value corresponding to the device type, wherein, in the process of determining the target device type of the Internet of Things device based on the target identification result of each feature match, the target identification system can determine whether the same device type exists in the target identification results of each feature match, and then, if the same device type exists in the target identification result, determine the target device type of the Internet of Things device based on the number of occurrences of each device type in the target identification result, and thus, if the same device type does not exist in the target identification result, determine the target device type of the Internet of Things device based on the target probability value in each target identification result.

[0100] Optionally, the target recognition result includes the device type to which the identified IoT device belongs and the target probability value corresponding to the device type. That is, each target recognition result includes only one device type. Therefore, the target recognition system can determine the target device type of the IoT device based on the principle of minority obeys majority. Optionally, the target recognition system can determine whether the same device type exists in the target recognition results of each feature match. If so, the target device type of the IoT device is determined according to the number of occurrences of each device type in the target recognition result. For example, the device type with the most occurrences is determined as the target device type of the IoT device. Optionally, if there are multiple device types with the most occurrences, the target device type can be determined based on the target probability value of the device type in each target recognition result. For example, the target probability value of the device type in each target recognition result is averaged to obtain the average value of the target probability value, so that the device type with the highest average value of the target probability value among the multiple device types with the most occurrences is determined as the target device type.

[0101] Optionally, if the same device type does not exist in the target recognition result, the target recognition system may determine that each device type appears at most once. In this case, the target recognition system may determine the device type with the highest target probability value as the target device type.

[0102] For example, Figure 5 is an exemplary embodiment of an optional method for determining the target device type according to an embodiment of the present invention. Figure 1 ,according to Figure 5 An optional process for determining the target device type is described. In the case where the device data includes device flow data, working audio data, and device information, such as Figure 5 As shown, the target recognition system can use the corresponding device recognition model to process the first feature of the device traffic data match, the second feature of the working audio data match, and the third feature of the device information match to obtain the target recognition result of the first feature match, the target recognition result of the second feature match, and the target recognition result of the third feature match. After that, the target recognition system can determine whether the principle of minority obeys majority can be adopted according to the device types in the three target recognition results. If so, the target device type of the Internet of Things device is determined according to the number of appearances of each device type in the target recognition results. If not, the device type with the largest target probability value is determined as the target device type.

[0103] It should be noted that by determining the target device type based on the principle of minority obeys majority, the amount of data calculation can be effectively reduced, thereby improving the efficiency of determining the target device type. By determining the target device type based on probability value judgment when the principle of minority obeys majority cannot be adopted, the accuracy of determining the target device type is improved.

[0104] In an optional embodiment, in the process of determining the target device type of the Internet of Things device based on the target feature set, the target recognition system can perform feature fusion on the features in the target feature set to obtain the target features, and then use the second device recognition model to process the target features to obtain the recognition result of the second device recognition model, and determine the target device type of the Internet of Things device based on the recognition result of the second device recognition model.

[0105] Optionally, the second device identification model can be based on the fourth training sample set trained, where the training samples in the fourth training sample set are obtained by feature fusion of the sample first feature, sample second feature and sample third feature of the above-mentioned sample Internet of Things devices, and the real labels in the fourth training sample set represent the real device type of the sample Internet of Things devices.

[0106] Optionally, the target recognition system may perform feature fusion on the features in the target feature set to obtain the target feature. Afterwards, the target recognition system may input the target feature into the second device recognition model to obtain the recognition result of the second device recognition model, thereby determining the target device type of the IoT device according to the recognition result of the second device recognition model.

[0107] In some embodiments, the recognition result of the second device recognition model includes probability values ​​of the IoT device belonging to different device types, and the target recognition system can determine the device type with the largest probability value in the recognition result of the second device recognition model as the target device type.

[0108] In some embodiments, the second device identification model may be one or more. In the case of multiple second device identification models, the target features may be input into each second device identification model respectively to obtain the identification results of each second device identification model, and then the probability values ​​in the identification results of each second device identification model may be integrated according to the device type, so as to determine the target device type of the IoT device according to the integrated probability values ​​corresponding to each device type.

[0109] In some embodiments, the second device model can be a deep learning model or a machine learning model, for example, the SVM, LR, RF, XGB, DT and other models mentioned above.

[0110] For example, Figure 6 is an exemplary embodiment of an optional method for determining the target device type according to an embodiment of the present invention. Figure 2 ,according to Figure 6 An optional process for determining the target device type is described. In the case where the device data includes device flow data, working audio data, and device information, such as Figure 6 As shown, the target recognition system can perform feature fusion on the first feature matched by the device traffic data, the second feature matched by the working audio data, and the third feature matched by the device information to obtain the target feature, and then input the target feature into the second device recognition model, thereby determining the target device type of the IoT device according to the recognition result of the second device recognition model.

[0111] It should be noted that, through the above process, it is possible to use relatively few models to process the features of IoT devices to identify the device types of IoT devices, thereby improving the recognition efficiency.

[0112] In some embodiments, Figure 7 Schematic diagram of an optional method for identifying an IoT device according to an embodiment of the present invention Figure 2 ,according to Figure 7 An optional identification process of an IoT device in this embodiment is described. Figure 7 As shown, when the device data includes device flow data, working audio data and device information, the target recognition system first extracts features from each data to obtain the first feature, the second feature and the third feature. Then, according to different actual application requirements, in one scenario, the target recognition system can use the device recognition model corresponding to the feature (i.e., the first device recognition model) to process the feature to obtain the recognition result, and process the recognition result to obtain the target recognition results corresponding to the first feature, the second feature and the third feature respectively. Then, the target device type of the IoT device is determined based on the target recognition result.

[0113] like Figure 7As shown, in another scenario, the target recognition system can perform feature fusion on the first feature, the second feature, and the third feature obtained above, and then use the second device recognition model to process the fused features to obtain a recognition result, thereby determining the target device type of the IoT device based on the recognition result.

[0114] Optionally, this application is applicable to the scene of IoT device identification, and can be extended to the identification of other device multi-classification scenes, or more general multi-classification scenes. In this application, by obtaining the device flow data, working audio data, and device information of the IoT device, and performing corresponding processing based on the data of different dimensions to obtain the corresponding features, models of different dimensions are constructed for identification processing or a unified model is used to identify the fused features to obtain the final device category, thereby realizing the consideration of the features of the IoT device in different dimensions, so that the characterization features of the device can be extracted more accurately, thereby improving the accuracy of device identification.

[0115] It can be seen that the solution provided in the present application achieves the purpose of identifying the device type of the Internet of Things device based on the multi-dimensional data of the Internet of Things device, thereby achieving the technical effect of improving the recognition accuracy, and further solving the technical problem of low recognition accuracy of the device type of the Internet of Things device in the related technology.

[0116] Example 2

[0117] According to an embodiment of the present invention, an embodiment of an identification device for an Internet of Things device is provided, wherein: Figure 8 is a schematic diagram of an optional identification device for an Internet of Things device according to an embodiment of the present invention, such as Figure 8 As shown, the device comprises:

[0118] An acquisition module 801 is used to acquire device data of an IoT device to be identified, wherein the device data includes at least one of device information of the IoT device and device flow data of the IoT device, and working audio data of the IoT device;

[0119] The feature extraction module 802 is used to extract features from device data to obtain a target feature set;

[0120] The determination module 803 is used to determine the target device type of the Internet of Things device according to the target feature set.

[0121] It should be noted that the above-mentioned acquisition module 801, feature extraction module 802 and determination module 803 correspond to steps S101 to S103 in the above-mentioned embodiment. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment 1.

[0122] Optionally, the feature extraction module 802 also includes: a first feature extraction submodule, which is used to, when the device data includes device traffic data, perform feature extraction on the device traffic data according to multiple preset feature dimensions, obtain feature values ​​corresponding to the multiple feature dimensions, and perform statistical feature extraction on the feature values ​​corresponding to the multiple feature dimensions; a second feature extraction submodule, which is used to, when the device data includes working audio data, extract Mel-frequency cepstral coefficients in the working audio data, and perform feature extraction on the extracted Mel-frequency cepstral coefficients through an audio feature extraction model; and a third feature extraction submodule, which is used to, when the device data includes device information, perform feature extraction on the device information through a target language model.

[0123] Optionally, the determination module 803 also includes: a first processing submodule, used to process each feature in the target feature set using at least one first device identification model to obtain an identification result corresponding to the feature under each first device identification model, wherein the first device identification model refers to a device identification model that matches the feature, and the feature is one of the following: a first feature that matches device traffic data, a second feature that matches working audio data, and a third feature that matches device information; a determination submodule, used to determine the target device type of the Internet of Things device according to the identification result of each feature under each first device identification model.

[0124] Optionally, the determination submodule also includes: a first determination unit, used to determine, for each feature, a target recognition result of feature matching according to the recognition result of the feature under each first device recognition model; and a second determination unit, used to determine the target device type of the Internet of Things device according to the target recognition result of each feature matching.

[0125] Optionally, the first determination unit also includes: a first determination subunit, used to determine, for each device type, a target probability value corresponding to the Internet of Things device under the device type according to the probability value of the device type in each identification result; and a second determination subunit, used to determine the target identification result according to the target probability value corresponding to the Internet of Things device under each device type.

[0126] Optionally, the second determination unit also includes: a first judgment subunit, used to judge whether the same device type exists in the target identification results of each feature match; a third determination subunit, used to determine the target device type of the Internet of Things device according to the number of occurrences of each device type in the target identification results when the same device type exists in the target identification results; and a fourth determination subunit, used to determine the target device type of the Internet of Things device according to the target probability value in each target identification result when the same device type does not exist in the target identification results.

[0127] Optionally, the determination module 803 also includes: a second processing submodule, used to perform feature fusion on the features in the target feature set to obtain the target features; a third processing submodule, used to process the target features using the second device recognition model to obtain the recognition result of the second device recognition model, and determine the target device type of the Internet of Things device based on the recognition result of the second device recognition model.

[0128] Example 3

[0129] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for identifying an Internet of Things device when running.

[0130] Example 4

[0131] According to another aspect of an embodiment of the present invention, an electronic device is provided, wherein: Fig. 9 is a schematic diagram of an optional electronic device according to an embodiment of the present invention, such as Fig. 9 As shown, the electronic device includes one or more processors; a memory for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to run the programs, wherein the programs are configured to execute the above-mentioned method for identifying Internet of Things devices when running.

[0132] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0133] The embodiments or examples of the present disclosure are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0138] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying an Internet of Things device, characterized in that: include: Acquire device data of the IoT device to be identified, wherein the device data includes at least one of device information of the IoT device and device traffic data of the IoT device, and working audio data of the IoT device; Performing feature extraction on the device data to obtain a target feature set; A target device type of the Internet of Things device is determined according to the target feature set.

2. The method according to claim 1, characterized in that Extracting features from the device data includes: In the case where the device data includes the device flow data, feature extraction is performed on the device flow data according to a plurality of preset feature dimensions to obtain feature values ​​corresponding to the plurality of feature dimensions, and statistical feature extraction is performed on the feature values ​​corresponding to the plurality of feature dimensions; In a case where the device data includes the working audio data, extracting Mel-frequency cepstral coefficients in the working audio data, and performing feature extraction on the extracted Mel-frequency cepstral coefficients through an audio feature extraction model; In a case where the device data includes the device information, feature extraction is performed on the device information using a target language model.

3. The method according to claim 1 or 2, characterized in that: Determining the target device type of the IoT device according to the target feature set includes: For each feature in the target feature set, the feature is processed using at least one first device recognition model to obtain a recognition result corresponding to the feature under each first device recognition model, wherein the first device recognition model refers to a device recognition model that matches the feature, and the feature is one of the following: a first feature matched by the device traffic data, a second feature matched by the working audio data, and a third feature matched by the device information; According to the recognition result of each feature under each first device recognition model, the target device type of the Internet of Things device is determined.

4. The method according to claim 3, characterized in that Determining the target device type of the IoT device according to the recognition result of each feature under each first device recognition model includes: For each feature, determining a target recognition result of the feature matching according to the recognition result of the feature under each first device recognition model; According to the target recognition result of each feature matching, the target device type of the Internet of Things device is determined.

5. The method according to claim 4, characterized in that The recognition result at least includes a probability value that the IoT device belongs to different device types, wherein, according to the recognition results of the feature under each first device recognition model, determining the target recognition result of the feature matching includes: For each device type, determine the target probability value of the IoT device corresponding to the device type according to the probability value of the device type in each recognition result; The target recognition result is determined according to the target probability value corresponding to each device type of the IoT device.

6. The method according to claim 4, characterized in that The target recognition result includes the device type to which the IoT device belongs and the target probability value corresponding to the device type, wherein the target device type of the IoT device is determined according to the target recognition result of each feature match, including: Determine whether the same device type exists in the target recognition results of each feature match; In the case where the same device type exists in the target recognition result, determining the target device type of the IoT device according to the number of occurrences of each device type in the target recognition result; In the case that the same device type does not exist in the target recognition result, the target device type of the Internet of Things device is determined according to the target probability value in each target recognition result.

7. The method according to claim 1, characterized in that Determining the target device type of the IoT device according to the target feature set includes: Performing feature fusion on the features in the target feature set to obtain target features; The target feature is processed using a second device recognition model to obtain a recognition result of the second device recognition model, and the target device type of the Internet of Things device is determined according to the recognition result of the second device recognition model.

8. An identification device for an Internet of Things device, characterized in that: include: An acquisition module, used to acquire device data of an IoT device to be identified, wherein the device data includes at least one of device information of the IoT device and device traffic data of the IoT device, and working audio data of the IoT device; A feature extraction module, used to extract features from the device data to obtain a target feature set; A determination module is used to determine the target device type of the Internet of Things device according to the target feature set.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method for identifying an Internet of Things device as described in any one of claims 1 to 7 when running.

10. An electronic device, characterized in that: The electronic device includes one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to run the programs, wherein the program is configured to execute the method for identifying an Internet of Things device as described in any one of claims 1 to 7 when running.

Images