System and method for determining device attributes using a hierarchy of classifiers

Abstract

A system and method for determining device attributes using a hierarchy of classifiers. The method includes: sequentially applying a plurality of sub-models of a hierarchy to a plurality of features extracted from device activity data, wherein the sequential application ends with applying a last sub-model of the plurality of sub-models, wherein each sub-model includes a plurality of classifiers, wherein each sub-model outputs a class when applied to at least a portion of the plurality of features, wherein each class is a classifier output representing a device attribute, wherein applying the plurality of sub-models further includes iteratively determining a next sub-model to apply based on the class output of the most recently applied sub-model and the hierarchy; and determining the device attribute based on the class output by the last sub-model.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Patent Application No. 16 / 729,823, filed December 30, 2019, the contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to device description, and more particularly to using machine learning to determine device attributes. Background Art

[0004] Whenever a new device is connected to an organization's network or activated in close proximity to an organization's physical location, the new device has the potential to be used by a malicious entity to cause damage to the organization, the network, or both. Due to the rapid increase in the number of new devices now capable of network connectivity, the number of potential problems that can occur due to network access has increased exponentially. In addition, many of these new devices are capable of not only connecting to a network, but also creating their own networks or hotspots.

[0005] Existing solutions for protecting against malicious devices include profiling the device while it is engaged in activities involving connecting to a network. This device profile can be used to determine whether the device is operating consistently with known typical behavior for that class of devices, and therefore whether the device may be exhibiting unusual behavior that indicates a threat.

[0006] Therefore, it would be advantageous to provide a solution that can overcome the above challenges. Summary of the invention

[0007] The following is an overview of several example embodiments of the present disclosure. This overview is provided for the convenience of providing the reader with a basic understanding of these embodiments and does not completely limit the scope of the present disclosure. This overview is not an extensive review of all contemplated embodiments, and is neither intended to identify the key or important elements of all embodiments nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a preface to a more detailed description presented later. For convenience, the term "some embodiments" or "certain embodiments" may be used herein to refer to a single embodiment or multiple embodiments of the present disclosure.

[0008] Certain embodiments disclosed herein include a method for determining device attributes using a hierarchy of classifiers. The method includes: sequentially applying a plurality of sub-models of a hierarchy to a plurality of features extracted from device activity data, wherein the sequential application ends with the application of a last sub-model of the plurality of sub-models, wherein each sub-model includes a plurality of classifiers, wherein each sub-model outputs a class when applied to at least a portion of the plurality of features, wherein each class is a classifier output representing a device attribute, wherein applying the plurality of sub-models further includes iteratively determining a next sub-model to apply based on a most recently applied sub-model and the class output of the hierarchy; and determining the device attribute based on the class output by the last sub-model.

[0009] Certain embodiments disclosed herein also include a non-transitory computer-readable medium having stored thereon a process that causes a processing circuit to execute a process, the process comprising: sequentially applying a plurality of sub-models of a hierarchy to a plurality of features extracted from device activity data, wherein the sequential application ends with the application of a last sub-model of the plurality of sub-models, wherein each sub-model comprises a plurality of classifiers, wherein each sub-model outputs a class when applied to at least a portion of the plurality of features, wherein each class is a classifier output representing a device attribute, wherein applying the plurality of sub-models further comprises iteratively determining a next sub-model to apply based on a most recently applied sub-model and the class output of the hierarchy; and determining the device attribute based on the class output by the last sub-model.

[0010] Certain embodiments disclosed herein also include a system for determining device attributes using a hierarchy of classifiers. The system includes: processing circuitry; and memory containing instructions that, when executed by the processing circuitry, configure the system to: sequentially apply a plurality of sub-models of a hierarchy to a plurality of features extracted from device activity data, wherein the sequential application ends with the application of a last sub-model of the plurality of sub-models, wherein each sub-model includes a plurality of classifiers, wherein each sub-model outputs a class when applied to at least a portion of the plurality of features, wherein each class is a classifier output representing a device attribute, wherein applying the plurality of sub-models further includes iteratively determining a next sub-model to apply based on a most recently applied sub-model and the class output of the hierarchy; and determining the device attribute based on the class output by the last sub-model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features and advantages of the disclosed embodiments will become apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0012] Figure 1 is a network diagram used to describe various disclosed embodiments.

[0013] Figure 2 is a flow chart illustrating a method of determining device attributes using a classifier hierarchy according to one embodiment.

[0014] Figure 3 is a diagram illustratively showing a classifier hierarchy according to one embodiment.

[0015] Figure 4 is a flow chart illustrating a method for performing mitigation actions based on device properties according to an embodiment.

[0016] Figure 5 is a schematic diagram of a device attribute hierarchical classification system according to one embodiment. DETAILED DESCRIPTION

[0017] It is important to note that the embodiments disclosed herein are merely examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed embodiments. In addition, some statements may apply to some inventive features but not to others. In general, unless otherwise stated, singular elements may be plural and vice versa without loss of generality. In the accompanying drawings, the same numerals refer to the same parts in several views.

[0018] Accurate device descriptions rely on complete and accurate device properties. In this regard, it has been recognized that device properties are often missing or conflicting, especially when a large amount of device property data is received from different sources. For example, network flow data may be only partially covered due to deployment or due to data sampling rather than continuous collection. As another example, when data is collected continuously over time, or when different parts of the data originate from different extraction libraries, device data may be conflicting.

[0019] To this end, the disclosed embodiments provide solutions for improving the accuracy of device attributes used for description. The disclosed embodiments include systems and methods for determining device attributes using machine learning. More specifically, the disclosed embodiments provide a decision tree-based integrated machine learning model intended to provide device identification. The decision tree-based integrated machine learning model is configured to determine device attributes using a hierarchical structure including classifier sub-models.

[0020] According to the disclosed embodiment, each level in the hierarchy includes multiple sub-models dedicated to attribute prediction. Each sub-model also includes one or more classifiers. More specifically, each classifier is trained to output a predicted class representing a device attribute and a confidence score for the prediction. An appropriate class is selected from the predicted classes. The appropriate class selected from each sub-model is used to determine the next relevant sub-model to be applied.

[0021] In one embodiment, device attributes may be determined when a branch of the hierarchy has been fully traversed (i.e., a leaf node has been reached and its classifier has been applied). In another embodiment, device attributes may be determined and traversal of the hierarchy may be stopped before a leaf node is reached (e.g., once a termination criterion occurs). In another embodiment, the termination criterion occurs when a confidence score below a threshold is output. That is, traversal may stop when one or more previous classifications have a confidence score above a threshold and a subsequent classification has a confidence score below a threshold. In such an embodiment, the most recent classification with a confidence score above a threshold may be used so that the last sub-model is the sub-model that outputs the most recent classification.

[0022] The disclosed embodiments provide techniques for improving the machine learning process, which can improve the accuracy of classification, reduce the extraction of unnecessary features, or both. The disclosed embodiments also provide techniques for preprocessing training data to further improve the resulting hierarchy and, therefore, the accuracy of the resulting classification. The disclosed embodiments can be used, for example, to populate device attribute fields of a device description file, which in turn can be used to detect anomalous activity compared to devices with similar description files. Thus, the disclosed embodiments allow for improved mitigation of network security threats posed by unknown devices.

[0023] Figure 1 1 is an example network diagram of a wireless environment 100 for describing various disclosed embodiments. The wireless environment 100 includes a plurality of wireless devices 110-1 to 110-m (hereinafter individually referred to as wireless devices 110 and collectively referred to as wireless devices 110 for simplicity purposes only) and a plurality of coexisting wireless networks 120-1 to 120-n (hereinafter individually referred to as wireless networks 120 and collectively referred to as wireless networks 120 for simplicity purposes only) to allow communication between the wireless devices 110. Also operable in the wireless environment 100 is an internal network 130 of an organization. The internal network 130 includes protected resources 131. Such protected resources 131 may include, but are not limited to, servers, computers, databases, services, applications, data, and the like.

[0024] Each wireless device 110 may be, but is not limited to, a laptop, a PC, a smartphone, an IoT device, a wearable device, a peripheral device (e.g., a printer, a mouse, a keyboard, a speaker, etc.), an appliance, a remote control, a wireless router (access point), or any other electrical or electronic device that can communicate via a wireless network.

[0025] The wireless network 120 can facilitate communication using wireless communication protocols, such as but not limited to Wi-Fi, nRF, cellular communication protocols (e.g., LTE, 2G, 3G, 4G, etc.), Bluetooth, Wi-Gi, Zigbee, Z-Wave, 5 Bluetooth Low Energy (BLE), EnOcean, DECT, 6LoWPAN, etc. It should be noted that application layer protocols can communicate via such wireless protocols. Examples of application layer protocols include, but are not limited to, AirDrop, HTTP, SMB, NTLM, OBEX, etc.

[0026] In an example implementation, a plurality of sensors 140-1 to 140-p (hereinafter individually referred to as sensor 140 and collectively referred to as sensors 140 for simplicity only) are deployed in the wireless network 100. Each sensor 140 is deployed as an out-of-band device and is configured to monitor and control wireless traffic sent by wireless devices 110 in the vicinity of where the sensor 140 is deployed. In some embodiments, the sensor 140 may be deployed as a device connected in the wireless network 100, a passive device that only listens to traffic, or as a tap device.

[0027] Each sensor 140 is configured to process the monitored traffic based on the corresponding wireless communication protocol. For example, if the monitored traffic is transmitted via a Bluetooth connection, the sensor 140 is configured to extract the data carried by the Bluetooth protocol through its entire stack. Such data may include header and payload data required to determine, for example, whether a new connection has been established, whether an existing connection has been identified, the type of data being transmitted, or a combination thereof. Therefore, each sensor 140 is configured to process the stack (all or some layers) of each wireless protocol supported by the sensor 140. Examples of such wireless protocols are mentioned above. In one embodiment, each sensor 140 can handle at least one type of wireless protocol.

[0028] As an example, sensor 140 can sniff traffic transmitted between two wireless devices 110 over a secure Wi-Fi network (e.g., network 120-1). In order to extract protocol data, sensor 140 needs to decrypt the data transmitted over the air. To do this, according to one embodiment, the password of the Wi-Fi network is uploaded and stored in sensor 140. Using the password, sensor 140 can act as a passive device that is configured to listen to data traffic on the Wi-Fi network and decrypt any data traffic transmitted between devices (e.g., wireless devices 110-3 and 110-2 in network 120-1).

[0029] Specifically, when a new wireless device 110 connects to a Wi-Fi network, a key exchange protocol is performed between the access point and the new device. The sensor 140 can be configured to capture this communication. In one example implementation, the key exchange protocol is a 4-way handshake process, which is a two-sided challenge-response designed to verify that both parties can derive the same encryption key without directly sending the key. This process can be performed for mutual authentication. Once the authentication packet is captured, the sensor 140 is configured to use a pre-configured password to derive the same key as the new wireless device 110. These keys are used to decrypt and encrypt any traffic transmitted between the new wireless device 110 and the access point.

[0030] The sensor 140 is also configured to capture BLE communications between two paired wireless devices 110 and extract protocol data. In order to discover the encryption key, a key entry (e.g., a 6-digit number when a key entry pairing procedure is used) is guessed or pre-configured by the sensor 140 or the system 150. For example, the password entry can be guessed by trying all possible options for a 6-digit code. Using the discovered key entry and the captured handshake, an encryption key can be derived for the session based on the BLE key exchange procedure. Using the encryption key, the communication between the two BLE devices 110 can be decrypted and the protocol data can be extracted.

[0031] In another example implementation, the sensor 140 may extract traffic parameters by querying a network device (not shown) installed or deployed in the environment 100. For example, the sensor 140 may query a wireless access point about a new connection being established and the wireless devices participating in the establishment of such a connection. For example, the network device may be queried about the type of connection being established, the protocol type, the wireless devices participating in the connection, the time when the connection was established, etc. In yet another embodiment, the sensor 140 may query the wireless device 110 to retrieve information about a specific wireless device 110. For example, the vendor and current OS type of the wireless device 110-2 may be retrieved by querying the wireless device 110-2.

[0032] The data extracted by each sensor 140 may be sent to the hierarchical classification system 150 for further analysis. As a non-limiting example, when the sensor 140 identifies a new wireless connection, the data associated with the new wireless connection is sent to the hierarchical classification system 150.

[0033] The information used to determine device attributes is processed by the sensor 140, the hierarchical classification system 150, or both. For example, in one embodiment, the sensor 140 can identify MAC addresses in traffic transmitted over the air. The MAC addresses are sent to the hierarchical classification system 150, which correlates the addresses to identify a single device associated with two or more MAC addresses. The MAC address can also provide an indication of the vendor of a given device. Determining the vendor of a device based on the MAC address can be performed by the sensor 140 or by the hierarchical classification system 150.

[0034] It should be noted that the information provided by the sensor 140 to the hierarchical classification system 150 may be decrypted and may contain relevant information for analysis by the hierarchical classification system 150. For example, such information may include network addresses (e.g., IP addresses, URLs, port numbers, etc.) accessed by the wireless device 110. The information may include data being accessed or transmitted by the wireless device 110. For example, the content of a Word document transmitted from the wireless device 110 to a location inside or outside the wireless environment would be provided to the hierarchical classification system 150.

[0035] In one embodiment, the sensor 140 is further configured to perform mitigation actions under the control of the hierarchical classification system 150. For example, the sensor 140 may terminate the wireless connection with the wireless device 110 determined to be vulnerable. In another embodiment, the hierarchical classification system 150 may be configured to perform mitigation actions or cause the sensor 140 to perform mitigation actions.

[0036] Each sensor 140 can be implemented as a physical device, a virtual device (executed by a processing system), or a combination thereof. Examples of virtual device implementations include, but are not limited to, virtual machines, microservices, software agents, and the like.

[0037] The hierarchical classification system 150 may be deployed in a data center, a cloud computing platform (eg, a public cloud, a private cloud, or a hybrid cloud), an organization's on-premise, or a combination thereof. The hierarchical classification system 150 or its functionality may be embedded in one or more sensors 140 .

[0038] In an example deployment, the hierarchical classification system 150 is configured to interface with one or more external systems 160. Examples of such external systems 160 may include, but are not limited to, an organization's active directory for retrieving user permissions, an access control system, a security information and event management (SIEM) system for reporting detected vulnerabilities, or other devices that may provide potential threats or known vulnerabilities to the hierarchical classification system 150. External systems 160 may also include inventory management systems, identity management systems, authentication servers, and the like.

[0039] In one embodiment, the hierarchical classification system 150 is configured to determine the device attributes described herein. Specifically, the hierarchical classification system 150 is configured to navigate the classifier hierarchy to determine each device attribute, such as a device attribute of one of the wireless devices 110. The hierarchical classification system 150 can be further configured to train sub-models of the hierarchy, as described herein.

[0040] The hierarchical classification system 150 may further be configured to detect wireless devices 110 that do not belong to the organization, which may perform malicious activities and, therefore, may pose a risk to the devices and networks in the wireless environment 100. Without limiting the scope of the disclosed embodiments, such devices will be referred to as vulnerable devices. Such detection may be performed based in part on data extracted by and received from the sensor 140. The hierarchical classification system 150 may be configured to describe each wireless device 110 found in the wireless environment 100 using device attributes determined as described herein. The description of the device is performed to determine whether the device is a legitimate device. In some embodiments, the hierarchical classification system 150 is configured to create a list of legitimate devices and a list of "unknown devices." A legitimate device is a wireless device 110 that is authorized to access the internal network 130 or any protected resource 131. An unknown device is a wireless device 110 that cannot be identified as a legitimate device.

[0041] In some implementations, a risk score may be determined for each device 110 (both unknown and legitimate) detected in the environment 110 based on the determined device attributes. The risk score is determined based on the activities performed by the device and the device's profile. These activities may relate to the type of vulnerability detected, the internal resources accessed, the type of data exploited, the type of data transmitted by the device, the device type, the connection type, or any combination thereof.

[0042] In order to discover new wireless devices 110, the sensor 140 can be configured to detect any new wireless connections being established in the wireless environment 100. For example, any new connection established with an access point (e.g., through a Wi-Fi network) or with another device (e.g., a Bluetooth connection) is detected. Alternatively or in combination, the wireless device 110 can be detected when the device is disconnected from the network. For example, if the wireless device 110 detects a probe request from a Wi-Fi network or a paging request in a Bluetooth network, such a request indicates that the device is disconnected. The wireless device 110 that initiates such a request is a discovered device.

[0043] The sensor 140 extracts or determines at least a unique identifier of the device initiating the connection. Based on the value of the extracted unique identifier, it is determined whether the device has been discovered. The unique identifier may include, but is not limited to, a MAC address. The list of previously discovered devices may be managed in a centralized location (e.g., in a hierarchical classification system 150) or distributed among the sensors 140.

[0044] It should be noted that a unique identifier is not always available when discovering a new wireless device 110. For example, Bluetooth piconets, slave devices, etc. are identified without knowing their MAC addresses. In this case, the unique identifier may be a temporary slot address assigned to the slave device by the master device using the temporary slot address. The MAC address of the slave device may be extracted later.

[0045] In one embodiment, the hierarchical classification system 150 may be configured to determine whether the wireless device 120 is vulnerable based on a profile of the device and data indicating the activity of the wireless device 120. Specifically, given the device attributes in the profile of the wireless device 120, the wireless device 120 may be determined to be vulnerable if, for example, there are anomalies in the activity of the wireless device 120. Such anomalies may be defined by using predetermined rules for different device attributes using a machine learning model, the machine learning model being trained using activity data from devices having different device attributes, or a combination thereof.

[0046] It should be understood that the embodiments disclosed herein are not limited to Figure 1 The specific architecture shown is not intended, and other architectures may be used without departing from the scope of the disclosed embodiments. Specifically, in one embodiment, there may be multiple hierarchical classification systems 150, which operate as described above and are configured to use one of them as a backup, to share the load between them, or to divide the functions between them.

[0047] Furthermore, in a typical wireless environment 100 or a typical deployment, there will be multiple internal networks 130 that can coexist, i.e., Figure 1 The single network 130 shown in the figure is only for the purpose of simplifying the discussion. Communications according to the disclosed embodiments can be carried out through a network, such as but not limited to a wireless, cellular or wired network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), the Internet, the World Wide Web (WWW), similar networks and any combination thereof.

[0048] It should also be noted that in some configurations, one or more sensors 140 may be embedded in the hierarchical classification system 150. Furthermore, the wireless device 120 is described as being wireless, but device attributes may also be determined for wired devices in accordance with the disclosed embodiments.

[0049] Figure 2 is an example flow chart 200 illustrating a method for determining device attributes using a classifier hierarchy according to one embodiment. In one embodiment, the method comprises Figure 1 The hierarchical classification system 150 is implemented.

[0050] At S210, training data for training a classifier of a hierarchy is preprocessed. The training data includes data related to a plurality of devices, such as but not limited to data obtained from the devices, data related to device activities (e.g., network activities), or a combination thereof. The network activity data may include but is not limited to traffic sent and received over a period of time, domains used, ports used, number of sessions, number of hosts called, etc. In one embodiment, the training data also includes class labels indicating device attributes of each device.

[0051] In one embodiment, S210 includes extracting features from the training data. The features to be extracted are based on the classifier to be applied.

[0052] In a further embodiment, S210 may further include cleaning the training data. The classifiers to be used may include a set of predetermined classifiers, each classifier being associated with a different device attribute. The classifiers include some classifiers that do not overlap in device attributes (e.g., a classifier associated with a mobile device and a classifier associated with a non-mobile device), and some classifiers that overlap in device attributes, such that one classifier is associated with a more specific device attribute than another (e.g., a classifier associated with a mobile device and a classifier associated with a smartphone, i.e., a specific type of mobile device).

[0053] In one embodiment, S210 also includes reducing the training data about the device. To this end, data related to at least some devices can be excluded. In another embodiment, data related to devices whose required device attributes are unknown (i.e., unlabeled), data related to devices that are inconsistent with the identified device attributes (i.e., inconsistent labels), or both can be excluded.

[0054] In another embodiment, S210 may further include removing one or more training device attribute classes indicated by the training tags. To this end, S210 may include excluding classes, modifying classes, or both. To this end, classifiers associated with excluded or modified categories may be excluded or modified. Excluded classes may include, but are not limited to, classes having a small number of samples (e.g., below a threshold) in the training data, i.e., classes for which a small number of devices use the class as a device attribute. Modifying a class may include merging multiple classes into a single class, such as classes with a low degree of variation in behavioral characteristics (e.g., classes for which the similarity of features used to determine the class is above a threshold).

[0055] At S220, sub-models of the hierarchy are trained and the hierarchy is established. In one embodiment, a gradient boosting tree is used to train the sub-models.

[0056] In one embodiment, the hierarchy includes multiple levels, at least some of which include multiple sub-models. The sub-models at different levels provide different granularities regarding device attributes. As a non-limiting example, a lower granularity sub-model can be trained to determine the device type (e.g., "printer"), and a higher granularity sub-model can be trained more specifically to determine the model of the device (e.g., "ABC Company Model 8 Printer").

[0057] In one embodiment, the hierarchy uses ensemble learning techniques. Specifically, different classifiers of the hierarchy's sub-models can be applied to features extracted from a data set, and a class representing a device attribute is selected based on the output of each sub-model's classifier. To this end, each classifier can be trained to output a confidence score for its respective class. The confidence score indicates the likelihood that the classifier's class represents the correct device attribute and can be compared to other confidence scores to determine which label should be selected.

[0058] Figure 3 An example hierarchy of classifiers is visually represented in . Figure 3 is a diagram illustratively showing a classifier hierarchy 300 according to one embodiment. Figure 3 , a hierarchy is a model including nodes 310 to 340. Each node represents a corresponding classifier associated with a corresponding attribute. Thus, nodes lower in the hierarchy 300 represent classifiers of higher granularity than nodes higher in the hierarchy. Nodes 320, 330, and 340 (i.e., nodes other than the root node) are grouped into sub-models 325, 335, and 345, respectively.

[0059] In the hierarchy 300, node 310 is a root (R) node representing an initial class (or lack thereof). When at the root node 310, a first sub-model 325 of classifiers 320-1 and 320-2 is applied, for example using an ensemble learning technique. Specifically, all classifiers 320 in the sub-model 325 are applied, and the appropriate class is determined based on the results of the classifiers 320.

[0060] Subsequent sub-models are applied based on the results of applying previous sub-models. More specifically, based on the class determined using one sub-model, the next sub-model to be used is determined. The next sub-model is a related sub-model (i.e., related to the last determined class) and is at the next level in the hierarchy (e.g., sub-model 335 is at the next level in the hierarchy compared to sub-model 325).

[0061] When determining device attributes, the hierarchy 300 is navigated by sequentially applying sub-models starting with the sub-model that includes the node directly below the root node 310 until a leaf node of the path being navigated is reached (in the example hierarchy 300, the leaf node of all paths is node 340) or until another termination criterion occurs (e.g., outputting a confidence score below a classification threshold for one of the sub-models, reaching a predetermined level in the hierarchy, or determining a confidence score for a device attribute above a threshold). By applying classifiers in stages, device attributes can be determined more accurately. Specifically, each classifier is trained with respect to a subset of a broader set of possible device attributes, so that the apex of the stage may be more accurate in determining device attributes than, for example, applying a single classifier. In addition, different features can be extracted for each sub-model, allowing the accuracy of the classification to be further improved by allowing the input to the classifier to be customized.

[0062] As a non-limiting example, when classifiers 320-1 and 320-2 of sub-model 325 are trained to classify a feature as a mobile device or a non-mobile device, the classifier associated with node 320-1 may represent the category "mobile device" while the classifier associated with node 320-2 may represent the category "non-mobile device." The result of applying sub-model 325 would be to select "mobile device" or "non-mobile device" as the appropriate device attribute for the device.

[0063] It should be noted that Figure 3 The described hierarchy is a non-limiting visual representation, and the hierarchy according to the disclosed embodiments need not be as described. Figure 3 Specifically, different numbers of total nodes, different numbers of nodes per level, different numbers of nodes per sub-model, etc. may be used without departing from the scope of the present disclosure. Likewise, the downward branching direction of the hierarchy is only used to visually describe the relationship between nodes and to distinguish nodes of different levels, and should not be construed as a limitation. For example, the hierarchy may also be viewed as an upward flowing hierarchy without departing from the scope of the present disclosure. In addition, not all leaf nodes will be at the same level, i.e., some device attributes may require more sub-model applications than other device attributes to reach the most specific device attribute.

[0064] Back to Figure 2 In one embodiment, the training of each classifier is a supervised machine learning process using training data, which includes training device data and labels indicating device attribute classifications. The training device data may include, but is not limited to, data obtained by inspecting the device, data obtained by monitoring traffic to and from the device, network data associated with the device, etc. Such training device data may include data obtained by a system monitoring the device (e.g., a sensor 140 monitoring the wireless device 110, Figure 1) captures the type of data.

[0065] At S230, features are extracted from the application data. The extracted features include features required by each classifier to be used as part of the hierarchical integrated classification. In some embodiments, features can be extracted as needed when navigating the hierarchy. For example, because navigating the hierarchy includes selecting a sub-model (i.e., a group of one or more classifiers) in the next level of the hierarchy to be used, features can be extracted only when required by the classifier of the next selected sub-model. This allows minimizing the amount of feature extraction. Alternatively, all potentially necessary features can be extracted before navigating the hierarchy.

[0066] At S240, the sub-model is applied to the features extracted from the application data to determine the appropriate class. In one embodiment, the first sub-model of the application is the one that includes the features directly at the root node (e.g., Figure 3 A submodel of the nodes under the root node 310).

[0067] In one embodiment, each classifier of the S240 sub-model is applied to the relevant features, and the appropriate class is determined based on the output of each classifier. In an example implementation, each classifier outputs a confidence score, and the class of the classifier that outputs a confidence score above a threshold is determined to be the appropriate class. Alternatively, the class of the classifier that outputs the highest confidence score can be determined to be the appropriate class. In some embodiments, if all classifiers output a confidence score below a threshold, no device attribute can be determined or a null value can be determined.

[0068] At S250, based on the output of the classifier applied at S240, the next relevant sub-model of the hierarchy is determined. In one embodiment, the next relevant sub-model is a sub-model in the next level of the hierarchy associated with the category determined using the previous sub-model.

[0069] At S260, it is determined whether the last sub-model of the current traversal of the hierarchy has been reached. If so, S270 is executed; otherwise, S240 is executed, where the features are input to the relevant sub-model in the next level of the hierarchy. In one embodiment, the last level of the hierarchy where the last sub-model is located may be different, depending on the path taken when traversing the hierarchy.

[0070] In one embodiment, when the termination criteria occurs, the last sub-model of the hierarchy of a particular path has been reached. In one embodiment, the termination criteria has occurred when there are no more sub-models on the path (e.g., when each classifier of the sub-model is represented by a leaf node). Note that the last sub-model in accordance with the disclosed embodiments need not be a sub-model in the lowest or highest level of the hierarchy, and traversal may stop before reaching such a level.

[0071] In one embodiment, the termination criterion may be that a sub-model outputs a low confidence classification with a confidence score below a threshold. In such an embodiment, the sub-model most recently applied before the sub-model outputting the low confidence classification is identified as the last sub-model, so that the classification of such most recently applied sub-model is used to determine the device attribute. Thus, any more specific classification that does not meet the confidence score threshold may be excluded from use, thereby further improving the accuracy of the device attribute determination.

[0072] At S270, device attributes are determined. In one embodiment, device attributes are determined based on the appropriate category determined for the most recently applied sub-model. For example, device attributes may be used as part of a device description file, which in turn may be used to identify anomalous activity of a device that may require mitigation. Thus, the device attribute identification described herein may be used to accurately describe a device, which in turn allows for more accurate identification of anomalies in device behavior by comparison with devices having the same device attributes or combination of device attributes. An example of such use is in Figure 4 Shown in.

[0073] As a non-limiting example, in Figure 3 , when applying submodel 325, the output of classifier 320-1 is a confidence score of 9 for the category "mobile device", and the output of classifier 320-2 is a confidence score of 5 for the category "non-mobile device", the category "mobile device" is appropriately selected, so that the next relevant submodel is determined to be submodel 335-1 (i.e., the submodel in the next level below submodel 325). The same process is performed by applying classifiers 330-1 and 330-2 of submodel 335-1, and based on the outputs of these classifiers, submodel 345-2 (e.g., representing the category "smartphone") is determined to be the next relevant submodel. By applying classifiers 340-3 to 340-5 of submodel 345-2, it is determined that the class of classifier 340-4 (e.g., representing the class "ABC brand smartphone") is the appropriate class. Since submodel 345-2 is at the last level of this path in the hierarchy, the class of classifier 340-4 is determined to be the device attribute.

[0074] Figure 4 is an example flow chart 400 illustrating a method for mitigating abnormal device activity based on device attributes according to one embodiment.

[0075] At S410, a device (eg, Figure 1 In one embodiment, at least some of the device attributes are determined using a classifier hierarchy, as described above with reference to Figure 2 Further described.

[0076] At S420 , a device description file is created for the device based on the determined device attributes.

[0077] At S430, device activities are monitored. The device activities may include but are not limited to network activities of the device.

[0078] At S440, anomalies are detected based on the monitored device activity and device profiles. For example, anomalies may be detected based on predetermined rules (e.g., rules defining a value or combination of values ​​of abnormal device activity data) or using a machine learning model trained based on historical device profile data. In one embodiment, different rules or models may be applied to different device attributes. As a non-limiting example, rules applied to a mobile device may be different from rules applied to a server.

[0079] At S450, one or more mitigation actions are performed to mitigate the detected anomaly. The mitigation actions may include, but are not limited to, disconnecting one or more connections of the device (eg, a connection to a network), sending a notification of the device indicating abnormal behavior, or a combination thereof.

[0080] Figure 5 1 is an exemplary diagram of a device attribute hierarchical classification system 150 according to one embodiment. The device attribute hierarchical classification system 150 includes a processing circuit 510 coupled to a memory 520, a storage device 530, and a network interface 540. In one embodiment, the components of the device attribute hierarchical classification system 150 may be communicatively connected via a bus 550.

[0081] Processing circuitry 510 may be implemented as one or more hardware logic components and circuits. For example, but not limited to, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), graphics processing units (GPUs), tensor processing units (TPUs), general purpose microprocessors, microcontrollers, digital signal processors (DSPs), etc., or any other hardware logic component that can perform computations or other information operations.

[0082] The memory 520 may be volatile (eg, RAM, etc.), non-volatile (eg, ROM, flash memory, etc.), or a combination thereof.

[0083] In one configuration, software for implementing one or more embodiments disclosed herein may be stored in storage device 530. In another configuration, memory 420 is configured to store such software. Software should be broadly interpreted as any type of instruction, whether referred to as software, firmware, middleware, microcode, hardware description language, or other. Instructions may include code (e.g., source code format, binary code format, executable code format, or any other suitable code format). When executed by processing circuit 510, the instructions cause processing circuit 510 to perform the various processes described herein.

[0084] Storage device 530 may be a magnetic storage device, an optical storage device, etc., and may be implemented, for example, as flash memory or other storage technology, CD-ROMs, digital versatile disks (DVDs), or any other medium that can be used to store the desired information.

[0085] The network interface 540 allows the device attribute classification system 150 to communicate with, for example, the wireless device 110 or the sensor 140 , for example, to receive data related to device attributes.

[0086] It should be understood that the embodiments described herein are not limited to Figure 5 Specific architectures are shown, and other architectures may equally be used without departing from the scope of the disclosed embodiments.

[0087] Various embodiments disclosed herein can be implemented as hardware, firmware, software or any combination thereof. In addition, the software is preferably implemented as an application program tangibly contained in a program storage unit or a computer-readable medium, which is composed of a combination of some or some devices and / or devices. The application program can be uploaded to a machine including any suitable architecture and executed by the machine. Preferably, the machine is implemented on a computer platform with hardware such as one or more central processing units ("CPUs"), a memory and an input / output interface. The computer platform may also include an operating system and microinstruction codes. The various processes and functions described herein may be a part of a microinstruction code or a part of an application program, or any combination thereof, and they may be executed by a CPU, regardless of whether such a computer or processor is explicitly shown. In addition, various other peripheral units may be connected to a computer platform, such as an additional data storage unit and a printing unit. In addition, a non-transitory computer-readable medium is any computer-readable medium other than a temporary propagation signal.

[0088] All examples and conditional language cited here are for teaching purposes, to help the reader understand the principles of the disclosed embodiments and the concepts contributed by the inventors to promote the art, and should be interpreted as not being limited to these specifically cited examples and conditions. In addition, all statements of the principles, aspects and embodiments of the disclosed embodiments and their specific examples are intended to cover their structural and functional equivalents. In addition, it is intended that such equivalents include currently known equivalents and equivalents developed in the future, that is, any element that performs the same function developed, regardless of the structure.

[0089] It should be understood that any reference to an element using names such as "first," "second," etc. herein generally does not limit the number or order of these elements. Rather, these names are generally used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to a first and a second element does not mean that only two elements may be used there, or that the first element must precede the second element in some manner. Furthermore, unless otherwise specified, a group of elements includes one or more elements.

[0090] As used herein, the phrase "at least one of" followed by a list of items means that any of the listed items may be used alone or in any combination of two or more of the listed items. For example, if a system is described as including "at least one of A, B, and C," the system may include only A alone; B alone; C alone; 2A; 2B; 2C; 3A; a combination of A and B; a combination of B and C; a combination of A and C; a combination of A, B, and C; a combination of 2A and C; a combination of A, 3B, and 2C; and so on.

Claims

1. A method for determining device attributes using a classifier hierarchy, comprising: Sequentially applying a plurality of sub-models of a hierarchy to a plurality of features extracted from the device activity data, wherein the hierarchy comprises: Multiple nodes; and a set of sub-models, wherein each sub-model in the set of sub-models comprises a set of classifiers, each classifier being represented by one of the plurality of nodes; wherein each sub-model in the set of sub-models is configured to output a class of device attributes when applied to the plurality of features, wherein each of the plurality of nodes is connected to at most one previous node in the hierarchy, and wherein each of the plurality of nodes represents a classifier having a higher granularity than a classifier represented by the at most one previous node to which the node is connected, wherein the sequential applying ends with applying a last sub-model of the plurality of sub-models, and wherein applying the plurality of sub-models further comprises iteratively determining a next sub-model to apply based on a class output by a most recently applied sub-model and based on a connection between the next sub-model in the hierarchy and a most recently referenced sub-model; and Determines device properties based on the class output by the last submodel.

2. The method according to claim 1, wherein: The device activity data includes network activity data.

3. The method according to claim 2, wherein: The network activity data includes at least one of the following: sent traffic, received traffic, used domains, used ports, number of sessions, and number of hosts in conversation.

4. The method of claim 1 , wherein each classifier is trained to output a class and a confidence score, wherein the class output by each sub-model is determined based on the class and confidence score output by each classifier in the set of classifiers of the corresponding sub-model.

5. The method of claim 4, wherein the class output by each sub-model is a first class output by a first classifier in the set of classifiers of the corresponding sub-model, wherein a first confidence score output by the first classifier is above a threshold.

6. The method according to claim 1, wherein: Training the set of sub-models using a training data set, the training data set comprising training device activity data and a plurality of training labels indicating a plurality of training device attribute classes, the method further comprising: The training data is preprocessed by removing training device activity data associated with at least one of: devices for which at least one device attribute is unlabeled, and devices for which at least one device attribute is inconsistently labeled.

7. The method according to claim 1, wherein: Training the set of sub-models using a training data set including a plurality of training labels indicating a plurality of training device attribute classes, the method further comprising: Remove at least one of a plurality of training device attribute classes.

8. The method according to claim 6, wherein: Removing at least one of the plurality of training device attribute classes further comprises: At least two of the plurality of training device attribute classes are merged into a merged training device attribute class, wherein a classifier in the hierarchy is trained to output a confidence score for the merged training device attribute class.

9. The method according to claim 1, further comprising: Creating a device description file using the determined device properties; Monitor device behavior based on device profiles to detect abnormal behavior; as well as At least one mitigation action is performed with respect to the device based on the detected abnormal behavior.

10. A non-transitory computer readable medium having stored thereon instructions for causing a processing circuit to perform a process, the process comprising: Sequentially applying a plurality of sub-models of a hierarchy to a plurality of features extracted from the device activity data, wherein the hierarchy comprises: Multiple nodes; and a set of sub-models, wherein each sub-model in the set of sub-models comprises a set of classifiers, each classifier being represented by one of the plurality of nodes; wherein each sub-model in the set of sub-models is configured to output a class of device attributes when applied to the plurality of features, wherein each of the plurality of nodes is connected to at most one previous node in the hierarchy, and wherein each of the plurality of nodes represents a classifier having a higher granularity than a classifier represented by the at most one previous node to which the node is connected, wherein the sequential applying ends with applying a last sub-model of the plurality of sub-models, wherein applying the plurality of sub-models further comprises iteratively determining a next sub-model to apply based on a class output by a most recently applied sub-model and based on a connection between a next sub-model in the hierarchy and a most recently referenced sub-model; and Determines device properties based on the class output by the last submodel.

11. A system for determining device attributes using a classifier hierarchy, comprising: Processing circuit; as well as a memory comprising instructions that, when executed by the processing circuitry, configure the system to: Sequentially applying a plurality of sub-models of a hierarchy to a plurality of features extracted from the device activity data, wherein the hierarchy comprises: Multiple nodes; and a set of sub-models, wherein each sub-model in the set of sub-models comprises a set of classifiers, each classifier being represented by one of the plurality of nodes; wherein each sub-model in the set of sub-models is configured to output a class of device attributes when applied to the plurality of features, wherein each of the plurality of nodes is connected to at most one previous node in the hierarchy, and wherein each of the plurality of nodes represents a classifier having a higher granularity than a classifier represented by the at most one previous node to which the node is connected, wherein the sequential applying ends with applying a last sub-model of the plurality of sub-models, wherein applying the plurality of sub-models further comprises iteratively determining a next sub-model to apply based on a class output by a most recently applied sub-model and a connection between a next sub-model in the hierarchy and a most recently referenced sub-model; and Determines device properties based on the class output by the last submodel.

12. The system of claim 11, wherein the device activity data comprises network activity data.

13. The system of claim 12, wherein the network activity data comprises at least one of: traffic sent, traffic received, domains used, ports used, number of sessions, and number of hosts in conversation.

14. The system of claim 11, wherein each classifier is trained to output a class and a confidence score, wherein the class output by each sub-model is determined based on the class and confidence score output by each classifier in the set of classifiers of the corresponding sub-model.

15. The system of claim 14, wherein the class output by each sub-model is a first class output by a first classifier in the set of classifiers of the corresponding sub-model, wherein a first confidence score output by the first classifier is above a threshold.

16. The system of claim 11, wherein the set of sub-models is trained using a training dataset, the training dataset comprising training device activity data and a plurality of training labels indicating a plurality of training device attribute classes, wherein the system is further configured to: The training data is preprocessed by removing training device activity data associated with at least one of: devices for which at least one device attribute is not labeled, and devices for which at least one device attribute is inconsistently labeled.

17. The system of claim 11, wherein the set of sub-models is trained using a training dataset comprising a plurality of training labels indicating a plurality of training device attribute classes, wherein the system is further configured to: Remove at least one of a plurality of training device attribute classes.

18. The system of claim 16, wherein the system is further configured to: At least two of the plurality of training device attribute classes are merged into a merged training device attribute class, wherein a classifier in the hierarchy is trained to output a confidence score for the merged training device attribute class.

19. The system of claim 11, wherein the system is further configured to: Creating a device description file using the determined device properties; monitoring the behavior of the device based on the device profile to detect abnormal behavior; and At least one mitigation action is performed with respect to the device based on the detected abnormal behavior.

Images