Application workload capture for IoT devices

Abstract

An application workload capture for an Internet of Things (IoT) device is disclosed. A target IoT device is selected. Flows associated with the target device are determined and tagged. Packets from the tagged flows are accepted into a ring buffer. An indication that extraction should be performed on a portion of the packets contained in the ring buffer is received.

Description

[Background technology]

[0001] Malicious individuals attempt to compromise computer systems in a variety of ways. As one example, such individuals may embed or otherwise include malicious software ("malware") in email attachments and send them to unsuspecting users. When executed, the malware compromises the victim's computer and can perform additional illicit tasks (e.g., extracting sensitive data, propagating to other systems, etc.). Various approaches can be used to harden computers against exposure to such and other risks. Unfortunately, existing techniques for protecting computers are not necessarily suitable for all computing environments. Additionally, malware authors are continually adapting their techniques to evade detection. And there is a continuing need for improved techniques for detecting malware and preventing its harm in a variety of situations. [Brief description of the drawings]

[0002] Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings. [Figure 1] FIG. 1 illustrates an example of an environment in which malicious activity is detected and its harm mitigated. [Figure 2A] FIG. 2A illustrates one embodiment of a data appliance. [Figure 2B] FIG. 2B is a functional diagram of the logical components of an embodiment of a data appliance. [Figure 2C] FIG. 2C shows one exemplary event path between the IoT Server and the IoT Module. [Figure 2D] FIG. 2D shows an example of a device discovery event. [Figure 2E] FIG. 2E shows an example of a session event. [Figure 2F] FIG. 2F illustrates one embodiment of an IoT module. [Figure 2G] FIG. 2G illustrates one exemplary method for performing IoT device analysis. [Diagram 3] FIG. 3 illustrates one embodiment of a process for passively providing AAA support for IoT devices in a network. [Figure 4A] FIG. 4A illustrates an example of a RADIUS message sent by an IoT server to an AAA server on behalf of an IoT device in various embodiments. [Figure 4B] FIG. 4BC shows an example of a RADIUS message sent by the IoT server on behalf of the IoT device to the AAA server in various embodiments. [Figure 4C] FIG. 4C illustrates an example of a RADIUS message sent by the IoT server to the AAA server on behalf of the IoT device in various embodiments. [Diagram 5] FIG. 5 illustrates one embodiment of an IoT module. [Figure 6] FIG. 6 shows an example of a process for classifying IoT devices. [Figure 7] FIG. 7 illustrates one embodiment of an IoT server. [Figure 8] FIG. 8 shows example messages that may be exchanged in conjunction with capturing an IoT device application workload. [Figure 9] FIG. 9 illustrates an example of a process for capturing an IoT device application workload. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0003] The present invention can be implemented in numerous ways, including as a process, an apparatus, a system, a composition of matter, a computer program product embodied on a computer-readable storage medium, and / or a processor configured to execute instructions stored in and / or provided by a memory coupled to the processor. These embodiments, or other forms in which the present invention may be employed, are referred to as techniques in this specification. In general, the order of steps of a disclosed process may be modified within the scope of the present invention. Unless otherwise indicated, components such as a processor or memory described as being configured to perform a task may be implemented as general components that are temporarily configured to perform the task at a given time, or as specific components that are manufactured to perform the task. As used herein, the term "processor" refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0004] A detailed description of one or more embodiments of the present invention is provided below along with the accompanying drawings that illustrate the principles of the invention. While the present invention will be described in connection with such embodiments, the present invention is not limited to any embodiment. The scope of the present invention is limited only by the claims, and the present invention encompasses numerous alternatives, modifications, and equivalents. Numerous specific details are set forth in the following description to provide a thorough understanding of the present invention. These details are provided for the purposes of example, and the present invention may be practiced according to the claims without some or all of these specific details. For the sake of clarity, technical material well known in the art related to the invention will not be described in detail so as not to unnecessarily obscure the invention.

[0005] I. Overview

[0006] A firewall generally allows authorized communications to pass through the firewall while protecting the network from unauthorized access. A firewall is typically a device, a set of devices, or software running on a device that provides firewall functionality for network access. For example, a firewall can be integrated into the operating system of a device (e.g., a computer, a smartphone, or other type of network-enabled device). Firewalls can also be integrated or run as software applications on various types of devices or security devices, such as computer servers, gateways, network / routing devices (e.g., network routers), or data appliances (e.g., security appliances, or other types of special purpose devices), and in some implementations, certain operations can be implemented in special purpose hardware, such as ASICs or FPGAs. do.

[0007] A firewall typically denies or allows network transmissions based on a set of rules. These sets of rules are often referred to as policies (e.g., network policies or network security policies). For example, a firewall can filter inbound traffic by applying a set of rules or policies to prevent unwanted external traffic from reaching a protected device. A firewall can also filter outbound traffic by applying a set of rules or policies (e.g., allow, block, monitor, notify, or log, and / or other actions that may be specified in a firewall rule or firewall policy, which may be triggered based on various criteria, as described herein). A firewall can also filter local network (e.g., intranet) traffic by applying a set of rules or policies as well.

[0008] Security devices (e.g., security appliances, security gateways, security services, and / or other security devices) may perform various security operations (e.g., firewalls, anti-malware, intrusion prevention / detection, proxies, and / or other security functions), network functions (e.g., routing, quality of service (QoS), workload balancing of network-related resources, and / or other network functions), and / or other security and / or network-related functions. For example, routing may be performed based on source information (e.g., IP addresses and ports), destination information (e.g., IP addresses and ports), and protocol information.

[0009] Basic packet filtering firewalls filter network communication traffic by inspecting each individual packet sent across a network (e.g., a packet filtering firewall or first generation firewall, which is a stateless packet filtering firewall). Stateless packet filtering firewalls typically inspect the individual packets themselves and then apply rules based on the inspected packets (e.g., using a combination of the packet's source and destination address information, protocol information, and port numbers).

[0010] Application firewalls can also perform application layer filtering (e.g., using an application layer filtering firewall, or a second generation firewall that functions at the application level of the TCP / IP stack). Application layer filtering firewalls or application firewalls can generally identify certain applications and protocols (e.g., web browsing using the Hypertext Transfer Protocol (HTTP), Domain Name System (DNS) requests, file transfers using the File Transfer Protocol (FTP), and various other types of applications and other protocols, such as Telnet, DHCP, TCP, UDP, and TFTP (GSS)). For example, an application firewall can block unauthorized protocols that attempt to communicate on standard ports (e.g., unauthorized / out-of-policy protocols that attempt to sneak through by using a non-standard port for that protocol can generally be identified using an application firewall).

[0011] Stateful firewalls can also perform stateful-based packet inspection, where each packet is inspected in the context of the set of packets associated with its network outgoing packet flow. This firewall technique is commonly referred to as stateful packet inspection, because it keeps a record of all connections that pass through the firewall and can determine whether a packet is the start of a new connection, part of an existing connection, or an invalid packet. For example, the state of a connection can itself be one of the criteria that triggers a rule in a policy.

[0012] Advanced or next-generation firewalls can perform stateless and stateful packet filtering and application layer filtering, as described above. Next-generation firewalls can also perform additional firewall technologies. For example, certain new firewalls, often referred to as advanced or next-generation firewalls, can also identify users and content. In particular, certain next-generation firewalls have expanded the list of applications that they can automatically identify to thousands of applications. Examples of such next-generation firewalls are commercially available from Palo Alto Networks (e.g., Palo Alto Networks PA series firewalls). For example, Palo Alto Networks next-generation firewalls use a variety of identification technologies to enable enterprises and service providers to identify and control applications, users, and content - not just ports, IP addresses, and packets. Various identification techniques include Application-ID (e.g., App ID) for precise application identification, User-ID (e.g., User ID) for user identification, and Content-ID (e.g., Content ID) for real-time content scanning (e.g., to control web surfing and limit data and file transfers). These identification techniques allow enterprises to safely enable application usage using business-relevant concepts instead of following the traditional approach provided by traditional port-blocking firewalls.Additionally, special-purpose hardware for next-generation firewalls (e.g., implemented as dedicated devices) generally provides higher performance levels for application inspection than software running on general-purpose hardware (e.g., security appliances from Palo Alto Networks, Inc., which utilize dedicated, function-specific processing that is tightly integrated with a single-pass software engine to maximize network throughput while minimizing latency, for Palo Alto Networks' PA Series Next-Generation Firewalls).

[0013] Advanced or next-generation firewalls can also be implemented using virtualized firewalls. An example of such a next-generation firewall is available from Palo Alto Networks (which runs on VMware® ESXi). TM and NSX TM , Citrix(R)Netscaler SDX TM It supports a variety of commercial virtualization environments, including VMware, KVM / OpenStack (Centos / RHEL, Ubuntu®), and Amazon Web Services (AWS). For example, the virtualized firewall can support similar or identical next-generation firewall and advanced threat prevention capabilities available in physical form factor devices, allowing enterprises to safely enable the influx of applications to private, public, and hybrid cloud computing environments. Automation capabilities such as VM monitoring, dynamic address groups, and REST-based APIs enable enterprises to dynamically monitor VM changes and update security policies with that context, thereby eliminating policy lag that can occur when VMs change.

[0014] II. Example Environment

[0015] Figure 1 illustrates an example of an environment in which malicious activity is detected and its harm mitigated. In the example shown in Figure 1, client devices 104-108 are a laptop computer, a desktop computer, and a tablet (respectively) that reside on an enterprise network 110 of a hospital (also referred to as "Acme Hospital"). Data appliance 102 is configured to enforce policies regarding communications between client devices, such as client devices 104 and 106, and nodes outside enterprise network 110 (e.g., reachable via external network 118).

[0016] Examples of such policies include those that govern traffic shaping, quality of service, and traffic routing. Other examples of policies include security policies, such as those that require scanning for threats in incoming (and / or outgoing) email attachments, website content, files exchanged via instant messaging programs, and / or other file transfers. In some embodiments, data appliance 102 is also configured to enforce policies with respect to traffic that remains within enterprise network 110.

[0017] The network 110 also includes a directory service 154 and an authentication, authorization, and accounting (AAA) server 156. In the example shown in FIG. 1, the directory service 154 (also called an identity provider or domain controller) utilizes the Lightweight Directory Access Protocol (LDAP) or other suitable protocol. The directory service 154 is configured to manage user identity and credential information. An example of the directory service 154 is a Microsoft Active Directory server. Instead of an Active Directory server, other types of systems, such as a Kerberos-based system, can also be used, and the techniques described herein are adapted accordingly. In the example shown in FIG. 1, the AAA server 156 is a network admission control (NAC) server. The AAA server 156 is configured to authenticate wired, wireless, and VPN users and devices to the network, evaluate and remediate devices for policy compliance before allowing access to the network, differentiate access based on role, and then audit and report on who is on the network. One example of an AAA server 156 is Cisco's Identity Services Engine (ISE) server that utilizes Remote Authentication Dial-In User Service (RADIUS). Other types of AAA servers, including those that use protocols other than RADIUS, can be used with the techniques described herein.

[0018] In various embodiments, the data appliance 102 is configured to listen for communications (e.g., passively monitor messages) to / from the directory service 154 and / or the AAA server 156. In various embodiments, the data appliance 102 is configured to communicate (i.e., actively communicate messages) with the directory service 154 and / or the AAA server 156. In various embodiments, the data appliance 102 is configured to communicate with an orchestrator (not shown) that communicates (e.g., actively communicates messages) with various network elements, such as the directory service 154 and / or the AAA server 156. Other types of servers may also be included in the network 110 and may communicate with the data appliance 102 where applicable, and the directory service 154 and / or the AAA server 156 may also be omitted from the network 110 in various embodiments.

[0019] 1 as having a single data appliance 102, a given network environment (e.g., network 110) may include multiple embodiments of data appliances, whether operating individually or in concert. Similarly, the term "network" is generally referred to herein in the singular (e.g., as "network 110") for simplicity, but the techniques described herein may be deployed in a variety of network environments of different sizes and topologies, including a variety of mixes of networking technologies (e.g., virtual and physical), using a variety of networking protocols (e.g., TCP and UDP) and infrastructures (e.g., switches and routers) across a variety of network layers, where applicable.

[0020] Data appliance 102 may be configured to operate in cooperation with remote security platform 140. Security platform 140 may provide various services including performing static and dynamic analysis on malware samples (e.g., via sample analysis module 124) and providing a list of signatures of known malicious files, domains, etc. to a data appliance, such as data appliance 102, as part of a subscription. As described in more detail below, security platform 140 may also provide information associated with the discovery, classification, management, etc. of IoT devices present in a network, such as network 110 (e.g., via IoT module 138). In various embodiments, the signatures, results of the analysis, and / or additional information (e.g., regarding samples, applications, domains, etc.) are stored in database 160. In various embodiments, security platform 140 includes one or more dedicated, commercially available hardware servers (e.g., having multi-core processors, 32G+ RAM, gigabit network interface adapters, and hard drives) running a typical server-class operating system (e.g., Linux). Security platform 140 may be implemented across a scalable infrastructure including multiple such servers, solid state drives or other storage 158, and / or other applicable high performance hardware. Security platform 140 may include several distributed components, including components provided by one or more third parties. For example, some or all of security platform 140 may be implemented using Amazon's Elastic Compute Cloud (EC2) and / or Amazon's Simple Storage Service (S3).Additionally, similar to data appliance 102, whenever security platform 140 is referred to as performing a task, such as storing data or processing data, it should be understood that a subcomponent or subcomponents of security platform 140 may cooperate (individually or in cooperation with a third party component) to perform that task. As an example, security platform 140 may cooperate with one or more virtual machine (VM) servers to perform static / dynamic analysis (e.g., via sample analysis module 124) and / or IoT device functionality (e.g., via IoT module 138). One example of a virtual machine server is a physical machine that includes commercially available server-class hardware (e.g., multi-core processors, 32+ gigabytes of RAM, and one or more gigabit network interface adapters) running commercially available virtualization software, such as VMware's ESXi, Citrix's XenServer, or Microsoft's Hyper-V. In some embodiments, the virtual machine server is omitted. Additionally, the virtual machine server may be under the control of the same entity that manages security platform 140, but may also be provided by a third party. As one example, the virtual machine servers may rely on EC2, with the remainder of security platform 140 provided by dedicated hardware owned and under the control of the operator of security platform 140.

[0021] One embodiment of a data appliance is shown in FIG. 2A. The illustrated example is a representation of the physical components included in the data appliance 102 in various embodiments. Specifically, the data appliance 102 includes a high-performance multi-core central processing unit (CPU) 202 and a random access memory (RAM) 204. The data appliance 102 also includes storage 210 (such as one or more hard disks or solid-state units). In various embodiments, the data appliance 102 stores (either in the RAM 204, the storage 210, and / or other suitable locations) information used to monitor the enterprise network 110 and to implement the disclosed techniques. Examples of such information include application identifiers, content identifiers, user identifiers, requested URLs, IP address mappings, policies and other configuration information, signatures, hostname / URL categorization information, malware profiles, machine learning models, IoT device classification information, and the like. The data appliance 102 may also include one or more optional hardware accelerators. For example, the data appliance 102 may include a cryptographic engine 206 configured to perform encryption and decryption operations, and one or more field programmable gate arrays 208 configured to perform matching, operate as a network processor, and / or perform other tasks.

[0022] The functionality described herein as being performed by data appliance 102 may be provided / implemented in a variety of ways. For example, data appliance 102 may be a dedicated device or set of devices. A given network environment may include multiple data appliances, each of which may be configured to serve a particular portion of the network, which may cooperate to serve a particular portion of the network, etc. The functionality provided by data appliance 102 may be integrated or executed as software on a general purpose computer, computer server, gateway, and / or network / routing device. In some embodiments, at least some of the functionality described as being provided by data appliance 102 is instead (or in addition) provided to a client device (e.g., client device 104 or client device 106) by software executing at the client device. Functionality described herein as being performed by data appliance 102 may also be performed, at least in part, by or in cooperation with security platform 140, and / or functionality described herein as being performed by security platform 140 may also be performed, at least in part, by or in cooperation with data appliance 102, where applicable. As one example, various functions described as being performed by IoT module 138 may be performed by an embodiment of IoT server 134.

[0023] Whenever the data appliance 102 is described as performing a task, a single component, a subset of the components, or all of the components of the data appliance 102 may cooperate to perform the task. Similarly, whenever a component of the data appliance 102 is described as performing a task, the subcomponent may perform the task and / or the component may perform the task with other components. In various embodiments, portions of the data appliance 102 are provided by one or more third parties. Depending on factors such as the amount of computing resources available to the data appliance 102, various logical components and / or features of the data appliance 102 may be omitted and the techniques described herein adapted accordingly. Similarly, additional logical components / features may be included in embodiments of the data appliance 102, as applicable. One example of a component included in the data appliance 102 in various embodiments is an application identification engine configured to identify applications (e.g., using various application signatures to identify applications based on packet flow analysis). For example, the application identification engine may determine the type of traffic a session involves, such as web browsing-social networking, web browsing-news, SSH, etc. Another example of a component included in the data appliance 102 in various embodiments is an IoT server 134, described in more detail below. The IoT server 134 can take a variety of forms, including as a standalone server (or set of servers), whether physical or virtualized, and may also be co-located / embedded with the data appliance 102 (e.g., as shown in FIG. 1 ), where applicable.

[0024] 2B is a functional diagram of the logical components of one embodiment of a data appliance. The example shown is a representation of the logical components that may be included in data appliance 102 in various embodiments. Unless otherwise specified, the various logical components of data appliance 102 may generally be implemented in a variety of ways, including as a set of one or more scripts (e.g., written in Java, python, etc., as applicable).

[0025] As shown, data appliance 102 includes a firewall and includes a management plane 212 and a data plane 214. The management plane is responsible for managing user interaction, such as by providing a user interface for setting policies and displaying log data. The data plane is responsible for data management, such as by performing packet and session processing.

[0026] The network processor 216 is configured to receive packets from client devices, such as the client device 108, and provide them to the data plane 214 for processing. The flow module 218 creates a new session flow whenever it identifies a packet as part of a new session. Subsequent packets are identified as belonging to the session based on the flow lookup. If applicable, SSL decryption is applied by the SSL decryption engine 220. If not, processing by the SSL decryption engine 220 is skipped. The decryption engine 220 helps the data appliance 102 inspect and control SSL / TLS and SSH encrypted traffic, and thus helps stop threats that may otherwise remain hidden in the encrypted traffic. The decryption engine 240 can also help prevent sensitive content from leaving the enterprise network 110. Decryption can be selectively controlled (e.g., enabled or disabled) based on parameters such as URL category, traffic source, traffic destination, user, user group, and port. In addition to a decryption policy (e.g., one that specifies which sessions to decrypt), a decryption profile can be assigned to control various options of the session controlled by the policy, for example, the use of specific cipher suites and encryption protocol versions can be required.

[0027] The application identification (APP-ID) engine 222 is configured to determine the type of traffic the session involves. As an example, the application identification engine 222 may recognize a GET request in the received data and conclude that the session requires an HTTP decoder. In some cases, such as during a web browsing session, the identified application may change and such changes may be noted by the data appliance 102. For example, a user may first browse a corporate Wiki (classified as “Web Browsing-Productivity” based on the URLs visited) and then browse a social networking site (classified as “Web Browsing-Social Networking” based on the URLs visited). Different types of protocols have corresponding decoders.

[0028] Based on the determination made by the application identification engine 222, the packets are sent by the threat engine 224 to the appropriate decoder, which is configured to assemble the packets (which may be received out of order) into the correct order and extract the information. The threat engine 224 also performs signature matching to determine what should happen to the packets. If necessary, the SSL encryption engine 226 can re-encrypt the decrypted data. The packets are forwarded using the forwarding module 228 for forwarding (e.g., to a destination).

[0029] As also shown in FIG. 2B, policies 232 are also received and stored in the management plane 212. The policies can include one or more rules, which can be specified using domain names and / or host / server names, and the rules can apply one or more signatures or other matching criteria or heuristics, such as for security policy enforcement, to subscriber / IP flows based on various extracted parameters / information from the monitored session traffic flows. An interface (I / F) communicator 230 is provided for management communications (e.g., via (REST) ​​APIs, messages, or network protocol communications, or other communication mechanisms). The policies 232 can also include policies for managing communications involving IoT devices.

[0030] III. IoT Device Discovery and Identification

[0031] Returning to FIG. 1 , assume that a malicious individual has created malware 130 (e.g., using system 120). The malicious individual wants vulnerable client devices to run copies of the malware 130, compromising the client device and causing the client device to become a bot in a botnet. The compromised client device can then be instructed to perform tasks (e.g., cryptocurrency mining, participating in denial of service attacks, and propagating to other vulnerable client devices) and to report information to an external entity (e.g., command and control (C&C) server 150) or otherwise extract data, as well as receive instructions from C&C server 150, if applicable.

[0032] Some of the client devices shown in FIG. 1 are commodity computing devices typically used within an enterprise organization. For example, client devices 104, 106, and 108 each run a typical operating system (e.g., macOS, Windows, Linux, Android, etc.). Such commodity computing devices are often provisioned and maintained by administrators (e.g., as corporate-issued laptops, desktops, and tablets, respectively), and often operate with user accounts (e.g., managed by a directory service provider (also called a domain controller) configured with user identity and certificate information). As an example, employee Alice is issued a laptop 104 to use to access her ACME-related email and perform various ACME-related tasks. Other types of client devices (generally referred to herein as Internet of Things or IoT devices) are also increasingly present within networks, and are often “unmanaged” by IT departments. Some such devices (e.g., teleconferencing devices) may be found across a variety of different types of enterprises (e.g., as IoT whiteboards 144 and 146). Such devices may also be vertical specific. For example, infusion pumps and computed tomography scanners (e.g., CT scanner 112) are examples of IoT devices that may be found within a healthcare enterprise network (e.g., network 110), and a robotic arm is an example of a device that may be found within a manufacturing enterprise network. Additionally, consumer IoT devices (e.g., cameras) may also exist within enterprise networks. Similar to commodity computing devices, IoT devices that exist within a network may communicate with resources both inside or outside (or both, if applicable) such networks.

[0033] Like commodity computing devices, IoT devices are targets for malicious individuals. Unfortunately, the presence of IoT devices in a network can present some unique security / management challenges. IoT devices are often low-power or dedicated devices, and are often deployed without the knowledge of network administrators. Even when known to such administrators, it may not be possible to install endpoint protection software or agents on IoT devices. IoT devices may be managed by and communicate solely / directly with third-party cloud infrastructure (e.g., with an industrial thermometer 152 communicating directly with the cloud infrastructure 126) using proprietary (or otherwise non-standard) protocols. This may confound attempts to monitor network traffic in and out of such devices to make a determination as to when a threat or attack is occurring against the device. Furthermore, some IoT devices (e.g., in a healthcare environment) are mission-critical (e.g., network-connected surgical systems). Unfortunately, the compromise of an IoT device (e.g., by malware 130) or the misapplication of security policies to traffic associated with an IoT device can have potentially devastating effects. Techniques described herein may be used to improve the security of heterogeneous networks that include IoT devices and reduce harm caused to such networks.

[0034] In various embodiments, the data appliance 102 includes an IoT server 134. The IoT server 134, in some embodiments, is configured to cooperate with an IoT module 138 of the security platform 140 to identify IoT devices within a network (e.g., the network 110). Such identification may be used, for example, by the data appliance 102 to help create and enforce policies regarding traffic associated with the IoT devices and to enhance the functionality of other elements of the network 110 (e.g., providing contextual information to the AAA 156). In various embodiments, the IoT server 134 incorporates one or more network sensors configured to passively sniff / monitor traffic. One exemplary way of providing such network sensor functionality is as a tap interface or a switch mirror port. Other approaches to monitoring traffic may also be used (additionally or alternatively) where applicable.

[0035] In various embodiments, the IoT server 134 is configured to provide logs or other data (e.g., passively collected from the monitoring network 110) to the IoT module 138 (e.g., via the front end 142). FIG. 2C illustrates one exemplary event path between the IoT server and the IoT module. The IoT server 134 sends device discovery events and session events to the IoT module 138. Exemplary discovery and session events are illustrated in FIGS. 2D and 2E, respectively. In various embodiments, discovery events are sent by the IoT server 110 whenever it observes a packet that can uniquely identify or confirm the identity of the device (e.g., whenever a DHCP, UPNP, or SMB packet is observed). Each session that the device has (with other nodes, either inside or outside the device's network) is described in a session event that summarizes information about the session (e.g., source / destination information, number of packets received / sent, etc.). If applicable, multiple session events may be batched together by the IoT server 134 before sending to the IoT module 138. In the example shown in Figure 2E, two sessions are included. The IoT module 138 provides (234) the device classification information to the IoT server 134 via a device verdict event.

[0036] One exemplary way to implement IoT module 138 is to use a microservices-based architecture. IoT module 138 may also be implemented, where applicable, using different programming languages, databases, hardware, and software environments, and / or as services that are messaging enabled, context-bounded, autonomously developed, independently deployable, distributed, and built and released using automated processes. One task performed by IoT module 138 is to identify IoT devices in data provided by IoT server 134 (and by other embodiments of data appliances, such as data appliances 136 and 148), and provide additional contextual information about those devices (e.g., back to the respective data appliances).

[0037] FIG. 2F illustrates one embodiment of an IoT module. Area 295 illustrates a set of Spark Applications that run at intervals (e.g., 5 minutes, hourly, and daily) across all tenants' data. Area 297 illustrates a Kafka message bus. Session event messages received by IoT module 138 (e.g., from IoT server 134) bundle multiple events together as they are observed at IoT server 134 (e.g., to save bandwidth). Transformation module 236 is configured to flatten received session events into individual events and publish them at 250. The flattened events are aggregated by aggregation module 238 using a variety of different aggregation rules. One exemplary rule is "aggregate all event data for a particular device and each application (APP-ID) it uses for a time interval (e.g., 5 minutes)." Another exemplary rule is "aggregate all event data for a particular device communicating with a particular destination IP address for a time interval (e.g., 1 hour)." For each rule, the aggregation engine 238 tracks a list of attributes that need to be aggregated (e.g., a list of applications used by a device or a list of destination IP addresses). A feature extraction module 240 extracts features (252) from the attributes. An analysis module 242 uses the extracted features to perform device classification (e.g., using supervised and unsupervised learning), and the results (254) are used to power other types of analytics (e.g., via an operational intelligence module 244, a threat analysis module 246, and an anomaly detection module 248). The operational intelligence module 244 provides analytics related to the OT framework and operational or business intelligence (e.g., how the device is being used). Alerts (256) can be generated based on the results of the analytics.In various embodiments, MongoDB 258 is used to store the aggregated data and feature values. Background services 262 receive the data aggregated by the Spark application and write the data to MongoDB 258. API server 260 pulls and merges data from MongoDB 258 to fulfill requests received from frontend 142.

[0038] FIG. 2G illustrates one exemplary method of implementing IoT device identification analysis (e.g., in IoT module 138 as one embodiment of analysis module 242 and related elements). Discovery and session events (e.g., as shown in FIG. 2D and FIG. 2E, respectively) are received as Kafka topics on the message bus and as raw data 264 on the message bus (and also stored in storage 158). Features are extracted by feature engine 276 (which may be implemented using Spark / MapReducer, for example). The raw data is enriched 266 with additional context information, such as geolocation information (e.g., of source / destination addresses), by security platform 140. During metadata feature extraction 268, features such as the number of packets sent from an IP address within a time interval, the number of applications used by a particular device during that time interval, and the number of IP addresses contacted by the device during that time interval are constructed. Features are both passed in real time (e.g., in JSON format) to the inline analytics engine 272 (e.g., over a message bus) and stored (e.g., in a suitable format such as Apache Parquet / Data Frame in the feature database 270) for subsequent querying (e.g., during offline modeling 299).

[0039] In addition to features constructed from metadata, a second type of features, referred to herein as analytic features, may be constructed 274 by the IoT module 138. One exemplary analytic feature is one that is constructed over time based on time series data using aggregated data. The analytic features are similarly passed to the analytics engine 272 in real time and stored in the feature database 270.

[0040] An inline analytics engine 272 receives features on the message bus via a message handler. One task it performs is activity classification (278), which attempts to identify activity related to the session (such as a file download, a login / authentication process, or a disk backup activity) based on the received feature values / session information and attaches any applicable tags. One way to implement activity classification 278 is via a neural network-based multi-layer perceptron combined with a convolutional network.

[0041] Assume that as a result of the activity classification, it is determined that a particular device is engaged in printing activities (i.e., using a printing protocol) and also periodically contacts resources owned by HP (e.g., to check for updates by calling an HP URL and using it to report status information). In various embodiments, the classification information is passed to both a clustering process (unsupervised) and a prediction process (supervised). If either process is successful in classifying the device, the classification is stored in the device database 286.

[0042] Devices may be clustered into multiple clusters (e.g., behaves like a printer, behaves like an HP device, etc.) by a stage one clustering engine 280 based on their attributes and other behavioral patterns. One way to implement the clustering engine 280 is to use an extreme gradient boosting framework (e.g., XGB). Stage one classifiers can be useful for classifying devices that have not been seen before but are similar to existing known devices (e.g., a new thermostat vendor begins selling a thermostat device that behaves similarly to a known thermostat).

[0043] As shown in FIG. 2G, the activity classification information is also provided to a set of classifiers 282, and a prediction is performed based on the provided features about the device. Two possibilities can arise. In the first scenario, it is determined that there is a high probability that the device matches a known device profile (i.e., a high confidence score). If so, information about the device is provided to a stage two classifier (284), which makes a final verdict on the device's identity (e.g., using the provided information and any additional applicable contextual information) and updates the device database 286 accordingly. One way to implement the stage two classifier is to use a gradient boosting framework. In the second scenario, assume that the confidence score is low (e.g., the device matches both an HP printer and an HP laptop with 50% confidence). In this scenario, the information determined by the classifier 282 can be provided to the clustering engine 280 as additional information usable in clustering.

[0044] Also shown in FIG. 2G is an offline modeling module 299. The offline modeling module 299 is not time-constrained, in contrast to the inline analysis engine 272, which attempts to provide device classification information in real time (e.g., as messages 234). Periodically (e.g., once a day or once a week), the offline modeling module 299 (e.g., implemented using Python) rebuilds the models used by the inline analysis module 272. The activity modeling engine 288 builds models for activity classification 278, which are also used for device type models (296), which are used by the classifier for device identification during inline analysis. The baseline modeling engine 290 builds models of baseline behavior of device models, which are also used when modeling specific types of device anomalies (292), such as kill chains, and specific types of threats (294). The generated models are stored in a model database 298 in various embodiments.

[0045] IV. Network Entity Identification AAA

[0046] As mentioned above, assume that Alice has been issued a laptop 104 by ACME. Various components of network 110 cooperate to authenticate Alice's laptop as she uses it to access various resources. As one example, when Alice connects laptop 104 to a wireless access point (not shown) located within network 110, the wireless access point can communicate (either directly or indirectly) with AAA server 156 while provisioning network access. As another example, when Alice uses laptop 104 to access her ACME email, laptop 104 can communicate (either directly or indirectly) with directory service 154 while fetching her inbox, etc. As a commodity laptop running a commodity operating system, laptop 104 can generate appropriate AAA messages (e.g., RADIUS client messages) that help laptop 104 gain access to the appropriate resources it needs.

[0047] As mentioned above, one problem posed by IoT devices (e.g., device 146) in a network such as 110 is that such devices are often “unmanaged” (e.g., not configured, provisioned, managed, etc., by a network administrator), do not support protocols such as RADIUS, and therefore cannot be integrated with AAA services such as other devices such as laptop 104. Various approaches can be employed to provide IoT devices with network access in network 110, each of which has drawbacks. One option is for ACME to restrict the IoT device to use a guest network (e.g., via a pre-shared key). Unfortunately, this can limit the usefulness of the IoT device if it cannot communicate with other nodes in network 110 to which it should legitimately have access. Another option is to allow the IoT device unrestricted access to network 110, mitigating the security benefits of having a segmented network. Yet another option is for ACME to manually specify rules governing how a given IoT device should be able to access resources in network 110. This approach is generally unacceptable / unworkable for a variety of reasons. As one example, administrators may often not be involved in the deployment of IoT devices and therefore may not know that policies for such devices should be included (e.g., in data appliance 102). Even if an administrator could, for example, manually configure policies for a particular IoT device (e.g., for a device such as device 112) within appliance 102, keeping such policies up to date is error-prone and generally unacceptable given the sheer number of IoT devices that may be present in network 110.Moreover, such policies would likely be simple (e.g., assigning the CT scanner 112 to a particular network by IP address and / or MAC address) and would not allow for more fine-grained control over connections / policies involving the CT scanner 112 (e.g., dynamically including policies applicable to surgical devices versus point of sales (POS) terminals). Furthermore, even if the CT scanner 112 were manually included in the data appliance 102, as mentioned above, IoT devices generally do not support technologies such as RADIUS, and the benefits of having such an AAA server manage the networking access of the CT scanner 112 would be limited compared to other types of devices (e.g., laptop 104) that more fully support such technologies. As described in more detail below, in various embodiments, the data appliance 102 (e.g., via the IoT server 134) is configured to provide support for AAA functions to IoT devices present in the network 110 in a passive manner.

[0048] In the following description, assume that Alice's department at ACME recently purchased an interactive whiteboard 144 so that Alice can collaborate with other ACME employees as well as individuals outside of ACME (e.g., Bob, a researcher at Beta University who has his own network 114, data appliance 136, and whiteboard 146). As part of the initial setup of the whiteboard 146, Alice connects it to a power source and provides it with a wired connection (e.g., to an outlet in a conference room) or wireless credentials (e.g., credentials for use by visitors to the conference room). When the whiteboard 146 provisions a network connection, the IoT server 134 recognizes the whiteboard 146 as a new device in the network 110 (e.g., via mechanisms such as network sensors as described above). One action taken in response to this detection is to communicate with security platform 140 (e.g., create a new record for whiteboard 146 in database 160 and retrieve any currently available contextual information associated with whiteboard 146 (e.g., get manufacturer of whiteboard 146, model of whiteboard 146, etc.)). Any contextual information provided by security platform 140 may be provided to (and stored by) data appliance 102, which in turn may be provided to directory service 154 and / or AAA server 156, if applicable. If applicable, IoT module 138 may provide updated contextual information regarding whiteboard 146 to data appliance 102 as it becomes available. Data appliance 102 may then (e.g., via IoT server 134) provide ongoing information regarding whiteboard 146 to security platform 140 as well.Examples of such information include observations regarding the behavior of the whiteboard 146 on the network 110 (e.g., statistics regarding the connections it makes) that may be used by the security platform 140 to build a behavior profile for a device such as the whiteboard 146. Similar behavior profiles may be built by the security platform 140 for other devices (e.g., the whiteboard 144). Such profiles may be used for a variety of purposes, including detecting anomalous behavior. As one example, the data appliance 148 may use the information provided by the security platform 140 to detect whether the thermometer 152 is behaving abnormally, compared to historical observations of the thermometer 152 and / or compared to other thermometers of a similar model, manufacturer (not shown), or more generally, including thermometers present in other networks. If anomalous behavior is detected (e.g., by the data appliance 148), appropriate corrective action may be automatically taken, such as restricting access of the thermometer 152 to other nodes on the network 116, generating an alert, etc.

[0049] 3 illustrates one embodiment of a process for passively providing AAA support for IoT devices in a network. In various embodiments, the process 300 is performed by the IoT server 134. The process begins at 302 when a set of packets transmitted by an IoT device is obtained. As one example, such packets may be passively received by the IoT server 134 at 302 when the whiteboard 146 is initially provisioned on the network 110. Packets may also be received at 302 during subsequent use of the whiteboard 144 (e.g., when Alice has a whiteboard session with Bob via the whiteboard 146). At least one packet in the set of data packets is analyzed at 304. As one example of the process performed at 304, the IoT server 134 determines that the packet received at 302 is transmitted by the whiteboard 146. One action that the IoT server 134 can take is to identify the whiteboard 146 as a new IoT device on the network 110 and obtain context information, if available, from the IoT module 138. At 306, the IoT server 134 sends an AAA message on behalf of the IoT device, including information associated with the IoT device. An example of such a message is shown in FIG. 4A. As mentioned above, the whiteboard 146 does not support the RADIUS protocol. However, the IoT server 134 can generate a message, such as that shown in FIG. 4A, on behalf of the whiteboard 146 (e.g., using the information received at 302 and also from the security platform 140, if applicable). As mentioned above, when the IoT server 134 provides information about the whiteboard 146 to the IoT module 138, the IoT module 138 can take various actions.4A , RADIUS messages generated by the IoT server 134 on behalf of the whiteboard 146 may include limited information. As additional contextual information is collected by the security platform 140, the profile may be updated and propagated to the data appliance 102. When the whiteboard 146 is initially provisioned in the network 110, the additional contextual information may not be available (e.g., the security platform 140 may not have such additional information or the security platform 140 may not provide such information to the IoT server 134 instantaneously). Thus, and as shown in FIG. 4A , RADIUS messages generated by the IoT server 134 on behalf of the whiteboard 146 may include limited information. As additional contextual information is received (e.g., by the IoT server 134 from the IoT module 138), subsequent RADIUS messages sent by the IoT server 134 on behalf of the whiteboard 146 may be enriched with such additional information. Examples of such subsequent messages are shown in Figures 4B and 4C. Figure 4B shows an example of a RADIUS message that the IoT server 134 can send on behalf of the whiteboard 146 once the context information for the whiteboard 146 (e.g., including a database of context information for a wide variety of IoT devices) is provided by the IoT module 138. In the example shown in Figure 4B, context information such as the manufacturer of the whiteboard (Panasonic) and the nature of the device (e.g., that it is an interactive whiteboard) is included. Such context information can be used by an AAA server, such as AAA server 156, to provide AAA services for the whiteboard 146 (without having to modify the whiteboard 146).4C , by automatically provisioning on a sub-network dedicated to teleconferencing equipment. Other types of IoT devices may also be automatically grouped based on attributes such as device type, purpose, etc. (e.g., critical surgical equipment is automatically provisioned on a sub-network dedicated to such equipment and thus isolated from other devices on the network). Such context information may be used to enforce policies such as traffic shaping policies, such as a policy (e.g., determined using the APP-ID) that gives preferential treatment to whiteboard 146 packets over social networking packets. Fine-grained policies may be applied to communications with critical surgical equipment as well (e.g., preventing any device communicating with such equipment from having an out-of-date operating system, etc.). In the example shown in FIG. 4C , additional context information is also included by the IoT server 134 in the RADIUS message on behalf of the whiteboard 146. Such additional context information includes additional attribute information such as device model, operating system, and operating version. When the whiteboard 146 is initially provisioned in the network 110, all of the context information shown in FIG. 4C may not be available. As the whiteboard 146 is used over time within the network 110, additional contextual information may be collected (e.g., as the IoT server 134 continues to passively observe packets from the whiteboard 146 and provide information to the security platform 140). This additional information may be leveraged (e.g., by the data appliance 102) to enforce fine-grained policies. As one example, as shown in FIG. 4C, the whiteboard 146 runs a particular operating system that is Linux-based and has version 3.16. Frequently, IoT devices run versions of operating systems that are not upgradeable / patchable.Such devices may pose security risks as exploits are developed for their operating systems. The data appliance 102 can implement security policies based on the context information, such as by isolating (or otherwise restricting access to) IoT devices with outdated operating systems from other nodes in the network 110, while allowing less restrictive network access to IoT devices with current operating systems.

[0050] 4A-4C show examples of RADIUS Access-Request messages. Where applicable, the IoT server 134 can generate various types of RADIUS messages on behalf of the whiteboard 146. As one example, a RADIUS accounting start message can be triggered when traffic from the whiteboard 146 is first observed. Periodic RADIUS accounting interim update messages can be sent while the whiteboard is in use, and a RADIUS accounting stop message can be sent when the whiteboard 146 goes offline.

[0051] V. IoT Device Discovery and Identification

[0052] As mentioned above, one task performed by the security platform 140 (e.g., via the IoT module 138) is IoT device classification. As one example, when the IoT server 134 sends a device discovery message to the IoT module 138, the IoT module 138 attempts to determine a classification for the device and respond (e.g., via decision 234 shown in FIG. 2C). The device is associated with a unique identifier by the IoT module 138, if applicable, such that subsequent classification of the device does not need to be performed (or, if applicable, is performed less frequently than it would otherwise be). Also, as mentioned above, the determined classification may be used (e.g., by the data appliance 102) to enforce policies on traffic to / from the device.

[0053] To classify devices, various approaches can be used. The first approach is to perform the classification based on a set of rules / heuristics that leverage static attributes of the device, such as its organizationally unique identifier (OUI), the type of application it runs, etc. The second approach is to perform the classification using machine learning techniques that leverage dynamic but predefined attributes of the device extracted from its network traffic (e.g., the number of packets sent per day). Unfortunately, both of these approaches have weaknesses.

[0054] A rule-based approach generally requires that a separate rule be manually created for each type of IoT device (describing which attributes / values ​​should be used as signatures for that type of device signature). One challenge presented by this approach is in determining which signatures are both relevant to identifying the device and unique among other device signatures. Furthermore, in a rule-based approach, a limited number of static attributes are available that can be easily obtained from the traffic (e.g., user agent, OUI, URL destination, etc.). The attributes generally need to be simple enough to appear in a pattern that a regular expression can match. Another challenge is in identifying new static attributes that may be present / identifiable as new devices enter the market (e.g., a new brand or model of CT scanner is offered). Another challenge is that all matching attributes in the network traffic need to be collected for the rule to be triggered. With fewer attributes, a verdict cannot be reached. As an example, a signature may require that a specific device with a specific OUI connects to a specific URL. While having the OUI itself may already be a sufficient indicator of the device's identity, the signature does not trigger until the URL is also observed. This causes additional delays in determining the identity of the device. Another challenge is maintaining and updating the signature as a static attribute of the device that changes over time (e.g., due to updates made to the device or the services used by the device). As an example, a particular device may have initially been manufactured using a first type of network card, but over time the manufacturer may have switched to a different network card (indicating a different OUI). If the rule-based system is unaware of the change, false positives may result. Yet another challenge is in scaling signature generation / verification as the number of new IoT devices brought online each day approaches millions of new device instances.As a result, newly created rules may conflict with existing rules and cause false positives in classification.

[0055] Machine learning based approaches generally involve creating training models based on static and / or dynamic features extracted from network traffic. Prediction results for network data from new IoT devices are based on pre-trained models that provide device identification information with associated accuracy. Examples of problems with machine learning approaches are as follows: If predictions are made for each new device or for devices that do not have a constant / unique ID (e.g., MAC address), the computation time required to reach the desired accuracy may be unacceptable. There are thousands or tens of thousands of features that need to be generated, and these features may vary over a given time window, taking a significant amount of time before a sufficient number of features are available to make a valid prediction (which may defeat the purpose of policy enforcement). Furthermore, if the goal is to minimize delays in predictions, the cost of building and maintaining large data pipelines for streaming network data may be high. Yet another problem is that noise introduced by irrelevant features specific to a given deployment environment may reduce the accuracy of the predictions. And there are challenges in maintaining and updating the models when the number of device types reaches tens of thousands or more.

[0056] In various embodiments, the security platform 140 addresses the problems of each of the two aforementioned approaches by using a hybrid approach to classification. In an exemplary hybrid approach, a network behavior pattern identifier (also referred to herein as a pattern ID) is generated for each type of device. In various embodiments, the pattern ID is a list of attributes or sequence features combined with their respective probabilities (as importance scores for features or behavior categories), which form a distinct network behavior description and can be used to identify the type of IoT device. The pattern ID can be stored (e.g., in a database) and used to identify / verify the identity of the device.

[0057] When training on a set of attributes, certain approaches such as the extreme gradient boosting framework (e.g., XGB) can provide a top list of important features (whether static attributes, dynamic attributes, and / or aggregated / transformed values). The pattern ID can be used to uniquely identify the device type once established. If certain features are dominant for the device (e.g., a certain static feature (such as contacting a very specific URL at boot time) identifies the device with 98% confidence), they can be used to automatically generate rules. Even if there are no dominant features, a representation of the top features can still be used as the pattern ID (e.g., if multiple sets of features are concatenated into a pattern). By training on a dataset that includes all known models (and all known IoT devices), potential conflicts between models / uniquely identifying features can be avoided. Furthermore, the pattern ID does not need to be human readable (but can be stored, shared, and / or reused for identification purposes). Significant time savings can also be realized with this approach, and it can be used for near real-time classification. Classification of a particular device can occur as soon as a dominant feature is observed (instead of having to wait until a large number of features occur).

[0058] One example of data that may be used to create a Pattern ID for a “Teem Room Display iPad®” device may include the following (with the complete list generated automatically through training a multivariate model or training multiple binary models): *Apple devices (100%) *Special iPad (>98.5%) *Teamroom app (>95%) *Meets volume pattern VPM-17 (>95%) *Servers in the cloud (>80%)

[0059] One exemplary way to implement a hybrid approach is as follows: A neural network-based machine learning system may be used for automated pattern ID training and generation. Examples of features that may be used to train the neural network model include both static features extracted from network traffic (e.g., OUI, hostname, TLS fingerprint, matched L7 payload signatures, etc.) and sequence features extracted from network traffic but not specific to the environment (e.g., application, L7 attributes of application, volume ranges converted to categorical features, etc.). A lightweight data pipeline may be used to stream selected network data for feature generation in real time. A prediction engine may be used to import the model and provide caching to minimize prediction latency. In prediction, short (e.g., minute-based) aggregation may be used to stabilize selected sequence features. Customized data normalization, enrichment, aggregation, and transformation techniques may be used to manipulate sequence features. For better accuracy, longer aggregation windows may be used in training. With features that have been merged and aggregated over time, the accuracy of predictions may be improved. A back-end feedback engine can be used to route the results of a “slow path” prediction system (e.g., a machine learning based approach including a device type modeling subsystem and a device group modeling subsystem) that helps expand the attributes used for pattern ID prediction. A device group model can be trained to compensate for issues with the device type model when sufficient samples or features are not available, improving accuracy beyond an acceptable threshold (e.g., assigning prediction results based on a set of predefined types, some of which are attached to another subsystem to cluster unlabeled devices of similar types).Finally, a decision module may be used to publish results from the real-time prediction engine.

[0060] Exemplary advantages of the hybrid approach to classification as described herein are: First, fast convergence occurs, allowing a given device to potentially be identified within minutes or seconds. Second, the invention addresses the individual problems of rule-based and machine learning-based systems. Third, it provides stability and consistency in prediction results. Fourth, it has the scalability to support tens of thousands (or more) of different types of IoT devices. Prediction is typically only required for new devices (even if a given device lacks a unique ID assignment, such as L3 network traffic-based identification).

[0061] An embodiment of module 138 is shown in Figure 5. One exemplary way to implement IoT module 138 is to use a microservices-based architecture where services are fine-grained and protocols are lightweight. Services may also be implemented using relatively small services that are context-bounded, autonomously developed, independently deployable, distributed, and built and released using automated processes, spanning different programming languages, databases, hardware and software environments, and / or messaging environments, where applicable.

[0062] As previously mentioned, in various embodiments, the security platform 140 periodically receives (e.g., from the data appliance 102) information about IoT devices on a network (e.g., the network 110). In some cases, the IoT device has been previously classified by the security platform 140 (e.g., a CT scanner installed on the network 110 last year). In other cases, the IoT device is newly seen by the security platform 140 (e.g., a whiteboard 146 is being installed for the first time). Assume that a given device has not been previously classified by the security platform 140 (e.g., there is no entry for the device in the database 286 that stores a set of unique device identifiers and associated device information). As shown in FIG. 5, information about a new device may be provided to two different processing pipelines for classification. Pipeline 504 represents a “fast path” classification pipeline (corresponding to a pattern ID-based approach), and pipeline 502 represents a “slow path” classification pipeline (corresponding to a machine learning-based approach).

[0063] In pipeline 504, fast path feature engineering is performed (508) to identify applicable static and sequence features of the device. Fast path prediction is performed (510) using the pattern ID or a previously built model (e.g., a model built based on top significant features and using offline processing pipeline 506). A confidence score for the device matching the particular pattern is determined (512). If the confidence score for the device meets a pre-trained threshold (e.g., based on the overall prediction accuracy of module 138 or its components, such as 0.9), a classification can be assigned or updated to the device (in device database 516), if applicable. Initially, the confidence score is based on near real-time fast path processing. The advantage of this approach is that the data appliance 102 can start applying policies to the device's traffic very quickly (e.g., within minutes of module 138 identifying the device as new / unclassified). The appliance 102 may be configured to be fail-safe (e.g., reduce / limit the device's ability to access various network resources) or fail-safe (e.g., allow broad access for the device) pending a classification decision from the system 140. As additional information becomes available (e.g., via slow-path processing), the confidence score can be based on that additional information, if applicable (e.g., increasing the confidence score or correcting / amending mistakes made during fast-path classification).

[0064] Examples of features (eg, static attributes and array features) that can be used include the following: The pattern ID can be any combination of these attributes with logical conditions, including: *OUI in mac address *Hostname string from decoded protocol *User agent strings from HTTP and other clear-text protocols *The system name string from the decoded SNMP response *OS, hostname, domain, and username from decrypted LDAP protocol *URL from decoded DNS protocol *SMB version, command, error from decoded SMB protocol *TCP flags *Decoded option string from the DHCP protocol *Strings from decoded IoT protocols, such as Digital Imaging and Communications in Medicine (DICOM) * List of inbound applications from the local network *List of inbound applications from the Internet * List of outbound applications to the local network *List of outbound applications to the Internet *List of inbound server ports from the local network *List of inbound server ports from the Internet *List of outbound server ports to the local network *List of outbound server ports to the Internet *List of inbound IPs from your local network *List of inbound URLs from the Internet *List of outbound IPs to your local network *List of outbound URLs to the Internet

[0065] In some cases, the confidence score determined at 512 may be very low. One reason this may occur is because the device is a new type (e.g., a new type of IoT toy or other type of product not previously analyzed by security platform 140) and there is no corresponding pattern ID available for the device on security platform 140. In such a scenario, information about the device and the classification results may be provided to the offline processing pipeline 506, which may perform clustering (514) on the behavior and other applicable information exhibited by the device (e.g., to determine that the device is a wireless device, acts like a printer, uses the DICOM protocol, etc.). The clustering information is applied as a label and, if applicable, flagged for further investigation 518, so that any similar devices subsequently seen are automatically grouped together. If, as a result of the investigation, additional information about a given device is determined (e.g., identified as corresponding to a new type of consumer IoT meat thermometer), the device (and all other devices with similar characteristics) can be relabeled accordingly (e.g., as a brand XYZ meat thermometer) and an associated pattern ID can be generated and made available by pipelines 502 / 504, if applicable (e.g., after the models have been rebuilt). In various embodiments, offline modeling 520 is a process that runs daily to train and update various models 522 used for IoT device identification. In various embodiments, the models are refreshed daily to cover new labeled devices, and rebuilt weekly (for the slow path pipeline 502) to reflect behavioral changes and adapt to new features and data insights added during the week.Note that when adding a new type of device to security platform 140 (i.e., creating a new device pattern), multiple existing device patterns may be affected, requiring either the list of features or their importance scores to be updated. This process can be performed automatically (and is a major advantage compared to rule-based solutions).

[0066] For fast path modeling, neural network-based models (e.g., FNN) and general classification models (e.g., XGB) are widely used for multivariate machine learning models. To help improve the results and provide input to the clustering, a binary model is also built for the selected profile. The binary model gives a yes / no answer to the identity of the device or a given behavior of the device. For example, a binary model can be used to determine whether a device is a type of IP phone or is unlikely to be an IP phone. A multivariate model has many outputs normalized to a probability of 1. Each output corresponds to a type of device. A binary model is generally faster, but requires the device to go through many of them in the prediction so that the correct "yes" answer can be found. A multivariate model can achieve that in one step.

[0067] The slow-path pipeline 502 is similar to pipeline 504 in that features are extracted (524). However, the features used by pipeline 502 typically take a period of time to build. As one example, a feature "bytes sent per day" requires one day to collect. As another example, a given usage pattern may take a period of time to develop / observe (e.g., a CT scanner is used hourly to perform scans (first behavior), backs up data daily (second behavior), and checks the manufacturer's website weekly for updates (third behavior)). The slow-path pipeline 502 invokes a multivariate classifier (526) in an attempt to classify new device instances against the complete set of features. The features used are not limited to static or sequence features, but include volume and time series based features as well. This is generally referred to as stage 1 prediction. If the results of stage 1 prediction are not optimal (low confidence), for a particular profile, stage 2 prediction is used to improve the results. The slow-path pipeline 502 invokes a set of decision tree classifiers (528) supported by additional imported device context to classify new device instances. The additional device context is imported from an external source. As one example, URLs connected to by the device may have been given a category and a risk-based reputation that may be included as a feature. As another example, applications used by the device may have been given a category and a risk-based score that may be included as a feature. By combining the results from stage 1 prediction 526 and stage 2 prediction 528, a final decision of slow-path classification may be achieved using the derived confidence score.

[0068] Generally, there are two stages included in the slow path pipeline 502. In the slow path pipeline, in some embodiments, a stage 1 model is built using a multivariate classifier based on neural network techniques. Stage 2 of the slow path pipeline is generally a set of decision-based models with additional logic to handle probability-related exceptions of stage 1. In prediction, stage 2 integrates inputs from stage 1, applies rules and context to validate the output of stage 1, and generates the final output of the slow path. The final output includes the device's identity, an overall confidence score, a pattern ID that can be used in future fast path pipelines 504, and an explanation list. The confidence score is based on the reliability and accuracy of the model (the model also has a confidence score) and the probability as part of the classification. The explanation list includes a list of features that contribute to the result. As mentioned above, an investigation can be triggered if the result deviates from a known pattern ID.

[0069] In some embodiments, two types of models are built for slow path modeling: one for individual identification and one for group identification. For example, telling the difference between two printers from different vendors or with different models is often more difficult than distinguishing a printer from a thermometer (e.g., because printers tend to exhibit network behavior, speak similar protocols, etc.). In various embodiments, various printers from various vendors are included in a group, and a "printer" model is trained for group classification. This group classification result can provide better accuracy than a specific model of a specific printer, and can be used to update the device's confidence score, if applicable, or to provide reference and validation for identity-based classification of individual profiles.

[0070] FIG. 6 illustrates an example of a process for classifying an IoT device. In various embodiments, the process 600 is performed by the security platform 140. The process 600 may also be performed by other systems (e.g., systems collocating on-premise with the IoT device) where applicable. The process 600 begins at 602 when information associated with the network communications of the IoT device is received. As an example, such information is received by the security platform 220 when the data appliance 102 sends a device discovery event for a given IoT device to the security platform 140. At 604, a determination is made that the device is not classified (or that reclassification should be performed, if applicable). As an example, the platform 140 may query the database 286 to determine whether the device is classified. At 606, a two-part classification is performed. As one example, at 606, a two-part classification is performed by the platform 140, providing information about the device to both the fast-path information provision pipeline 504 and the slow-path classification pipeline 502. Finally, at 608, the results of the classification process performed at 606 are provided to a security appliance configured to enforce policies on the IoT device. As discussed above, this allows highly fine-grained security policies to be implemented in potentially mission-critical environments with minimal administrative effort.

[0071] In a first example of implementing the process 600, assume that an Xbox One game console is connected to the network 110. During classification, a determination may be made that the device has the following key features: a “vendor=Microsoft” feature with 100% confidence, a “communicates with Microsoft cloud server” feature with 89.7% confidence, and a “game console” match feature with 78.5% confidence. These three features / confidence scores may be collectively matched against a set of profile IDs (a process driven by neural network-based prediction) to identify the device as an Xbox One game console (i.e., at 512, a profile ID match that meets a threshold is found). In a second example, assume that an AudioCodes IP phone is connected to the network 110. During classification, a determination can be made that the device matches the "vendor=AudioCodes" feature with 100% confidence, the "is IP audio device" feature with 98.5% confidence, and the "act as local server" feature with 66.5%. These three features / confidence scores are also matched against a set of profile IDs, assuming in this scenario that there are no existing profile IDs that match with sufficient confidence. The information about the device can then be provided to a clustering process 514, and if applicable, a new profile ID can finally be generated and associated with the device (and used to classify future devices).

[0072] If applicable, the security platform 140 may recommend certain policies based on the determined classification. The following are examples of policies that may be implemented: *Deny internet traffic for all infusion pumps (regardless of vendor). *Deny Internet traffic for all GE ECG machines except from / to specific GE hosts. *Allow only internal traffic to Picture Archiving and Communication System (PACS) servers for all CT scanners (regardless of vendor).

[0073] VI. Application Workload Capture for IoT Devices

[0074] Some types of security threats may be detected and / or prevented in real time based on observing a single packet or event. As one example, the data appliance 102 may detect / block an attempted transmission of a known malicious file (e.g., as an attempted file download made by the client device 104 from a malicious website or sent to the client device 104 as an attachment) by comparing the file's signature to a list of known bad files. Other types of security threats require larger data sets (e.g., packet captures) to identify.

[0075] Network traffic analysis may be performed in real-time (e.g., by the data appliance 110 as traffic flows through the data appliance 102) and also offline (e.g., by the security platform 140 analyzing previously stored traffic) in conjunction with post-incident forensics, research, and the like. Unfortunately, while the availability of some computing resources continues to grow rapidly (e.g., cloud-based storage and cloud-based computing power), other resources (e.g., the storage and computing power of the data appliance 102) are limited. As one example, an embodiment of the data appliance 102 may process thousands of network transactions per second. This translates into tens of thousands or more individual packets (potentially terabytes of packets per day). Storing and processing this amount of data in real-time may be difficult / infeasible, for example, based on the hardware specifications of the data appliance 102. Moreover, even if it were possible to store and process every packet, it would likely be prohibitively expensive (e.g., the cost of purchasing cloud storage and associated computing resources) and / or would otherwise have adverse effects (e.g., reduced network speed / performance experienced by end users due to the computational overhead imposed on the data appliance 102). Thus, there is currently a need to be able to prioritize what network traffic is captured to best support complex tasks such as real-time traffic analysis and post-incident forensic analysis, while at the same time being able to scale with the amount of network traffic being processed by a data appliance, such as data appliance 102, and its resources.

[0076] FIG. 7 illustrates one embodiment of an IoT server. As previously mentioned, the IoT server can take various forms, including as a standalone server (or set of servers), whether physical or virtualized, and can also be co-located / embedded with the data appliance 102 (e.g., as IoT server 134), if applicable. The IoT server 134 includes a real-time priority ring buffer 702 (which may also be referred to as a circular buffer, circular queue, or circular buffer). The buffer 702 stores raw IoT network session data. In one exemplary implementation of the buffer 702, the total buffer size is 500 MB of RAM, optionally with additional space reserved for snapshots. The size of the buffer can vary. However, a constraint on the buffer size is that the IoT network data in the ring buffer typically has a low information density, so sorting through a larger buffer would not be worth the effort. In an exemplary implementation, the ring buffer 702 is divided into 50 batches, each with a size of 10 MB. A given batch contains traffic data (e.g., raw traffic data) for a period of time, and when the batch size limit of 10 MB is reached, a new batch is filled with the traffic data. In an alternative example, a time-based ID is used for each file, and the batches are indexed by time. The ring buffer 702 does not have to shuffle its elements as one is consumed, and can have a fixed maximum size and can have a constant-time queue behavior, overwriting old data with new data in a manner reminiscent of a first-in-first-out (FIFO) queue, in that the first data written is the first data overwritten.

[0077] As incoming packets arrive (710), they may be made available by the kernel for capture (e.g., via a Berkeley Packet Filter (BPF)). The IoT server 134 performs various tasks in conjunction with the ring buffer 702. A first task is to selectively place a subset of the packets it receives (e.g., from the BPF) into the ring buffer 702. A second task is to selectively extract flows of packets from the ring buffer 702 (e.g., to send to offline storage 704 in response to a trigger activation). The IoT server 134 can conveniently store traffic corresponding to the flows of most interest and provide the packets (e.g., as context information) for persistent storage and subsequent analysis (e.g., when an alert is triggered).

[0078] A. Target Acquisition

[0079] The rules engine 706 is configured to provide instructions to the IoT server 134 for determining when the IoT server 134 should accept packets into the ring buffer 702 (e.g., under what circumstances IoT traffic should be captured and for which IoT devices). In one exemplary embodiment, the rules engine 706 is included in the IoT module 138 (e.g., as one or more Spark applications 295) and helps the IoT server 134 identify capture targets. The target manager 708 (e.g., implemented as a set of one or more scripts) uses information provided by the rules engine 706 (and, if applicable, observations of the devices and network traffic of the data appliance 102 itself) to maintain a list of targets for traffic capture. As described in more detail below, traffic capture may be performed on all network traffic associated with a given target IoT device. It may also be selectively narrowed (e.g., to traffic relating to one or more particular types of traffic, such as traffic using particular protocols, to / from particular IP addresses and / or ports, and / or to traffic related to particular applications associated with the target device).

[0080] One exemplary consideration in determining whether IoT traffic should be captured is the risk level associated with the IoT device. One example of a risk level is a score on a 0-100 point scale, such as: *0-39 Device poses normal / low risk. *40-69 The device poses a medium risk (e.g., poor user practices have been observed, such as using manufacturer's default passwords). *70-89 The device poses a high risk (e.g., exploits have been attempted by or against the device, or the device has attempted to access a risky Internet site). *90-100 The device poses a significant hazard (e.g., a known breach of the device has occurred). In various embodiments, the risk level is determined by the IoT module 138 (e.g., based on various information analyzed and stored by the IoT module 138, as described above) and may be used to select IoT traffic for capture in various ways. In a first example, a rule may be set to target capturing traffic of all IoT devices in the network 110 that have a risk score of 80 or greater. If a given device regularly and consistently has a score of 60 or less, its traffic will not be captured (unless other applicable conditions are met). However, if an observation is made that the risk of the device has risen to 80 points (e.g., the device is observed taking a particular risky action, the device's vendor has issued a vulnerability warning, or the device's firmware is out of date), the device is added to a target list and capture of at least a portion of its traffic begins. In an alternative example, a rule may be set to target capturing traffic of IoT devices whose security scores increase by a threshold amount (e.g., a gain of 50 points or more), regardless of whether a particular score has been reached. As one example, if a device typically has a risk score of 20, but then later has a risk score of 70 (e.g., based on adherence to certain risky activities), it may be added to the target list.

[0081] Another example consideration in determining whether IoT traffic should be captured is configuration / behavior changes detected for an IoT device (e.g., if an IoT device begins using existing applications in a different way or begins using new applications). If applicable, the capture may be limited in scope to only traffic related to the changed application.

[0082] Some specific example scenarios for performing selective IoT device traffic capture are as follows: *An IV pump has its risk score increased (e.g., from 50 to 80) by IoT module 138 due to the publication of an applicable new Common Vulnerabilities and Exposures (CVE) entry and the observation of anomalous connections that do not match the pump's typical behavior. *A set of security cameras are relocated from a first subnet to a different subnet that allows inbound connections from a previously unobserved network. This configuration change (along with an existing CVE related to the cameras) makes it a target for capture. *The X-Ray machine has a routine heartbeat connection every hour from the installed security agent to the security server. The X-Ray machine begins to lose heartbeats. This observed change makes it a target for capture. *Microsoft Remote Desktop Protocol (RDP) vulnerabilities have been discovered and published by Microsoft. All IoT devices utilizing the RDP protocol have an elevated risk score, potentially making them eligible targets for capture.

[0083] Using the RDP vulnerability as an example, a rule to target traffic capture could be specified as follows: if risk_score changes and risk_score>medium and application==RDP then tag=on and hours=24

[0084] This rule specifies that if a device's risk score changes to high or critical and the application it is using is RDP, then the associated packets (also referred to herein as the "RDP workload") should be captured for a 24-hour period. Other examples of rules that may be used to limit packet capture include rules that specify that for a given IoT device in the target list, only traffic to / from a particular IP address (or exchanged through a particular website), or traffic that occurs during a particular time window (e.g., Monday 8:00-8:30 AM) is captured.

[0085] Each application running on a device has a corresponding flow that can be observed by a data appliance (e.g., a TCP flow or a UDP flow observed by the data appliance 102). As mentioned above, while all packets associated with a given IoT device can be captured, the overall load on the packet capture can be reduced by capturing only packets associated with a particular flow. At the transport layer, a flow can be uniquely identified by a 5-tuple, i.e.,<src_ip,src_port,dst_ip,dst_port,prot> where src_ip is the source IP address, src_port is the source port number, dst_ip is the destination IP address, dst_port is the destination port number, and prot is the protocol number. A hash table can be used to map each unique 5-tuple to an integer flow identifier. When a packet arrives, the 5-tuple in its transport layer header is used to look up the hash table to find the flow identifier if it exists, or create a new one if it does not. One way that the IoT server 134 can focus very specifically on particular flows is through tagging them (and their packets) with a packet tagger 712 (which can be implemented, for example, as a set of scripts). When the first packet in a flow is received (in the case of TCP, e.g., a TCP SYN packet is sent by a client to a server), the packet tagger 712 tags the flow (e.g., adds the 5-tuple to a tagging list) if the packet matches an applicable targeting rule. As subsequent packets are received, if their 5-tuples appear in the tagging list, they may be accepted into the ring buffer 702 accordingly.

[0086] In the above example, and conversely, the target may be removed when the risk score returns to normal or when other conditions (which may be set by the same rule) are met (e.g., a defined number of sessions have been captured or a certain period of time has passed).

[0087] B. Packet Extraction

[0088] As packets continue to arrive, the ring buffer 702 eventually begins to automatically overwrite older packets in its storage with new packets (i.e., data remains on the ring buffer for a relatively short period of time). If a particular attack or other security-related problem occurs over a longer period of time, it may avoid detection (e.g., a network traffic analysis that requires four days' worth of traffic may not be successful if only two hours' worth of traffic is available). In addition to selectively accepting packets into the ring buffer 702, another task performed by the IoT server 134 is to selectively extract packets from the ring buffer 702 for longer-term storage. Those extracted packets may be saved locally and then, optionally, transmitted to an offline storage 704 (e.g., stored within the network 110 and / or by the security platform 140 for aggregation and subsequent analysis).

[0089] In various embodiments, the packet extractor 714 (which may be implemented, for example, as a set of scripts) extracts packets from the ring buffer 702 and stores them (for example, in local storage that may be copied externally, if applicable). Extraction may be performed in various ways. As one example, the packet extractor 714 may use a set of smart triggers to determine which packets to extract. A first example of a smart trigger is a time-based trigger (e.g., once an hour, or every day at 15:00). A second example of a smart trigger is an alert-based trigger (e.g., the IoT module 138 generates an alert and provides it to the IoT server 134, which extracts packets implied by the alert). One example of an alert-based trigger is as follows: Assume that a particular IV pump is known to be vulnerable to a particular exploit described in a CVE (and packets associated with the IV pump have been captured in the ring buffer 702). Further assume that an active exploit targeting an IV pump is observed (e.g., by the appliance 102 or by the security platform 140). An alert may be generated in response (e.g., by the security platform 140) and ultimately routed (directly or indirectly) to the IoT server 134, which then triggers the extraction of packets associated with the IV pump from the ring buffer 702 for offline storage (and analysis, if applicable). Such an alert may likewise be used to initiate packet capture, if applicable (e.g., if the IoT server 134 has not yet accepted packets associated with the IV pump into the ring buffer 702).

[0090] Triggers can be configured based on a variety of factors, including whether a device is designated as "mission critical" (e.g., either making the trigger more sensitive or making the trigger an automatic trigger for continuous workload capture), anomalous behavior (triggering workload extraction when a device deviates from its expected behavior), timeliness (files are indexed by flow time and workloads can be extracted using timestamps), new patterns (devices without a priori behavior models have traffic with start times but with patterns that are not yet known. If configured to do so, continuous workload capture can be triggered until patterns are learned and categorized as not requiring workload capture), and threats (greater threats, including devices with high risk scores, can make the trigger more sensitive or make the trigger an automatic trigger for continuous workload capture).

[0091] FIG. 8 illustrates an example of messages that may be exchanged in conjunction with capturing an IoT device application workload. The message exchange illustrated in FIG. 8 corresponds to the RDP vulnerability scenario described above. Events occurring in the scenario include an RDPCVE being issued, RDP traffic being observed, and the device's risk score being increased (802). These events collectively cause the rules engine 706 to instruct the target manager 708 to begin capturing packets associated with the device's RDP usage in the ring buffer 702 (804), and also to instruct the packet extractor 714 on the conditions under which packets should be extracted from the ring buffer 702 (806). After one day has passed, the rules engine 706 instructs the target manager 708 to remove the device from its target list (i.e., stop capturing its packets), and similarly instructs the extraction manager 810 to stop extracting associated packets from the ring buffer 702 (810). Packets associated with the device that remain in the ring buffer 702 (and that have not been extracted) will eventually be overwritten with other packets.

[0092] FIG. 9 illustrates an example of a process for capturing an IoT device application workload. In various embodiments, the process 900 is performed by the IoT server 134. The process 900 begins at 902 when a target IoT device is selected. An example of such selection occurs when the rules engine 706 (e.g., as provided by the IoT module 138) instructs the IoT server 134 to add a given IoT device to a target list. At 904, a flow associated with the device is determined and tagged. The tag is used (e.g., at 906) to accept packets (i.e., packets associated with the tagged flow) into a ring buffer (e.g., the ring buffer 702). At 908, an indication is received that extraction should be performed on a portion of the packets contained in the ring buffer. An example of such an indication is the triggering of a smart trigger. Finally, at 910, the packets are extracted from the ring buffer. Generally, all packets belonging to the tagged flow are extracted. As mentioned above, the extracted packets may be temporarily stored in local storage (if applicable) and then archived offline. One advantage of this approach is that packet captures, potentially spanning long periods of time (e.g., days or weeks), may be efficiently constructed for offline analysis.

[0093] Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the present invention is not limited to the details provided. There are many alternative ways of implementing the present invention. The disclosed embodiments are illustrative and not restrictive.

Claims

1. 1. A system comprising: a processor; Select the target IoT device, determining and tagging flows associated with the target device; Accepting packets from said tagged flows into a ring buffer; receiving an indication that extraction is to be performed on a portion of the packets contained in the ring buffer; and extracting the portion of the packet; a processor configured to Memory, coupled to the processor; configured to provide instructions to the processor; Memory, Including, The target device is selected based, at least in part, on a configuration change associated with the target device; the configuration change includes a change in the use of an application by the target device; system.

2. The target device is selected based, at least in part, on detection of an elevated risk score. The system of claim 1 .

3. the risk score is increased based, at least in part, on a determination of the applicability of known exploits to the target device; The system of claim 2.

4. the risk score is increased based, at least in part, on observation of exploits attempted on the target device; The system of claim 2.

5. The application has not been previously used by the target device; and the modification includes the target device using the application; The system of claim 1 .

6. The target device is selected based, at least in part, on a URL with which the target device communicates. The system of claim 1 .

7. The indication is received as part of a time-based trigger. The system of claim 1 .

8. The indication is received in response to generating an alert. The system of claim 1 .

9. the extracted portion of the packet is provided to a network traffic analysis system. The system of claim 1 .

10. The processor further comprises: receiving an instruction to stop accepting packets associated with the target device into the ring buffer; The system of claim 1 .

11. the indication to stop accepting packets is received in response to a prescribed number of sessions associated with the target device being captured. The system of claim 10.

12. 1. A method comprising: selecting a target IoT device; determining and tagging flows associated with a target device; accepting packets from said tagged flows into a ring buffer; receiving an indication that extraction is to be performed on a portion of the packets contained in the ring buffer; extracting the portion of the packet; Including, The target device is selected based, at least in part, on a configuration change associated with the target device; the configuration change includes a change in the use of an application by the target device; method.

13. A computer program stored on a non-transitory computer readable medium and comprising computer instructions, The computer instructions, when executed, cause the computer to: selecting a target IoT device; determining and tagging flows associated with a target device; accepting packets from said tagged flows into a ring buffer; receiving an indication that extraction is to be performed on a portion of the packets contained in the ring buffer; extracting the portion of the packet; Implement the following: The target device is selected based, at least in part, on a configuration change associated with the target device; the configuration change includes a change in the use of an application by the target device; Computer program.

Images