Inferring firewall rules from network traffic

Abstract

Aspects of the disclosed technology include generating firewall rules based on traffic, outputting the generated firewall rules to an output file, and using the output file to set firewall rules in a network. The firewall rules can be generated without a priori knowledge of the network; instead, the firewall rules are not required. The generated rules can be adjusted for user preferences to adjust the number of generated firewall rules, and their over- or under-inclusivity for non-historical traffic data.

Description

[0001] Cross-reference to related applications

[0002] This application is a continuation-to-file of U.S. Patent Application No. 17 / 121,082, filed December 14, 2020. That application claims the benefit of U.S. Provisional Patent Application No. 63 / 117,810, filed November 24, 2020, the disclosure of which is incorporated herein by reference. Background Technology

[0003] Modern computing environments typically comprise various computing devices that communicate with each other over a network to share information. This communication usually takes place within a framework of rules or permissions that define security features such as firewalls and other characteristics such as packet size. Such computing environments can include corporate networks (e.g., between different offices within a company) and communications involving cloud computing resources. Modern cloud environments can contain a very large number of virtual machines (VMs). Cloud systems or cloud environments are maintained by cloud operators or owners. Typically, a portion of a cloud environment or VM belongs to different users or user groups. Each VM on a cloud environment can deploy various applications specific to its user or user group.

[0004] Users in cloud environments may expect to configure firewalls for individual machines or groups of machines within the cloud. A firewall is a security program, device, or module that monitors incoming and outgoing network traffic. Firewalls can be configured to allow or block certain data packets based on rule sets. Therefore, a firewall allows incoming or outgoing data packets that should be blocked or permitted to pass through. Thus, a firewall can block unwanted traffic, such as viruses, executable code, or responses to requests from fraudulent or unauthenticated sources. Configuring a firewall is a complex task for network administrators, requiring the manual monitoring, configuration, and initialization of hundreds or thousands of individual rules to manage the firewall. The interactions between multiple rules create unexpected complexities in effectively managing, diagnosing, and using firewalls.

[0005] In cloud environments, users may want to migrate network operations from their corporate network to the cloud, or between different cloud environments supported by different providers, or within a given cloud computing environment. In such examples, existing rules may not be properly translated to the new cloud environment. Furthermore, the sheer number and complexity of rules that may have been specifically created may not translate into the expected behavior of the firewall. Additionally, old or managed rules may have unexpected effects in the new environment and, given the often self-organizing nature of their creation, may pose security risks or lead to inefficient operations.

[0006] New users of cloud environments may want to create or establish firewall rule sets for new virtual machines. Summary of the Invention

[0007] The disclosed technical aspects include methods, systems, and apparatus for generating firewall rules based on historical business information within the network, such as outputting firewall rules in an output file and using the output file to set up, establish, or instantiate firewall rules within the network.

[0008] The disclosed technical aspects include: generating firewall rules by receiving data traffic flow information associated with computing devices connected via a network; generating a network graph with nodes and edges from the received data traffic flow information; grouping the nodes of the network graph to generate a new graph; generating a firewall rule set from the new graph; and outputting the firewall rule set as an output file, data, or sending those rules to the firewall.

[0009] The disclosed technical aspects include: generating firewall rules by receiving data traffic flow information associated with computing devices connected via a network; generating a network graph with nodes and edges from the received data traffic flow information; grouping the nodes of the network graph to generate a new graph; generating a firewall rule set from the new graph; and outputting the firewall rule set as an output file, data, or sending those rules to the firewall.

[0010] Packetization can occur through machine learning algorithms. It can automatically configure network devices associated with another network in the corresponding network graph to communicate or include firewall rules, based on a set of firewall rules output as an output file or other format. Packetization can occur without prior knowledge of business data. Nodes in the graph can be grouped, and each group is a node in the new graph. Packetization can be iterative, and can occur without prior knowledge of business data. Parameters associated with the packetization algorithm can be changed to minimize or otherwise alter the number of types of firewall rules generated.

[0011] It can validate firewall rule sets based on historical business data and raw firewall rule sets. It can also validate firewall rule sets based on synthesized business data and raw firewall rule sets. Furthermore, it can generate validation reports summarizing metrics related to the generated firewall rules.

[0012] Firewall rule sets can be implemented automatically or in response to user input after validation or generation. Firewall rule sets can be adjusted or periodically updated and implemented within the network environment. Firewall rule sets can be loaded into firewall components, and firewall components can be used to control network traffic through the firewall component or the network.

[0013] The machine learning method used for grouping can be one of the following: (i) community detection algorithm, (ii) algorithm based on centrality metric, (iii) visual machine learning method, (iv) clique-based method, (v) pagerank algorithm, (vi) generative adversarial network, (vii) machine learning classification algorithm. Attached Figure Description

[0014] The accompanying drawings are not intended to be drawn to scale. The same reference numerals and names in the various drawings indicate the same elements. For clarity, not every component may be labeled in every drawing.

[0015] In the attached diagram:

[0016] Figure 1 An example computing system according to various aspects of this disclosure is illustrated.

[0017] Figure 2 The diagram illustrates a network according to various aspects of this disclosure.

[0018] Figure 3 An example network diagram is illustrated according to various aspects of this disclosure.

[0019] Figure 4A The diagram illustrates an example network diagram after graph analysis based on various aspects of this disclosure.

[0020] Figure 4B The diagram illustrates an example network diagram after graph analysis based on various aspects of this disclosure.

[0021] Figure 5 This is an exemplary schematic system architecture based on aspects of this disclosure.

[0022] Figure 6 Example methods according to various aspects of this disclosure are illustrated.

[0023] Figure 7 These are example methods that can be used to generate graphs based on various aspects of this disclosure.

[0024] Figure 8 These are example methods that can be used for graph analysis based on aspects of this disclosure.

[0025] Figure 9 This is an example method that can be used to generate firewall rules based on aspects of this disclosure. Detailed Implementation

[0026] Overview

[0027] As used in this disclosure, points, vertices, and nodes are used interchangeably in a graph theory context. Business data is information relating to the entry or exit of information from a virtual machine or computing device to another virtual machine or computing device, which may be within or outside a specific cloud environment.

[0028] Historical business data, both inside and outside the cloud, can be represented as a mathematical graph. Each object, such as a virtual machine, router, device, or computer, can be represented as a node, and the connection between two objects can be represented as an edge between those nodes. In some examples, for a network of "n" objects, n*(n-1) directed edges representing the direction of data flow can exist in the graph, and n*(n-1) / 2 undirected edges can exist.

[0029] Graph analysis algorithms can be used on the network representation to group vertices within the network into groups or otherwise analyze the graph. In some examples, each group can be c. Each group in the first graph can be used to transform or create new nodes in the new graph. Edges within the previous graph can be combined into a single edge. This new graph can represent a new, simplified set of firewall rules. This new set of firewall rules will generally be more computationally efficient, easier for human users such as network administrators to understand, and more intuitively represented in a graphical format. The new set of firewall rules can be compiled into a file, which can then be used to configure the network resulting from the migration.

[0030] Graph analysis algorithms can be based on a variety of parameters. In some examples, the analysis or grouping can be more "aggressive," resulting in fewer groups and fewer edges. In other examples, less aggressive grouping can lead to more groups, a larger number of edges, and more generated firewall rules. Since the set of groups and the generated firewall rules are not "unique," the generated firewall rules can be optimized based on user input, preferences, requirements, or criteria. Because nodes in the graph can be grouped in different ways, the corresponding set of nodes and the resulting firewall rules can be different for different groups. Different groupings lead to different technical trade-offs in the resulting rule set, such as between the overall size and complexity of the resulting rule set and the granularity that allows it to distinguish traffic from different sources and destinations.

[0031] It can validate and adjust the rules and parameters generated related to grouping or graph analysis algorithms based on historical and synthetic business data. Synthetic business data can test connections not part of the historical business data. Furthermore, it can allow or deny behavior based on the generated rules for unknown or non-historical business patterns. In some examples, it can test the over-inclusivity or under-inclusivity of the generated rules based on synthetic data and adjust them accordingly. In other examples, it can iteratively execute the above processes, such as iteratively creating groups using grouping methods, or additionally or in combination with supervised or unsupervised machine learning, to generate multiple firewall rule sets. For example, in each iteration, it can perform grouping to generate a new graph, and then iteratively group that new graph again.

[0032] In some examples, synthetic traffic data can be used to test generated firewall rules against business flows that were not included in the historical information used to generate firewall rules, and to validate firewall rules against unseen business data or unknown historical business data.

[0033] The aspects of the disclosed technology allow the generated firewall rule sets to be (1) correct because they match historical traffic logs, (2) secure because they are more general than ungenerated firewall rule sets (more general because they are more applicable to unknown types of traffic that are not or have not yet been part of the historical logs, since all unknown types of data can be rejected), or (3) simple because the number of generated rules can be less and more concise than a regular firewall rule set, and more understandable to users or network administrators. Firewall rules generated using the technology disclosed herein generally increase the security of firewall rules created by administrators or other conventional technologies because the generated traffic rules can generally reject traffic not encountered in the historical traffic logs. Furthermore, the aspects of the disclosed technology allow for periodic updates and automatic adjustment of firewall rules as traffic patterns to the network or traffic patterns within the network change over time. Therefore, the generated firewall rules can be periodically updated or automatically implemented within the network environment.

[0034] The disclosed technologies can be combined with various “Internet of Things” devices, such as routers, smart thermostats, or other devices that communicate with each other and contain private or sensitive information, and can be used in conjunction with or dedicated to communication between various “Internet of Things” devices in virtual machines.

[0035] The disclosed aspects of the technology allow the use of verification output or obfuscation matrices to verify firewall rules, which can be used to "adjust" or influence the generated rules.

[0036] The disclosed techniques allow for the use of packet processing and firewall rule generation without knowledge of the type of network data used to generate firewall rules. Therefore, the disclosed techniques allow for the use of network data without prior knowledge of the historical network data being used, and the method is independent of the type of network data being used.

[0037] Example System

[0038] The following figures illustrate various aspects of the disclosed technology. Those skilled in the art will understand that the various disclosed components can be electrically coupled through one or more of the disclosed electronics, processors, and computing devices to perform the methods and techniques disclosed herein. For simplicity and clarity, not every electronic device or data link is illustrated.

[0039] In some examples, existing cloud users can use historical business logs to create optimized or new sets of firewall rules. In other examples, users migrating to a new cloud environment can import or log business data to generate new firewall rule sets in the new environment.

[0040] Figure 1 This is a block diagram 100 illustrating an exemplary computer system 110, which enables the implementation of various aspects of this disclosure, including the machine learning systems and techniques described herein and any components thereof. In some aspects, the computer system 110 may be implemented using hardware or a combination of software and hardware, or implemented in a dedicated server, or integrated into another entity, or distributed across multiple entities. In some examples, the exemplary computing system 110 may be a user computing system or device. In other examples, such as... Figure 2 As shown, cloud 201 can consist of one or more example computer systems similar to computing system 110, which are coupled or linked via software and hardware components to operate together as a cloud.

[0041] In summary, computing system 110 includes at least one processor 150 for performing actions according to instructions and one or more memory devices 170 or 175 for storing instructions and data. The illustrated exemplary computing system 110 includes one or more processors 150, memory 170, and any other devices 180 (e.g., I / O interfaces) that communicate via bus 115 with at least one network interface driver controller 120 having one or more network interface cards 122 connected to one or more network devices 124. The network interface card 122 may have one or more network interface driver ports for communicating with connected devices or components. Typically, processor 150 executes instructions received from memory. The illustrated processor 150 is either incorporated into or directly connected to cache memory 175.

[0042] More specifically, processor 150 can be any logic circuit that processes instructions, such as instructions fetched from memory 170 or cache 175. In many embodiments, processor 150 is a microprocessor unit or a dedicated processor. Computing device 110 can be based on any processor or collection of processors capable of operating as described herein. Processor 150 can be a single-core or multi-core processor. Processor 150 can be multiple processors. In some embodiments, processor 150 can be configured to run multi-threaded operations. In some embodiments, processor 150 can host one or more virtual machines or containers, and a hypervisor or container manager for managing the operations of the virtual machines or containers. In such embodiments, the methods shown and described in Figures 4-6 can be implemented within a virtualized or containerized environment provided on processor 150.

[0043] Memory 170 can be any device suitable for storing computer-readable data. Memory 170 can be a device with fixed storage or a device for reading removable storage media. Examples include all forms of non-volatile memory, media and memory devices, semiconductor memory devices such as EPROM, EEPROM, SDRAM, and flash memory devices, magnetic disks, magneto-optical disks, and optical disks such as CD ROM, DVD-ROM, and... Disk. The computing system 110 may have any number of storage devices 170. In some embodiments, the storage 170 supports virtualized or containerized storage that can be accessed by a virtual machine or container execution environment provided by the computing system 110.

[0044] Cache memory 175 is typically a form of computer memory placed adjacent to processor 150 for fast read times. In some embodiments, cache memory 175 is part of processor 150 or on the same chip as processor 150. In some embodiments, there are multiple levels of cache 175, such as L2 and L3 cache layers.

[0045] Network interface driver controller 120 manages data exchange via network interface driver 122 (also referred to as a network interface driver port). Network interface driver controller 120 handles the physical and data link layers of the OSI model for network communication. In some embodiments, some of the tasks of the network interface driver controller are handled by processor 150. In some embodiments, network interface driver controller 120 is part of processor 150. In some embodiments, computing system 110 has multiple network interface driver controllers 120. The network interface driver port configured in network interface card 122 is a connection point for a physical network link. In some embodiments, network interface controller 120 supports wireless network connectivity, and the interface port associated with network interface card 122 is a wireless receiver / transmitter. Typically, computing device 110 exchanges data with other network devices 124 via a physical or wireless link interfaced with the network interface driver port configured in network interface card 122. In some embodiments, network interface controller 120 implements network protocols such as Ethernet.

[0046] Other network devices 124 are connected to computing device 110 via network interface driver ports included in network interface card 122. Other network devices 124 can be peer-to-peer computing devices, network devices, or any other computing device with networking capabilities. For example, the first network device 124 could be a device that connects computing device 110 to, for example, a peer-to-peer computing device, a network device, or any other computing device with networking capabilities. Figure 1 Network devices for the Internet or cloud data network shown in Figure 201, such as hubs, bridges, switches, or routers.

[0047] Other devices 180 may include I / O interfaces, external serial device ports, and any additional coprocessors. For example, computing system 110 may include interfaces (e.g., a Universal Serial Bus (USB) interface) for connecting input devices (e.g., a keyboard, microphone, mouse, or other pointing devices), output devices (e.g., a video display, speaker, or printer), or additional memory devices (e.g., a portable flash drive or external media drive). In some embodiments, computing device 100 includes additional devices 180 such as coprocessors; for example, a math coprocessor can assist processor 150 in performing high-precision or complex calculations.

[0048] Instructions on computing system 110 can control various components and functions of computing system 110. For example, instructions can be executed to perform any method indicated in this disclosure. In some examples, algorithms can be included as a subset of the instructions on computing system 110 or otherwise as part of the instructions. Instructions can include algorithms for performing any method or subset of methods described within this disclosure.

[0049] The user interface on the computing system 110 may include a screen, such as a touchscreen or buttons, that allows users to interact with the computing system 110. It may also include a display, such as an LCD, LED, mobile phone display, e-ink, or other display, to display information about the computing system 110. The user interface is capable of allowing input from and output to the user. One or more communication interfaces may include hardware and software to enable data communication via standards such as Wi-Fi, Bluetooth, infrared, radio waves, and / or other analog and digital communication standards. One or more communication interfaces allow updates to the computing system 110 and allow information generated by the computing system 110 to be shared with other devices. In some examples, one or more communication interfaces are capable of sending information stored in memory to another user device for display, storage, or further analysis.

[0050] Figure 2 The diagram illustrates an example of a network 200 having a cloud 201, virtual machines 211-215, and devices 231-235 associated with users 221-225 respectively. The cloud 201 can include hardware, such as networking devices like switches, routers, firewalls, load balancers, storage arrays, backup devices, and servers. The cloud 201 can be considered an abstraction that connects servers together, partitioning and abstracting resources so that they can be accessed by users via virtual machines (VMs). The cloud 201 can include a firewall module 250, a hypervisor 240, and virtual machines 211-215.

[0051] Although Cloud 201 is represented as a single entity, those skilled in the art will understand that Cloud 201 is a conceptualization of a distributed hardware and software system. Cloud 201 can be composed of other clouds. In other examples, Cloud 201 can be a virtual machine or a virtual cloud itself located within another cloud. In some examples, Cloud 201 can be distributed or partitioned across multiple physical locations such as data centers that can be interconnected or linked together. In other examples, portions of Cloud 201 can be off-site hosted. For example, in some examples, the computer processing or computing hardware of Cloud 201 can be located in one location, while the storage media can be located in other regions.

[0052] Cloud 201 can also be configured to allow control over various aspects of the cloud environment. For example, Cloud 201 can include software that responds to user needs or requests, such as increasing or decreasing the size of virtual machines, allocating resources specifically for virtual machines, or the number of virtual machines available to a given user.

[0053] Cloud 201 can contain multiple virtual machines 211-215. Typically, a virtual machine is an emulation of a computer system or computer network. Virtual machines are based on a computer architecture and can provide the functionality of a physical computer. Implementation methods can involve dedicated hardware, software, or a combination thereof. Each virtual machine 211-219 can be hosted or run on the cloud. In some examples, virtual machines can be instantiated in response to user requests. In some examples, each virtual machine can be a group of virtual machines or a network of virtual machines.

[0054] Cloud 201 may also contain hypervisor 240 or more hypervisors. A hypervisor is also known as a virtual machine monitor, VMM, or virtualizer. A hypervisor is a set of computer software, firmware, or hardware that can create, run, or monitor virtual machines. In some examples, hypervisor 240 may only have access to certain types of information about the virtual machines in cloud 201.

[0055] Each virtual machine can be managed by users 221-225. Each user can access their corresponding virtual machine through tools provided by the cloud provider, such as through user devices 231-235. In some examples, this occurs using a web interface typically provided by the cloud provider, such as web interface 241. In other examples, dedicated software installed on the user device can be used to interact with the cloud or a specific virtual machine. User devices 231-235 can be similar to those referenced above. Figure 1 The computing system 110 is described.

[0056] User device 236 may be a device that does not control or subscribe to virtual machines on cloud 201 but is able to access information or resources on cloud 201. In some examples, user device 236 may be able to request or attempt to access resources hosted on cloud 201. For example, user device 236 may attempt to make a specific request using a web interface, which may then be routed to a specific virtual machine on cloud 201.

[0057] Each virtual machine or group of virtual machines can run one or more applications, software, operating systems, and store data. Additionally, requests from users to the cloud, or to one or more virtual machines, can generate network data or services.

[0058] Figure 3Figure 300 illustrates a graphical representation of the permissible connections between virtual machines. A graph is generally considered a mathematical structure used to illustrate or conceptualize pairwise relationships between objects. A graph can consist of points (also called nodes). An edge is a connection between two nodes, indicating a relationship between them. A graph typically has a single edge between any two nodes, while a multigraph can have more than one edge between any two nodes. Those skilled in the art will understand that a graph can refer to various objects in graph theory, including, for example, multigraphs. Those skilled in the art will also understand that equivalent mathematical structures can be used in place of or in combination with Figure 300 as described herein.

[0059] In Figure 300, each virtual machine can be represented as a node. A permissible connection between two virtual machines can be represented as an edge or connection between two nodes or vertices. In some examples, Figure 300 can be a directed graph, where each edge has an associated direction. In other examples, the edges can be bidirectional. In other examples, Figure 300 can be a multigraph. In other examples, Figure 300 can be a mixed graph, where some edges are directed and others are undirected.

[0060] Additional information, such as the port or protocol of the virtual machine used, can be encoded or represented within Figure 300. For example, the 5-tuple {VM1, port1, VM2, port2, protocol} results in an edge between virtual machines VM1 and VM2, where the port and protocol serve as edge metadata. This metadata can be used for grouping and for rule generation, as further described herein.

[0061] Figure 4A A graphical representation of permissible connections between virtual machines is illustrated, where virtual machines have been categorized, grouped, or clustered together as shown in Figure 400. Example methods for generating or categorizing virtual machines by group are further described in this disclosure.

[0062] Node groups are delimited by dashed lines. In some examples, these groups can be formed using classification or grouping methods, such as those described in this disclosure. In some examples, spatial grouping algorithms can be used to generate or classify vertices within Graph 400 into groups. For example, graphs can be abstracted or represented in two-dimensional, three-dimensional, or other geometric or measurement spaces, whether Euclidean or non-Euclidean. The location and extent of spatial objects can define implicit relationships in spatial neighborhoods, such as topological, distance, and directional relationships, which can be used by spatial grouping algorithms or spatial data mining algorithms. Additional example algorithms and methods are described herein.

[0063] Figure 4B The figure illustrates a graphical representation of permissible connections, where the edges between two sets of nodes have been replaced by a single edge, Figure 450. Figure 4BEach edge in the equation is associated with a generated firewall rule, as explained further below.

[0064] Figure 5 An exemplary architecture 500 and an exemplary processing flow related to the disclosed technology are illustrated. Figure 5 The diagram illustrates the processing, generation, and firewall engine 505, machine learning engine 510, business logs 515, data processing module 520, network graph 525, rule generation module 530, firewall rules 535, business records 540, configuration verification engine 545, and verification report 550. Those skilled in the art should understand that the names and descriptions used with respect to the reference architecture 500 are merely examples, and other architectures, names, and combinations of modules and engines are possible.

[0065] Business log 515 can be stored on computer system 110, within network 200, or in the cloud 201. As an example, business log 515 can include multiple logs, which can include information related to business direction, source machine or virtual machine, target virtual machine or machine, or source port, Internet or communication protocol used, destination port and request time, time request for approval or rejection, requested data type, and data volume.

[0066] Data processing module 520 can include software that reads, parses, interprets, and extracts information from business logs 515. In some examples, data processing module 520 can also combine information extracted from certain business logs based on a rules database. As an example, data processing module 520 can combine logs sequentially or within a certain amount of time, such as within a fixed number of seconds or minutes, into the same record of processed data. In other examples, business records can be sampled to reduce log volume, simplify analysis, or remove duplicate business records. In other examples, the data processing module can update the results of previously processed data with new business records, or update processed data based on records obtained periodically, such as weekly, monthly, or daily. In other examples, the data processing module can generate comparative information related to the business logs, such as the amount of data within a certain time frame, the amount of data within a specific time frame, or the amount of data from one node to another. In some examples, sampling of business log data can be based on this comparative information.

[0067] The data processing module 520 can also generate business records 540. Business records 540 can be associated with business logs 515 and can be generated by removing duplicates, filtering data, or otherwise processing the data in business logs 515.

[0068] A network graph 525 can be generated from data obtained through processing by data processing module 520. In some examples, one or more graph generation modules can be used to generate the network graph 525. In other examples, the network graph 525 can be schematically represented in volatile or non-volatile memory as an equivalent mathematical representation of the graph, such as binary code in graph theory data structures. Other non-limiting examples of the network graph 525 include graphs of differences between graphs, generated as time series of time intervals, such as weeks or months.

[0069] The rule generation module 530 can process or interpret network graph 525 or related data to generate a set of firewall rules, such as firewall rule 535, that represents or is equivalent to network graph 525. In some examples, the rule generation module 530 can work in conjunction with machine learning (ML) engine 510.

[0070] ML Engine 510 can include machine learning-related models and software modules. ML Engine 510 can include grouping or graph analysis algorithms, such as role extraction, community detection, centrality measurement, and connected components. As an example, ML Engine 510 can be based on supervised or unsupervised training methods using a training dataset. The training dataset can contain a set of training nodes. This training node set can also include a graph constructed from the training node set, including edges, such as reference edges. Figure 3 The edges are shown. The training dataset can also include the output of the training, such as an output graph that can represent a firewall rule set or is associated with a firewall rule set. For example, the output graph can resemble a reference graph. Figure 4B Figure 450 describes this. In some examples, the training data can contain a set of nodes and the edges between those nodes, representing endpoints within the network and the business data between those endpoints.

[0071] Firewall rule 535 can represent a set of rules generated from rule generation module 520 or ML engine 510. Firewall rule 535 can be optimized for specific criteria, such as the number of rules, complexity, priority for certain machines or ports, processing time or complexity, or latency within the network.

[0072] The configuration verification engine 545 processes business logic and compares the generated business logs with historical business logs to obtain information related to the generated firewall rules. In some examples, the configuration verification engine 545 can work in parallel or in conjunction with the processing, generation, and firewall engines 505. In some examples, the configuration verification engine 545 can use two firewall rule sets, such as the original firewall associated with the cloud environment and the generated firewall rule set, such as firewall rule 535, and "run" business logs using both rule sets. A first response can be generated using the original firewall rule set, and a second response can be generated using the second firewall rule set. In some examples, the first and second responses can contain information about which connections to and from virtual machines within the cloud environment are permitted or denied. The first and second responses can be compared as part of the verification result.

[0073] The verification report 550 may include summary information or other information related to the generated firewall rules for presentation to end users or for storage on computing devices. An example verification report may take the following forms:

[0074]

[0075] Example Method

[0076] Figure 6 The illustration illustrates example method 600. Method 600 can be used to generate firewall rule sets that can be stored and implemented within a cloud environment. In some examples, method 600 can be executed automatically at predetermined intervals. In other examples, method 600 can be executed in response to user input or user commands. In still other examples, method 600 can be executed upon predetermined events, such as the addition of virtual machines, an increase in requests to the cloud environment, or exceeding thresholds, such as an increase or decrease in the number of requests or the percentage of requests to a particular virtual machine.

[0077] In some examples, certain clouds or virtual machines can have persistent rules associated with them. For instance, it's possible to know that certain virtual machines or cloud environments contain publicly accessible information, and to set firewall rules associated with such virtual machines or cloud environments to allow incoming requests. In other examples, certain virtual machines can only allow connections between other pre-identified machines.

[0078] In some examples, it is possible to display the generated firewall rule set to users or network administrators for use, implementation, modification, or otherwise interaction. In other examples, network administrators can manually modify subsets of firewall rules or indicate additional network-related intents that can "enforce" specific types of classification or grouping for specific vertices.

[0079] In some examples, it is possible to "tune" or optimize the number of groups or the grouping algorithm for certain metrics, such as the number of firewall connections, the number of rules, or another user-defined or testable metric that the user can define or test. Depending on the grouping algorithm used, the grouping threshold allows users to customize it to meet their requirements. For example, coarse-grained grouping logic results in a smaller number of groups, and therefore a smaller number of firewall rules, providing coarse-grained security. On the other hand, fine-grained grouping results in a larger number of groups and firewall rules that closely mimic the underlying business model, resulting in a more stringent security profile.

[0080] At box 610, business logs, such as business log 515, can be obtained. These business logs can be historical logs and can be associated with one or more virtual machines on a cloud environment. The business logs can contain information as described herein. In some examples, firewall logs can be obtained instead of business logs, and these firewall logs can contain additional metadata. Business logs are received via data processing module 520 and can be pushed or pulled depending on the network design.

[0081] At box 620, a graph can be generated based on the obtained business logs. The generated graph is equivalent to other mathematical representations. Additional boxes can be executed at box 620, such as references. Figure 7 The boxes that are described.

[0082] At box 630, the network graph can be grouped. At this box, various nodes or vertices of the network graph can be grouped together. Additional boxes, such as reference boxes, can be executed at box 630. Figure 8 The boxes that are described.

[0083] At box 640, rules can be generated from the network graph. Additional boxes, such as reference boxes, can be executed at box 640. Figure 9 The boxes that are described.

[0084] At box 650, firewall rules can be optimized. In some examples, firewall optimization can occur based on two virtual machines that share a common identifier, such as being identified as being in the same group. In some examples, firewall rules can be saved as a file to be implemented within the output file.

[0085] At box 660, business verification can occur. At this box, two additional firewall rule sets can be analyzed, such as the original firewall rule set associated with the cloud environment and the generated firewall rule set, or an optimized firewall rule set, such as firewall rule 535. Both rule sets can be used for synthetic business or historical business logs. A first response and a second response can be generated at this step. The first and second responses can correspond to firewall responses generated using the generated firewall rules and synthetic firewall rules. In some examples, a "confusion matrix," such as the one illustrated in Figure 10, can be generated or used as part of business verification. A confusion matrix, also known as an error matrix, is a specific table layout that allows visualization of the performance of machine learning or grouping algorithms.

[0086] In box 670, firewall recommendations can be generated. This box also allows you to display firewall recommendations to firewall users or administrators.

[0087] At box 680, the generated firewall rules can be implemented. In some examples, the generated firewall rules can be implemented automatically. In other examples, the generated firewall rules can be implemented after receiving a positive response or request from the user or administrator. In still other examples, the generated firewall rules can be compared with the core firewall rule set, and if the generated rules are at least as strict as the core firewall rule set, the generated firewall rules are implemented automatically.

[0088] Figure 7 The diagram illustrates a method 700 that can be used to generate graphs.

[0089] In box 710, network data or network logs can be obtained. Network data can contain information indicating the number of virtual machines, their identities, how long they have been instantiated, the IP addresses of the virtual machines, or other identifiers. Network data can also include which connections or transmissions are allowed between two virtual machines, and whether those connections or transmissions are within the cloud environment or between two cloud environments. In some examples, network data may have been pre-processed to remove duplicate connections or transmissions between two sets of virtual machines. In other examples, traffic can be normalized.

[0090] At box 720, each virtual machine can be identified as a node in the graph. In some examples, each virtual machine will form a node in the graph. In other examples, multiple nodes or vertices can correspond to virtual machines, where each of the multiple nodes corresponds to a specific metadata set, such as a virtual machine's port or a protocol used for communication.

[0091] At box 730, each node can be labeled or identified.

[0092] At box 740, each incoming or outgoing connection from a node, reflected in the obtained network logs or network data, can be represented as an edge between two vertices or nodes of a graph. In some examples, the edge can be directed. In other examples, the edge can be bidirectional. In still other examples, each edge can be undirected.

[0093] Figure 8 The illustration shows a method 800 that can be used, for example, for grouping. (See reference...) Figure 8 Exemplary methods for grouping have been discussed, but those skilled in the art will understand that other methods and techniques can be used to group or associate vertices of a graph, including but not limited to machine learning techniques, generative adversarial networks, Gaussian techniques, density techniques, binary techniques, Bayesian classifiers, binary forests, random forests, neural networks, neural networks, nearest neighbor techniques, and boosting trees.

[0094] At box 810, a graph representing historical information can be received. The received graph can be similar to Figure 300 shown above.

[0095] At box 820, graph analysis algorithms or grouping techniques can be used to analyze the graph, which can include the use of machine learning or deep learning techniques. Those skilled in the art will recognize that the described methods are exemplary and can be used in various combinations or otherwise adapted or modified. At this box, the method can result in the elements of the graph being grouped into multiple groups.

[0096] In some examples, community detection algorithms can be used. In some examples, minimum cut methods can be used. In other examples, the number of "minimum cuts" can vary based on the desired characteristics of the final graph, such as the desired number of firewall rules. In some examples, hierarchical methods can be used. Other community methods that can be used include, but are not limited to, the Girvan-Newman algorithm, modular maximization methods, statistical inference methods, or small-clique-based methods. Modular maximization methods can be selected or chosen based on the computational complexity of log data, the sensitivity of information in the underlying nodes, or other methods. Statistical inference methods can include generative models. Small-clique-based methods can be used with small-cliques of fixed or varying sizes.

[0097] In other examples, algorithms based on centrality metrics can be used. In some examples, the centrality measure between nodes can be used as a feature or input to a machine learning model. Other metrics can also be used. As an example, "intermediate centrality" can be used. In other examples, "degree centrality" can be used. Based on algorithms using centrality metrics, nodes can be grouped into sets that form the basis of a new graph. In some examples, connected component methods, including connected component labeling, region labeling, or region extraction, can be used, where node groups can be uniquely labeled based on a provided heuristic.

[0098] In other examples, when grouping nodes in the generation of firewall rules, other machine learning-based classification methods can be used to group nodes based on a determination of over-inclusivity and under-inclusivity between edges and business data. Machine learning classification methods can be configured or trained on a training set, ensuring that the provided groupings or classifications are neither over-inclusive nor under-inclusive, and satisfy the conditions specified for the output groupings and / or associated firewall rules.

[0099] In some examples, one or more of the following techniques can be used. In some examples, probabilistic methods can be used. For example, Gaussian mixture models can be used. Gaussian mixture models are probabilistic models used to represent normally distributed subpopulations within a population. In Gaussian mixture models, it is not required that the observed dataset should characterize or specify which subpopulation a particular observation within the distribution belongs to.

[0100] Examples of machine learning techniques that can be used include the following. In some examples, a mixture of supervised and unsupervised learning techniques can be used. In some examples, generative adversarial networks (GANs) can be used to predict or detect network anomalies. GANs use two networks, an adversarial network and a generative network, to attempt to deceive the adversarial network by means of objects generated by the generative network. In some examples, graph analysis methods can be used to group inputs, network parameters, trained models, or virtual machines. Graph analysis or grouping methods can be used in real time to classify and match models or groups of models with virtual machines or groups of virtual machines. Some other examples of techniques include dimensionality reduction. Dimensionality reduction can be used to remove information that has the least impact or is statistically least significant. In networks that generate large amounts of data and can observe many types of data, dimensionality reduction can be used in combination with any of the techniques described in this paper. An example dimensionality reduction method is Principal Component Analysis (PCA). PCA can be used to reduce the dimension or number of variables in the “space” by finding new vectors that maximize the linear change of the data. PCA also allows for observation of the amount of lost information and allows for adjustment in the new vectors to be selected. Another example technique is t-random neighborhood embedding (t-SNE), which can be used to group nodes within a graph based on relative spatial relationships and one or more spatial mappings from endpoints to the graph. Ensemble methods can be used, primarily employing the idea of ​​combining several predictive models, which can be supervised or unsupervised ML, to obtain higher quality predictions than each model alone can provide. As an example, random forest algorithms, neural networks, and deep learning techniques can also be used with the aforementioned techniques. Neural networks typically attempt to replicate the behavior of the biological brain by “opening” or “closing” connections between inputs and outputs in an effort to maximize a chosen objective.

[0101] In box 830, when assigning each group, the graph can be partitioned into groups based on the resulting groups. Mathematically, graph partitioning is the process of simplifying or transforming a graph into a new or smaller graph by dividing the nodes of the original graph into mutually exclusive groups. Edges of the original graph that intersect between groups will generate edges in the partitioned graph.

[0102] Figure 9The illustration shows aspects of a method 900 that can be used for rule generation. As illustrated in method 900, a set of edges from a graph, such as set 910, can be combined with a set of grouped edges or graph analysis results, such as set 920, to transform or map that set of edges into a set of groups with edges, such as through grouped edges and analysis in box 930. For example, duplicate edges can be removed during 930 to generate an information set 940. 940 can include information about the newly generated graph. Information from set 940 can be used to generate firewall rules, such as rule 950 and rule 960. Rule 950 can correspond to "Group 1 → Group 3" in set 940, while rule 960 can correspond to "Group 1 → Group 2" in set 940. From Figure 9 As can be seen, the results of grouping or graph analysis subsequently alter the transformation from set 910 and set 920 to set 940, and thus affect the number and information contained within the generated firewall rules. When no "allow" rule is generated for this set, a deny firewall rule can be generated by default between two groups or elements in the graph. In some examples, a "deny" rule can therefore supplement the "allow" rule in the graph. In other examples, "deny" rules can be generated using other machine learning algorithms.

[0103] Although the reference virtual machine (VM) diagram shows Figure 9 However, it should be understood that other network labels can be used instead of VMs, such as external IP addresses. Similarly, based on IP address characteristics, groups or units can exist for sets of external IP addresses. For example, a portion of an IP address can refer to the "network portion" of the IP address, while another portion can refer to the "host" portion. As an example of group mapping, all addresses with the same "network" portion can be considered to be in one group. In other examples, addresses with the same "network" and "host" portions can be used together, but grouped based on metadata.

[0104] In some examples, the generated firewall rules can be tested against historical business logs by replaying the data and against synthetic business data. It is expected that the generated firewall rules will "match" the historical business logs, as they will reject all previously rejected services and allow all previously rejected services. Similarly, it is expected that for synthetic data, the generated firewall rules will reject all data that the original firewall rules had already rejected. This information can be visualized in the table below.

[0105]

[0106]

[0107] In the table above, each piece of synthetic data will fall into one of the four quadrants. In some examples, it is possible to know or configure what results are expected for a specific piece of synthetic data logs replayed against both the original firewall rule and the generated firewall rule.

[0108] The table above allows for the categorization of generated firewall rules based on their desirability. "Ideal" indicates the expected or required outcome, while "unacceptable" indicates the result of specific synthetic data or the behavior of the firewall rule, suggesting that the generated firewall rule lacks important or critical features or exhibits unexpected behavior.

[0109] Similarly, in the table above, several rules can fall into the "not ideal but acceptable" category, where the resulting firewall behavior is still acceptable but not ideal because stricter security would cause that particular synthetic data to be rejected. "Ideal" indicates behavior that represents stricter security for the resulting firewall rule. In some examples, it is possible to divide the number of data points into the quadrants above and to "tune" the firewall rules and associated algorithms to result in more or fewer "not ideal but acceptable" outcomes.

[0110] While the above classification is exemplary, those skilled in the art will understand that additional variations and test configurations of the generated firewall rules are possible, including larger tables or more granular tests, and that additional parameters can be “adjusted” in the generated firewall rules.

[0111] While this disclosure contains numerous specific implementation details, these details should not be construed as limiting the scope of possible claims, but rather as descriptions of features specific to particular embodiments. Certain features described in this specification within the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, one or more features from a claimed combination may be removed from the combination in some cases, and the claimed combination may involve sub-combinations or variations of sub-combinations.

[0112] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring that they be performed in the specific order shown or sequentially, or that all the illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous.

[0113] A reference to "or" can be interpreted as inclusive, meaning that any term described using "or" can refer to any one, more than one, or all of the terms described. Labels such as "first," "second," "third," etc., do not necessarily indicate order and are generally used only to distinguish identical or similar items or elements.

[0114] Various modifications to the embodiments described in this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit or scope of this disclosure. Therefore, the claims are not intended to be limited to the embodiments shown herein, but are accorded the widest scope consistent with this disclosure, the principles disclosed herein, and the novel features.

[0115] The non-limiting aspects of the disclosed technology can include the following features:

[0116] Feature 1. A method for generating firewall rules, the method comprising:

[0117] Receive data service flow information associated with computing devices connected via a network;

[0118] Generate a network graph with nodes and edges from the received data service flow information;

[0119] The nodes of the network graph are grouped to generate a new graph;

[0120] Generate a firewall rule set from the new graph; and

[0121] Output the firewall rule set as an output file.

[0122] Feature 2. Based on the method of Feature 1, where grouping occurs via a machine learning algorithm.

[0123] Feature 3. The method according to features 1-2 includes automatically configuring network devices associated with another network for communication based on the firewall rule set, wherein the other network is implemented as corresponding to the network graph.

[0124] Feature 4. According to the method of features 1-3, the grouping occurs without prior knowledge of the business data.

[0125] Feature 5. According to the method of features 1-3, wherein the nodes are grouped, and each group is a node of the new graph.

[0126] Feature 6. According to the method of Feature 5, grouping occurs based on iterative grouping.

[0127] Feature 7. The method according to features 1-3 includes changing parameters associated with the grouping algorithm to minimize the firewall rule set.

[0128] Feature 8. The method according to Features 1-3 includes verifying the firewall rule set based on historical business data and the original firewall rule set.

[0129] Feature 9. The method according to Features 1-3 includes verifying the firewall rule set based on synthetic business data and the original firewall rule set.

[0130] Feature 10. Based on the methods of Features 7-8, this includes generating a verification report that summarizes the metrics associated with the generated firewall rules.

[0131] Feature 11. According to the method of Features 1-8, the firewall rule set is automatically implemented after verification.

[0132] Feature 12. The method according to features 1-8 includes loading the firewall rules into a firewall component and using the firewall component to control network services passing through the firewall component or the network.

[0133] Feature 13. According to the method of features 1-12, wherein the machine learning method used for grouping is one of the following: (i) community detection algorithm, (ii) algorithm based on centrality metric, (iii) visual machine learning method, (iv) clique-based method, (v) pagerank algorithm, (vi) generative adversarial network, (vii) machine learning classification algorithm.

[0134] Feature 14. A system comprising:

[0135] One or more processing devices, said one or more processing devices being coupled to a memory containing instructions, said instructions causing said one or more processing devices to:

[0136] Receive data service flow information associated with computing devices connected via a network;

[0137] Generate a network graph with nodes and edges from the received data service flow information;

[0138] The nodes of the network graph are grouped to generate a new graph;

[0139] Generate a firewall rule set from the new graph; and

[0140] Output the firewall rule set as an output file.

[0141] Feature 15. The system based on Feature 14, wherein grouping occurs via a machine learning algorithm.

[0142] Feature 16. The system according to features 14-15 includes instructions that cause the one or more processing devices to automatically configure network devices associated with another network for communication based on the firewall rule set, the other network being implemented corresponding to the network graph.

[0143] Feature 17. The system according to features 14-16, wherein grouping occurs without prior knowledge of the business data.

[0144] Feature 18. The system according to features 14-16, wherein the nodes are grouped, and each group is a node of a new graph.

[0145] Feature 19. The system according to features 14-18 includes instructions that cause the one or more processing devices to change parameters associated with a packet algorithm to minimize the firewall rule set.

[0146] Feature 20. A system according to features 14-19, including instructions that cause the one or more processing devices to verify the firewall rule set based on historical business data and the original firewall rule set.

[0147] Feature 21. The system according to features 14-19 includes instructions that cause the one or more processing devices to verify the firewall rule set based on synthetic business data and the original firewall rule set.

[0148] Feature 22. The system according to features 14-19 includes instructions that cause the one or more processing devices to generate a verification report summarizing metrics related to the generated firewall rules.

[0149] Feature 23. The system according to features 14-19 includes instructions that cause the one or more processing devices to automatically implement the firewall rule set after verification.

[0150] Feature 24. The system according to features 14-19 includes instructions that cause the one or more processing devices to load the firewall rules into a firewall component and use the firewall component to control network services passing through the firewall component or the network.

Claims

1. A method of generating firewall rules, the method comprising: receiving data traffic flow information associated with a set of virtual machines connected through a network; generating a network graph with nodes and edges from the received data traffic flow information, wherein the nodes of the network graph represent virtual machines; grouping the nodes of the network graph to generate a new graph; generating a set of firewall rules from the new graph based on common identifiers between two virtual machines from the set of virtual machines; verifying the set of firewall rules by comparing first traffic records generated using the set of firewall rules with second traffic records generated using an original set of firewall rules; outputting the set of firewall rules as an output file; and loading the set of firewall rules into a firewall component and using the firewall component to control network traffic through the firewall component or the network.

2. The method of claim 1, wherein, The grouping occurs through a machine learning algorithm.

3. The method of claim 1, comprising automatically provisioning network devices associated with another network corresponding to the network graph to communicate based on the set of firewall rules.

4. The method of claim 1, wherein, The grouping occurs without a priori knowledge of traffic data.

5. The method of claim 1, wherein, The nodes are grouped into groups and each group is a node of the new graph.

6. The method of claim 5, wherein, The grouping occurs based on iterative grouping.

7. The method of claim 5, comprising varying parameters associated with the grouping algorithm to minimize the set of firewall rules.

8. The method of claim 1, wherein, The verifying further comprises verifying the set of firewall rules based on historical traffic data and the original set of firewall rules.

9. The method of claim 1, wherein, The verifying further comprises verifying the set of firewall rules based on synthetic traffic data and the original set of firewall rules.

10. The method of claim 7, comprising generating a verification report summarizing metrics related to the generated firewall rules.

11. The method of claim 1, comprising automatically implementing the set of firewall rules after verification.

12. The method of claim 2, wherein, The machine learning algorithm for grouping is one of: (i) a community detection algorithm, (ii) an algorithm based on centrality measures, (iii) a visual machine learning method, (iv) a method based on small groups, (v) a web page rank algorithm, (vi) a generative adversarial network, (vii) a machine learning classification algorithm.

13. A system for generating firewall rules, comprising: one or more processing devices coupled to a memory containing instructions that cause the one or more processing devices to: receive data traffic flow information associated with a set of virtual machines connected through a network; generate a network graph with nodes and edges from the received data traffic flow information, wherein the nodes of the network graph represent virtual machines; group the nodes of the network graph to generate a new graph; generate a set of firewall rules from the new graph based on common identifiers between two virtual machines from the set of virtual machines; verify the set of firewall rules by comparing first traffic records generated using the set of firewall rules with second traffic records generated using an original set of firewall rules; output the set of firewall rules as an output file; and The firewall rules are loaded into a firewall component and the firewall component is used to control network traffic through the firewall component or the network.

14. The system of claim 13, wherein, The grouping occurs through a machine learning algorithm.

15. The system of claim 14, wherein, The machine learning algorithm used for grouping is one of: (i) a community detection algorithm, (ii) an algorithm based on centrality measures, (iii) a visual machine learning method, (iv) a small group based method, (v) a web page rank algorithm, (vi) a generative adversarial network, (vii) a machine learning classification algorithm.

16. The system of claim 13, comprising instructions that cause the one or more processing devices to automatically provision network devices associated with another network corresponding to the network graph to communicate based on the firewall rule set.

17. The system of claim 13, wherein, The nodes are grouped into groups and each group is a node of a new graph.

18. The system of claim 13, comprising instructions that cause the one or more processing devices to change parameters associated with a grouping algorithm to minimize the firewall rule set.

19. The system of claim 13, comprising instructions that cause the one or more processing devices to automatically implement the firewall rule set after validation.

Images