System and method for generating standardized event logs for cloud detection and response in multi-layer cloud environment

Abstract

A system and method improve cloud detection and response by generating standardized event logs from multiple cloud computing layers. The method includes receiving a plurality of events, where a first event is generated in a first cloud layer of a cloud computing environment provided by a cloud service provider (CSP) and a second event is generated in a second cloud layer of the cloud computing environment; extracting data from each event; generating a standardized event based on the extracted data and further based on a predefined data pattern, the predefined pattern comprising a plurality of data fields, at least a portion of the plurality of data fields being associated with the cloud layer; storing the standardized event in a transaction database, wherein a standardized event log is stored in the transaction database; and applying rules from the rules engine to the standardized events to detect cyber-security threats in the cloud computing environment.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. non-provisional patent application No. 17 / 816,609, filed on August 1, 2022, the contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to network security threat detection, and particularly to network security threat detection across multiple cloud tiers. Background Art

[0004] Cloud computing infrastructure (such as Amazon Web Services) Web Services, AWS), Microsoft Azure ( Azure), Google Cloud Platform ( Cloud platforms, etc.) offer many computing advantages. Namely, the ability to flexibly control the amount of computing resources an organization needs and pay only for the actual resources used is a significant advantage when the alternative is that the organization would pay for a server (e.g., lease or purchase real estate to physically house the server) and continue to pay for maintenance (such as IT services, electricity, etc.) whether the server is in use or not.

[0005] Each cloud computing infrastructure provides and delivers services that are different from the others, as occurs naturally in most markets where competitors offer different offerings at different prices. Therefore, it is not uncommon for organizations to deploy multiple cloud computing environments across cloud computing infrastructures in order to better utilize the offerings provided by each cloud computing infrastructure. For example, an organization may utilize Azure for an organizational repository and utilize AWS to provide services, such as a web server backend.

[0006] As more and more organizations leverage different aspects of different cloud computing infrastructures, managing multiple cloud environments quickly becomes challenging. Even a relatively small deployment may include thousands of users and hundreds of resources, all of which may scale up or down based on unforeseen demand. As a result, many cybersecurity risks may also arise as a result, especially when an organization has a portion of its cloud computing infrastructure that is used internally and should not be exposed publicly and a portion of its infrastructure needs to be exposed publicly (such as the network servers in the above example).

[0007] To further complicate matters, a cloud computing environment includes multiple layers. For example, a cloud computing environment may include an infrastructure layer that provides infrastructure as a service (IaaS), an operating system (OS) and middleware layer that provides platform as a service (PaaS), and an application layer that provides software as a service (SaaS). Each cloud layer exposes different resources to users, and each layer may include its own unique cybersecurity risks.

[0008] Solutions that address a single cloud computing infrastructure or a single layer of the cloud may be effective for that specific infrastructure or layer, but they do not communicate with other cloud computing infrastructures. Such solutions require that each cloud computing environment deployed on a different infrastructure has its own solution, and each such solution needs to be managed independently.

[0009] Cloud detection and response (CDR) solutions attempt to detect and provide responses to cybersecurity threats, sometimes as part of attack surface management (ASM). Some CDR solutions attempt to detect cybersecurity threats by reading event logs and performing anomaly detection on them. As event logs and data sets grow larger, these CDR solutions leverage machine learning (ML) and artificial intelligence (AI) solutions. However, this solution has significant drawbacks. For example, an AI model can change its output based on processing inputs, so that if the same input is provided twice, the first time may produce a different result than the second time the input is provided. This is obviously a problem if cybersecurity threats cannot be detected continuously.

[0010] Additionally, AI and ML solutions are not transparent. It is often impossible to trace the decision tree that led to a specific input generating a specific output. While this may help with anomaly detection, it makes it more difficult to tune the model for false positive and false negative detections.

[0011] Therefore, it would be advantageous to provide a solution that can overcome the above challenges. Summary of the invention

[0012] The following is an overview of several example embodiments of the present disclosure. This overview is provided to facilitate the reader's basic understanding of these embodiments and does not completely limit the scope of the present disclosure. This overview is not a broad overview of all contemplated embodiments, and is neither intended to identify the key or important elements of all embodiments nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a preface to a more detailed description presented later. For convenience, the term "some embodiments" or "certain embodiments" may be used herein to refer to a single embodiment or multiple embodiments of the present disclosure.

[0013] Certain embodiments disclosed herein include a method for improving cloud detection and response (CDR) by generating a standardized event log based on multiple cloud computing layers. The method includes: receiving multiple events, wherein a first event of the multiple events is generated in a first cloud layer of a cloud computing environment provided by a cloud service provider (CSP), and a second event of the multiple events is generated in a second cloud layer of the cloud computing environment; extracting data from each of the multiple events; generating a standardized event based on the extracted data and further based on a predefined data pattern, the predefined pattern including multiple data fields, at least a portion of the multiple data fields being related to a cloud layer of the cloud computing environment; storing the standardized event in a transaction database, the transaction database storing a standardized event log; and applying rules from a rule engine to the standardized events stored in the transaction database to detect network security threats in the cloud computing environment.

[0014] Certain embodiments disclosed herein also include a non-transitory computer-readable medium having stored thereon a process for causing a processing circuit to execute the process, the process comprising: receiving a plurality of events, wherein a first event of the plurality of events is generated in a first cloud layer of a cloud computing environment provided by a cloud service provider (CSP), and a second event of the plurality of events is generated in a second cloud layer of the cloud computing environment; extracting data from each of the plurality of events; generating a standardized event based on the extracted data and further based on a predefined data pattern, the predefined pattern comprising a plurality of data fields, at least a portion of the plurality of data fields being related to a cloud layer of the cloud computing environment; storing the standardized event in a transaction database, wherein a standardized event log is stored in the transaction database; and applying rules from a rule engine to the standardized events stored in the transaction database to detect network security threats in the cloud computing environment.

[0015] Certain embodiments disclosed herein also include a system for improving cloud detection and response (CDR) by generating a standardized event log based on multiple cloud computing layers. The system includes: a processing circuit; and a memory, the memory containing instructions, which, when executed by the processing circuit, configure the system to: receive multiple events, wherein a first event of the multiple events is generated in a first cloud layer of a cloud computing environment provided by a cloud service provider (CSP), and a second event of the multiple events is generated in a second cloud layer of the cloud computing environment; extract data from each of the multiple events; generate a standardized event based on the extracted data and further based on a predefined data pattern, the predefined pattern including multiple data fields, at least a portion of the multiple data fields being related to a cloud layer of the cloud computing environment; store the standardized event in a transaction database, the transaction database storing a standardized event log; and apply rules from a rule engine to the standardized events stored in the transaction database to detect network security threats in the cloud computing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims of this specification.The above and other objects, features and advantages of the disclosed embodiments will become apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0017] Figure 1 is a schematic diagram of multiple cloud computing environments connected to an inspection environment according to an embodiment

[0018] Figure 2 is a schematic diagram of the flow of events through an event log normalizer implemented according to an embodiment.

[0019] Figure 3 is a schematic diagram of a cloud computing environment having multiple layers for describing an embodiment.

[0020] Figure 4 is a flow chart of a method for generating a standardized event log from multiple cloud tiers according to an embodiment.

[0021] Figure 5 is a schematic diagram of a log normalizer according to an embodiment.

[0022] Figure 6 is a flow chart of a method for detecting event clusters from a standardized event log implemented according to an embodiment. DETAILED DESCRIPTION

[0023] It is important to note that the embodiments disclosed herein are merely examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various embodiments claimed. In addition, some statements may apply to some inventive features but not to other features. In general, unless otherwise stated, singular elements may be plural and vice versa without loss of generality. In the accompanying drawings, the same reference numerals represent the same parts in several views.

[0024] The disclosed embodiments include a method and system for standardizing event logs across multiple cloud tiers. Standardized event logs allow for accurate representation of the state of an organization's entire cloud technology stack, including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). In addition, such standardized logs can be used to apply a unified rule engine to events of the standardized logs. In some embodiments, event logs can be standardized across stacks (i.e., from IaaS to SaaS) and across cloud platforms (i.e., between a first type of cloud platform and a different type of cloud platform).

[0025] This is particularly advantageous because it allows a single rule to be applied across multiple cloud tiers, rather than having to maintain a different rule engine for each tier, ensuring that those rules are all compatible, and in some embodiments, ensuring that this compatibility is also cross-platform across multiple cloud platforms. In some embodiments, standardized logs are generated based on a unified data schema. In an embodiment, the data schema specifies a data structure for storing events in a log. In some embodiments, the data schema also includes rules for generating standardized log entries based on log entries for a particular cloud tier. In an embodiment, log events are received from a queue of a cloud tier by an event log normalizer ("normalizer") (the event log normalizer parses events), extracts data from the received events, and generates standardized log events based on the extracted data and a predefined data schema.

[0026] In this regard, it is recognized that entering data into a log is an activity that can be performed by humans. However, humans cannot receive events at the speed at which cloud computing environments generate events. The time required for humans to manually enter each event and standardize each event based on predefined data patterns will make standardized logs impractical because the information will likely be irrelevant. In addition, human operators will be required to consistently apply data pattern rules to many different types of events received from different cloud environments and different cloud layers. If the rules are inconsistently applied and differences occur, the standardized event log cannot accurately reflect the state of the cloud computing environment, thereby making the log invalid. The disclosed embodiments provide a system that is configured to consistently apply predefined data patterns to events received from multiple cloud layers and in some embodiments further across multiple cloud computing environments. The system is further configured to provide the standardized event log to a unified rule engine that applies rules, controls, etc. to the events of the standardized event log, such as to detect network security threats. Since the predefined data patterns are consistently applied to the received events, the rule engine is also consistently applied to the generated standardized event log.

[0027] Figure 1 1 is an example diagram of multiple cloud computing environments connected to an inspection environment according to an embodiment. A first cloud computing environment 110 and a second cloud computing environment 120 (generally referred to as cloud computing environments) are each connected to an inspection environment 130 .

[0028] In an embodiment, the cloud computing environment can be implemented as a virtual private cloud (VPC) on a cloud computing infrastructure, also known as a cloud service provider (CSP). The cloud computing infrastructure can be, for example, Amazon Web Services (AWS), Google Cloud Platform ( Cloud Platform, GCP), Microsoft Azure, etc. In an embodiment, the cloud computing environment includes multiple resources and entities.

[0029] For example, a first cloud computing environment 110 deployed on a first cloud computing infrastructure includes a first resource 112, a first subject 114, and an event store 116 specific to the cloud infrastructure. In an embodiment, the event store 116 is implemented as a database for storing events, which correspond to actions performed in the first cloud computing environment 110. For example, if the first subject 114 accesses the first resource 112, such access is recorded as an event, and the event is stored in the event store 116. In an embodiment, the event store 116 stores events based on a predefined data pattern that is unique to the cloud computing infrastructure on which the first cloud computing environment 110 is deployed. For example, the event store 116 may be used to store events. Event storage 116 is implemented.

[0030] In an embodiment, the first resource 112 is hardware provisioned by a cloud computing environment, such as a processor, memory, storage, etc. In certain embodiments, the first resource 112 is a virtual workload, such as a virtual machine, a container, a serverless function, etc. A resource is a cloud entity that provides a service or provides access to hardware.

[0031] In some embodiments, principal 114 is a user account, a service account, a role, etc. A principal is a cloud entity that is authorized to perform actions on resources, initiate actions in a cloud computing environment, etc. In some embodiments, a cloud entity is both a corresponding resource for some principals and a corresponding principal for some resources. For example, a load balancer can be a resource from the perspective of a user account, and a load balancer can be a principal from the perspective of a network server accessed by the load balancer.

[0032] As another example, the first resource 112 can be a virtual machine deployed in a GCP cloud computing environment. The virtual machine is configured to access a cloud recording application programming interface (API) and generate events by providing data via a specified sink, which is then recorded as an event in a log. The data is received by the cloud recording API and routed to the sink according to the specification provided by the virtual machine. In an embodiment, the sink is associated with a cloud resource. The sink routes log events to a log storage bucket. In an embodiment, the log storage bucket is a virtual storage device. In certain embodiments, a service account is generated for each sink in the cloud computing environment, and the service account initiates writing log events to a sink associated with the service account.

[0033] The second cloud computing environment 120 includes a hardware layer 122, a middleware layer 124, and an application layer 126. In an embodiment, the second computing environment 120 is deployed on a cloud computing infrastructure that is different from the cloud computing infrastructure of the first cloud computing environment 110. For example, the first cloud computing environment 110 is deployed on AWS, while the second cloud computing environment 120 is deployed on Azure. In an embodiment, each cloud computing environment generates events specific to the cloud computing infrastructure on which their respective cloud computing environments are deployed. Further, different layers in different cloud computing environments may generate events differently.

[0034] The hardware layer 122 is used to provide IaaS services. For example, Google Compute Engine ( Compute Engine) is an IaaS provided through a cloud computing environment in GCP. In an embodiment, IaaS provides scalable self-service access to hardware resources such as processors, storage devices, networking, etc.

[0035] The middleware layer 124 is used to provide PaaS services. Google App Engine ( App Engine), AWS Elastic Beanstalk, and OpenShift are examples of PaaS services. In an embodiment, PaaS provides a platform for creating software tools (such as operating systems, development kits, storage, etc.).

[0036] The application layer 126 is used to provide SaaS services. For example, etc. are examples of SaaS services. In an embodiment, SaaS provides software applications through a network interface, which does not require users to install software, worry about updates, version management, etc.

[0037] In an embodiment, each cloud layer generates events. Events may be generated based on, for example, actions initiated by a user account in that cloud layer. In some embodiments, events in a hardware layer of a first cloud computing environment are not generated in the same manner as events in a hardware layer of a different second cloud computing environment.

[0038] In an embodiment, an event is a data record corresponding to an action initiated in a cloud computing environment. For example, accessing a resource, adding a subject, associating a subject with a privilege, starting a machine, shutting down a machine, writing to a storage bucket, extracting an image from a repository, etc. are all examples of actions initiated in a cloud computing environment. Each such action may be recorded as an event. In an embodiment, an event includes an identifier corresponding to the action (e.g., a descriptor of the action) and a timestamp. In some embodiments, an event may further include: a resource identifier, a user account identifier, a service account identifier, an action identifier, a network address, a namespace identifier, a timestamp, additional information fields, an identifier of the cloud computing environment, an identifier of the cloud computing infrastructure, etc.

[0039] The first cloud computing environment 110 and the second cloud computing environment 120 are each connected to an inspection environment 130. In an embodiment, the inspection environment 130 is a cloud computing environment deployed on a cloud computing infrastructure. In some embodiments, the cloud computing infrastructure of the inspection environment 130 is the same as the first cloud computing environment 110 or the second cloud computing environment 120. In some embodiments, the inspection environment 130 is deployed as a VPC on a cloud computing infrastructure (such as GCP).

[0040] In an embodiment, the inspection environment 130 includes a log normalizer 132, a rule engine 134, a security graph 136, and a standardized event log 138. In some embodiments, the unified data schema is stored as a schema of, for example, a database, and the standardized event log 138 is stored on the database. Although the elements of the inspection environment 130 are shown as individual elements in a single environment, it should be understood that this is merely one possible implementation according to an embodiment, and other implementations utilizing other elements may be equally implemented.

[0041] The log normalizer 132 is configured to receive events from a plurality of cloud tiers, wherein at least a first cloud tier is different from a second cloud tier. For example, in an embodiment, the log normalizer 132 is configured to receive a first plurality of events (e.g., IaaS events) from a first cloud tier and a second plurality of events (e.g., SaaS events) from a second cloud tier. In some embodiments, the log normalizer 132 is configured to pull events from event streams generated by cloud computing environments. In an embodiment, the log normalizer 132 is configured to pull events from a plurality of event streams, wherein a first event stream is generated according to a first cloud computing environment and a second event stream is generated according to a second cloud computing environment, the second cloud computing environment being deployed on an infrastructure different from an infrastructure on which the first cloud computing environment is deployed. In certain embodiments, the log normalizer 132 is configured to receive a plurality of events from an event queue of each of a plurality of cloud computing environments.

[0042] In an embodiment, the log normalizer 132 is configured to extract data from the event and store the event as a standardized event in the standardized event log 138. In some embodiments, the log normalizer 132 is further configured to store the event in the standardized event log 138 based on a predefined data pattern. In an embodiment, the extracted data includes a resource identifier, a user account identifier, a service account identifier, an action identifier, a network address, a namespace identifier, a timestamp, additional information fields, etc. In some embodiments, the standardized event may be further generated based on an identifier of a cloud computing environment, an identifier of a cloud computing infrastructure, an identifier of a cloud tier, etc.

[0043] The rules engine 134 is configured to apply conditions to events. For example, a rule may include a trigger, a condition, and an action. In an embodiment, a trigger is a keyword, a combination of keywords, a series of events, etc., that causes an action to be initiated when the condition is met. For example, a rule may specify that an alert should be generated if an event is detected that a user is given administrator privileges.

[0044] The rule engine 134 applied to the normalized event log 138 is actually applied to events from multiple cloud tiers. Thus, rules and controls can be generated that are applied equally across multiple cloud tiers and across multiple cloud computing environments.

[0045] In an embodiment, rules are used to detect events, and controls are used to ensure that a positive response is initiated in response to a rule being triggered. This is advantageous because it reduces redundancy in, for example, generating similar rules for each cloud computing environment, for each cloud tier, etc. In addition, when a rule is updated, it is updated across all cloud computing environments and across all tiers.

[0046] Redundancy of having the same rules for different cloud environments and different cloud tiers creates a risk that when such rules are updated, they may be updated for some cloud environments but not all, creating a potential cybersecurity risk between the gap between how administrators believe their cloud environments are defined and how they are actually defined. Attackers may exploit this gap and gain unauthorized access to the cloud environments.

[0047] In an embodiment, the inspection environment 130 further includes a security graph 136. The security graph 136 is used to represent the cloud computing environment in a graph database, which is configured to store the security graph 136 therein. In an embodiment, the security graph 136 may include a predefined data schema to store cloud entities (such as subjects, resources, etc.) as nodes in the security graph 136. The predefined data schema may be applied to unify the representation, such that, for example, a subject from a first cloud computing environment and a subject from a second cloud computing environment will each be represented by a master node according to the predefined data schema in the security graph 136.

[0048] In an embodiment, an event may be associated with a resource, a subject, or both of a cloud computing environment. In some embodiments, an event may be associated with a single cloud tier, or connected to multiple cloud tiers. In an embodiment, a security graph may be queried to detect nodes connected to nodes representing resources or other cloud entities for which standardized events are generated. Examples of methods for generating a security graph are discussed in further detail in U.S. non-provisional patent application Ser. No. 17 / 524,410, the entire contents of which are incorporated herein by reference.

[0049] In some embodiments, network security threats can be detected based on standardized events. For example, cross-layer (i.e., between two different layers of the same cloud platform, or between two different layers of different cloud platforms) access can be detected. For example, a service account from a first cloud computing environment can assume a role in a second cloud computing environment and start a new virtual machine with a cryptominer or other malware installed. Cryptocurrency mining is malware that uses the resources of a cloud computing environment to mine cryptocurrencies. In this example, a SaaS layer application from a first cloud computing environment provides a service account with the ability to assume a role in a second cloud computing environment and deploy virtual machines using the PaaS layer of the second cloud computing environment.

[0050] A standardized event corresponding to a new virtual machine, deployed by a service account with a role assumed, may be defined as a network security threat. In an embodiment, the security graph 136 is traversed to detect nodes corresponding to identifiers extracted from the standardized event. For example, the identifier may be an identifier of a cloud entity, such as an identifier of a user account, a service account, a resource, etc. In some embodiments, the node representing the cloud entity may be further associated with a network security risk. For example, a service account may be associated with a weak password. As another example, a resource may be associated with a misconfiguration, such as opening a port that allows a cryptocurrency mining application to connect to an external network outside the network of the cloud computing environment. In an embodiment, a network security risk may be represented as a node in the security graph 136. In some embodiments, the network security risk may be stored as metadata, data, etc. of the detected node.

[0051] Figure 2 is an example diagram of an event flow through an event log normalizer implemented according to an embodiment. The log normalizer 132 receives events from a plurality of queues 210-1 through 210-N (generally referred to as queues 210, individually referred to as queues 210), where 'N' is an integer with a value of '2' or greater. For example, queue 201 can be implemented as an Amazon Simple Queue Service (SQS). In some embodiments, queue 210 can be implemented as an event stream, such as Amazon Motion Data Stream ( Kinesis Data Stream).

[0052] The log normalizer 132 is configured to access the data schema 205. In an embodiment, the data schema 205 is implemented as part of a database storing a standardized event log, such as the standardized event log 138. In certain embodiments, the data schema 205 includes predefined data fields, such as a resource identifier, a user account identifier, a service account identifier, an action identifier, a network address, a namespace identifier, a timestamp, additional information fields, an identifier about a cloud computing environment, an identifier of a cloud computing infrastructure, and the like.

[0053] In an embodiment, log normalizer 132 is configured to extract data from events received via queue 210 and generate normalized events based on data patterns 205. In certain embodiments, log normalizer 132 is configured to generate normalized events from a plurality of received events.

[0054] In some embodiments, the log normalizer 132 is configured to write the generated normalized events to a normalized event log 138. In an embodiment, the normalized event log 138 is stored in a transactional database 220 having the properties of atomicity, consistency, isolation and durability (ACID).

[0055] In some embodiments, a standardized event log 138 is provided to the rule engine 134. In some embodiments, the generated standardized events are provided to the rule engine 134 and written to the standardized event log 138. Providing the standardized event log 138 to the rule engine 134 allows the rules of the rule engine 134 to be applied equally and consistently to all events generated from multiple different cloud environments. This is advantageous because it reduces, for example, redundant rule engines, each with rules corresponding to other rule engines, all managed by a single organization for each cloud tier of each cloud computing environment of the organization. By having a single point of applying rules through the rule engine 134, maintenance of the rule engine is significantly reduced. Rule updates only need to occur on a single engine, thereby reducing points of failure.

[0056] Figure 3 is an example diagram of a cloud computing environment 300 with multiple layers for describing an embodiment. In an embodiment, the cloud computing environment includes a hardware layer 310, a middleware layer 320, and an application layer 330. Each layer of the cloud computing environment exposes different resources and allows the use of these resources. In an embodiment, the cloud layer is built on top of the previous layer and provides support for the next layer. For example, the middleware layer 320 is built on top of the hardware layer 310 and provides the ability to deploy the application layer 330.

[0057] In an embodiment, the hardware layer 310 is used to provide IaaS services 315. For example, Google Compute Engine is an IaaS provided through a cloud computing environment in GCP. In an embodiment, IaaS provides scalable self-service access to hardware resources such as processors, storage devices, networking, etc.

[0058] In some embodiments, the middleware layer 320 is used to provide PaaS services 325. Google App Engine, AWS Elastic Beanstalk, and OpenShift are examples of PaaS services. In an embodiment, PaaS provides a platform for creating software tools (such as operating systems, development kits, storage, etc.). In an embodiment, PaaS services 325 are provided on top of IaaS services 315.

[0059] In some embodiments, the application layer 330 is used to provide SaaS services 335. For example, etc. are examples of SaaS services. In an embodiment, SaaS provides software applications through a network interface, which does not require users to install software, worry about updates, version management, etc. In an embodiment, SaaS service 335 is provided on top of PaaS service 325.

[0060] In an embodiment, each cloud layer generates events. Events may be generated based on, for example, actions initiated by a user account in that cloud layer. In some embodiments, events in a hardware layer of a first cloud computing environment are not generated in the same manner as events in a hardware layer of a different second cloud computing environment.

[0061] Figure 4 is an example flow chart 400 of a method for generating a standardized event log from multiple cloud tiers implemented according to an embodiment.

[0062] At S410, a plurality of events are received. In an embodiment, the plurality of events include events from a first cloud layer (e.g., a hardware layer) and events from a second cloud layer (e.g., an application layer). In some embodiments, some events are received via a push queue, a pull queue, an event stream, and any combination thereof. For example, a first set of events from a first cloud layer is received by accessing an event queue, and a second set of events from a second cloud layer is received by accessing an event stream.

[0063] At S420, a standardized event is generated based on the received events. In an embodiment, a standardized event is generated for each received event. In some embodiments, a standardized event is generated for a group of received events. For example, a first received event may include generating a new user account, and a second received event may include providing administrator privileges for the new user account. In an embodiment, a single standardized event corresponding to a new user account having administrator privileges being generated may be generated using the first received event and the second received event.

[0064] In some embodiments, the standardized event is generated based on a data pattern. In an embodiment, the data pattern includes a data structure for the event. In some embodiments, data is extracted from the received event, and the standardized event is generated based on the extracted data and the data pattern.

[0065] In some embodiments, the standardized event is generated based on a data pattern. In an embodiment, the data pattern includes a data structure for the event. In some embodiments, data is extracted from the received event, and the standardized event is generated based on the extracted data and the data pattern.

[0066] In some embodiments, a standardized event is generated for the received group of events. For example, a first received event may include generating a new user account, and a second received event may include allocating resources in a cloud computing environment through the user account, such as a user account requesting that a container node be deployed in the cloud computing environment. In an embodiment, the first received event and the second received event may be used to generate a single standardized event corresponding to a new user account requesting deployment of a container node.

[0067] For example, a virtual machine deployed in a GCP cloud computing environment is configured to access a cloud recording application programming interface (API) and generate events by providing data via a specified receiver, which is then recorded as an event in a log. Data is received by the cloud recording API and routed to the receiver according to the specification provided by the virtual machine. In an embodiment, the receiver is associated with a cloud resource. The receiver routes log events to a log storage bucket. In an embodiment, the log storage bucket is a virtual storage device. In certain embodiments, a service account is generated for each receiver in a cloud computing environment, and the service account initiates writing log events to a receiver associated with the service account. In an embodiment, an event normalizer is configured to access a cloud recording API and read events of records, such as those stored in a storage bucket. In an embodiment, standardized events are generated by extracting data from recorded events, generating data records according to a predefined data pattern, and filling the generated records with the extracted data. The generated records are then stored as standardized events.

[0068] As another example, a virtual machine deployed in a cloud computing environment hosted on AWS (such as Amazon Elastic Compute Cloud (EC2)) includes a logging agent. In an embodiment, when the orchestrator of the cloud computing environment provides the virtual machine, the logging agent is deployed on the virtual machine. The logging agent can be implemented as an executable software application that, when executed by the virtual machine, monitors actions performed by the virtual machine and generates log events containing records of the actions performed. For example, the record may include an identifier and a timestamp of the action. The record is sent to CloudWatch Log destination. In one embodiment, the log is provided to Kinesis DataStream, where AWS Lambda functions can write records to Amazon Simple Storage Service (S3) and retrieve the records from it.

[0069] In an embodiment, the event normalizer is configured to access S3 and read the record. In an embodiment, the standardized event is generated by extracting data from the record, generating a data record according to a predefined data schema, and filling the generated record with the extracted data. The generated record is then stored as a standardized event.

[0070] It should be noted that for the above examples, a human operator can configure the system to access event logs in different computing environments and extract data from the records, manually entering the data into a standardized event log. However, it is also worth noting at this point that such event logs are typically stored as petabytes of data, and processing the data must occur within a time frame that humans cannot process because the raw data is typically only stored for a limited time. Although multiple humans can be assigned to this task, humans are known to be unreliable in repetitive tasks, i.e., receiving event logs where events may be slightly different, appropriately and consistently applying predefined data patterns to each element of the data extracted from each event, and errors may occur with such a large amount of data that needs to be processed.

[0071] In contrast, by configuring a system to perform methods as described herein, a standardized event log can be generated that consistently applies predefined data patterns and does not introduce errors due to conditions that plague human operators, such as personal judgment, bias, fatigue, etc. Event logs must reliably reflect actions performed in a cloud computing environment, and anything less reliable greatly reduces the value of having such information.

[0072] At S430, the standardized events are stored in a standardized event log. In an embodiment, the standardized event log is stored on a transaction database. In some embodiments, the transaction database includes a data pattern based on which the standardized events are generated. In certain embodiments, the standardized events, the standardized event log, and combinations thereof are provided to a rule engine. Optionally, rules are applied to the standardized events. In an embodiment, rules are utilized to detect network security threats. For example, if a condition of a rule is met, this is an indication that a network security threat exists. In an embodiment, the condition of meeting a rule is an indication of a network security event.

[0073] At S440, a check is performed to determine whether additional standardized events should be generated. In an embodiment, if 'yes', execution continues at S420. In certain embodiments, if 'no', a check may be performed to further determine whether additional events should be received. If 'yes', execution continues at S410. In some embodiments, if either check returns 'no', execution terminates.

[0074] Figure 5is an example schematic diagram of a log normalizer 132 according to an embodiment. The log normalizer 132 (also referred to as normalizer 132) includes a processing circuit 510 coupled to a memory 520, a storage device 530, and a network interface 540. In an embodiment, the components of the normalizer 132 may be communicatively connected via a bus 550.

[0075] Processing circuit 510 may be implemented as one or more hardware logic components and circuits. For example, but not limited to, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), graphics processing units (GPUs), tensor processing units (TPUs), general-purpose microprocessors, microcontrollers, digital signal processors (DSPs), etc., or any other hardware logic components that can perform calculations or other manipulations of information.

[0076] The memory 520 may be volatile (eg, random access memory, etc.), non-volatile (eg, read-only memory, flash memory, etc.), or a combination thereof.

[0077] In one configuration, software for implementing one or more embodiments disclosed herein may be stored in storage device 430. In another configuration, memory 420 is configured to store such software. Software should be broadly interpreted to mean any type of instructions, whether referred to as software, firmware, middleware, microcode, hardware description language or other. Instructions may include code (e.g., source code format, binary code format, executable code format, or any other suitable code format). Instructions, when executed by processing circuit 510, cause processing circuit 510 to perform various processes described herein.

[0078] The storage device 530 may be a magnetic storage device, an optical storage device, etc., and may be implemented as, for example, flash memory or other memory technology, a compact disk-read only memory (CD-ROM), a digital versatile disk (DVD), or any other medium that may be used to store the desired information.

[0079] The network interface 540 allows the normalizer 132 to communicate with, for example, the inspection environment 130, a security map, a database for storing normalized event logs, and the like.

[0080] It should be understood that the embodiments described herein are not limited to Figure 5 The specific architecture shown in , and other architectures may be equally used without departing from the scope of the disclosed embodiments. In addition, in embodiments, other workloads described herein (such as security graphs, normalized event log databases, etc.) may be used in conjunction with Figure 5 The invention can be implemented on a computing architecture similar to or the same as the computing architecture discussed in .

[0081] Figure 6 600 is an example of a flow chart of a method for detecting event clusters from a standardized event log according to an embodiment. An advantage provided by the standardized event log is the ability to detect event clusters based on a plurality of received events. The received events are used to generate standardized events, such as described in detail above. In an embodiment, a portion of the standardized events are clustered into event clusters.

[0082] At S610, a plurality of standardized events are received. In an embodiment, standardized events are generated based on the received events and according to a data schema. Examples of methods for generating standardized events are discussed in more detail above. In an embodiment, the standardized events include data attributes such as: a resource identifier, a user account identifier, a service account identifier, an action identifier, a network address, a namespace identifier, a timestamp, additional information fields, an identifier about a cloud computing environment, an identifier of a cloud computing infrastructure, and the like.

[0083] At S620, an event cluster is generated. In an embodiment, the cluster is generated based on a group of multiple standardized events based on data attributes (or data fields) of standardized events with the same value. For example, the cluster can be based on the following values: resource identifier, user account identifier, service account identifier, action identifier, network address, namespace identifier, timestamp, additional information field, identifier about cloud computing environment, identifier of cloud computing infrastructure, cloud layer identifier, combinations thereof, etc.

[0084] In an embodiment, a first standardized event of an event cluster is from a first cloud tier and a second standardized event of the event cluster is from a second cloud tier that is not the first cloud tier. In some embodiments, an event cluster can trigger a rule, and each of the events comprising the event cluster will not trigger the rule by itself.

[0085] For example, the first standardized event indicates access to a workload (such as a virtual machine) in a first cloud computing environment. In an embodiment, the first standardized event is received from a PaaS layer, such as from a Kubernetes cluster ( The second standardized event indicates, for example, reading data from a SaaS implemented on a Kubernetes node. In general, these events together form an event cluster, for example based on an identifier of a Kubernetes node or cluster, which indicates that a cross-layer operation has occurred. When viewing these events alone, these events lack context, but when they are clustered and presented together, a single action (or session) is obvious. Therefore, providing context between actions that occur on different layers but are connected to each other provides the benefit of allowing the time to detect network security threats to be reduced. In addition, in some embodiments, events from a first layer in a first cloud computing environment may be clustered with events from a second layer in a second cloud computing environment. In an embodiment, the first layer and the second layer are different layers (e.g., the first layer is a SaaS layer and the second layer is a PaaS layer), and the first cloud computing environment is different from the second cloud computing environment (e.g., the first cloud computing environment is AWS and the second cloud computing environment is GCP).

[0086] At S630, a check is performed to determine whether another event cluster should be generated. In an embodiment, if there are additional standardized events, an event cluster may be generated. In some embodiments, if another event cluster should be generated, execution continues at S610 to receive additional standardized events that can be clustered into an event cluster. In other embodiments, if another event cluster should not be generated, execution may terminate.

[0087] In some embodiments, the disclosed technology improves cloud detection and response (CDR) technology. In an embodiment, a CDR system provides the ability to detect threats in a cloud computing environment and respond to those threats. Specifically, the disclosed technology improves the ability to detect threats by generating a standardized event log. The standardized event log allows increased visibility into actions that occur across the entire stack on a single cloud platform, from hardware to software provisioning, and allows visibility into previously unavailable cross-cloud platform solutions. When viewed individually (i.e., only within the context of its local cloud), cross-platform events may be viewed as insignificant, but when viewed in the context of another event from another cloud platform, cross-platform threats that were not otherwise visible are allowed to be detected. Similarly, layer events may be insignificant when viewed individually, but may indicate different events in the context of additional layers in the same or different cloud computing environments.

[0088] In addition, the response time to network security threats is improved. This is because standardized logs reduce detection time. For example, without cross-layer and cross-platform event detection, it may take several weeks for operators to find an event in one layer of the cloud platform and link it to an event in another layer of the cloud platform. By utilizing standardized event logs, a single rule engine can be used to detect network security events from standardized events. For example, when a detection rule is applied to a standardized event, the triggering of the rule (e.g., a data field of the standardized event has a value that satisfies the condition of the rule) indicates that a network security threat has been detected. In some embodiments, remedial actions, mitigation actions, etc. are initiated in response to a rule that triggers the detection of a network security threat. In an embodiment, a mitigation action is to generate a notification that a network security threat has been detected.

[0089] In an embodiment, providing a standardized event log may be performed in lieu of or in conjunction with applying machine learning (ML) and artificial intelligence (AI) techniques. In the context of CDR, ML and AI techniques often attempt to perform anomaly detection from seemingly unrelated data to identify events that correspond to the behavior of an attacker. However, a disadvantage of these techniques is that they are opaque and may not be consistent (i.e., an AI model that receives the same input a second time does not necessarily generate the same output as the first time). This means that an AI model processing an event may detect a threat based on the input, or may not detect a threat based on the same input, without providing an explanation. In cybersecurity, where predictability and consistency are important, this lack of predictability is undesirable.

[0090] Therefore, there is a need for a standardized event log on which a rule engine can be applied while ensuring transparency. Rules are applied identically and consistently, and can be monitored, changed, updated, etc. as needed from a single point.

[0091] Various embodiments disclosed herein can be implemented as hardware, firmware, software or any combination thereof. In addition, the software is preferably implemented as an application program, which is tangibly embodied in a program storage unit or a computer-readable medium, which is composed of a combination of part or some devices and / or devices. The application program can be uploaded to a machine including any suitable architecture and executed by the machine. Preferably, the machine is implemented on a computer platform, and the computer platform has hardware such as one or more central processing units ("CPU"), memory and input / output interfaces. The computer platform can also include an operating system and microinstruction codes. The different processes and functions described herein can be a part of a microinstruction code or a part of an application program or any combination thereof, which can be executed by a CPU, regardless of whether such a computer or processor is explicitly shown. In addition, various other peripheral units can be connected to the computer platform, such as additional data storage units and printing units. In addition, a non-transitory computer-readable medium is any computer-readable medium other than a temporary propagation signal.

[0092] All examples and conditional language listed herein are intended for teaching purposes to help the reader understand the principles of the disclosed embodiments and the concepts contributed by the inventors to promote the art, and are to be interpreted as not being limited to such specifically listed examples and conditions. In addition, the principles, aspects, and embodiments of the disclosed embodiments listed herein, together with all statements of their specific examples, are intended to cover both their structural and functional equivalents. In addition, it is intended that such equivalents include currently known equivalents as well as equivalents developed in the future, i.e., any developed element that performs the same function regardless of structure.

[0093] It should be understood that any reference to an element using names such as "first", "second", etc. herein does not generally limit the number or order of these elements. Instead, these names are generally used herein as a convenient method of distinguishing between two or more elements or element instances. Thus, a reference to a first and a second element does not mean that only two elements can be used there, or that the first element must be in some way before the second element. In addition, unless otherwise specified, a group of elements includes one or more elements.

[0094] As used herein, the phrase "at least one of" followed by a list of items means that any one of the listed items may be used alone, or any combination of two or more of the listed items may be used. For example, if a system is described as including "at least one of A, B, and C," the system may include: only A; only B; only C; 2A; 2B; 2C; 3A; a combination of A and B; a combination of B and C; a combination of A and C; a combination of A, B, and C; a combination of 2A and C; a combination of A, 3B, and 2C; and so on.

Claims

1. A method for improving cloud detection and response (CDR) by generating a standardized event log based on multiple cloud computing layers, comprising: Receiving a plurality of events, wherein a first event of the plurality of events is generated in a first cloud layer of a cloud computing environment provided by a cloud service provider CSP, and a second event of the plurality of events is generated in a second cloud layer of the cloud computing environment; extracting data from each of the plurality of events; generating a standardized event based on the extracted data and further based on a predefined data schema, the predefined schema comprising a plurality of data fields, at least a portion of the plurality of data fields being related to a cloud tier of a cloud computing environment; storing the standardized events in a transaction database, wherein the transaction database stores a standardized event log; and Rules from a rules engine are applied to standardized events stored in the transactional database to detect network security threats in the cloud computing environment.

2. The method according to claim 1, wherein: The normalized event is further based on data extracted from the plurality of received events.

3. The method according to claim 1, further comprising: The condition of the rule is triggered by determining that the value of the data field of the standardized event matches the condition.

4. The method according to claim 1, wherein: The first cloud layer is any one of the following layers: a software as a service SaaS layer, a platform as a service PaaS layer, and an infrastructure as a service IaaS layer.

5. The method according to claim 4, wherein: The second cloud layer is any one of the following layers, which is different from the first cloud layer: a SaaS layer, a PaaS layer, and an IaaS layer.

6. The method according to claim 1, further comprising: receiving an event from a second cloud tier of a second cloud computing environment deployed on a second CSP; extracting additional data from events from the second cloud layer; generating another standardized event based on the extracted data and the extracted additional data; as well as Applying another rule from the rules engine to detect another network security threat in any of: the cloud computing environment, the second cloud computing environment, and combinations thereof.

7. The method according to claim 1, wherein: The predefined data pattern includes attributes, which are any of the following: a resource identifier, a user account identifier, a service account identifier, an action identifier, a network address, a namespace identifier, a timestamp, an additional information field, an identifier of a cloud computing environment, an identifier of a cloud tier, and an identifier of a cloud computing infrastructure.

8. The method according to claim 1, wherein: The plurality of events are received from any of: a queue, an event stream, and a combination thereof.

9. The method according to claim 1, further comprising: A unique normalized event is generated for each unique event in the plurality of events received.

10. The method according to claim 1, further comprising: An event cluster including a first standardized event and a second standardized event is generated based on any of: a data field of the predefined data pattern, a value of a data field of the predefined data pattern, and any combination thereof.

11. The method according to claim 10, wherein: The first standardized event corresponds to an event from the first cloud layer, and the second standardized event corresponds to an event from the second cloud layer.

12. The method according to claim 1, further comprising: Detect network security threats based on standardized events.

13. The method according to claim 12, further comprising: extracting an identifier of a cloud entity from the standardized event; as well as The security graph is traversed to detect nodes representing the cloud entities.

14. The method according to claim 13, further comprising: determining that the detected node is associated with a network security risk; as well as An alert is generated based on the cybersecurity risk and the detected cybersecurity threat.

15. A non-transitory computer readable medium having stored thereon instructions for causing a processing circuit to perform a process for improving cloud detection and response (CDR) by generating a standardized event log according to a plurality of cloud computing tiers, the process comprising: Receiving a plurality of events, wherein a first event of the plurality of events is generated in a first cloud layer of a cloud computing environment provided by a cloud service provider CSP, and a second event of the plurality of events is generated in a second cloud layer of the cloud computing environment; extracting data from each of the plurality of events; generating a standardized event based on the extracted data and further based on a predefined data schema, the predefined schema comprising a plurality of data fields, at least a portion of the plurality of data fields being related to a cloud tier of a cloud computing environment; storing the standardized events in a transaction database, wherein the transaction database stores a standardized event log; and Rules from a rules engine are applied to standardized events stored in the transactional database to detect network security threats in the cloud computing environment.

16. A system for improving cloud detection and response (CDR) by generating a standardized event log based on multiple cloud computing layers, comprising: Processing circuit; as well as a memory comprising instructions that, when executed by the processing circuit, configure the system to: Receiving a plurality of events, wherein a first event of the plurality of events is generated in a first cloud layer of a cloud computing environment provided by a cloud service provider CSP, and a second event of the plurality of events is generated in a second cloud layer of the cloud computing environment; extracting data from each of the plurality of events; generating a standardized event based on the extracted data and further based on a predefined data schema, the predefined schema comprising a plurality of data fields, at least a portion of the plurality of data fields being related to a cloud tier of a cloud computing environment; storing the standardized events in a transaction database, wherein the transaction database stores a standardized event log; and Rules from a rules engine are applied to standardized events stored in the transaction database to detect network security threats in the cloud computing environment.

17. The system of claim 16, wherein: The normalized event is further based on data extracted from the plurality of received events.

18. The system of claim 16, wherein: The memory includes further instructions which, when executed by the processing circuit, further configure the system to: The condition of the rule is triggered by determining that the value of the data field of the standardized event matches the condition.

19. The system of claim 16, wherein: The first cloud layer is any one of the following layers: a software as a service SaaS layer, a platform as a service PaaS layer, and an infrastructure as a service IaaS layer.

20. The system of claim 19, wherein: The second cloud layer is any one of the following layers that is different from the first cloud layer: a SaaS layer, a PaaS layer, and an IaaS layer.

21. The system of claim 16, wherein: The memory includes further instructions which, when executed by the processing circuit, further configure the system to: receiving an event from a second cloud tier of a second cloud computing environment deployed on a second CSP; extracting additional data from events from the second cloud layer; generating another standardized event based on the extracted data and the extracted additional data; as well as Applying another rule from the rules engine to detect another network security threat in any of: the cloud computing environment, the second cloud computing environment, and combinations thereof.

22. The system of claim 16, wherein: The predefined data pattern includes attributes, which are any of the following: a resource identifier, a user account identifier, a service account identifier, an action identifier, a network address, a namespace identifier, a timestamp, an additional information field, an identifier of a cloud computing environment, an identifier of a cloud tier, and an identifier of a cloud computing infrastructure.

23. The system of claim 16, wherein: The plurality of events are received from any of: a queue, an event stream, and a combination thereof.

24. The system of claim 16, wherein: The memory includes further instructions which, when executed by the processing circuit, further configure the system to: A unique normalized event is generated for each unique event in the plurality of events received.

25. The system of claim 16, wherein: The memory includes further instructions which, when executed by the processing circuit, further configure the system to: An event cluster including a first standardized event and a second standardized event is generated based on any of: a data field of the predefined data pattern, a value of a data field of the predefined data pattern, and any combination thereof.

26. The system of claim 25, wherein: The first standardized event corresponds to an event from the first cloud layer, and the second standardized event corresponds to an event from the second cloud layer.

27. The system of claim 16, wherein: The memory includes further instructions which, when executed by the processing circuit, further configure the system to: Detect network security threats based on standardized events.

28. The system of claim 27, wherein: The memory includes further instructions which, when executed by the processing circuit, further configure the system to: extracting an identifier of a cloud entity from the standardized event; and The security graph is traversed to detect nodes representing the cloud entities.

29. The system of claim 28, wherein: The memory includes further instructions which, when executed by the processing circuit, further configure the system to: determining that the detected node is associated with a network security risk; and An alert is generated based on the cybersecurity risk and the detected cybersecurity threat.

Images