System and method for cloud infrastructure security

Abstract

A system retrieves security audit logs from a cloud-based warehousing platform and transfers them to a SIEM platform for analysis. The system consists of a connector that is installed on the Snowflake platform, and a data transfer module that is installed on the SIEM platform. The system collects the partial data sets, consolidates the data and transforms the data into universal API format JavaScript Object Notation Lines (JSONL). The converted data is then exported into a cloud provider bucket. Formatting the audit data into JSONL and placing the data into a bucket creates a reusable and universal method for ingesting security data into any Security Information and Event Management (SIEM) platform that is capable of monitoring Software as a Service (SaaS) assets.

Description

SYSTEM AND METHOD FOR CLOUD INFRASTRUCTURE SECURITYBACKGROUND

[0001] Cloud-based data warehousing platforms provide data storage and analysis capabilities. Examples include the Snowflake platform from Snowflake Inc., of Bozeman, Montana. These cloud-based data warehousing platforms provide internal security, but generally lack a reliable method for ingesting security audit logs from them for analysis as part of a user’s broader Security Ingestion and Event Management (SIEM) platform. Using built-in features, security audit logs may be exported in two ways. The first is by an application programming interface (API) and a virtual machine or a serverless function. The second is to use external integration. Using an API requires a static API key, whereas external integration requires custom cloud components. It would be desirable to develop a system for ingesting security logs that required the fewest external components and processing possible.

[0002] The limitations just described have created a significant problem for cybersecurity teams, as they are unable to effectively defend their infrastructure and data without access to these logs while the cloud-based data warehousing platform remains a part of their overall system. The current cybersecurity threat landscape has created a demand for establishing and maintaining threat monitoring capabilities for Software-as-a-Service (SaaS), cloud, hybrid, and on-premise platforms. It would be desirable therefore to solve the logging and visibility challenges encountered by users and organizations that utilize cloud-based data warehousing platforms.

[0003] References mentioned in this background section are not admitted to be prior art with respect to the present invention.SUMMARY

[0004] The invention disclosed in this patent application provides a secure and reliable method for retrieving and transmitting security audit logs from a cloud-based data warehousing platform into a SIEM platform. In certain embodiments, the export and import method provides the ability to manipulate data into a cloud-agnostic platform. With the exception of a cloud storage bucket for data export, certain embodiments establish a conversion and export process that eliminates the requirement for custom or cloud-provider resources. In a particular embodiment, the system includes a connector that is installed on the cloud-based data warehousing platform, and a data transfer module that is installed on the SIEM platform. This embodiment of the invention collects the partial data sets, consolidates the data, and transforms the data into universal application programming interface (API) format JavaScript Object Notation Lines (JSONL). The converted data is then exported into a cloud provider bucket, such as one based on the top three cloud providers (Amazon AWS, Microsoft Azure, Google GCP) used today. It may be seen then that the invention in this embodiment gives users of cloud-based warehousing platforms the ability to monitor their platform externally and centrally.

[0005] The invention in certain embodiments provides several advantages and benefits over existing methods for ingesting security audit logs into a SIEM platform. By providing a secure and reliable method for retrieving and transmitting security audit logs, it enables cyber security teams to effectively defend their infrastructure and data, and respond quickly to potential security threats. Additionally, the system's configurable connector and processing capabilities ensure that the audit logs are transmitted andanalyzed in a timely and efficient manner.

[0006] These and other features, objects and advantages of the present invention will become better understood from a consideration of the following detailed description of the preferred embodiments and appended claims in conjunction with the drawings as described following:DRAWINGS

[0007] Fig. 1 is an architectural diagram of a cloud-based data warehousing platform environment as used with respect to an embodiment of the present invention.

[0008] Fig. 2 is a data flow diagram illustrating a method according to an embodiment of the present invention.DETAILED DESCRIPTION

[0009] Before the present invention is described in further detail, it should be understood that the invention is not limited to the particular embodiments described, and that the terms used in describing the particular embodiments are for the purpose of describing those particular embodiments only, and are not intended to be limiting, since the scope of the present invention will be limited only by the claims in a subsequent nonprovisional patent application.

[0010] The invention comprises a system that retrieves security audit logs from a cloudbased warehousing platform (in a particular non-limiting embodiment, the Snowflake platform), and transfers them to a SIEM platform for analysis. The system uses the cloud-based data warehousing platform’s security event logging feature to capture security-related events, such as login attempts, privilege changes, and data accessattempts.

[0011] As shown in Fig. 1 , the system consists of a connector 12 that is installed on the cloud-based data warehousing platform 10, and a data transfer module 16 that is installed on the SIEM platform 14. The connector 12 retrieves the audit logs 18 from the cloud-based data warehousing platform 10 and transmits them to a pre-ingestion storage bucket 11 , which then notifies the data transfer module 16 on the SIEM platform 14, and which in turn then retrieves the audit logs and processes the data. The data transfer module 16 then processes the logs and sends them to the appropriate storage modules 20 within the SIEM platform 14 for analysis and storage. The system may also parse and normalize 22 the logs into a format that can be understood by the SIEM platform 14 as a part of this process.

[0012] As shown in Fig. 2, a method according to an embodiment of the invention using the configuration of Fig. 1 may now be described. (The set-up portion of Fig. 2 will be further described below.) This embodiment of the invention collects the partial data sets 50 from cloud-based data warehousing platform 10. It then consolidates the data 52. Next, it transforms the data into universal API format JavaScript Object Notation Lines (JSONL) 54. The converted data is then exported into a cloud provider bucket 56. The cloud-provider bucket may be based on one of the top three cloud providers (AWS, Azure, GCP), or another cloud provider, as desired in a particular implementation.

[0013] In a particular example in the Snowflake platform, instructions may be generated to run on a schedule to export the security audit logs 18. The CREATE TASK statement may be used to create the task, specifying the schedule and the SQL script to run. A script selects the security logs 18 from theACCOUNT_USAGE. SECURITY_EVENT_LOG table. ACCOUNT-USAGE is a schema that collects data from views. The SELECT statement may be used to retrieve the logs 18, filtering on the desired criteria such as time range, user, or event type. The COPY INTO statement may be used to export the logs 18 through connector 12. This allows the user to specify the file format, location, and other parameters as needed. Views such as ACCESS_HISTORY and QUERY_HISTORY may be used, but other views can be added to the script based on the user’s needs. For example, LOGIN_HISTORY may be added into the script.

[0014] To execute the SQL script, the CREATE EXTERNAL FUNCTION statement may be used to define a function, specifying the input and output parameters as needed. CREATE TASK may be used to define this task, specifying the schedule and the function to call. For example, the task may be designed to run every five minutes, checking if new data is available in the SECURITY_EVENT_LOG table at audit logs 18, and then calling the export_security_logs() function to export the logs 18 to the specified location. In this way, the connector 12 may automatically export security audit logs 18 on a schedule, making it easier to integrate with the SIEM platform 1 .

[0015] Formatting the audit data into JSONL format at step 54 and placing the data into a bucket 56 creates a reusable and universal method for ingesting security data into the SIEM platform 14 that is capable of monitoring Software as a Service (SaaS) assets. JSONL allows for batch event collection into a single file and allows for SIEM 14 to parse events based on key-value pairs. Additionally, this method allows for managing and monitoring for more than one account at a time by using an Infrastructure-as-Code (laC) tool, such as Terraform from HashiCorp of San Francisco, California. This allowscentralized monitoring into a single SIEM 14, and further allows the user to make script changes in a single location that may be deployed to all connected accounts. The Snowflake audit log external export method may be implemented within a provider platform system using Terraform, which deploys the Snowflake resources into the Snowflake account (typically when a new account is created). Finally, this method uses cloud identity maturity model for identity and access management (IAM) best practices by using trusted relationships rather than static credentials (API Keys), which makes the system more secure when transporting sensitive audit data. For example, in response to a security event, the provider will not need to rotate API Keys because the system relies upon bi-directional trust relationships between cloud accounts.

[0016] Once deployed, an individual accesses the cloud-provider account at step 30 through the console 24, as shown in Fig. 1 , and describes the external integration to obtain the IAM information needed to build a trust relationship with a cloud account or project that the SIEM platform 14 is connected to for log ingestion at step 32. The individual then completes the bi-directional permissions with the IAM information obtained at step 34 to authorize the cloud-based data warehousing platform 10 to unload data into the cloud account or project where log ingestion for the SIEM 14 is set up.

[0017] Once the permissions have been added, the audit log data gets exported into the cloud bucket at step 56, the SIEM 14 is notified through a bucket object creation notification at step 58, and SIEM 14 ingests the data at step 60. The security export method uses pre-existing cloud components that are integrated together with new JavaScript code. For laC implementation within a Snowflake environment, SNOW SQL(the Snowflake-specific SQL command-line client) is ported over to Terraform utilizing the Snowflake provider.

[0018] In order to utilize the embodiment of the invention described herein, a user must have a cloud-based data warehousing platform account and a cloud provider account, such as one of the following cloud accounts: AWS, Azure, or GCP. The user must have the grant for Account Admin to run the export or to create a custom role with the appropriate permissions. The user must have the IAM permissions in the cloud account to create and modify IAM permissions and to create and modify a bucket.

[0019] In a Snowflake implementation, export method uses Snowflake’s object construct function (which converts table data into JSONL), external integration, and the account usage view. When porting over to Terraform, some grants must be explicitly defined compared to SNOW SQL, which would create grants automatically based on the resources or Snowflake objects created, as the Snowflake Terraform provider is still at an early stage of development.

[0020] The systems and methods described herein may in various embodiments be implemented by any combination of hardware and software. For example, in one embodiment, the systems and methods may be implemented by a collection of networked computer systems, each of which includes one or more processors executing program instructions stored on a computer-readable storage medium coupled to the processors. The program instructions may implement the functionality described herein. The various systems and displays as illustrated in the figures and described herein represent example implementations. The order of any method may be changed, and various elements may be added, modified, or omitted.

[0021] A computing system or computing device as described herein may implement a hardware portion of a cloud computing system, as forming parts of the various implementations of the present invention. Each computer system in the cloud computing system may be any of various types of devices, including, but not limited to, a commodity server, personal computer system, desktop computer, laptop or notebook computer, mainframe computer system, handheld computer, workstation, network computer, a consumer device, application server, storage device, telephone, mobile telephone, or in general any type of computing node, compute node, compute device, and / or computing device. The computing system includes one or more processors (any of which may include multiple processing cores, which may be single or multi-threaded) coupled to a system memory via an input / output (I / O) interface. The computer system further may include a network interface coupled to the I / O interface.

[0022] In various embodiments, the computer system may be a single processor system including one processor, or a multiprocessor system including multiple processors. The processors may be any suitable processors capable of executing computing instructions. For example, in various embodiments, they may be general-purpose or embedded processors implementing any of a variety of instruction set architectures. In multiprocessor systems, each of the processors may commonly, but not necessarily, implement the same instruction set. The computer system also includes one or more network communication devices (e.g., a network interface) for communicating with other systems and / or components over a communications network, such as a local area network, wide area network, or the Internet. For example, a client application executing on the computing device may use a network interface to communicate with a serverapplication executing on a single server or on a cluster of servers that implement one or more of the components of the systems described herein in a cloud computing or noncloud computing environment as implemented in various sub-systems. In another example, an instance of a server application executing on a computer system may use a network interface to communicate with other instances of an application that may be implemented on other computer systems.

[0023] The computing device also includes one or more persistent storage devices and / or one or more I / O devices. In various embodiments, the persistent storage devices may correspond to disk drives, tape drives, solid state memory, other mass storage devices, or any other persistent storage devices. The computer system (or a distributed application or operating system operating thereon) may store instructions and / or data in persistent storage devices, as desired, and may retrieve the stored instruction and / or data as needed. For example, in some embodiments, the computer system may implement one or more nodes of a control plane or control system, and persistent storage may include the SSDs attached to that server node. Multiple computer systems may share the same persistent storage devices or may share a pool of persistent storage devices, with the devices in the pool representing the same or different storage technologies.

[0024] The computer system includes one or more system memories that may store code / instructions and data accessible by the processor(s). The system’s memory capabilities may include multiple levels of memory and memory caches in a system designed to swap information in memories based on access speed, for example. The interleaving and swapping may extend to persistent storage in a virtual memoryimplementation. The technologies used to implement the memories may include, by way of example, static random-access memory (RAM), dynamic RAM, read-only memory (ROM), non-volatile memory, or flash-type memory. As with persistent storage, multiple computer systems may share the same system memories or may share a pool of system memories. System memory or memories may contain program instructions that are executable by the processor(s) to implement the routines described herein. In various embodiments, program instructions may be encoded in binary, Assembly language, any interpreted language such as Java, compiled languages such as C / C++, or in any combination thereof; the particular languages given here are only examples. In some embodiments, program instructions may implement multiple separate clients, server nodes, and / or other components.

[0025] In some implementations, program instructions may include instructions executable to implement an operating system (not shown), which may be any of various operating systems, such as UNIX, LINUX, Solaris™, MacOS™, or Microsoft Windows™. Any or all of program instructions may be provided as a computer program product, or software, that may include a non-transitory computer-readable storage medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to various implementations. A non-transitory computer-readable storage medium may include any mechanism for storing information in a form (e.g., software, processing application) readable by a machine (e.g., a computer). Generally speaking, a non-transitory computer-accessible medium may include computer-readable storage media or memory media such as magnetic or optical media, e.g., disk or DVD / CD-ROM coupled to thecomputer system via the I / O interface. A non-transitory computer-readable storage medium may also include any volatile or non-volatile media such as RAM or ROM that may be included in some embodiments of the computer system as system memory or another type of memory. In other implementations, program instructions may be communicated using optical, acoustical or other form of propagated signal (e.g., carrier waves, infrared signals, digital signals, etc.) conveyed via a communication medium such as a network and / or a wired or wireless link, such as may be implemented via a network interface. A network interface may be used to interface with other devices, which may include other computer systems or any type of external electronic device. In general, system memory, persistent storage, and / or remote storage accessible on other devices through a network may store data blocks, replicas of data blocks, metadata associated with data blocks and / or their state, database configuration information, and / or any other information usable in implementing the routines described herein.

[0026] In certain implementations, the I / O interface may coordinate I / O traffic between processors, system memory, and any peripheral devices in the system, including through a network interface or other peripheral interfaces. In some embodiments, the I / O interface may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory) into a format suitable for use by another component (e.g., processors). In some embodiments, the I / O interface may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, for example. Also, in some embodiments, some or all of the functionality of the I / O interface, such as an interface tosystem memory, may be incorporated directly into the processor(s).

[0027] A network interface may allow data to be exchanged between a computer system and other devices attached to a network, such as other computer systems (which may implement one or more storage system server nodes, primary nodes, read-only node nodes, and / or clients of the database systems described herein), for example. In addition, the I / O interface may allow communication between the computer system and various I / O devices and / or remote storage. Input / output devices may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for entering or retrieving data by one or more computer systems. These may connect directly to a particular computer system or generally connect to multiple computer systems in a cloud computing environment, grid computing environment, or other system involving multiple computer systems. Multiple input / output devices may be present in communication with the computer system or may be distributed on various nodes of a distributed system that includes the computer system. The user interfaces described herein may be visible to a user using various types of display screens, which may include CRT displays, LCD displays, LED displays, and other display technologies. In some implementations, the inputs may be received through the displays using touchscreen technologies, and in other implementations the inputs may be received through a keyboard, mouse, touchpad, or other input technologies, or any combination of these technologies.

[0028] In some embodiments, similar input / output devices may be separate from the computer system and may interact with one or more nodes of a distributed system thatincludes the computer system through a wired or wireless connection, such as over a network interface. The network interface may commonly support one or more wireless networking protocols (e.g., Wi-Fi / IEEE 802.11 , or another wireless networking standard). The network interface may support communication via any suitable wired or wireless general data networks, such as other types of Ethernet networks, for example. Additionally, the network interface may support communication via telecommunications / telephony networks such as analog voice networks or digital fiber communications networks, via storage area networks such as Fibre Channel SANs, or via any other suitable type of network and / or protocol.

[0029] Any of the distributed system embodiments described herein, or any of their components, may be implemented as one or more network-based services in the cloud computing environment. For example, a read-write node and / or read-only nodes within the database tier of a database system may present database services and / or other types of data storage services that employ the distributed storage systems described herein to clients as network-based services. In some embodiments, a network-based service may be implemented by a software and / or hardware system designed to support interoperable machine-to-machine interaction over a network. A web service may have an interface described in a machine-processable format, such as the Web Services Description Language (WSDL). Other systems may interact with the networkbased service in a manner prescribed by the description of the network-based service’s interface. For example, the network-based service may define various operations that other systems may invoke, and may define a particular application programming interface (API) to which other systems may be expected to conform when requestingthe various operations.

[0030] In various embodiments, a network-based service may be requested or invoked through the use of a message that includes parameters and / or data associated with the network-based services request. Such a message may be formatted according to a particular markup language such as Extensible Markup Language (XML), and / or may be encapsulated using a protocol such as Simple Object Access Protocol (SOAP). To perform a network-based services request, a network-based services client may assemble a message including the request and convey the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the web service, using an Internet-based application layer transfer protocol such as Hypertext Transfer Protocol (HTTP). In some embodiments, network-based services may be implemented using Representational State Transfer (REST) techniques rather than message-based techniques. For example, a network-based service implemented according to a REST technique may be invoked through parameters included within an HTTP method such as PUT, GET, or DELETE.

[0031] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of the exemplary methods and materials are described herein. It will be apparent to those skilled in the art that many more modifications are possible without departing from the inventive concepts herein.

[0032] All terms used herein should be interpreted in the broadest possible mannerconsistent with the context. When a grouping is used herein, all individual members of the group and all combinations and sub-combinations possible of the group are intended to be individually included. When a range is stated herein, the range is intended to include all subranges and individual points within the range. All references cited herein are hereby incorporated by reference to the extent that there is no inconsistency with the disclosure of this specification.

[0033] The present invention has been described with reference to certain preferred and alternative embodiments that are intended to be exemplary only and not limiting to the full scope of the present invention, as set forth in the claims of a subsequent nonprovisional patent application claiming priority to this application.

Claims

CLAIMS:1 . A system for ingesting security audit logs from a cloud-based data warehousing platform into a Security Ingestion and Event Management (SIEM) platform, comprising: a connector installed on the cloud-based data warehousing platform; and a data transfer module installed on the SIEM platform, wherein the connector is configured to retrieve a plurality of security audit logs from the cloud platform and transmit the plurality of security audit logs to the data transfer module across a network, and further wherein the data transfer module is configured to process the plurality of security audit logs and send the plurality of security audit logs to a storage location which is then ingested into the SIEM.

2. The system of claim 1 , wherein the connector is further configured to retrieve the plurality of security audit logs at a specified interval.

3. The system of claim 1 , wherein the data transfer module is further configured to parse and normalize the plurality of security audit logs before sending the security audit logs to the storage location and which is then ingested into the SIEM platform.

4. The system of claim 1 , wherein the cloud-based data warehousing system is a Snowflake system.

5. The system of claim 4, further comprising an infrastructure-as-code (laC) tool configured to manage a first account for the Snowflake system within the SIEM platform and to further manage at least one additional account within the SIEM platform.

6. The system of claim 1 , wherein the data transfer module is further configured to collect partial data sets from the plurality of security audit logs from the connector.

7. The system of claim 6, wherein the data transfer module is further configured to consolidate the partial data sets into a complete data set from the plurality of security audit logs from the connector.

8. The system of claim 7, wherein the data transfer module is further configured to transform the complete data set into universal application programming interface (API) format.

9. The system of claim 8, wherein the universal API format comprises JavaScript Object Notation Lines (JSONL).

10. The system of claim 9, wherein the data transfer module is further configured to export the complete data set into a cloud provider bucket.

11. A method for exporting security logs from a cloud-based data warehousing platform to a Security Information and Event Management (SIEM) platform, comprising: at a connector in the cloud-based data warehousing platform, retrieving a plurality of security audit logs from a database table in the cloud-based data warehousing platform; sending the plurality of security audit lots from the connector to cloud storage which then is ingested using a data transfer module at the SIEM platform; parsing and normalizing the security audit logs in the SIEM platform for use in security monitoring and analysis; and storing the parsed and normalized plurality of security audit logs in a storagearea at the SIEM platform.

12. The method of claim 11 , further comprising the step of retrieving the plurality of security audit logs with the connector according to a specified schedule.

13. The method of claim 11 , further comprising the step of, at the data transfer module, parsing and normalizing the plurality of security audit logs before sending the security audit logs to the storage area in the SIEM platform.

14. The method of claim 11 , wherein the cloud-based data warehousing system is a Snowflake system.

15. The method of claim 14, further comprising the step of managing the Snowflake system through an infrastructure-as-code (laC) tool.

16. The method of claim 11 , further comprising the step of, at the data transfer module, collecting partial data sets from the plurality of security audit logs from the connector.

17. The method of claim 16, further comprising the step of, at the data transfer module, consolidating the partial data sets into a complete data set from the plurality of security audit logs from the connector.

18. The method of claim 17, further comprising the step of, at the data transfer module, transforming the complete data set into universal application programming interface (API) format.

19. The method of claim 18, wherein the universal API format comprises JavaScript Object Notation Lines (JSONL).

20. The method of claim 19, further comprising the step of, at the data transfer module, exporting the complete data set into a cloud provider bucket.

Images