Method and system for filtering sensitive data in cloud-native application logs

Abstract

This application relates to filtering sensitive data in logs of cloud-native applications. A computer-implemented method for limiting access to sensitive information by filtering log files. The method includes deploying a first container pod on a node of a cloud computing system, where the first container pod includes a first container configured to run an application. The method also includes generating a first log file for the first container, where the first log file includes a set of actions performed by the application over a period of time. The method further includes filtering the first log file by a filter, where the filter is configured to remove a type of sensitive data from the first log file. The method includes exporting the first log file to the node in response to the filtering. Advantageously, this can prevent parties from accessing sensitive data contained in the log file.

Description

Technical Field

[0001] This disclosure relates to cloud computing, and more specifically, to filtering sensitive information from cloud-native application logs. Background Technology

[0002] Most modern applications include logging mechanisms. Logs track and record actions that occur within an application during operation. Logs are particularly useful for troubleshooting and monitoring cluster activity. For example, a single application can generate standard output logs and standard error stream logs. Each log can be configured to log a dataset based on the application's configuration. Standard output logs can include information about usage patterns, activities, operating system operations, server operations, application usage, etc. Error logs can record errors encountered by the application that prevent proper execution or indicate exception handling. Data may include corrupted / missing files / tables, configuration errors, and other similar data.

[0003] Containers and container orchestration are used to efficiently and effectively accomplish cloud computing tasks. A container is a ready-to-run software package that can be sent from a host and run on a node. Each container can include all the features required to run an application. Containers can be mirrored onto any set of hardware capable of running the software contained within them. Each container is contained within a container pod. Each container pod can include more than one container. In some embodiments, a container pod can be defined as a self-contained deployable unit managed by a container orchestration solution. Containers and / or cloud-native applications can output one or more logs in one or more log streams.

[0004] In the current system, when containers output log files, these files may include sensitive data and / or other data that the data owner wants to keep private. The log files are stored outside the container pod and on the system storage of the node on which the container is running. When stored on the node, the data is potentially accessible to containers operating in different pods on the same node and / or to user accounts that have access to view data stored on the node (specifically, log files). Therefore, sensitive data in the log files can potentially be accessed by parties who otherwise do not have access rights or permission to view the data. It is necessary to reduce or eliminate the possibility of sensitive data in the log files being viewed by unauthorized parties. Summary of the Invention

[0005] A computer-implemented method is disclosed for restricting access to sensitive information by filtering log files. The method includes deploying a first container pod on a node of a cloud computing system, wherein the first container pod includes a first container configured to run an application. The method also includes generating a first log file for the first container, wherein the first log file includes a set of actions performed by the application over a period of time. The method further includes filtering the first log file using a filter configured to remove a type of sensitive data from the first log file. The method includes exporting the first log file to the node in response to the filtering. Advantageously, this prevents parties from accessing sensitive data contained in the log file.

[0006] Additionally, the method can be configured such that the filter includes one or more providers, with a first provider configured to remove sensitive data of one type and a second provider configured to remove sensitive data of a second type. This allows filtering multiple log types and multiple types of sensitive data. Another optional feature is that the filter includes a user interface configured to receive one or more filter scripts from the data owner, where each filter script is configured to remove one type of data from the first log file.

[0007] Advantageously, this allows data owners to define and update the data types to be removed from individual log files. Other aspects of this disclosure relate to computer program products that incorporate functionality consistent with the methods described above.

[0008] The present invention also discloses a system for filtering log files. The system includes: a processor; a node of a cloud computing system configured to operate one or more container pods received from one or more hosts; and a computer-readable storage medium communicatively coupled to the processor and storing program instructions executable by the processor. The program instructions, when executed, are configured to cause the processor to receive a first container pod from the node of the cloud computing system. The first container pod includes a first container configured to run an application. The first container pod also generates a first log file for the first container, wherein the first log file includes a set of actions performed by the application over a period of time. Further, the first container pod filters the first log file using a filter configured to remove a type of sensitive data from the first log file. Moreover, the first container pod exports the first log file to the node in response to the filtering of the first log file. Advantageously, the system can prevent or limit the opportunity for third parties to access sensitive data stored in the log file because the sensitive data is removed.

[0009] This invention is not intended to illustrate every aspect, every implementation and / or every embodiment of the present disclosure. Attached Figure Description

[0010] This document describes various embodiments with reference to different subjects. Specifically, some embodiments may be described with reference to methods, while others may be described with reference to apparatuses and systems. However, those skilled in the art will conclude from the above and below description that, unless otherwise indicated, any combination of features related to different subjects (specifically, features of methods) and features of apparatuses and systems, in addition to any combination of features belonging to one type of subject, is also considered to be disclosed in this document.

[0011] The aspects defined above, and other aspects disclosed herein, are clear from examples of one or more embodiments described below and are explained with reference to those examples, but the invention is not limited thereto. Various embodiments are described by way of example only and with reference to the following figures:

[0012] Figure 1 A cloud computing environment according to an embodiment of the present invention is described.

[0013] Figure 2 An abstract model layer according to an embodiment of the present invention is described.

[0014] Figure 3 This is a block diagram of a data processing system (DPS) according to one or more embodiments disclosed herein.

[0015] Figure 4 A functional diagram illustrating a computing environment suitable for the operation of a log file filter according to some embodiments of the present disclosure is shown.

[0016] Figure 5 A flowchart is shown of an example method for filtering log files according to some embodiments of this disclosure.

[0017] Figure 6A The illustration shows an example of generating container files using a log filter.

[0018] Figure 6B An example of a filter that can be applied to log files is shown. Detailed Implementation

[0019] General cloud computing

[0020] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings set forth herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0021] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing power, memory, storage devices, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0022] The characteristics are as follows:

[0023] On-demand self-service: Cloud consumers can unilaterally and automatically supply computing power, such as server time and network storage, as needed, without requiring manual interaction with the service provider.

[0024] Extensive network access: Capabilities are available on the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin or thick client platforms, such as mobile phones, laptops, and personal digital assistants (PDAs).

[0025] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is a sense of location agnosticness because consumers typically do not control or know the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0026] Rapid and flexible: Capacity can be automatically and rapidly supplied in some situations to quickly expand outwards, and rapidly released to quickly expand inwards. For consumers, the capacity available for supply often appears unlimited and can be purchased at any time and in any quantity.

[0027] Measurement services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the providers and consumers of the services being utilized.

[0028] The service model is as follows:

[0029] Software as a Service (SaaS): The capability offered to consumers is the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from various client devices through a thin client interface such as a web browser (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0030] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created or acquired by the consumer onto cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and potentially over the configuration of the application hosting environment.

[0031] Infrastructure as a Service (IaaS): This provides consumers with the capability to deliver processing, storage, networking, and other basic computing resources that enable them to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do have control over the operating system, storage, deployed applications, and possibly limited control over chosen networking components (e.g., host firewalls).

[0032] The deployment model is as follows:

[0033] Private cloud: Cloud infrastructure operated solely by an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0034] Community cloud: Cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0035] Public cloud: Cloud infrastructure available to the general public or large industrial groups and owned by organizations that sell cloud services.

[0036] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0037] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0038] Now for reference Figure 1The diagram illustrates an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 to which local computing devices used by cloud consumers can communicate, such as personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software-as-a-service without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 1 The types of computing devices 54A-N shown are for illustrative purposes only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device on any type of network and / or network-addressable connection (e.g., using a web browser).

[0039] Now for reference Figure 2 This demonstrates a cloud computing environment of 50 ( Figure 1 This provides a set of functional abstractions. It should be understood beforehand that... Figure 2 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0040] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) based server 62; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0041] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage device 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0042] In one example, management layer 80 may provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost tracking for utilizing resources within the cloud computing environment, as well as billing or invoicing for consuming these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, and protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to meet required service levels. Service Level Agreement (SLA) Planning and Fulfillment 85 provides pre-scheduling and procurement of cloud resources, where future needs are anticipated according to the SLA.

[0043] The workload layer 90 provides examples of functionalities that can be leveraged in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis and processing 94; transaction processing 95; and log file filtering 96.

[0044] General data processing systems

[0045] Figure 3 This is a block diagram of an example data processing system (DPS) according to one or more embodiments. The DPS can be used as a cloud computing node 10. In this illustrative example, the DPS 100 may include a communication bus 102 that can provide communication between a processor unit 104, a memory 106, a persistent storage device 108, a communication unit 110, an input / output (I / O) unit 112, and a display 114.

[0046] Processor unit 104 is used to execute instructions for software that can be loaded into memory 106. Processor unit 104 may be multiple processors, a multi-core processor, or some other type of processor, depending on the specific implementation. As used in the referenced items herein, multiple means one or more items. Furthermore, processor unit 104 may be implemented using multiple heterogeneous processor systems, in which the master processor and the secondary processor reside on a single chip. As another illustrative example, processor unit 104 may be a symmetric multiprocessor system containing multiple processors of the same type.

[0047] Memory 106 and persistent storage device 108 are examples of storage device 116. A storage device can be any hardware capable of temporarily and / or permanently storing information (e.g., but not limited to, data, program code in functional form, and / or other suitable information). In these examples, memory 106 can be, for example, random access memory or any other suitable volatile or non-volatile storage device. Persistent storage device 108 can take various forms depending on the specific implementation.

[0048] For example, persistent storage device 108 may include one or more components or devices. For example, persistent storage device 108 may be a hard disk drive, flash memory, rewritable optical disk, rewritable magnetic tape, or a combination thereof. The media used in persistent storage device 108 may also be removable. For example, a removable hard disk drive may be used in persistent storage device 108.

[0049] In these examples, communication unit 110 can provide communication with other DPS or devices. In these examples, communication unit 110 is a network interface card. Communication unit 110 can provide communication using either or both physical and wireless communication links.

[0050] Input / output unit 112 allows for the input and output of data to other devices that can be connected to DPS 100. For example, input / output unit 112 can provide connectivity for user input via a keyboard, mouse, and / or other suitable input devices. Furthermore, input / output unit 112 can send output to a printer. Display 114 provides a mechanism for displaying information to the user.

[0051] Instructions for operating systems, applications, and / or programs may reside in storage device 116, which communicates with processor unit 104 via communication bus 102. In these illustrative examples, the instructions are in functional form on persistent storage device 108. These instructions may be loaded into memory 106 for execution by processor unit 104. Processes in different embodiments may be executed by processor unit 104 using computer-implemented instructions, which may reside in memory (such as memory 106).

[0052] These instructions are referred to as program code, computer-usable program code, or computer-readable program code that can be read and executed by the processor in processor unit 104. The program code in different embodiments may be implemented on different physical or tangible computer-readable media, such as on memory 106 or persistent storage device 108.

[0053] Program code 118 may be functionally located on selectively removable computer-readable medium 120 and may be loaded onto or transferred to DPS 100 for execution by processor unit 104. In these examples, program code 118 and computer-readable medium 120 may form computer program product 122. In one example, computer-readable medium 120 may be computer-readable storage medium 124 or computer-readable signal medium 126. Computer-readable storage medium 124 may include, for example, an optical disc or disk that is inserted into or placed into a drive or other device that is part of persistent storage device 108 for transfer to a storage device (such as a hard disk drive) that is part of persistent storage device 108. Computer-readable storage medium 124 may also take the form of a persistent storage device connected to DPS 100, such as a hard disk drive, thumb drive, or flash memory. In some instances, computer-readable storage medium 124 may not be removable from DPS 100.

[0054] Alternatively, program code 118 can be transmitted to DPS 100 using computer-readable signal medium 126. Computer-readable signal medium 126 can be, for example, a propagated data signal containing program code 118. For example, computer-readable signal medium 126 can be an electromagnetic signal, an optical signal, and / or any other suitable type of signal. These signals can be transmitted via a communication link (such as a wireless communication link, fiber optic cable, coaxial cable, wire, and / or any other suitable type of communication link). In other words, in the illustrative example, the communication link and / or connection can be physical or wireless.

[0055] In some illustrative embodiments, program code 118 may be downloaded from another device or DPS via a network to persistent storage device 108 for use within DPS 100 via computer-readable signal medium 126. For example, program code stored in a computer-readable storage medium in a server DPS may be downloaded from a server to DPS 100 via a network. The DPS providing program code 118 may be a server computer, a client computer, or some other device capable of storing and transmitting program code 118.

[0056] The illustrations of different components for DPS 100 are not intended to provide an architectural limitation on how different embodiments can be implemented. Different illustrative embodiments can be implemented in the DPS, which includes components other than or in lieu of those shown for DPS 100. Figure 1 Other components shown.

[0057] This disclosure relates to cloud computing, and more specifically, to filtering sensitive information from logs of cloud-native applications. While this disclosure is not necessarily limited to such applications, its various aspects can be understood through the discussion of various examples in this context.

[0058] Most modern applications include logging mechanisms. Logs track and record actions that occur within an application during operation. Logs are particularly useful for troubleshooting and monitoring cluster activity. Various applications can generate multiple log types and any number of individual logs. For example, a single application can generate standard output logs and standard error stream logs. Each log can be configured to record (non-exclusive) datasets based on the application's configuration. Standard output logs can include information about usage patterns, activities, operating system operations, server operations, application usage, etc. Error logs can record errors encountered by the application that prevent correct execution. Data can include corrupted / missing files / tables, configuration errors, and other similar data.

[0059] Leverage containers and container orchestration to efficiently and effectively accomplish cloud computing tasks. A container is a ready-to-run software package that can be sent from a host and run on a node. Each container can include all the features required to run an application. Containers can be mirrored onto any set of hardware capable of running the software contained within them. Each container is contained within a container pod. Each container pod can include more than one container. In some embodiments, a container pod can be defined as a self-contained, deployable unit managed by a container orchestration solution.

[0060] A container orchestration solution (“container manager”) can be designed to distribute containers / container pods to one or more remote compute nodes (or “nodes”). Various embodiments allow for specifying constraints and / or rules on how and when container pods are distributed and executed on one or more different nodes. This distribution can be based on several factors. Factors may include CPU type, node type, disk type, node hardware, container pod configuration, node configuration, and other similar factors.

[0061] Applications running within containers can also generate various log files. As part of the cloud-native architecture used for various applications, log files are stored on nodes. When on a node, logs can be accessible to other containers on the same host and / or various user accounts with appropriate permission levels. Sometimes, there may be sensitive, confidential, or other data that the data owner wants to keep private.

[0062] Embodiments of this disclosure include log filters. Embodiments of this disclosure can restrict access to any type of data when it can be accessed in other ways. It provides data owners with the opportunity to have greater control over their data to comply with organizational and / or government regulations. This can thereby make cloud computing more secure.

[0063] Embodiments of this disclosure may include a log manager. The log manager may be configured to generate and / or filter one or more log types for an application running in a container. In some embodiments, the log manager may include one or more log filters. Log filters may be implemented to remove any data that the data owner does not want others to access before the logs are stored on the node. In some embodiments, log filters may be built into the application container. In some embodiments, filters may be built into a sidecar container. Any number of unique filters may be added to the filtering module to remove any type of data. In some embodiments, the same and / or different filters may exist for different log types.

[0064] The advantages described above are exemplary advantages, and there are embodiments that may include all of the advantages described above, some of the advantages described above, or none of the advantages described above while remaining within the spirit and scope of this disclosure.

[0065] Reference will now be made in more detail to the various embodiments of this disclosure. Figure 4 This is a representation of a computing environment 400 capable of running container clients according to one or more embodiments of the present disclosure. Many modifications can be made to the described environment by those skilled in the art without departing from the scope of the present disclosure.

[0066] Computing environment 400 includes host 410, node 430, and network 450. Network 450 may be, for example, a telecommunications network, a local area network (LAN), a wide area network (WAN) (such as the Internet), or a combination of all three, and may include wired, wireless, or fiber optic connections. Network 450 may include one or more wired and / or wireless networks capable of receiving and transmitting data, voice, and / or video signals (including multimedia signals containing voice, data, and video information). Generally, network 450 may be any combination of connections and protocols that support communication between host 410, node 430, and other computing devices within computing environment 400 (such as additional nodes not shown). In some embodiments, each of host 410 and / or node 430 may include a computer system, such as... Figure 3 Data processing system 100.

[0067] Host 410 may be a standalone computing device, management server, web server, mobile computing device, or any other electronic device or computing system capable of receiving, sending, and processing data. In other embodiments, host 410 may represent a server computing system, such as one utilizing multiple computers as server systems in a cloud computing environment (e.g., cloud computing environment 50). In some embodiments, host 410 includes a container manager 412 and an application 416.

[0068] Container manager 412 can be any combination of hardware and / or software configured to operate the lifecycle of containers (e.g., application container 442). In some embodiments, container manager 412 controls and automates tasks, including but not limited to container provisioning and deployment, container redundancy and availability, resource allocation between containers, movement of containers across host infrastructure, and load balancing between containers and / or nodes 430. In some embodiments, container manager 412 includes a container orchestration system (e.g., In some embodiments, application 416 is included in container manager 412. They are shown separately for purposes of description. In some embodiments, container manager 412 identifies container pods and assigns them to nodes. The identified container pods may be newly created container pods and / or terminated / failed container pods (e.g., node failure).

[0069] Application 416 can be any combination of hardware and / or software configured to perform functions on a computing device (e.g., host 410). In some embodiments, application 416 is a web application. In some embodiments, application 416 can be wrapped in one or more container pods. In some embodiments, application 416 can represent any number of individual applications. These applications can be combined / grouped into one or more container pods or containers. In some embodiments, application 416 can initiate the creation of container pods.

[0070] Node 430 can be any combination of hardware and / or software configured to run one or more container pods. In some embodiments, node 430 can be a standalone computing device, management server, web server, mobile computing device, or any other electronic device or computing system capable of receiving, sending, and processing data. In some embodiments, node 430 can represent, for example, a server computing system utilizing multiple computers as server systems in a cloud computing environment. In some embodiments, node 430 can support two or more container pods from two or more different hosts. These two hosts can be managed by different parties utilizing node 430. In some embodiments, computing environment 400 can include one or more additional nodes. Additional nodes may be consistent with node 430 and may support container pods / containers from various hosts. All container pods may be managed by a common container manager. In some embodiments, node 430 includes container client 431, logging agent 436, log rotator 437, log storage device 438, and container pod 440.

[0071] Container client 431 can be any combination of hardware and / or software configured to run containers on a node. In some embodiments, container client 431 works complementarily to container manager 412. Container client 431 can be a node agent and / or container runtime agent running on each node (e.g., As (A proxy). Container client 431 can interface with container manager 412 to receive and execute containers as instructed by container manager 412. Container client 431 can receive one or more container pods from container manager 412 and / or a computing device separate from host 410.

[0072] The logging agent 436 can be any combination of hardware and / or software configured to generate and log files. In some embodiments, logs are stored in log storage device 438. In some embodiments, the logging agent 436 can be node-level and / or cluster-level (node ​​cluster). In some embodiments, the logging agent 436 generates logs based on instructions included in application container 442 and / or container client 431.

[0073] Log rotator 437 can be any combination of hardware and / or software configured to rotate logs. In some embodiments, log rotator 437 can remove old logs. Removing / rotating logs can limit the amount of storage used by the logs. In some embodiments, logs can be rotated based on instructions from an application, container pod, and / or container. Rotation can be continuous (e.g., maintaining a fixed number of log cycles) and / or occur after a predetermined flip. Events can be time periods, cycle counts, errors, container pod or container startup / shutdown, etc.

[0074] Log storage device 438 may be a collection of one or more log files currently stored on node 430. Any number of individual files may exist. In some embodiments, log storage device 438 includes logs for all or some containers / container pods running on node 430. Each application may have a unique log stream. In some embodiments, log storage device 438 may include two or more types of logs. For example, standard output logs and error logs may exist for each application. In some embodiments, log storage device 438 may be accessible by the appropriate credential account on node 430 and / or any container location on node 430. In some embodiments, log storage device 438 includes filtered log files. In some embodiments, log storage device 438 is located outside any containers and / or container pods operating on node 430.

[0075] Container pod 440 may be a virtual structure configured to transfer containers between a host (e.g., host 410) and a node (e.g., node 430). In some embodiments, container pod may include one or more containers. In the depicted embodiment, container pod 440 includes a container client 431, an application container 442, a sidecar container 443, and a log filter 444.

[0076] Application container 442 can be any combination of hardware and / or software configured to run an application on a remote node. The container can be a software package containing the necessary instructions and / or data for performing a specified task. It may include the runtime, all system libraries, and application libraries required to fully complete the application's task. In some embodiments, application container 442 is configured to generate one or more log types. Log types may include standard output logs, error logs, and other similar logs. Each log may include relevant information about the operations of the application within application container 442. In some embodiments, log files include a set of actions performed by the application over a period of time. This period may be based on time, the number of actions, and / or file size. For example, each log file may have a similar amount of data.

[0077] Sidecar container 443 can be any combination of hardware and / or software configured to provide additional functionality to application container 442. Sidecar container 443 can extend and / or enhance the functionality of application container 442 without requiring changes to / updates to application container 442. In some embodiments, sidecar container 443 can be configured to run in the same container pod as application container 442. In some embodiments, sidecar container 443 can include one or more of logging agent 436 and log rotator 437. In some embodiments, sidecar container 443 can be a streaming container. The streaming container can be configured to stream data from application container 442 and / or application 416. In some embodiments, sidecar container 443 can include log filter 444. Container pod 440 can include any number of sidecar containers configured to run alongside application container 442.

[0078] Log filter 444 can be any combination of hardware and / or software configured to remove data from log files. In some embodiments, the data can be sensitive data. Sensitive data can be any data that the data owner deems sensitive. Sensitive data can include usernames, passwords, emails, application data, specific words, workload data, and other types of data.

[0079] In some embodiments, log filter 444 may include one or more different providers. Each provider may identify and filter different types of data / different data fragments. Log filter 444 may pass a log file through each of the series of providers. In some embodiments, each provider may include a script configured to identify data fragments and remove / modify those data fragments. In some embodiments, each provider and each filter script may identify the type of sensitive data and replace the type of sensitive data with a generic value. The generic value may be a null value, an indication of specific filtering operations (e.g., placing "REMOVED" in the log), or a generic term (e.g., placing "email address" in the log). In some embodiments, log filter 444 outputs a log file with filtered data, or outputs a filtered log.

[0080] In some embodiments, log filter 444 can be implemented at any point before the log file is saved on node 430. Log filter 444 can be implemented on the output of application container 442 and / or the output of container pod 440. The output of application container 442 can be the input of sidecar container 443. In some embodiments, log filter 444 can be utilized by adding a log filter driver to another markup language "YAML" file that defines the container. The YAML file can be a human-readable file suitable for creating (e.g., for containers) configuration files. The driver can be instructions regarding the name and path used to locate the filter and / or provider.

[0081] Figure 5 A flowchart is depicted of an example method (method 500) for filtering log files that can be executed in a computing environment (e.g., computing environment 400 and / or cloud computing environment 50). According to various embodiments of this disclosure, one or more of the aforementioned advantages and improvements for filtering log files can be achieved through method 500.

[0082] Method 500 may be implemented by one or more processors, host 410, container manager 412, application 416, node 430, container client 431, logging agent 436, log rotator 437, log storage device 438, container pod 440, application container 442, sidecar container 443, log filter 444, and / or different combinations of hardware and / or software. In various embodiments, the various operations of method 500 are executed by one or more of host 410, container manager 412, application 416, node 430, container client 431, logging agent 436, log rotator 437, log storage device 438, container pod 440, application container 442, sidecar container 443, and log filter 444. For illustrative purposes, method 500 will be described as being executed by container client 431.

[0083] In operation 502, container client 431 deploys a container on a node (e.g., node 430). In some embodiments, the container is deployed in a container pod. A container pod may include additional containers and / or one or more sidecar containers. In some embodiments, the node on which the container is deployed includes at least one additional container pod. Additional container pods may be deployed from a second / standby host. Additional container pods may be managed by a different organization than the container pod. In some embodiments, additional container pods may be based on a separate YAML file. In some embodiments, they may be accessible by one or more authorized accounts. Access allows retrieval of data stored outside the container pod / container.

[0084] In some embodiments, operation 502 includes defining one or more filters. Filters may include one or more providers. Filters may include log filter directives with containers / container pods. These directives may be included in the container's YAML file.

[0085] In operation 504, container client 431 generates log files. In some embodiments, container client 431 generates two or more streams of log files. Standard logs, error logs, and any number of additional logs may exist. Streams may include sequences of generated logs. Logs may include information about the application's operations. In some embodiments, separate outputs may exist for each type of log file.

[0086] In operation 506, container client 431 filters log files. Log files can be filtered before they exit the container pod and / or before they are stored on the node. In some embodiments, the filter is implemented as the container's output. The filter can be implemented before the input to the sidecar container. In some embodiments, the filter is implemented at the container pod's exit.

[0087] In some embodiments, log files are passed through a filtering component (e.g., log filter 444). The filtering component may have one or more providers. Each provider may be configured to identify and remove a type of data. The type may be a general type, a specific word, etc. For example, the type may include email address, username, password, secret word, security information, name, date, etc. In some embodiments, the log file is passed through each provider sequentially. For example, the output of the first provider is the input of the second provider, the output of the second provider is input to the third provider, and so on. Filtered logs are output after the last provider in the series. In some embodiments, each log stream has inputs and outputs to the filtering component. In some embodiments, each log stream passes through the same filtering component and all the same providers. In some embodiments, each log stream may pass through a different set of providers. Each set may have some overlap. For example, a standard log stream may pass through filters A and B, and an error log stream may pass through providers A and C.

[0088] In operation 508, container client 431 exports log files. In some embodiments, the exported log files are filtered so that any filtered data is not exposed if they are accessed by different container pods of the node and / or authorized users. Logs can still be used for troubleshooting, error correction, and all other standard functions for logging.

[0089] Figure 6A Includes document 600. Document 600 includes an embodiment of a YAML file for generating a container pod with filtering capabilities. Figure 6B Includes filter 650. Filter 650 includes one embodiment of a filter that can be applied to log files. Filter 650 can remove and replace various parts of the log file to protect them from unwanted access.

[0090] Computer technology and computer-readable media

[0091] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0092] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or raised structures in recesses on which instructions are recorded, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0093] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0094] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., Smalltalk, C++, etc.) and conventional procedural programming languages ​​(e.g., the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing state information from the computer-readable program instructions.

[0095] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0096] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0097] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function(s). In some alternative embodiments, the functions mentioned in the blocks may occur in a non-linear order as shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0099] Various embodiments of this disclosure have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles of the embodiments, their practical application, or technical improvements to existing technologies on the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0100] In summary, various embodiments have been discussed, which are specified again in the following numbered clauses:

[0101] Clause 1 is as follows. A computer-implemented method includes: deploying a first container pod on a node of a cloud computing system, wherein the first container pod includes a first container configured to run an application; generating a first log file for the first container, wherein the first log file includes a set of actions performed by the application over a period of time; filtering the first log file with a filter, wherein the filter is configured to remove a type of sensitive data from the first log file; and exporting the first log file to the node in response to the filtering.

[0102] Clause 2 is as follows. As described in Clause 1, the export includes storing the first log file on the node.

[0103] Clause 3 is as follows. As described in Clause 1 or 2, wherein the second container in the second container pod has access to data stored on the node.

[0104] Clause 4 is as follows. The method as described in any of the preceding clauses, wherein the second container in the second container pod has access to data stored on the node.

[0105] Clause 5 is as follows. The method as described in any of the preceding clauses, wherein the first log file stored on the node is accessible by the node's user account.

[0106] Clause 6 is as follows. The method as described in any of the preceding clauses further includes: generating a second log file, wherein the first log file is a standard output log and the second log file is a standard error log.

[0107] Clause 7 is as follows. The method as described in any of the preceding clauses, wherein the filter comprises one or more providers, and a first provider is configured to remove sensitive data of that type, and a second provider is configured to remove sensitive data of a second type.

[0108] Clause 8 is as follows. The method as described in any of the preceding clauses, wherein the first log file and the second log file are filtered by a common set of providers.

[0109] Clause 9 is as follows. The method as described in any of the preceding clauses, wherein the first log file is filtered by a first provider, and the second log file is filtered by a second provider.

[0110] Clause 10 is as follows. The method as described in any of the preceding clauses, wherein the filter includes a user interface configured to receive one or more filter scripts from the data owner, wherein each filter script is configured to remove one type of data from the first log file.

[0111] Clause 11 is as follows. The method as described in any of the preceding clauses, wherein each filter script is capable of identifying a type of sensitive data and replacing that type of sensitive data with a generic value.

[0112] Clause 12 is as follows. A system includes: a processor; a node of a cloud computing system configured to operate one or more container pods received from one or more hosts; and a computer-readable storage medium communicatively coupled to the processor and storing program instructions, which, when executed by the processor, are configured to cause the processor to: receive a first container pod destined for a node of the cloud computing system, wherein the first container pod includes a first container configured to run an application; generate a first log file for the first container, wherein the first log file includes a set of actions performed by the application over a period of time; filter the first log file through a filter, wherein the filter is configured to remove a type of sensitive data from the first log file; and, in response to the filtering of the first log file, export the first log file to the node.

[0113] Clause 13 is as follows. The system as described in Clause 12, wherein a node is configured to, in response to the export of a first log file, store a set of log files generated on the node, including the first log file.

[0114] Clause 14 is as follows. A system as described in Clauses 12 and 13, wherein a node is operating a second container pod from a second host, and the second container in the second host has access to a set of log files.

[0115] Clause 15 is as follows. In the system described in Clauses 12 through 14, a set of log files is accessible to accounts authorized to access the nodes.

[0116] Clause 16 is as follows. A system as described in Clauses 12 through 15, wherein the filter comprises a series of two or more filter scripts, and each filter script is configured to remove different types of data from the first log.

[0117] Clause 17 is as follows. A computer program product includes a computer-readable storage medium having program instructions contained therein, the program instructions being executable by a processing unit to cause the processing unit to: deploy a first container pod on a node of a cloud computing system, wherein the first container pod includes a first container configured to run an application; generate a first log file for the first container, wherein the first log file includes a set of actions performed by the application over a period of time; filter the first log file through a filter, wherein the filter is configured to remove a type of sensitive data from the first log file; and in response to the filtering, export the first log file to the node.

[0118] Clause 18 is as follows. A computer program product as described in Clause 17, wherein the program instructions are further configured to cause the processing unit to: generate a second log file, wherein the first log file is a standard output log and the second log file is a standard error log.

[0119] Clause 19 is as follows. A computer program product as described in Clauses 17 and 18, wherein the filter comprises one or more providers, and a first provider is configured to remove sensitive data of that type, and a second provider is configured to remove sensitive data of a second type.

[0120] Clause 20 is as follows. For computer program products as described in Clauses 17 through 19, wherein the first log file and the second log file are filtered by a common set of providers.

Claims

1. A computer-implemented method, comprising: The first container pod is deployed on a node of the cloud computing system, wherein the first container pod includes a first container configured to run an application. Deploy the second container pod on the node; Generate a first log file for the first container, wherein the first log file includes a set of actions performed by the application over a period of time; The first log file is filtered by a filter, wherein the filter is configured to remove one type of sensitive data from the first log file; In response to the filtering, the first log file is exported from the first container pod to a log storage device on the node, wherein the log storage device is outside the first container pod, and wherein the second container in the second container pod has access to the log storage device on the node; as well as In response to an event, the first log file is rotated from the log storage device, wherein the event is closing the first container pod, and the rotation includes removing the first log file from the node.

2. The computer-implemented method according to claim 1, wherein, The export includes storing the first log file on the log storage device.

3. The computer-implemented method according to claim 1, wherein, The first log file stored on the log storage device can be accessed by the user account of the node.

4. The computer-implemented method according to claim 1 further includes: Generate a second log file, wherein the first log file is the standard output log and the second log file is the standard error log.

5. The computer-implemented method according to claim 4, wherein, The filter includes one or more providers, with a first provider configured to remove sensitive data of the first type and a second provider configured to remove sensitive data of the second type.

6. The computer-implemented method according to claim 5, wherein, The first log file and the second log file are filtered by a set of common providers.

7. The computer-implemented method according to claim 5, wherein, The first log file is filtered by the first provider, and the second log file is filtered by the second provider.

8. The computer-implemented method according to claim 1, wherein, The filter includes a user interface configured to receive one or more filter scripts from the data owner, wherein each filter script is configured to remove one type of data from the first log file.

9. The computer-implemented method according to claim 8, wherein, Each filter script can identify a type of sensitive data and replace that type of sensitive data with a generic value.

10. A computer system, comprising: processor; A node in a cloud computing system is configured to operate one or more container pods received from one or more hosts, wherein the node includes a log storage device. as well as A computer-readable storage medium communicatively coupled to the processor and storing program instructions that, when executed by the processor, are configured to cause the processor to: Receives a first container pod from a plurality of container pods destined for the node in the cloud computing system, wherein the first container pod: This includes the first container configured to run the application; Generate a first log file for the first container, wherein the first log file includes a set of actions performed by the application over a period of time; The first log file is filtered by a filter, wherein the filter is configured to remove one type of sensitive data from the first log file; as well as In response to the filtering of the first log file, the first log file is exported from the first container pod to the log storage device on the node, wherein the log storage device is outside the first container pod; Receive a second container pod from a plurality of container pods destined for the node of the cloud computing system, wherein a second container in the second container pod is able to access the log storage device on the node; as well as In response to an event, the first log file is rotated from the log storage device, wherein the event is closing the first container pod, and the rotation includes removing the first log file from the node.

11. The computer system according to claim 10, wherein, The node is configured to store a set of log files generated on the node in response to the export of the first log file, the set of log files including the first log file.

12. The computer system according to claim 11, wherein, The set of log files can be accessed by accounts authorized to access the node.

13. The computer system according to claim 10, wherein, The filter comprises a series of two or more filter scripts, and each filter script is configured to remove different types of data from the first log.

14. A computer program product comprising program instructions executable by a processing unit to cause the processing unit to perform the method according to any one of claims 1 to 9.

Images