Preventing unauthorized package deployment in a cluster

Abstract

A method for verifying the integrity of an object deployed in a cluster is provided. The method detects a resource creation request. In response to the request being an initial resource creation request for the object, the method verifies the integrity of the object based on properties in the request to create a release secret in the cluster if the integrity verification results in a positive integrity verification for the object. The release secret represents a specific deployment configuration of the object on the cluster. In response to the request not being an initial resource request, the method verifies whether the request corresponds to a specific deployment configuration of the object by matching it with a release secret in the cluster. In response to the request corresponding to a deployment of the object and the release secret existing in the cluster, the method creates the resource requested by the request in the cluster.

Description

[Technical field]

[0001] The present invention relates generally to information processing, and more particularly to preventing unauthorized Helm-based application package deployments and resource modifications in a cluster. [Background technology]

[0002] Kubernetes (also known as "k8s") is an open source container orchestration system for automating the deployment, scaling, and management of applications. Kubernetes aims to provide a platform for automating the deployment, scaling, and operation of application containers across a cluster of hosts.

[0003] The goal of cluster-side integrity is to verify that (a) the package contents are signed with the key of an authorized signer, and (b) all admission requests have not been altered (tampered with) from resource creation requests derived from the signed package contents.

[0004] Existing approaches to cluster-side integrity rely on client-side validation (e.g., Helm validation), which is vulnerable to malicious clients. Cluster-side validation is nontrivial because it is difficult to (a) identify package contents from individual admission requests on the cluster side, (b) distinguish a package's admission request from all other admission requests, and (c) validate the integrity of the admission request because the resource creation request needs to be visible only by the client. Summary of the Invention

[0005] According to one aspect of the present invention, a computer-implemented method is provided that is executed by a server in a cluster to verify the integrity of an object deployed in the cluster. The method includes detecting a resource creation request from a client. The method further includes, in response to the resource creation request being an initial resource creation request for the object, verifying the integrity of the object based on a property in the resource creation request to create a release secret in the cluster in case of a positive integrity verification result for the object. The release secret represents a particular deployment configuration of the object on the cluster. In response to the resource creation request not being an initial resource request, the method also includes verifying whether the resource creation request corresponds to a particular deployment configuration of the object by matching against a release secret in the cluster. In response to the resource creation request corresponding to a deployment of the object and the release secret being present in the cluster, the method further includes creating the resource requested by the resource creation request in the cluster.

[0006] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a non-transitory computer-readable storage medium having program instructions embodied therein, the program instructions being executable by a computer to cause a computer to perform a method executed by a server in a cluster to verify the integrity of an object deployed to the cluster. The method includes detecting a resource creation request from a client. In response to the resource creation request being an initial resource creation request for the object, the method further includes verifying the integrity of the object based on a property in the resource creation request to create a release secret in the cluster in case of a positive integrity verification result for the object. The release secret represents a specific deployment configuration of the package on the cluster. In response to the resource creation request not being an initial resource request, the method also includes verifying whether the resource creation request corresponds to a specific deployment configuration of the object by matching against a release secret in the cluster. In response to the resource creation request corresponding to a deployment of the object and the release secret being present in the cluster, the method further includes creating the resource requested by the resource creation request in the cluster.

[0007] According to yet another aspect of the present invention, a server in a cluster for verifying the integrity of an object deployed in a cluster is provided. The server includes a memory configured to store program code. The server further includes a hardware processor configured to execute the program code to detect a resource creation request from a client. In response to the resource creation request being an initial resource creation request for the object, the hardware processor further executes the program code to verify the integrity of the package based on the properties in the resource creation request to create a release secret in the cluster in case of a positive integrity verification result of the object, the release secret representing a particular deployment configuration of the object on the cluster. In response to the resource creation request not being an initial resource request, the hardware processor also executes the program code to verify whether the resource creation request corresponds to a particular deployment configuration of the object by matching with the release secret in the cluster. In response to the resource creation request corresponding to the deployment of the package and the release secret being present in the cluster, the hardware processor further executes the program code to create the resource requested by the resource creation request in the cluster.

[0008] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.

[0009] The following description sets forth details of the preferred embodiment with reference to the following drawings: [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary computing device, in accordance with one embodiment of the present invention. [Diagram 2] FIG. 1 is a block diagram illustrating an exemplary Kubernetes architecture to which the present invention can be applied, according to an embodiment of the present invention. [Diagram 3] 1 is a block diagram illustrating an exemplary computing environment, according to one embodiment of the present invention. [Figure 4] FIG. 1 is a block diagram illustrating an example Helm application package and deployment scenario, according to one embodiment of the present invention. [Diagram 5] 1 is a flow diagram illustrating an example method for preventing unauthorized Helm-based application package deployments and resource modifications in a k8s cluster, according to an embodiment of the present invention. [Figure 6] FIG. 9 is a schematic block diagram illustrating an example system / method in which a method (as described in FIG. 7 and FIG. 8) for preventing unauthorized Helm-based application package deployment and resource modification in a k8s cluster is applied, according to an embodiment of the present invention. [Figure 7] FIG. 1 illustrates an example method for preventing unauthorized Helm-based application package deployment and resource modifications in a k8s cluster, according to an embodiment of the present invention. [Figure 8] FIG. 1 illustrates an example method for preventing unauthorized Helm-based application package deployment and resource modifications in a k8s cluster, according to an embodiment of the present invention. [Figure 9] FIG. 13 illustrates exemplary pseudocode for releasing secrets, according to one embodiment of the present invention. [Figure 10] FIG. 13 illustrates an exemplary pseudocode for a decoded data release according to one embodiment of the present invention. [Figure 11] FIG. 2 illustrates a cluster-side integrity enforcement step according to one embodiment of the present invention. [Figure 12] FIG. 2 illustrates a cluster-side integrity enforcement feature according to one embodiment of the present invention. [Figure 13] FIG. 1 is a block diagram illustrating an exemplary cloud computing environment having one or more cloud computing nodes with which local computing devices used by cloud consumers communicate, according to one embodiment of the present invention. [Figure 14] FIG. 2 is a block diagram illustrating a set of functional abstraction layers provided by a cloud computing environment in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Embodiments of the present invention are directed to preventing unauthorized Helm-based application package deployments and resource modifications in a k8s cluster.

[0012] Embodiments of the present invention provide a cluster-side integrity verification mechanism that (a) identifies any admission requests resulting from a Helm installation, (b) identifies package contents from each individual admission request, and (c) detects unauthorized modifications of admission requests without relying on client-side verification.

[0013] One or more embodiments of the invention can include identifying the application package contents from the individual admission requests by detecting an initial admission request with a release secret and correlating other admission requests corresponding to a particular package deployment to create individual resources contained in the package. Properties of the request can include, for example, a Helm release name and an owner reference.

[0014] One or more embodiments of the present invention may distinguish an admission request that corresponds to a particular package deployment from among multiple admission requests received at a cluster using properties in the request.

[0015] One or more embodiments of the present invention may use an integrity chain to verify the integrity of an application package deployment: for example, for each resource creation request, a signed Helm package (template) and a signed deployment configuration are used to (a) create an expected resource definition and (b) compare it to the resource definition in the incoming admission request.

[0016] One or more embodiments of the present invention (a) provide cluster-side integrity enforcement for application packages without relying on any client-side validation, (b) address variations in application packaging and deployment, and (c) prevent unauthorized resource configuration changes to application packages over their entire lifetime. Additionally, embodiments of the present invention make no changes to the application bundle or packaging tools while enforcing integrity.

[0017] Figure 1 is a block diagram illustrating an exemplary computing device 100, in accordance with one embodiment of the present invention. The computing device 100 represents a device / node in a k8s cluster 210 in the environment 200 illustrated in Figure 2. The computing device 100 is configured to prevent unauthorized Helm-based application package deployment and resource modifications in the K8s cluster.

[0018] Computing device 100 may be implemented as any type of computational or computing device capable of performing the functions described herein, including, but not limited to, a computer, a server, a rack server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, or a consumer electronic device, or any combination thereof. Additionally or alternatively, computing device 100 may be implemented as one or more compute sleds, memory sleds, or other racks, sleds, computing chassis, or other components of a physically partitioned computing device. As shown in FIG. 1, computing device 100 illustratively includes a processor 110, an input / output subsystem 120, a memory 130, a data storage device 140, and a communication subsystem 150, or other components and devices, such as those typically found in a server or similar computing device, or any combination thereof. Of course, in other embodiments, computing device 100 may include other or additional components such as those typically found in a server computer (e.g., various input / output devices). Also, in some embodiments, one or more of the illustrated components may be incorporated into or otherwise form a part of another component. For example, in some embodiments, memory 130, or portions thereof, may be incorporated into processor 110.

[0019] Processor 110 may be implemented as any type of processor capable of performing the functions described herein, including but not limited to a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuitry.

[0020] Memory 130 may be implemented as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. In operation, memory 130 may store various data and software used during operation of computing device 100, such as an operating system, applications, programs, libraries, and drivers. Memory 130 is communicatively coupled to processor 110 via I / O subsystem 120, which may be implemented as circuits and / or components that facilitate input / output operations with processor 110, memory 130, and other components of computing device 100. For example, I / O subsystem 120 may be implemented as or otherwise include a memory controller hub, an input / output control hub, a platform controller hub, an integrated control circuit, a firmware device, a communication link (e.g., a point-to-point link, a bus link, a wire, a cable, a light guide, a printed circuit board trace, etc.), or other components and subsystems that facilitate input / output operations or combinations thereof. In some embodiments, I / O subsystem 120 may form part of a system-on-chip (SOC) and may be integrated with processor 110, memory 130, and other components of computing device 100 on a single integrated circuit chip.

[0021] The data storage device 140 may be implemented as any type of device configured for short-term or long-term storage of data, such as, for example, memory devices and circuits, memory cards, hard disk drives, solid state drives, or other data storage devices. The data storage device 140 may store program code for preventing unauthorized Helm-based application package deployment and resource modification in a k8s cluster. The communication subsystem 150 of the computing device 100 may be implemented as any network interface controller or other communication circuitry, device, or collection thereof that may enable communication between the computing device 100 and other remote devices over a network. The communication subsystem 150 may be configured to use any one or more communication technologies (e.g., wired or wireless communication) and associated protocols (e.g., Ethernet®, InfiniBand®, Bluetooth®, Wi-Fi®, WiMAX, etc.) that effect such communication.

[0022] As shown, computing device 100 may also include one or more peripheral devices 160. Peripheral devices 160 may include any number of additional input / output devices, interface devices, or other peripheral devices, or combinations thereof. For example, in some embodiments, peripheral devices 160 may include a display, a touch screen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0023] Of course, the computing device 100 may include other elements (not shown) or omit certain elements, as would be readily contemplated by one of ordinary skill in the art. For example, various other input and / or output devices may be included in the computing device 100, depending on the particular implementation of the computing device 100, as would be readily apparent to one of ordinary skill in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Additional processors, controllers, memories, etc. may also be used in various configurations. In another embodiment, a cloud configuration may be used. These and other variations of the computing device 100 will be readily contemplated by one of ordinary skill in the art given the teachings of the present invention provided herein.

[0024] As employed herein, the term "hardware processor subsystem" or "hardware processor" may refer to a processor, memory (including RAM, cache, etc.), software (including memory management software), or combinations thereof, that cooperate to perform one or more specific tasks. In useful embodiments, the hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The one or more data processing elements may be included in a central processing unit, a graphics processing unit, or a separate processor or computing element-based controller (e.g., logic gates, etc.), or combinations thereof. The hardware processor subsystem may include one or more on-board memories (e.g., cache, dedicated memory arrays, read-only memories, etc.). In some embodiments, the hardware processor subsystem may include one or more memories (e.g., ROM, RAM, Basic Input / Output System (BIOS), etc.), which may be on-board or off-board or may be dedicated for use by the hardware processor subsystem.

[0025] In some embodiments, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications, or specific code for producing a specified result, or any combination thereof.

[0026] In other embodiments, the hardware processor subsystem may include specialized circuitry dedicated to performing one or more electronic processing functions to achieve a specified result. Such circuitry may include one or more application specific integrated circuits (ASICs), FPGAs, or PLAs, or combinations thereof.

[0027] A hardware processor may embody one or more of the elements in the clusters shown in FIGS.

[0028] These and other variations of hardware processor subsystems are contemplated by embodiments of the present invention.

[0029] FIG. 2 is a block diagram illustrating an exemplary Kubernetes architecture 200 to which the present invention may be applied, in accordance with one embodiment of the present invention.

[0030] The Kubernetes architecture 200 includes a Kubernetes master 210. The Kubernetes master 210 includes a controller 211, an ETCD 212, a scheduler 213, and a Kubernetes API server 214. The Kubernetes API server 214 includes an authentication stage 214A, an authorization stage 214B, and an admission stage / controller 214C.

[0031] A user can initiate a request to create or update or delete a k8s resource via a client tool (kubectl, Helm, etc.). Once the request reaches the k8sAPI server 214, it passes through several stages: authentication 214A, authorization 214B, and admission control 214C. After the authentication and authorization steps, the request is intercepted by the admission controller 214C before the persistence of the object specified in the request. The admission controller 214C can modify or reject the request. Various of these elements are described in further detail below.

[0032] Admission webhooks may be used as part of the admission chain along with the admission controller 214C. k8s defines admission webhooks as HTTP callbacks based on the resource type and the request method and registers them. A custom admission webhook may process the incoming request sent by the k8sAPI server 214 and return a decision (such as allow or deny) to the k8sAPI server 214 based on its logic. Also, of the two types of admission webhooks, a mutating admission webhook may modify the review request based on custom logic. One example is that an annotation may be added to the resource that is created or updated via the incoming request.

[0033] FIG. 3 is a block diagram illustrating an exemplary computing environment 300, according to one embodiment of the present invention.

[0034] Environment 300 includes multiple k8s clusters (collectively designated by reference numeral 310) and multiple Helm client devices (collectively designated by reference numeral 320).

[0035] Communications between entities of environment 300 may occur via one or more networks 330. For purposes of illustration, a wireless network 330 is shown. In other embodiments, either wired, wireless, or a combination thereof may be used to facilitate communications between the entities.

[0036] The k8s cluster 310 can observe all incoming requests (admission requests).

[0037] The Helm client device 310 includes a package installer 310A that includes (a) a template engine 310A1 for creating resource creation requests from package contents and (b) a k8sAPI call handler 310A2 for processing k8sAPI calls to send the resource creation requests to a k8s cluster.

[0038] The Helm client device 310 is capable of (a) installing Helm packages and (b) updating Helm packages.

[0039] The Helm client device 310 provides the above functionality through the following components: (a) helm, a command line tool that provides a user interface to all Helm functionality; (b) a Helm packaging format called a chart; and (c) a curated chart repository with pre-packaged charts for popular open source software projects.

[0040] A Helm package, called a chart, contains a few YAML configuration files and some templates that are turned into Kubernetes manifest files. A chart can contain the following directories and / or files: - charts: This directory can contain chart dependencies that are managed manually, but it is typically better to use requirements.yaml to dynamically link dependencies. - templates / : This directory contains template files that are combined with configuration values ​​(from values.yaml and the command line) and turned into Kubernetes manifests. - Chart.yaml: A YAML file with metadata about the chart, such as the chart name and version, maintainer information, associated websites, and search keywords.

[0041] The Helm client tool allows you to verify the integrity and provenance of a chart by comparing the chart to its corresponding provenance record before deployment. A provenance file contains (a) the YAML content of the chart and information needed for verification, such as the chart's signature, what was signed, etc.

[0042] In the context of a k8s cluster, nodes, also known as workers or minions, are the machines on which containers (workloads) are deployed. Every node in the cluster runs a container runtime such as Docker, and other components for communication with the primary for network configuration of these containers.

[0043] FIG. 4 is a block diagram illustrating an example Helm application package and deployment scenario 400, according to one embodiment of the present invention.

[0044] Scenario 400 involves a client 401 and a k8s cluster 440.

[0045] The client 401 includes a package (Helm chart) 410, a deployment configuration (values.yaml) 420, and a template engine / application installer (Helm) 430.

[0046] The k8s cluster 440 includes an ETCD 441, a Kubernetes application programming interface (API) 442, an admission controller 443, and an artifact (signature) repository 450.

[0047] A Helm-based package deployment makes a series of resource creation requests to the k8s cluster 440.

[0048] The template engine / installer 430 provides k8s resource definitions (YAML) 477 to the Kubernetes API 442.

[0049] The ETCD 441 is a persistent, lightweight, distributed key-value data store that reliably stores the cluster's configuration data, which represents the overall state of the cluster at any point in time. The configuration data is retrievable from the ETCD 441 via an API 442.

[0050] The API 442 may use, for example, JSON over HTTP, which provides both an internal and external interface to Kubernetes. The API 442 processes and validates REST requests and updates the state of API objects in ETCD 441, thereby allowing clients to configure workloads and containers across worker nodes. In one or more embodiments, the invention intercepts incoming create requests to the k8s API server. The invention then enforces integrity on the resources in the request by allowing or denying the creation of the resources.

[0051] The admission controller 443 performs integrity enforcement. The admission controller 443 enforces the integrity of the application package by (a) verifying that it is from a trusted source, i.e., whether the owner's identity can be verified and trusted using the owner's signature, and (b) ensuring that the contents have not been altered in transit since being packaged and signed. After the initial deployment, the admission controller 443 also enforces integrity on any configuration changes to the deployed application package or individual resources.

[0052] In case of an integrity violation, the admission controller 443 blocks both unauthorized workload deployment and resource modifications, and logs all enforcement actions for auditing purposes.

[0053] The Controller Manager (shown in Figure 2 as element 211) is essentially a coordination loop that drives the actual cluster state towards the desired cluster state, and communicates with the API server to create, update and delete the resources it manages (pods, service endpoints, etc.). The Controller Manager manages a set of core Kubernetes controllers. For example, one type of controller is the Admission Controller 443.

[0054] Next, a description of Helm-based application deployment will be provided, according to one embodiment of the present invention.

[0055] When deploying an application package, Helm converts the combination of chart templates and deployment configurations into k8s manifests. Helm then deploys them via the k8s API. This creates a release object (also called a "release secret" in k8s) as a first step, which represents a specific configuration and deployment of the charts on the cluster.

[0056] Next, a description of the unique features of the present invention is provided.

[0057] In Kubernetes, a secret is an object that contains a small amount of sensitive data, such as a password, token, or key. Such information might otherwise be put into a pod specification or image. Users can create secrets, and the system also creates some secrets.

[0058] A Helm "release secret" is a special type of secret in accordance with the present invention that is used to facilitate the release of a Helm package deployment in response to one or more validations using the release secret. In one embodiment of the present invention, the release manifest is encoded and included as the release secret (see, e.g., element 607 in FIG. 6).

[0059] FIG. 5 is a flow diagram illustrating an example method 500 for preventing unauthorized Helm-based application package deployments and resource modifications in a k8s cluster, according to an embodiment of the present invention. The method is performed by an admission control webhook in the cluster. For purposes of illustration and concreteness, the method is described with respect to a Helm application, but those skilled in the art will readily be able to use these teachings for other objects, such as, for example, a single YAML-based deployment and / or a k8s operator-based application bundle. The following description, such as, for example, the description with respect to FIG. 11 and FIG. 12, addresses these other object types in detail.

[0060] At block 505, a resource creation request from a client is detected.

[0061] In block 510, in response to the resource creation request being an initial resource creation request for an object (which in this illustrative example is a package, but which may otherwise be a single YAML-based deployment and / or k8s operator-based application bundle), verify the integrity of the package based on properties in the resource creation request to create a release secret on the cluster in case of a positive integrity verification result for the package. The release secret represents a particular configuration and deployment of the package on the cluster. Block 510 corresponds to blocks 710 through 714 in FIG. 7 and processes request 1 in FIG. 6.

[0062] In response to the resource creation request not being an initial resource request, block 515 determines whether the resource creation request corresponds to that particular deployment configuration of the package by matching against a release secret in the cluster. Block 515 corresponds to blocks 720 through 723 in Figure 7 and processes requests 2 through 4 in Figure 6.

[0063] Block 520 creates the resource requested by the resource create request in the cluster in response to the resource create request corresponding to that particular deployment configuration of the package and the release secret existing in the cluster. Block 520 corresponds to blocks 720 through 723 of Figure 7 and processes requests 2 through 4 of Figure 6.

[0064] The package contents are located and integrity is verified by looking up the owner references in the cluster at block 525. Block 525 corresponds to blocks 730 through 732 in Figure 7 and processes requests 5 and 6 in Figure 6.

[0065] 6 through 8 below essentially provide further details on method 500.

[0066] 6 is a schematic block diagram illustrating an example system / method 600 in which the method (described in FIGS. 7 and 8) for preventing unauthorized Helm-based application package deployment and resource modification in a k8s cluster is applied, according to an embodiment of the present invention. FIG 6 may also be an environment in which one or more embodiments of the present invention may be implemented.

[0067] The system / method 600 includes a Helm package 601 (e.g., a Helm chart), a deployment configuration (values.yaml) 602, a chart repository 603, a Helm client 604, a k8sAPI 605 for a k8s cluster 606, a package release (manifest) 607, a custom resource definition 608, and an artifact repository 609.

[0068] The Helm package 601 is signed with the package owner's key (or a notary service, for example) to obtain signed package contents 601A.

[0069] The deployment configuration 602 is signed by a signing authority to obtain a signed deployment configuration 602A.

[0070] The Helm client 604 publishes a release (manifest), from which the following requests are generated by the Helm client 604: Req.1: CREATE Secret Release Request 2 (Req.2): CREATE ServiceAccount Req.3: CREATE Deployment

[0071] The k8sAPI605 receives the above three requests and generates the following request: Request 5 (Req.5): CREATE ReplicaSet Request 6 (Req.6): CREATE PoD

[0072] The resources listed above are not limited and are provided for illustrative purposes only, in other embodiments, the resources may include, for example, a ConfigMap.

[0073] Because of the owner dependency, the following requirements are generated: Owner->Dependencies (creating an owner creates its dependents) Deployment -> ReplicaSet ReplicaSet->Pod

[0074] Again, this is an internal mechanism of k8s.

[0075] 7 and 8 illustrate an example method 700 for preventing unauthorized Helm-based application package deployments and resource modifications in a k8s cluster in accordance with an embodiment of the present invention.

[0076] In block 701, a Helm package 601 is provided.

[0077] In block 702, the Helm package 601 is signed with the package owner's key and the Helm package 601 and the signature (i.e., signed package 601A) are stored in the chart repository 603.

[0078] In block 703, the deployed configuration 602 is signed by a signing authority to obtain a signed deployed configuration 602A, and the signature is maintained as a CRD in the cluster 606.

[0079] In block 710, in response to receiving request 1, the first admission request having a release secret is detected and the integrity is verified.

[0080] In one embodiment, block 710 includes one or more of blocks 711 through 714.

[0081] In block 711, the name of the resource in the admission request (e.g., sh.helm.release.v1.ac-test-app1.v1s) is verified.

[0082] In block 712, the signature of the package contents retrieved from the chart repository 603 is verified and matched against the Helm template definition in the signed Helm package 601A stored in the chart repository 603.

[0083] At block 713, the signed deployment configuration is content-matched against the deployment configuration in the request.

[0084] In block 714, a release secret is created in the cluster.

[0085] At block 720, in response to receiving all of requests 2 through 4, the package contents are identified and the integrity is verified by matching them against existing release secrets in the cluster.

[0086] In one embodiment, block 720 includes one or more of blocks 721 through 723.

[0087] Block 721 checks whether requests 2 through 4 all correspond to a particular application package deployment by matching against the release secrets in cluster 606 .

[0088] In block 722, (a) an expected resource definition is created from the signed Helm package (template) 601A and the signed deployment configuration 602A, and (b) the signed Helm package 601A (template) and the signed deployment configuration are used to verify integrity by comparing the expected resource definition to the resource definition in the incoming admission request.

[0089] At block 723, a specially annotated resource is created in the cluster 606. The special annotation indicates a positive integrity verification result.

[0090] At block 730, request 5 and request 6 are all received and the package contents are identified and verified for integrity by looking up the ownerReference in cluster 606.

[0091] In one embodiment, block 730 includes one or more of blocks 731 through 734.

[0092] Block 731 determines whether the request corresponds to a particular application package deployment by checking the owner reference.

[0093] In response to being able to find the owner in cluster 606 and validating the integrity of the request (by verifying that it contains a special annotation), block 732 deploys the signed Helm package 601A to cluster 606 and creates all package resources (e.g., secrets, services, deployments, etc.). Block 732 processes the request according to request 5 and request 6, respectively, to create internal resources such as ReplicaSets and Pods. The above steps process Secrets and ServiceAccounts.

[0094] At block 740, the resource signature is obtained to verify the integrity of the updated resource in the request.

[0095] FIG. 9 illustrates example pseudocode 900 for releasing secrets, according to one embodiment of the present invention.

[0096] Pseudocode 900 includes a "data.release" section 910 that contains the encoded release manifest 607, which is decoded as shown in FIG.

[0097] FIG. 10 illustrates an example pseudocode 1000 for a decoded data release, according to one embodiment of the present invention.

[0098] Thus, in accordance with the present invention, the data can be used to identify subsequent requests by decoding the "data.release" portion 910 of the release secret into pseudocode 1000 as shown in Figure 10. For example, in pseudocode 1000, "templates / deployment.yaml" would be used by request #4 in Figure 6.

[0099] Integrity enforcement is described herein with respect to the following three cases (as shown in Figures 11 and 12): (1) Single YAML-based deployment (2) Helm-based application packages (3) k8s operator-based application bundles (this is similar to case #1, as deploying a k8s operator-based application package to a cluster involves creating a number of k8s manifests in YAML)

[0100] For illustrative purposes, this detailed description focuses on Case #2, however, one of ordinary skill in the art, given the teachings of the present invention provided herein, can readily apply the present invention to Cases #1 and #3.

[0101] The following provides a description of a k8s operator according to one embodiment of the present invention.

[0102] A k8s operator is a way to package, deploy and automatically manage a k8s application over its lifetime. The operator SDK generates a bundle that contains several manifest files for deploying the operator to a k8s cluster.

[0103] One of the manifest files contains a Custom Resource (CR), which is an endpoint to k8s that is attached by the operator. A Custom Resource Definition (CRD) models the underlying business logic of the application. A CRD is a specification of what constitutes a CR. Deploying an operator bundle involves creating the resources in the manifest file in the cluster.

[0104] The following provides an overview of cluster-side integrity enforcement in accordance with one embodiment of the present invention.

[0105] The present invention is configured to (a) provide cluster-side integrity enforcement for application packages without relying on any client-side validation, (b) accommodate variations in application packaging and deployment, and (c) prevent unauthorized resource configuration changes to an application package over its entire lifetime. Additionally, the method of the present invention aims to enforce integrity without making changes to the application bundle or packaging tools.

[0106] Figure 6 shows the overall architecture for enforcing integrity for application workloads (plain YAML manifests or Helm charts or k8s operators) deployed to a cluster.

[0107] First, one or more embodiments of the present invention may require (a) plain k8s manifests in YAML format, (b) Helm charts, and (c) k8s operators to be digitally signed with attestations recognized and approvable by an authority, but the present invention does not depend on client tools verifying the integrity.

[0108] Second, the invention enforces integrity on the cluster side against variations in application packages that are deployed to the cluster by (a) verifying that the application package is from a trusted source, i.e., can be trusted by verifying the owner's identity using the owner's signature, and (b) ensuring that the contents have not been altered in transit since being packaged and signed. Additionally, after the initial deployment, the mechanism enforces integrity against configuration changes to the deployed application package or individual resources.

[0109] In the event of an integrity violation, the present invention prevents both unauthorized workload deployment and resource modification, and records all enforcement actions for auditing purposes.

[0110] Below we provide an explanation of integrity enforcement using the Mutate Webhook Admission Controller.

[0111] One or more embodiments of the present invention encapsulate the core concept of enforcing integrity on the cluster side and implement it as custom logic in a mutate admission webhook in k8s as shown in Figure 6. This is because the deployment of all variations of an application package must go through a common gateway in the cluster, i.e. the k8s API server, which contains the webhook in the admission control chain as shown in Figure 2.

[0112] During deployment of a plain YAML manifest or an application package or a k8s operator, a built-in cluster-side enforcement mechanism encapsulated in a mutate admission webhook intercepts requests received by the k8sAPI server and imposes integrity on them. The enforcement mechanism processes each incoming request and determines its verdict (allow or deny) based on the logic shown in Figures 11 and 12. Our approach also annotates resources in incoming requests by adding a flag "integrityVerified=true" if it has verified their integrity.

[0113] Below, we describe digital signatures for three types of workloads: (a) plain k8s manifests in YAML format, (b) Helm charts, and (c) k8s operators. We identify the three scenarios as Case A, Case B, and Case C, respectively.

[0114] The following is a description of client-side individual resource signing (Case A) according to one embodiment of the present invention.

[0115] Assume that the intent is to create a single resource manifest in YAML format on a cluster protected by the integrity enforcement mechanism of the present invention. To this end, the present invention requires that the resource manifest be digitally signed and that the signature be made available to the cluster. As with creation, updates to existing resources also require signing.

[0116] According to one embodiment, signing a resource (either a new resource or an update to an existing resource) in a plain k8s manifest requires an authorized owner to create a ResourceSignature on the cluster. A ResourceSignature is a special resource type that stores the signature of a resource on the cluster. Our cluster-side enforcement mechanism uses the ResourceSignature to ensure the integrity of the resource.

[0117] To create a ResourceSignature, the present technique provides a template. The ResourceSignature template contains several key fields: MESSAGE, SIGNATURE, and RSIG_SIGNATURE.

[0118] The authorized entity MUST fill out the "ResourceSignature" template with the values ​​of the target resource they wish to sign (such as name, API version, type, etc.). The "MESSAGE" is generated by creating the encoded (base64) content of the target resource. The authorized entity MAY use standard tools such as PKI, GnuPG, etc. to create the signature to be filled in the "SIGNATURE" field.

[0119] The ResourceSignature also contains another field called RSIG_SIGNATURE, which points to the signature of the ResourceSignature itself. This is necessary because the ResourceSignature is also a k8s resource that needs to be created on the cluster and its integrity must be enforced. To generate the RSIG_SIGNATURE, standard tools such as PKI, GnuPGK, etc. can be used. After filling out the ResourceSignature, the authorized entity needs to generate a signature for the specification section of the ResourceSignature.

[0120] The following is a description of client-side signing of an application package (Case B) according to one embodiment of the present invention.

[0121] The general structure of an application package includes (a) a k8s resource template and (b) a deployment configuration.

[0122] The application package installer (e.g., Helm) converts the combination of k8s resource templates and deployment configurations into k8s manifests in YAML format. The application package installer then deploys them to the cluster by sending individual create requests to the k8sAPI server. Therefore, signing the application package requires that each of the converted k8s manifests be signed (i.e., to create a "ResourceSignature") so that the cluster-side mechanisms can use the signature to enforce the integrity when they arrive as requests to the k8sAPI server. This approach requires modifications to the application package installer, since the converted k8s manifests are an internal representation of the installer.

[0123] The goal of the inventive approach is to make no changes to the application package installer or bundle. Therefore, the inventors design the inventive method to sign the application package as a single artifact. As a specific example, in the case of Helm, the inventive approach uses a provenance record that includes a signature to enforce integrity (Case B).

[0124] Below is a description of client-side signing for a k8s operator (Case C) according to one embodiment of the present invention.

[0125] Deploying a k8s operator bundle to a cluster involves creating the resources specified in the k8s manifest files in the bundle (i.e. the 'deploy' directory of the operator bundle). Therefore, the present invention requires that these manifest files be digitally signed. Since these are plain k8s manifests in YAML format, an authorized entity needs to generate a 'ResourceSignature' for each of them using a template. After the 'ResourceSignature' manifest is generated, it needs to be created in the cluster where the operator will be deployed.

[0126] A more detailed description of cluster-side integrity enforcement in accordance with one embodiment of the present invention is provided below.

[0127] 11 and 12 show pseudo-code steps that may be included in various embodiments of the cluster-side enforcement mechanism according to the present invention. The mechanism handles different workloads such as plain k8s manifests in YAML format (case A) or Helm charts (case B) or k8s operators (case C). How the present invention handles each case is described below.

[0128] (1) Input: Figure 11 shows that input is a request received by the k8s API. The request conveys information about the resource to be created, updated or deleted (such as name type, kind, namespace, etc.).

[0129] (2) Output: Figure 12 shows that the output is an "ALLOW" or "REJECT" decision. In the "ALLOW" case, the method may annotate the resource with a flag indicating that its integrity has been verified (section R6 in Figure 12). In the case of rejection, the response will include a reason (section R7 in Figure 12).

[0130] (3) Internal Request by k8s: The first step of the enforcement mechanism checks if the incoming request was generated by an internal component of k8s as shown in Figure 11. This type of request can be identified based on request properties such as users, service accounts, etc. internal to k8s that are considered trusted.

[0131] Next, for other requests, the technique of the present invention checks whether the type of operation is CREATE or UPDATE.

[0132] (4) Creation Request: For cases A or B or C, "HandleCreationRequest" includes steps to enforce integrity on the creation request. We describe the steps in sections R1, R2, R3 and R4 of FIG.

[0133] (a) Handle the initial creation request (R1 in Fig. 12): In case B, "HandleCreationRequest" first checks if the request indicates the start of a package deployment (section R1 in Fig. 12). For example, when deploying a chart, Helm sends a request to create a resource called "Release Secret". This is always the first request sent by the Helm installer to the k8sAPI and therefore indicates the start of a chart deployment. Furthermore, the initial request carries information about the bundle (e.g., the resources in it). For example, the Helm release secret remembers the release, i.e. the running instance of the chart in the cluster.

[0134] The next step in R1 is to enforce the integrity of the resource in the initial request. More specifically, in the case of Helm, the enforcement mechanism retrieves the release name from the request. This release name is then used to fetch the corresponding chart and its provenance record from the artifact repository. The mechanism enforces the integrity of the resource in the initial request using the signature in the provenance record. Specifically, the mechanism checks that (1) the chart found in the repository is signed by an authorized owner and (2) its contents have not been tampered with since it was bundled. For (2), the mechanism compares the contents of the release resource in the request with the expected contents of the chart found in the repository to ensure that no tampering occurred during transmission. If the integrity enforcement is successful, a release object is created in the cluster and the remainder of the request is then sent by the installer to the k8sAPI server. If the initial request does not pass the integrity check, the package installation is terminated.

[0135] (b) Processing Follow-up Requests (R2 in Fig. 12): In case B, there will be a set of follow-up requests to create resources in the package after the initial object is created. For such resources, our approach tries to associate them with a validated resource object that indicates the start of the package installation (e.g., "Release Secret" in the case of Helm). Assume that the chart contains a "ConfigMap" resource and Helm sends a request to the k8sAPI to create it. We check whether we can associate "ConfigMap" with any of the validated "Release Secret" objects in the cluster. Since "Release Secret" contains all the resources in the chart, it is possible to check whether "Release Secret" contains a particular resource by matching its properties such as name, type, API version, etc.

[0136] (c) Processing a Request with Dependent Resources (R3 in FIG. 12): The steps in section R3 concern creating a resource that depends on other resources. k8s maintains owner dependencies between some of the resources. For example, a "ReplicaSet" is the owner of a set of "Pods", which is owned by a "Deployment". Each dependent object has an "OwnerReference" metadata that references the owning object. For a dependent resource in case A or case B or case C, if an owner with integrity verified can be found in the ownership hierarchy, the present invention determines its integrity.

[0137] (d) Processing Requests with Other Resources (R4 in FIG. 12): For other resource creation requests not covered by sections R1 or R2 or R3, the inventive technique requires that a ResourceSignature is available in the cluster before creating the resource. If a ResourceSignature can be found (by matching resource properties such as name, type, API version, etc.), it enforces the integrity of the resource in the request. The ResourceSignature includes a MESSAGE field that contains the encoded contents of the resource manifest. The invention compares the contents of the resource manifest in the request with the MESSAGE field in the corresponding ResourceSignature. If the match is successful, the inventive technique establishes integrity for the resource in the request.

[0138] (5) Update Request (R5 in FIG. 12): For a resource update, one or more embodiments of the present invention may require that a "ResourceSignature" be present in the cluster before updating the resource. For a resource update request, the invention finds the corresponding "ResourceSignature". If the resource signature is present, it is matched against the contents of the changed resource manifest in the request and the "MESSAGE" field in the corresponding "ResourceSignature". If successful, it establishes the integrity of the changed resources and persists those changes. If not successful, the invention rejects the resource update with a reason of "valid signature not found".

[0139] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0140] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with a service provider. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0141] Its features are as follows.

[0142] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time and network storage, automatically as needed, without the need for human interaction with the service provider.

[0143] Broad network access: Functionality is available over the network and can be accessed through standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., cell phones, laptops, and PDAs).

[0144] Resource Pooling: To serve multiple consumers using a multi-tenant model, a provider's computing resources are pooled, and different physical and virtual resources are dynamically allocated and reallocated depending on demand. Consumers generally have no control or knowledge over the exact location of the resources provided, but there is a sense of location independence in that it may be possible to specify a higher level of abstraction of location (e.g., country, state, or data center).

[0145] Rapid Elasticity: Rapid, elastic, and sometimes automatically provisioning capabilities to quickly scale out, and rapid release of capabilities to quickly scale in. To the consumer, the capabilities available for provisioning often seem infinite, and they can buy as much or as little as they want, at any time.

[0146] Metered Services: The cloud system automatically controls and optimizes resource utilization by utilizing metering capabilities at some level of abstraction appropriate for the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, giving both providers and consumers transparency of the services utilized.

[0147] The service model is as follows:

[0148] Software as a Service (SaaS): The functionality provided to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer has no management or control over the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings.

[0149] Platform as a Service (PaaS): The capability offered to the consumer is the deployment of consumer-created or acquired applications, written using programming languages ​​and tools supported by the provider, on a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the application hosting environment configuration.

[0150] Infrastructure as a Service (IaaS): The capability offered to the consumer is to provision processing, storage, network and other basic computing resources onto which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and possibly limited control over selected network components (e.g., host firewalls).

[0151] The deployment model is as follows:

[0152] Private Cloud: This cloud infrastructure is operated solely for the organization. It can be managed by the organization or a third party and can exist on-premise or off-premise.

[0153] Community Cloud: This cloud infrastructure is shared by several organizations to support a specific community with common concerns (e.g., mission, security requirements, policies, and compliance concerns). It can be managed by the organization or a third party and can reside on-premise or off-premise.

[0154] Public Cloud: This cloud infrastructure is available to the public or large industry organizations and is owned by an organization that sells cloud services.

[0155] Hybrid Cloud: This cloud infrastructure is a composite of two or more clouds (private, community or public) that remain their own entity but are joined by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0156] Cloud computing environments are service-oriented with a focus on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0157] 13, an exemplary cloud computing environment 1350 is illustrated. As shown, the cloud computing environment 1350 includes one or more cloud computing nodes 1310 with which local computing devices used by cloud consumers, such as, for example, a personal digital assistant (PDA) or mobile phone 1354A, a desktop computer 1354B, a laptop computer 1354C, or an automobile computer system 1354N, or combinations thereof, can communicate. The nodes 1310 can communicate with each other. The nodes 1310 may be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud, or combinations thereof, as described above. This enables the cloud computing environment 1350 to provide infrastructure, platform, and / or software as a service for which the cloud consumer does not need to maintain resources on a local computing device. It should be understood that the types of computing devices 1354A-1354N shown in FIG. 13 are intended to be illustrative only, and that the computing node 1310 and cloud computing environment 1350 can communicate with any type of computerized device via any type of network connection and / or network addressable connection (e.g., using a web browser).

[0158] Referring now to Figure 14, there is shown a set of functional abstraction layers provided by cloud computing environment 1350 (Figure 13). It should be understood in advance that the components, layers and functions shown in Figure 14 are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0159] Hardware and software layer 1460 includes hardware and software components. Examples of hardware components include mainframe 1461, reduced instruction set computer (RISC) architecture based servers 1462, servers 1463, blade servers 1464, storage devices 1465, and networks and networking components 1466. In some embodiments, software components include network application server software 1467 and database software 1468.

[0160] The virtualization layer 1470 provides an abstraction layer that can provide examples of virtual entities such as virtual servers 1471, virtual storage 1472, virtual networks including virtual private networks 1473, virtual applications and operating systems 1474, and virtual clients 1475.

[0161] In one example, the management layer 1480 may provide the following functions: Resource provisioning 1481 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 1482 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks and protection of data and other resources. User portal 1483 provides access to the cloud computing environment for consumers and system administrators. Service level management 1484 provides cloud computing resource allocation and management to ensure required service levels are met. Service Level Agreement (SLA) planning and fulfillment 1485 provides advance arrangement and procurement for cloud computing resources for anticipated future demand according to SLAs.

[0162] The workload layer 1490 provides examples of functionality that can utilize a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include mapping and navigation 1491, software development and lifecycle management 1492, virtual classroom instructional delivery 1493, data analytics processing 1494, transaction processing 1495, and unauthorized package deployment prevention 1496.

[0163] The invention may be a system, method, or computer program product, or combination thereof, of any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to implement aspects of the invention.

[0164] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves with instructions recorded thereon, and any suitable combination thereof. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through a fiber optic cable), or electrical signals transmitted over wires.

[0165] The computer readable program instructions described herein may be downloaded from the computer readable storage medium into the respective computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions to a computer readable storage medium within the respective computing / processing device for storage.

[0166] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​such as Smalltalk®, C++, and traditional procedural programming languages ​​such as the “C” programming language, or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer as a stand-alone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer readable program instructions to carry out aspects of the invention by personalizing the electronic circuitry using state information of the computer readable program instructions.

[0167] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams that illustrate methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0168] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, executed by the processor of the computer, or other programmable data processing apparatus, form means for implementing the functions / operations specified in the blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium such that the computer readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions that implement aspects of the functions / operations specified in the blocks of the flowcharts and / or block diagrams, and can direct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner.

[0169] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to create a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / operations specified in the flowchart and / or block diagram blocks.

[0170] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the figures. For example, two blocks shown in succession may in fact be executed substantially in parallel, or the blocks may be executed in reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may embody a combination of dedicated hardware and computer instructions.

[0171] Reference herein to "one embodiment" or "an embodiment" of the invention and other variations thereof means that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment of the invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" and any other variations thereof in various places throughout the specification are not necessarily all referring to the same embodiment.

[0172] For example, in the cases of "A / B," "A or B or both," and "at least one of A and B," the use of any of " / ," "or or both," and "at least one of" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the cases of "A, B, or C, or combinations thereof," and "at least one of A, B, and C," such phrases are intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). As will be readily apparent to those of skill in the art and those of skill in the relevant art, this may be extended to include as many items as are listed.

[0173] Although preferred embodiments of the system and method have been described, which are intended to be illustrative and not limiting, it should be noted that modifications and variations are possible for those skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments disclosed which are within the scope of the invention as outlined by the appended claims. Although aspects of the invention have been described above with the detail and thoroughness required by the Patent Laws, what is claimed and desired to be protected by Letters Patent is set forth in the appended claims.

Claims

1. 1. A computer-implemented method executed by a server in a cluster to verify integrity of objects deployed in the cluster, comprising: Detecting a resource creation request from a client; in response to the resource creation request being an initial resource creation request for the object, verifying the integrity of the object based on properties in the resource creation request to create a release secret at the cluster in case of a positive integrity verification result for the object, the release secret representing a particular deployment configuration of the object on the cluster; in response to the resource creation request not being the initial resource request, verifying whether the resource creation request corresponds to the particular deployment configuration of the object by matching against the release secret in the cluster; responsive to the resource creation request corresponding to unpacking the object and the release secret existing in the cluster, creating in the cluster the resource requested by the resource creation request.

2. verifying the integrity of the object, ascertaining a name of the resource being requested by the resource creation request; verifying signatures on object contents retrieved from the repository; 2. The computer-implemented method of claim 1, further comprising: matching a template definition in the signature of the object with a definition in the resource creation request; and in response to finding the name and signature of the resource, matching a signed deployment configuration in a database with a deployment configuration in the resource creation request.

3. determining whether the resource creation request corresponds to the particular deployment configuration of the object; creating an expected resource definition from the signed template and the particular deployment configuration; and comparing the expected resource definition to a resource definition in the resource creation request.

4. determining whether the resource creation request corresponds to the particular deployment configuration of the object by determining whether an object content owner identifier of the object can be found within the cluster; in response to the identifier of the owner being found within the cluster, verifying the integrity of the object and creating the resource in the cluster with an annotation indicating a positive integrity verification result; 2. The computer-implemented method of claim 1, further comprising: in response to finding the annotation absent from the resource, blocking any incoming update requests.

5. The computer-implemented method of claim 1 , wherein verifying the integrity of the object occurs only in the cluster.

6. 2. The computer-implemented method of claim 1, wherein verifying the integrity of the object is performed using a mutating admission webhook in the cluster.

7. 7. The computer-implemented method of claim 6, wherein the mutating admission webhook intercepts the resource creation request to subject the resource creation request to an integrity check.

8. 2. The computer-implemented method of claim 1, wherein verifying the integrity of the object using the release secret occurs for both creating the object and updating the object.

9. 2. The computer-implemented method of claim 1, wherein verifying the integrity of the object comprises adding an annotation indicating the positive integrity verification result of a package.

10. The computer-implemented method of claim 1 , wherein verifying the integrity of the object uses a signature of a resource signature.

11. The computer-implemented method of claim 10 , wherein the signature is detected as an artifact associated with the particular deployment configuration.

12. 2. The computer-implemented method of claim 1, wherein the release secret stores a release that embodies a running instance of the object in the cluster.

13. verifying the integrity of the object, determining whether a chart found in the repository corresponding to said object is signed by an authorized owner; and verifying that the content of the chart has not been tampered with since the content of the chart was bundled.

14. 14. The computer-implemented method of claim 13, wherein verifying that content of the chart has not been tampered with comprises comparing content from a release source in the resource creation request with expected content from the chart found in the repository.

15. 2. The computer-implemented method of claim 1, further comprising addressing variations in object packaging and deployment by ensuring that the variations are associated with and matched against a signature from a repository or within the cluster.

16. 2. The computer-implemented method of claim 1, wherein the object is a software package.

17. 2. The computer-implemented method of claim 1, wherein the object is a single YAML Ain't Markup Language (YAML)-based deployment.

18. The computer-implemented method of claim 1 , wherein the object is an operator-based application bundle.

19. A computer program product that causes a computer to carry out a method according to any one of claims 1 to 18.

20. a server in the cluster for verifying the integrity of objects deployed in the cluster, a memory configured to store program code; a hardware processor configured to execute the program code; The hardware processor includes: Detects a resource creation request from a client, in response to the resource creation request being an initial resource creation request for the object, verifying the integrity of the object based on properties in the resource creation request to create a release secret at the cluster representing a particular deployment configuration of the object on the cluster in the event of a positive integrity verification result for the object; in response to the resource creation request not being the initial resource request, verifying whether the resource creation request corresponds to the particular deployment configuration of the object by matching against the release secret in the cluster; a server that executes the program code to, in response to the resource creation request corresponding to the deployment of the object and the release secret existing in the cluster, create in the cluster the resource requested by the resource creation request.

Images