Cluster resource processing method, related apparatus and medium

Abstract

This disclosure provides a cluster resource processing method, related apparatus, and medium. The method includes: in response to receiving a resource deletion request for a target cluster, determining the target cluster resource to which the resource deletion request points and target event information associated with the resource deletion request; determining resource tag information of the target cluster resource; if the resource deletion request is determined to be of a first type based on the target event information, performing a first verification using a first verification strategy and the resource tag information, so that the resource deletion request is executed after the first verification passes; if the resource deletion request is determined to be of a second type based on the target event information, performing a second verification using a second verification strategy and the resource tag information, so that the resource deletion request is executed after the second verification passes. This disclosure can improve the security of cluster resources. This disclosure can be applied to various scenarios such as databases, artificial intelligence, and cloud technology.

Description

Technical Field

[0001] This disclosure relates to the field of database technology, and in particular to a cluster resource processing method, related apparatus and medium. Background Technology

[0002] Currently, in cloud computing environments, container orchestration platforms (Kubernetes, K8s clusters) are often used to manage cluster resources and automate deployment. Cluster resources are mainly used for the execution of various cloud computing tasks.

[0003] To enhance the security of cluster resources, related technologies consider pre-setting resource operation permissions for different objects on various containers within the Kubernetes cluster. When an object initiates a deletion request for a specific container, and it is determined that the object has valid deletion permissions, the container's resources are deleted. This approach can meet most access control and resource management needs.

[0004] However, some container deletions in Kubernetes clusters are triggered by changes in the cluster's own state. For example, these deletions can occur due to control plane component failures or automatic deletion decisions based on node resource scarcity. Since the aforementioned methods protect against container deletion by verifying the permissions of the object initiating the deletion request, this approach often fails to intercept deletions caused by abnormal situations. This leads to significant deletion risks for Kubernetes cluster resources, resulting in low cluster resource security. Summary of the Invention

[0005] This disclosure provides a cluster resource processing method, related apparatus, and medium that can improve the security of cluster resources.

[0006] According to one aspect of this disclosure, a cluster resource processing method is provided, the method comprising:

[0007] In response to receiving a resource deletion request for a target cluster, determine the target cluster resource to which the resource deletion request points and the target event information associated with the resource deletion request;

[0008] Determine the resource tag information of the target cluster resources;

[0009] If the resource deletion request is determined to be of the first type based on the target event information, then a first verification is performed using the first verification strategy and the resource tag information, so that the resource deletion request is executed after the first verification is passed;

[0010] If the resource deletion request is determined to be of the second type based on the target event information, a second verification is performed using the second verification strategy and the resource tag information, so that the resource deletion request is executed after the second verification is passed.

[0011] According to one aspect of this disclosure, a cluster resource processing apparatus is provided, the apparatus comprising:

[0012] The first determining unit is configured to, in response to receiving a resource deletion request for a target cluster, determine the target cluster resource to which the resource deletion request points and the target event information associated with the resource deletion request;

[0013] The second determining unit is used to determine the resource tag information of the target cluster resources;

[0014] The first verification unit is configured to perform a first verification using a first verification strategy and the resource tag information if the resource deletion request is determined to be of the first type based on the target event information, so as to execute the resource deletion request after the first verification is passed.

[0015] The second verification unit is configured to perform a second verification using a second verification strategy and the resource tag information if the resource deletion request is determined to be of the second type based on the target event information, so as to execute the resource deletion request after the second verification is passed.

[0016] Optionally, the first verification unit is used for:

[0017] Determine the reference event information corresponding to the first type;

[0018] The similarity between the target event information and the reference event information is calculated to obtain the event similarity.

[0019] In response to the event similarity being greater than a first threshold, the resource deletion request is determined to be of the first type.

[0020] Optionally, the first verification unit includes:

[0021] The determination module is used to determine the initiating object of the resource deletion request;

[0022] The first verification module is used to perform a first sub-verification on the initiating object based on the first verification strategy;

[0023] The second verification module is used to perform a second sub-verification on the resource tag information based on the first verification strategy.

[0024] Optionally, the first verification module is used to:

[0025] Based on the first verification strategy, obtain the authorized operation objects of the target cluster and the baseline object information of the authorized operation objects;

[0026] Determine the target object information of the initiating object;

[0027] Based on the baseline object information and the target object information, the initiating object undergoes a first sub-verification.

[0028] Optionally, the second verification module is used for:

[0029] Based on the first verification strategy, the tag value corresponding to the target cluster resource and the associated business of the target cluster resource are extracted from the resource tag information;

[0030] If the tag value is the first value, then the initiating object of the resource deletion request is determined, and the authorized service list of the initiating object is determined, wherein the first value is used to indicate that the target cluster resource is authorized to be deleted;

[0031] If the associated service is within the list of authorized services, then the second sub-verification is deemed successful.

[0032] Optionally, the first verification module is used to:

[0033] Based on the first verification strategy, the authorization certificate is extracted from the resource deletion request, and the validity period of the authorization certificate is determined;

[0034] If the baseline object information is consistent with the target object information, and the current time is within the validity period of the authorization certificate, then the first sub-verification is determined to be successful.

[0035] Optionally,

[0036] The first verification unit is used for:

[0037] If the target cluster resource is a container group, the target cluster resource and the resources associated with the target cluster resource are unbound in order to reclaim the resources associated with the target cluster resource;

[0038] Stop the container group from running, and after the container group stops running, delete the container group and its associated log information.

[0039] Optionally, the first verification unit includes:

[0040] The rate limiting verification module is used to perform rate limiting verification on the resource deletion request after the first verification is passed;

[0041] The detection module is used to perform associated resource detection on the target cluster resources if the rate limit verification passes, and obtain the detection result, wherein the detection result is used to indicate whether the target cluster resources depend on other cluster resources of the target cluster;

[0042] An execution module is configured to execute the resource deletion request if the detection result indicates that the target cluster resource does not depend on the other cluster resources.

[0043] Optionally, the speed limit verification module is used for:

[0044] Determine the number of resources to be deleted from the target cluster and the number of remaining tokens in the preset token bucket;

[0045] If the number of resources to be deleted is less than or equal to the second threshold and the number of remaining tokens is greater than or equal to the first number, then the rate limit verification is determined to be successful.

[0046] Optionally, the detection module is used for:

[0047] Determine the resource hierarchy architecture of the target cluster;

[0048] Locate the target cluster resources in the resource hierarchy architecture to obtain the resource dependencies of the target cluster resources;

[0049] If, based on the resource dependency relationship, it is determined that the target cluster resource does not have any lower-level cluster resources in the resource hierarchy architecture, then the detection result is determined to mean that the target cluster resource does not depend on the other cluster resources.

[0050] Optionally, the second verification unit is used for:

[0051] Based on the second verification strategy, a preset protection component is invoked, and the preset protection component is used to extract the tag value corresponding to the target cluster resource and the associated business of the target cluster resource from the resource tag information;

[0052] If the tag value is the first value, then the initiating object of the resource deletion request is determined, and the authorized service list of the initiating object is determined, wherein the first value is used to indicate that the target cluster resource is authorized to be deleted;

[0053] If the associated service is in the list of authorized services, then the second verification is deemed successful.

[0054] Optionally, the node agent component of the target cluster communicates with the container management component of the target cluster through the preset protection component;

[0055] The cluster resource processing device further includes an update unit, which is used for:

[0056] Detect the current state of the preset protection component;

[0057] If the current status indicates that the preset protection component is deployed abnormally or that the preset protection component is undergoing a version upgrade, then the communication connection between the preset protection component and the node agent component and the container management component is cut off, and the node agent component and the container management component are directly connected.

[0058] Optionally, the cluster resource processing device further includes a verification unit, which is used for:

[0059] In response to the failure of the second verification, the resource deletion request is sent to the operation and maintenance service platform, so that when the operation and maintenance service platform determines that the resource deletion request meets the first condition, it adds a target field identifier to the resource deletion request to obtain an updated resource deletion request and returns the updated resource deletion request.

[0060] The updated resource deletion request is received, and the request is verified using a first verification strategy and the resource tag information of the updated resource deletion request, so that the updated resource deletion request is executed after the request verification is successful.

[0061] Optionally, the resource deletion request is generated in one of the following three ways:

[0062] The resource deletion request is generated based on the command code written by the target object using a preset command-line tool;

[0063] The resource deletion request is generated based on the resource deletion operation initiated in the target application by the development toolkit;

[0064] The control manager of the target cluster periodically probes the cluster status of the target cluster, and when it determines that the cluster status meets the second condition, the control manager generates the resource deletion request.

[0065] Optionally, the resource deletion request is generated by the node proxy component in the target cluster in one of the following three ways:

[0066] In response to the detection that a target node among multiple nodes in the target cluster is experiencing resource stress, the node proxy component determines the target container group among multiple container groups associated with the target node and generates a resource deletion request for the target container group.

[0067] In response to the detection that the backend storage path associated with the target cluster resource of the target cluster has been deleted, the node proxy component generates a resource deletion request for the target cluster resource.

[0068] In response to a resource version incompatibility issue arising after an upgrade of the API server in the target cluster, the node agent component generates a resource deletion request for the target container group.

[0069] According to one aspect of this disclosure, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the cluster resource processing method as described above.

[0070] According to one aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program that, when executed by a processor, implements the cluster resource processing method described above.

[0071] According to one aspect of this disclosure, a computer program product is provided, the computer program product including a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the cluster resource processing method as described above.

[0072] In this embodiment, resource tagging information is set for various resources of the target cluster to identify whether different types of cluster resources are allowed to be deleted. Specifically, when a resource deletion request for the target cluster is received, the request is classified according to the target event information corresponding to the request. If the resource deletion request is determined to be triggered normally, it is considered to be of the first type, and a first verification is performed according to the first verification strategy and the resource tagging information. The resource deletion request is only executed after the first verification passes. Conversely, if the resource deletion request is determined to be triggered abnormally, it is considered to be of the second type, and a second verification is performed according to the second verification strategy and the resource tagging information. The resource deletion request is only executed after the second verification passes. This ensures that corresponding verification processes are executed for deletion operations caused by various situations, effectively intercepting and protecting cluster resource deletions, reducing the risk of abnormal deletion of cluster resources in the target cluster, and thus improving the security of the cluster resources in the target cluster.

[0073] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0074] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0075] Figure 1 This is a system architecture diagram of the cluster resource processing method applied according to the embodiments of this disclosure;

[0076] Figures 2A-2B A schematic diagram is shown illustrating the application of a cluster resource processing method according to an embodiment of the present disclosure in a container deletion scenario;

[0077] Figure 3 This is a flowchart of a cluster resource processing method according to an embodiment of the present disclosure;

[0078] Figure 4 This is a schematic diagram illustrating various deletion types protected by interception according to an embodiment of the present disclosure;

[0079] Figure 5 This is a schematic diagram of various types of clusters protected by interception according to an embodiment of the present disclosure;

[0080] Figure 6 This is a flowchart illustrating the determination of a resource deletion request as a first type according to an embodiment of the present disclosure;

[0081] Figure 7 This is a schematic diagram illustrating specific scenarios corresponding to various deletion types according to an embodiment of this disclosure;

[0082] Figure 8 This is a flowchart illustrating a first verification process based on a first verification strategy and resource tag information, according to an embodiment of this disclosure.

[0083] Figure 9 This is a flowchart of the first sub-verification according to an embodiment of the present disclosure;

[0084] Figure 10 This is a flowchart of the second sub-verification according to an embodiment of the present disclosure;

[0085] Figures 11A-11C This is a schematic diagram illustrating the implementation process of performing a first verification using a first verification strategy and resource tag information according to an embodiment of this disclosure;

[0086] Figure 12 This is a flowchart of the first sub-verification according to another embodiment of this disclosure;

[0087] Figure 13 This is a flowchart of executing a resource deletion request according to an embodiment of the present disclosure;

[0088] Figure 14 This is a flowchart of associated resource detection according to an embodiment of the present disclosure;

[0089] Figure 15 This is a schematic diagram of a resource hierarchy architecture according to an embodiment of the present disclosure;

[0090] Figures 16A-16B This is a schematic diagram illustrating the implementation process of the first verification according to an embodiment of the present disclosure;

[0091] Figure 17 This is a flowchart illustrating a second verification process based on an embodiment of the present disclosure, using a second verification strategy and resource tag information.

[0092] Figure 18 This is a schematic diagram illustrating the implementation process of the second verification according to an embodiment of the present disclosure;

[0093] Figure 19 This is a flowchart illustrating the modification of the communication connection of a preset protection component according to an embodiment of the present disclosure;

[0094] Figure 20 This is a flowchart illustrating the verification of a resource deletion request according to an embodiment of the present disclosure;

[0095] Figure 21 This is a block diagram of a cluster resource processing apparatus according to an embodiment of the present disclosure;

[0096] Figure 22 This is a terminal structure diagram of a cluster resource processing method according to an embodiment of the present disclosure;

[0097] Figure 23 This is a server structure diagram of a cluster resource processing method according to an embodiment of the present disclosure. Detailed Implementation

[0098] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0099] The following explains the terms and concepts related to the embodiments of this disclosure.

[0100] Kubernetes clusters are open-source container orchestration platforms used to automate the deployment, scaling, and management of containerized applications. A Kubernetes cluster typically consists of a control plane and a set of worker nodes. The control plane is primarily responsible for managing the worker nodes and container groups (Pods) within the Kubernetes cluster, while the worker nodes host the container groups (Pods) that make up the application workload. A Kubernetes cluster requires at least one worker node to run container groups (Pods). In production environments, the control plane often runs across multiple machines to provide fault tolerance and high availability.

[0101] In the Kubernetes cluster workflow, users interact with the Kubernetes cluster via API, submitting application configuration information. The API layer is responsible for receiving and processing user requests and forwarding them to the control layer. The control layer contains various controllers, such as the Deployment Controller and ReplicaSet Controller, which are responsible for monitoring the cluster status and adjusting it according to the user's desired state. The node layer runs containers and node agent components such as kubelet, and is responsible for the actual creation and management of containers.

[0102] Kubernetes cluster management involves several core components, specifically: the distributed key-value store database etcd, the Controller Manager, the node agent component Kubelet, and the scheduling component Scheduler. The distributed key-value store database etcd stores the cluster's metadata; the Controller Manager runs various controller processes to ensure the cluster state meets expectations; the Scheduler schedules container groups (Pods) to appropriate nodes for execution; and the node agent component Kubelet is responsible for maintaining and managing container groups on nodes.

[0103] The system architecture and scenarios in which this disclosure is applied are described below.

[0104] Figure 1 This is a system architecture diagram of the cluster resource processing method applied according to an embodiment of the present disclosure. It includes an object terminal 140, an Internet 130, a gateway 120, a target cluster, etc., the target cluster including a cluster API server 110, multiple container groups, and resource storage space 150.

[0105] The target terminal 140 includes various forms such as desktop computers, laptops, PDAs (personal digital assistants), mobile phones, vehicle terminals, home theater terminals, and dedicated terminals. Furthermore, it can be a single device or a collection of multiple devices. The target terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data. The target terminal 140 includes a resource processing platform, which is used to provide the target with various operational instructions to issue to the target cluster.

[0106] Cluster API server 110 refers to a computer system that can provide certain services to object terminal 140. Compared to ordinary object terminal 140, cluster API server 110 has higher requirements in terms of stability, security, and performance. Cluster API server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), a combination of portions of multiple high-performance computers (e.g., virtual machines), or a cloud server, etc. Cluster API server 110 contains various types of services, and the implementation of each service of cluster API server 110 is often associated with some intermediate databases or storage media. Cluster API server 110 is used to interact with object terminal, receiving and processing various requests sent by object through object terminal. Resource storage space 150 is used to store various cluster resources in the target cluster. Container group is used to run different business processes or applications.

[0107] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from target terminal 140 to cluster API server 110 are forwarded to the corresponding cluster API server 110 via gateway 120. Messages sent from cluster API server 110 to target terminal 140 are also forwarded to the corresponding target terminal 140 via gateway 120.

[0108] The embodiments disclosed herein can be applied in various scenarios, such as Figures 2A-2B The example shown is a container deletion scenario.

[0109] like Figure 2AAs shown, the target cluster includes a cluster API server 110, resource storage space 150, and multiple nodes. Each node runs a corresponding container group; node 1 runs container group 1, node 2 runs container group 2, ..., node n runs container group n. When an object sends a deletion command "delete cluster resources of container group 2" to the target cluster through the resource processing platform of the object terminal, the deletion command will first be received by the target interface of the target cluster.

[0110] like Figure 2B As shown, when the target interface receives a deletion command sent by an object, it first forwards the command to the deletion verification module in the target cluster. The verification module verifies the deletion command according to preset verification logic, and only after successful verification is the command sent to the cluster API server for execution. Furthermore, to align with the preset verification logic in the verification module, each container group running in the target cluster carries a tag indicating whether the cluster resources of that container group are allowed to be deleted. Specifically, the cluster resources of container group 1 and container group n are allowed to be deleted, while the cluster resources of container group 2 are not allowed to be deleted. In addition, when each node in the target cluster runs each container group, it needs to pass through a protection component to ensure that the deletion of resources in each container group is protected by this component, thus achieving multiple layers of resource deletion protection for the target cluster.

[0111] The embodiments of this disclosure are described in general below.

[0112] According to one embodiment of this disclosure, a cluster resource processing method is provided.

[0113] This cluster resource handling method is generally applied in business scenarios where it is necessary to intercept and protect the deletion operations of certain critical resources in the Kubernetes cluster, for example... Figures 2A-2B The illustration depicts a container deletion scenario. This disclosure provides a scheme for intercepting and protecting various types of resource deletions within a target cluster, thereby improving the security of cluster resources.

[0114] like Figure 3 As shown, a cluster resource processing method according to an embodiment of this disclosure may include:

[0115] Step 310: In response to receiving a resource deletion request for the target cluster, determine the target cluster resource to which the resource deletion request is directed and the target event information associated with the resource deletion request;

[0116] Step 320: Determine the resource tag information of the target cluster resources;

[0117] Step 330: If the resource deletion request is determined to be of the first type based on the target event information, then the first verification is performed using the first verification strategy and the resource tag information, so that the resource deletion request is executed after the first verification is passed;

[0118] Step 340: If the resource deletion request is determined to be of the second type based on the target event information, then a second verification is performed using the second verification strategy and the resource tag information, so that the resource deletion request is executed after the second verification is passed.

[0119] Steps 310-340 are described in detail below.

[0120] In step 310, in response to receiving a resource deletion request for the target cluster, the target cluster resource to which the resource deletion request is directed and the target event information associated with the resource deletion request are determined.

[0121] The target cluster in this embodiment of the disclosure refers to a Kubernetes cluster.

[0122] A resource deletion request refers to a request to delete resources related to container groups, persistent volumes, cluster jobs, etc. in a target cluster.

[0123] For example, a resource deletion request is a request initiated by an object on the control plane to delete container A in container group 1 of the target cluster.

[0124] The target cluster resource refers to the cluster resource specified in the resource deletion request to be deleted. Cluster resources are various resources that already exist and are running in the target cluster. For example, namespaces, persistent volumes (PVs), container groups (Pods), scheduled jobs (CronJobs), daemon sets, etc., all correspond to different types of resources in the target cluster.

[0125] The target event information is used to indicate the specific content of the resource deletion event corresponding to the resource deletion request. For example, the target event information corresponding to a resource deletion request is "Due to the resource utilization rate of node 1 exceeding the predetermined range, the container K with the fewest uses in node 1 is deleted".

[0126] In this specific implementation, the resource deletion request records the event on which it was triggered, who triggered it, and the type and location of the cluster resource to be deleted. Therefore, when a resource deletion request for a target cluster is received, the target cluster, in response to receiving the request, determines the target cluster resource and the associated target event information based on the specific content recorded in the request.

[0127] In step 320, the resource tag information of the target cluster resources is determined.

[0128] Resource tag information is used to indicate whether the target cluster resource is allowed to be deleted, and to indicate the specific business branch served by the target cluster resource, etc.

[0129] In this specific implementation, the resource tag information of the target cluster resource is closely bound to the target cluster resource. When a target cluster resource is specified in a resource deletion request, the resource tag information of the target cluster resource will be included in the request. Based on this, with authorization, the resource tag information of the target cluster resource can be extracted from the resource deletion request.

[0130] For example, the resource tag information of the target cluster resource is "The target cluster resource is allowed to be deleted, and the target cluster resource supports / serves business K".

[0131] In step 330, if the resource deletion request is determined to be of the first type based on the target event information, then the first verification is performed using the first verification strategy and the resource tag information, so that the resource deletion request is executed after the first verification is passed.

[0132] The first type indicates that the resource deletion request was triggered under normal circumstances. For example, the first type often refers to a resource deletion request generated based on a deletion command initiated by the developer or user.

[0133] The first verification strategy is a custom-based verification mechanism. In this embodiment, the first verification strategy refers to a deletion protection strategy that relies on a Webhook.

[0134] It should be noted that in a K8s cluster, there are often two types of webhooks: one is MutatingAdmissionWebhook, which is used to inject specific fields or default values ​​when modifying resources; the other is ValidatingAdmissionWebhook, which is used to verify the legality of resource modifications. The first verification strategy in this embodiment relies on K8sValidatingAdmissionWebhook.

[0135] To save space, the specific implementation process of determining the resource deletion request as a first type based on the target event information, the specific implementation process of performing the first verification through the first verification strategy and resource tag information, and the specific implementation process of executing the resource deletion request after the first verification is passed, according to the embodiments of this disclosure, will be described in detail below. They will not be repeated here.

[0136] In step 340, if the resource deletion request is determined to be of the second type based on the target event information, a second verification is performed using the second verification strategy and the resource tag information so that the resource deletion request can be executed after the second verification is passed.

[0137] The second type indicates that the resource deletion request was triggered under abnormal circumstances, typically generated by the node agent component of the target cluster. For example, when the node agent component detects resource shortages on a node or an abnormal situation such as a version change in the cluster API server, it automatically triggers and decides to delete certain cluster resources, and the resulting resource deletion request falls into the second type.

[0138] The second verification strategy is used to instruct the specific verification process and logic for resource deletion triggered by anomalies. This second verification strategy relies on pre-deployed protection components in the target cluster.

[0139] In this specific implementation, the process of determining the resource deletion request as the second type based on the target event information is similar to the process of determining the resource deletion request as the first type based on the target event information in step 330 above. Similarly, the process of executing the resource deletion request after the second verification is passed is similar to the process of executing the resource deletion request after the first verification is passed in step 330 above. For the sake of brevity, these details will not be elaborated further.

[0140] To save space, the specific implementation process of the second verification using the second verification strategy and resource tag information in this embodiment will be described in detail below. It will not be repeated here.

[0141] Through steps 310-340 above, this embodiment of the disclosure considers setting resource tag information for various resources of the target cluster, using resource tag information to identify whether different types of cluster resources are allowed to be deleted. Specifically, when a resource deletion request for the target cluster is received, the received resource deletion request is classified according to the target event information corresponding to the resource deletion request. If it is determined that the resource deletion request is triggered normally, it is considered to be of the first type, and a first verification is performed according to the first verification strategy and the resource tag information. The resource deletion request is only executed after the first verification passes. At the same time, if it is determined that the resource deletion request is triggered abnormally, it is considered to be of the second type, and a second verification is performed according to the second verification strategy and the resource tag information. The resource deletion request is only executed after the second verification passes. In this way, corresponding verification processes are performed for deletion operations caused by multiple situations, thereby effectively achieving interception and protection of cluster resource deletion, reducing the risk of abnormal deletion of cluster resources of the target cluster, and thus improving the security of cluster resources of the target cluster.

[0142] The above is a general description of steps 310-340. The following will provide a detailed description of the specific implementation of steps 310, 320, 330, and 340.

[0143] Steps 310-320 are described in detail below.

[0144] In steps 310-320, in response to receiving a resource deletion request for the target cluster, the target cluster resource to which the resource deletion request points and the target event information associated with the resource deletion request are determined; the resource tag information of the target cluster resource is determined.

[0145] like Figure 4 The diagram illustrates the categorization of various deletion types. Specifically, based on whether each deletion type involves the cluster API server, they are divided into two main categories. Deletion types involving the cluster API server are classified as the first type, while those not involving the cluster API server are classified as the second type. Deletions involving the cluster API server include deletions based on command-line tools (kubectl), deletions based on the control manager (kube-controller-manager), and deletions based on target applications (SDK), etc. The first type is often considered to be normally triggered deletions. Deletions involving the cluster API server include deletions based on node agent components (Kubelet), deletions based on resource storage (etcd), etc. The second type is often considered to be abnormally triggered deletions.

[0146] In one embodiment, when a resource deletion request is triggered normally, the resource deletion request is generated in one of the following three ways:

[0147] Method 1: Generate a resource deletion request based on command code written using a preset command-line tool for the target object.

[0148] Here, the command-line tool refers to the kubectl command-line tool in the target cluster. The command code is the code segment used to instruct resource deletion.

[0149] Specifically, users can send delete commands using the command-line tool kubectl to attempt to delete resources in the target cluster. Based on this, a specific resource deletion request can be generated according to the specific command code written for the target object.

[0150] Method 2: Generate a resource deletion request based on the resource deletion operation initiated in the target application by the development toolkit.

[0151] The target application refers to the application in the various language client libraries of the target Kubernetes cluster, including but not limited to the Go language client.

[0152] Specifically, the object can use the development kit to interact with the cluster API server (Kubernetes API) in the Go application (target application) of the Go language client, and generate a resource deletion request for the target cluster resource by initiating a resource deletion operation in the target application.

[0153] Method 3: The control manager of the target cluster periodically probes the cluster status of the target cluster, and when it is determined that the cluster status meets the second condition, the control manager generates a resource deletion request.

[0154] Here, "Controller Manager" refers to the Controller Manager. The cluster status indicates the overall operational status of the target cluster. The second condition specifies the state the target cluster must meet before its resources can be deleted.

[0155] Specifically, the control manager of the target cluster periodically probes the cluster status of the target cluster. When it determines that the cluster status meets the second condition, it considers that resources of the target cluster need to be deleted. At this time, the control manager generates a resource deletion request.

[0156] In this embodiment, when a resource deletion request is triggered abnormally, the resource deletion request is generated by the node proxy component in the target cluster through one of the following three methods:

[0157] Method 1: In response to the detection of resource shortage on a target node among multiple nodes in the target cluster, the node agent component determines the target container group among multiple container groups associated with the target node and generates a resource deletion request for the target container group.

[0158] Here, a node refers to a worker node (i.e., a Kubernetes node) in the target cluster.

[0159] Resource shortage on the target node refers to a shortage of resources such as disk space, memory, and process IDs on the target node.

[0160] The target container group is the lowest priority business container group selected by the node agent component among multiple container groups.

[0161] Specifically, firstly, the node agent component in the target cluster periodically checks the resource status of each node in the target cluster. Then, when the node agent component detects that the target node's disk space resources, memory resources, process ID resources, etc. are under resource pressure (for example, the target node's memory usage is high), the node agent component selects the lowest priority business container group from the multiple container groups maintained by the target node as the target container group and generates a resource deletion request for the target container group.

[0162] Method 2: In response to the detection that the backend storage path associated with the target cluster resource has been deleted, the node proxy component generates a resource deletion request for the target cluster resource.

[0163] The backend storage path associated with the target cluster resources refers to the etcd resource path in the target cluster.

[0164] Specifically, when the node agent component detects that the backend storage path associated with the target cluster resource of the target cluster has been deleted (the etcd resource path of the target cluster resource has been deleted), the node agent component will generate a resource deletion request for the target cluster resource.

[0165] Method 3: In response to resource version incompatibility issues arising after an upgrade of the API server in the target cluster, the node agent component generates a resource deletion request for the target container group.

[0166] The API server refers to the kube-apiserver in the target cluster.

[0167] Specifically, when an upgrade of the API server in the target cluster is detected and a resource version field incompatibility occurs (the version number or version file of the current API server is inconsistent with the version number or version file recorded locally), the node agent component will automatically generate a resource deletion request for the target container group.

[0168] The advantage of this embodiment is that it defines the specific methods for generating resource deletion requests under normal circumstances and under abnormal circumstances, enabling the interception and protection scheme of this embodiment to be applicable to resource deletion under various circumstances. This provides a more reliable and comprehensive interception and protection against cluster resource deletion as a whole, thereby improving the security of cluster resources and reducing the risk of critical resources being accidentally or maliciously deleted.

[0169] It should be noted that the various types of deletion requests triggered normally and abnormally are merely illustrative examples. In practical applications, all resource deletion requests that pass through the cluster API server can be considered as type 1, and all resource deletion requests that do not pass through the cluster API server can be considered as type 2.

[0170] like Figure 5 The diagram illustrates a simplified representation of various deletion types for deleting different types of cluster resources. Specifically, deletion requests based on command-line tools (kubectl), controllers (kube-controller-manager), target applications (SDK), node agent components (Kubelet), and resource storage (etcd) can all request the deletion of various cluster resources, including namespaces, custom resource definitions (CRDs), container groups (Pods), persistent volume declarations (PVCs), persistent volume declarations (PVs), deployments (Deployments), stateful sets (StatefulSets), daemon sets (DaemonSets), jobs, and scheduled jobs (CronJobs). Furthermore, this disclosure provides deletion protection based on a first verification strategy (protector-Webhook) and two deletion protection methods based on a second verification strategy (protector-cri) to determine whether the target cluster resource specified in the deletion request is allowed to be deleted.

[0171] Step 330 will be described in detail below.

[0172] In step 330, if the resource deletion request is determined to be of the first type based on the target event information, then the first verification is performed using the first verification strategy and the resource tag information, so that the resource deletion request is executed after the first verification is passed.

[0173] Please refer to Figure 6 In one embodiment, the process of determining that a resource deletion request is of the first type based on target event information includes, but is not limited to, the following steps 610-630:

[0174] Step 610: Determine the reference event information corresponding to the first type;

[0175] Step 620: Calculate the similarity between the target event information and the reference event information to obtain the event similarity.

[0176] Step 630: In response to the event similarity being greater than the first threshold, determine that the resource deletion request is of the first type.

[0177] Steps 610-630 are described in detail below.

[0178] In step 610, the reference event information corresponding to the first type is determined.

[0179] Reference event information is used to indicate the specific circumstances included in a normally triggered deletion type.

[0180] In the specific implementation of this embodiment, before deleting and protecting cluster resources, various scenarios included in normally triggered deletion types and various scenarios included in abnormally triggered deletion types can be collected and summarized in advance, and the specific content of different scenarios can be stored corresponding to the first type and the second type, respectively. Based on this, with authorization, the various specific scenarios corresponding to the first type can be determined as reference event information corresponding to the first type.

[0181] It should be noted that since there are many specific scenarios for normally triggered deletion types, there are often multiple reference event information for the first type.

[0182] In step 620, the similarity between the target event information and the reference event information is calculated to obtain the event similarity.

[0183] Event similarity indicates how similar the target event information of a resource deletion request is to a normally triggered deletion type. The higher the event similarity, the closer the target event information of the resource deletion request is to a normally triggered deletion type.

[0184] In this specific implementation, for each reference event information, both the target event information and the reference event information are vectorized. Then, a cosine similarity algorithm or similar method is used to calculate the similarity between the vectorized target event information and the vectorized reference event information to obtain the event similarity. Specifically, the closer the specific content between the target event information and the reference event information is, the greater the event similarity.

[0185] In step 630, in response to the event similarity being greater than a first threshold, the resource deletion request is determined to be of the first type.

[0186] The first threshold is used to indicate the minimum requirement for classifying a resource deletion request as a normally triggered deletion type.

[0187] In this specific implementation, firstly, the event similarity between the target event information and each reference event information is compared with a first threshold. Next, if the event similarity between the target event information and a certain reference event information is greater than the first threshold, it indicates that the target event information fits the situation corresponding to that reference event information, and the resource deletion request can be identified as a normally triggered deletion type. Therefore, the resource deletion request is determined to be of the first type. If the event similarity between the target event information and any reference event information is less than or equal to the first threshold, it indicates that the target event information does not match the situation corresponding to any of the reference event information, and the resource deletion request is not a normally triggered deletion type. Therefore, the resource deletion request is determined not to be of the first type.

[0188] like Figure 7 The diagram illustrates a simplified representation of different deletion types. Specifically, deletions based on command-line tools (kubectl), deletions based on the control manager (kube-controller-manager), and deletions based on target applications (SDK) are often considered to fall under the first type (normally triggered deletion). Specific scenarios for command-line tool-based deletion include "Xxxxx, sending a delete command via kubectl to delete resources in the cluster," etc.; deletions based on the control manager include "Xxx, deletion triggered by the control manager handling tasks such as node crashes, network disconnections, or kubelet exceptions," etc.; and deletions based on target applications include "deletion triggered by the developer interacting with the Kubernetes API in a Go application, xxxx," etc. Deletions based on resource storage (etcd) and deletions based on node agent components (Kubelet) are often considered to fall under the second type (abnormally triggered deletion). Among them, the specific scenarios corresponding to deletion based on resource storage include "Xxxxx, etcd resource path is deleted", etc.; the specific scenarios corresponding to deletion based on node agent components include "node resource shortage triggers Pod eviction and reconstruction, upgrade the version of cluster API server", etc.

[0189] The advantage of this embodiment is that it takes into account that various deletion types triggered normally may include multiple scenarios. By calculating the similarity between the reference event information corresponding to different scenarios and the target event information corresponding to the resource deletion request, the similarity of the events is used to measure whether the deletion type of the resource deletion request is one of the normally triggered deletion types. This can improve the efficiency and accuracy of identifying the specific type of resource deletion request, so as to take corresponding interception and protection measures for normally triggered / abnormally triggered resource deletion, thereby improving the security of cluster resource deletion.

[0190] Please refer to Figure 8 In one embodiment, the process of performing the first verification through the first verification strategy and resource tag information includes, but is not limited to, the following steps 810-830:

[0191] Step 810: Determine the object that initiated the resource deletion request;

[0192] Step 820: Perform the first sub-verification on the initiating object based on the first verification strategy;

[0193] Step 830: Perform a second sub-verification on the resource tag information based on the first verification strategy.

[0194] Steps 810-830 are described in detail below.

[0195] In step 810, the object that initiated the resource deletion request is determined.

[0196] The initiating object refers to the user, developer, or control process that triggers the resource deletion request.

[0197] In this specific implementation, since various requests, such as resource deletion requests, carry a source identifier, the resource deletion request is first parsed with authorization to extract a field that identifies the specific source of the request. Then, based on the value of the extracted field, the specific object pointed to by that field value is located, and this located object is identified as the initiating object of the resource deletion request.

[0198] In step 820, the initiating object is subjected to the first sub-verification based on the first verification strategy.

[0199] The first sub-verification is used to verify whether the object initiating the resource deletion request has the legal authority to delete the cluster resources of the target cluster.

[0200] To save space, the specific process of performing the first sub-verification on the initiating object based on the first verification strategy in this embodiment will be described in detail below. It will not be repeated here.

[0201] In step 830, the resource tag information is subjected to a second sub-verification based on the first verification strategy.

[0202] The second sub-verification is used to verify whether the target cluster resource specified in the resource deletion request to be deleted is a resource type that is allowed to be deleted.

[0203] To save space, the specific process of performing the second sub-verification of resource tag information based on the first verification strategy in this embodiment will be described in detail below. It will not be repeated here.

[0204] The advantage of this embodiment is that, for resource deletion requests triggered under normal circumstances, it takes into account both the legitimate permission verification of the initiating object and the permission deletion verification of the cluster resource to be deleted. It can achieve interception and protection of resource deletion under multi-dimensional verification, reduce the risk of erroneous deletion of cluster resources that are not allowed to be deleted, and improve the security of cluster resources.

[0205] Please refer to Figure 9 In one embodiment, the process of performing a first sub-verification on the initiating object based on a first verification strategy includes, but is not limited to, the following steps 910-930:

[0206] Step 910: Based on the first verification strategy, obtain the authorized operation objects and the baseline object information of the authorized operation objects in the target cluster;

[0207] Step 920: Determine the target object information of the initiating object;

[0208] Step 930: Based on the baseline object information and the target object information, perform the first sub-verification on the initiating object.

[0209] Steps 910-930 are described in detail below.

[0210] In step 910, based on the first verification strategy, the authorized operation objects of the target cluster and the baseline object information of the authorized operation objects are obtained.

[0211] An object with authorized operation rights refers to an object that has the legal authority to delete cluster resources of the target cluster.

[0212] Baseline object information is used to indicate the object identifier, terminal address, business branch, etc. of the object that is authorized to operate.

[0213] In the specific implementation of this embodiment, the object data of the authorized operation objects in the target cluster are often recorded in the backend server or database. Based on this, with authorization, data such as the object identifier, object terminal address, and business branch to which each authorized operation object belongs can be extracted from the backend server or database, and the object identifier, object terminal address, and business branch to which they belong can be directly determined as the base object information.

[0214] Furthermore, since the object identifier, object terminal address, and business branch to which it belongs are all in plaintext, in order to improve the information privacy of each authorized object, the extracted object identifier, object terminal address, and business branch to which it belongs can be concatenated to obtain the concatenation result. A preset digest algorithm is then used to perform a digest operation on the concatenation result, converting the plaintext object data into ciphertext. The digest value obtained from the digest operation is then used as the base object information of the authorized object.

[0215] In step 920, the target object information of the initiating object is determined.

[0216] The target object information is used to indicate the object identifier, terminal address, business branch, etc. of the initiating object.

[0217] In the specific implementation of this embodiment, the process of step 920 is similar to that of step 910 described above. To save space, it will not be described again.

[0218] In step 930, the first sub-verification is performed on the initiating object based on the baseline object information and the target object information.

[0219] In this specific implementation, firstly, the target object information of the initiating object is compared with the baseline object information of each authorized operation object. Next, if the target object information of the initiating object is completely consistent with the baseline object information of a certain authorized operation object, it indicates that the initiating object is indeed the authorized operation object, and the initiating object has the legitimate permission to delete the cluster resources of the target cluster. Therefore, the first sub-verification is deemed successful. If the target object information of the initiating object is consistent with the baseline object information of all authorized operation objects, it indicates that the initiating object does not have the legitimate permission to delete the cluster resources of the target cluster. Therefore, the first sub-verification is deemed unsuccessful, and the resource deletion request is rejected.

[0220] The advantage of this embodiment is that when intercepting and protecting the target cluster resource specified in the resource deletion request, it takes into account the comparison between the baseline information of the authorized operation object and the target object information of the initiating object to verify the permission of the initiating object. This realizes the interception and protection of cluster resource deletion through object permission verification, which can improve the accuracy of interception and protection and reduce the risk of cluster resources being maliciously or mistakenly deleted by objects without operation permissions, thereby improving the security of cluster resources in the target cluster.

[0221] Please refer to Figure 10 In one embodiment, the process of performing a second sub-verification on resource tag information based on the first verification strategy includes, but is not limited to, the following steps 1010-1030:

[0222] Step 1010: Based on the first verification strategy, extract the tag value corresponding to the target cluster resource and the associated business of the target cluster resource from the resource tag information;

[0223] Step 1020: If the tag value is the first value, determine the initiating object of the resource deletion request and determine the list of authorized services of the initiating object;

[0224] Step 1030: If the associated business is in the list of authorized businesses, then the second sub-verification is confirmed to be successful.

[0225] Steps 1010-1030 are described in detail below.

[0226] In step 1010, based on the first verification strategy, the tag value corresponding to the target cluster resource and the associated business of the target cluster resource are extracted from the resource tag information.

[0227] Tag values ​​are used to indicate whether a target cluster resource is allowed to be deleted, either numerically or alphabetically.

[0228] Related services are used to indicate business processes or applications that require the resources of the target cluster to function properly.

[0229] For example, the target cluster contains container group A and container group B. Container group A is used for executing service 1, while container group B is used for executing service 2. Based on this, the cluster resources corresponding to container group A allow deletion operations on objects belonging to service 1, but do not allow deletion of objects belonging to other services.

[0230] In this specific implementation, since the tag values ​​or annotations indicating whether the target cluster resource is allowed to be deleted are all in the resource tag information, and the business information of the specific services supported by the target cluster resource is also recorded in the resource tag information, based on this, with authorization, the tag value corresponding to the target cluster resource is extracted from the resource tag information based on the first verification strategy, and the associated services of the target cluster resource are extracted from the resource tag information.

[0231] In step 1020, if the tag value is the first value, the initiating object of the resource deletion request is determined, and the list of authorized services of the initiating object is determined.

[0232] The first value indicates that the target cluster resource has been deleted.

[0233] The list of authorized services is used to indicate the business branch to which the initiating object belongs. When the initiating object belongs to a certain business branch, it may have operation permissions for various resources related to that business in some cases.

[0234] In this specific implementation, if the tag value is the first value, it indicates that the target cluster resources have been deleted from the target cluster. Next, the initiating object of the resource deletion request is determined, and based on the business branch to which the initiating object belongs, the list of authorized businesses for that initiating object is determined. This process is similar to step 810 above. For brevity, it will not be elaborated further.

[0235] Furthermore, if the tag value is not the first value, it indicates that the target cluster resource is a type of resource that is not allowed to be deleted by default in the target cluster. Deleting this target cluster resource would significantly affect the normal operation of the target cluster; therefore, the resource deletion request is directly rejected.

[0236] In step 1030, if the associated service is in the list of authorized services, then the second sub-verification is confirmed to have passed.

[0237] In this specific implementation, if the associated service is in the list of authorized services, it indicates that the associated service is consistent with a certain service in the list of authorized services, and the initiating object has the permission to read, write, add, delete, or modify the cluster resources related to the associated service. Based on this, the target cluster resources corresponding to the target container group are allowed to be deleted according to the resource deletion request issued by the initiating object, so the second sub-verification is confirmed to be successful.

[0238] Furthermore, if the associated service is not in the list of authorized services, it indicates that the associated service is inconsistent with the various services in the list, and the initiating object does not have the permission to read, write, add, delete, or modify the cluster resources related to the associated service. Therefore, the deletion of the target cluster resources corresponding to the target container group based on the resource deletion request issued by the initiating object is not permitted. Thus, the second sub-verification is deemed unsuccessful, and the resource deletion request is rejected.

[0239] The advantage of this embodiment is that when verifying resource deletion requests, it considers whether cluster resources are allowed to be deleted, and takes into account the consistency between cluster resources associated with different businesses and the business branch to which the initiating object belongs. Based on a custom verification mechanism (first verification strategy), the deletion verification of cluster resources is considered to be successful (second sub-verification successful) only when the cluster resource is allowed to be deleted and the initiating object belongs to the business branch corresponding to the cluster resource. This approach can effectively reduce the erroneous deletion of cluster resources associated with other businesses based on resource deletion requests from objects of a certain business branch, improve the reliability of resource deletion interception and protection, and thus improve the security of cluster resources.

[0240] The following is combined with Figures 11A-11C An example is given to describe the process of intercepting and protecting resource deletion under normal circumstances.

[0241] like Figure 11AThe diagram illustrates the authorized operation object lookup table for the target cluster. Specifically, authorized operation objects in the target cluster possess legitimate read, write, add, delete, and modify permissions for various resources within the target cluster. These authorized operation objects include Object 1 (base object information: bwd1341), Object 2 (base object information: dq13d56), Object 3 (base object information: rp78oar), Object 4 (base object information: m5qr9ad), Object 5 (base object information: mqy5dq1), and Object 6 (base object information: Mtsar08). The base object information is obtained by concatenating the object identifier, object address, and associated business branch of the authorized operation object. Based on this, when a resource deletion request is received, if it is determined to be of the first type, the target object information of the initiating object of the resource deletion request is compared one by one with the base object information of the aforementioned six objects to determine if any object's base object information is identical to the initiating object's target object information.

[0242] like Figure 11B The diagram illustrates the resource tag mapping table for various types of cluster resources in the target cluster. Specifically, the target cluster has 10 different types of cluster resources: Namespace, Custom Resource Definition (CRD), Container Pod, Persistent Volume Declaration (PVC), Persistent Volume (PV), Deployment, Stateful Set, Daemon Set, Job, and CronJob. Each type of cluster resource has a tag value indicating whether it is allowed to be deleted (Allow) or denied to be deleted (Deny). In some specific scenarios, Container Pod, Deployment, Daemon Set, Stateful Set, Job, and CronJob are cluster resources that are allowed to be deleted; while Namespace, Custom Resource Definition (CRD), Persistent Volume Declaration (PVC), and Persistent Volume (PV) are cluster resources that are denied to be deleted. Based on this, when a resource deletion request is received, the target cluster resource is compared with the above 10 types of cluster resources to determine the resource type of the target cluster resource. Based on the tag value corresponding to the determined resource type, it can be determined whether the target cluster resource can be deleted.

[0243] like Figure 11CThe diagram illustrates the list of associated services for container groups in the target cluster. The associated services for a container group refer to objects within that container group that are allowed to perform operations such as modification and deletion. Specifically, container group 1 is associated with service 1; container group 2 is associated with services 1 and 2; container group 3 is associated with service 2; container group 4 is associated with services 3 and 4; and container group 5 is associated with service 5. Therefore, when a resource deletion request is initiated by an object belonging to service 2, and the request is "delete resources from container group 2 in the target cluster," since container group 2 allows deletion, the object belongs to service 2, and container group 2 serves service 2, the object has the permission to delete cluster resources in container group 2. Verification shows that the resource deletion request is executed normally.

[0244] To improve the accuracy of verifying resource deletion requests, this disclosure also provides a verification scheme based on a combination of permission verification and certificate verification. This scheme can only consider executing the resource deletion request when it is verified that the initiating object of the resource deletion request has the necessary permissions and that the resource deletion request is valid, thereby improving the security of request execution.

[0245] Please refer to Figure 12 In one embodiment, the process of performing a first sub-verification on the initiating object based on the baseline object information and the target object information includes, but is not limited to, the following steps 1210-1220:

[0246] Step 1210: Based on the first verification strategy, extract the authorization certificate from the resource deletion request and determine the validity period of the authorization certificate;

[0247] Step 1220: If the baseline object information is consistent with the target object information, and the current time is within the validity period of the authorization certificate, then the first sub-verification is confirmed to be successful.

[0248] Steps 1210-1220 are described in detail below.

[0249] In step 1210, based on the first verification strategy, the authorization certificate is extracted from the resource deletion request, and the validity period of the authorization certificate is determined.

[0250] Authorization proof refers to credential information that can prove that the initiated resource deletion request meets the established requirements and is recognized as verifiable and executable.

[0251] The authorization certificate for the resource deletion request is usually provided by a trusted credential issuing platform or the management of the target cluster when the resource deletion request is generated, and is bound to the resource deletion request.

[0252] The validity period indicates the time interval from when the resource deletion request is generated to when it is deemed invalid.

[0253] In this specific implementation, since the authorization certificate for the resource deletion request is bound to the resource deletion request, the authorization certificate can be extracted from the resource deletion request after authorization, and then parsed to obtain the validity period of the authorization certificate. Here, the authorization certificate often refers to an authorization token.

[0254] In step 1220, if the reference object information is consistent with the target object information and the current time is within the validity period of the authorization certificate, then the first sub-verification is determined to be successful.

[0255] In this specific implementation, firstly, the baseline object information is compared with the target object information, and the current time is compared with the validity period of the authorization certificate. Next, if the baseline object information matches the target object information, and the current time is within the validity period of the authorization certificate, it indicates that the initiating object has the authority to operate on the object, and the resource deletion request initiated by the initiating object is still a valid request; therefore, the first sub-verification is deemed successful. If the baseline object information does not match the target object information, it indicates that the initiating object does not have the authority to operate on the object; if the current time is not within the validity period of the authorization certificate, the resource deletion request initiated by the initiating object is an expired request, and expired requests should not be executed. Based on this, if the baseline object information does not match the target object information, and / or the current time is not within the validity period of the authorization certificate, the first sub-verification is deemed unsuccessful, and the resource deletion request is directly rejected.

[0256] The advantage of this embodiment is that, in the first sub-verification, permission verification and certificate verification are combined. After verifying the consistency between the baseline object information and the target object information, and verifying the validity of the authorization certificate, the resource deletion request can be executed only when it is verified that the object initiating the resource deletion request has the right to operate and that the resource deletion request is valid. This method can improve the accuracy of the verification of the resource deletion request, thereby improving the security of the request execution.

[0257] In one embodiment, the process of executing a resource deletion request after the first verification is successful may include the following steps:

[0258] If the target cluster resource is a container group, unbind the target cluster resource and the resources associated with the target cluster resource in order to reclaim the resources associated with the target cluster resource;

[0259] Stop the container group and delete the container group and its associated log information after the container group stops running.

[0260] Among them, the associated log information is used to indicate the log content related to the container group. This associated log information includes the container group's creation timestamp, container resource type, and program execution details of the application associated with the container group, etc.

[0261] Specifically, first, if the target cluster resource is a container group, unbind the target cluster resource (container group) from its associated resources, so that the associated resources no longer support the operation of the container group. Then, reclaim the associated resources to the target storage space, allowing them to be reused in other container groups. The target storage space can be the distributed key-value store database etcd of the Kubernetes cluster. Next, stop the container group, terminating the application associated with it. Further, after the container group stops running, delete it, so that it no longer exists in the target cluster. Simultaneously, delete the associated log information of the container group to free up log storage space in the target cluster.

[0262] The advantage of this embodiment is that when deleting a container group in the target cluster, the process first unbinds the container group and its associated resources, then reclaims the resources associated with the container group, next stops the container group, and finally deletes the container group and its associated log information. This method limits the order of operations when deleting a container group in the target cluster, improving the systematic and standardized process of container deletion, thereby achieving comprehensive deletion of container resources and improving the security of resource deletion. Furthermore, this method considers the recycling of cluster resources and the clearing of log information, which can improve the reuse rate of cluster resources and the effective utilization rate of cluster storage space to a certain extent.

[0263] Because excessive traffic to the target cluster can often disrupt its normal operation, leading to resource waste or system overload, this disclosure provides a solution that incorporates a rate-limiting verification mechanism into resource deletion protection. This mechanism enables multi-layered interception protection for critical resource deletion, improves the rationality of resource deletion requests to the target cluster, and ultimately enhances cluster resource security and operational security.

[0264] Please refer to Figure 13 In one embodiment, the specific process of executing the resource deletion request after the first verification is passed may include, but is not limited to, the following steps 1310-1330:

[0265] Step 1310: After the first verification passes, rate-limit verification is performed on the resource deletion request;

[0266] Step 1320: If the rate limit verification passes, perform associated resource detection on the target cluster resources and obtain the detection results;

[0267] Step 1330: If the detection result shows that the target cluster resources do not depend on other cluster resources, then execute the resource deletion request.

[0268] Steps 1310-1330 are described in detail below.

[0269] In step 1310, after the first verification passes, the resource deletion request is rate-limited.

[0270] In a specific implementation of this embodiment, step 1310 may include, but is not limited to, the following steps:

[0271] Determine the number of resources to be deleted in the target cluster and the number of remaining tokens in the preset token bucket;

[0272] If the number of resources to be deleted is less than or equal to the second threshold and the number of remaining tokens is greater than or equal to the first number, then the rate limit verification is considered successful.

[0273] The number of resources to be deleted indicates the amount of target cluster resources that the resource deletion request requests to be deleted.

[0274] A preset token bucket refers to a token bucket initialized for each resource type to implement traffic control over the target cluster. For example, if the target cluster is preset to allow 10 requests per second, then 10 tokens will be added to the preset token bucket every second.

[0275] It's important to note that when a request arrives, it attempts to consume one token from the pre-set token bucket. If the pre-set token bucket contains tokens, the request can proceed; otherwise, it is rejected. The state of the pre-set token bucket is updated after each request consumes a token. If the pre-set token bucket is empty, new requests will wait for it to be refilled. Rejected requests can be placed in a queue and retried when the pre-set token bucket has sufficient tokens. Furthermore, the client-side can implement a retry mechanism after receiving a rejection response, waiting a certain period before resending the request.

[0276] The number of remaining tokens indicates the total number of tokens remaining in the preset token bucket.

[0277] The second threshold is used to limit the maximum number of cluster resources that can be deleted in a single resource deletion.

[0278] The first number is used to limit the minimum number of tokens required for a resource deletion request to proceed normally.

[0279] Specifically, firstly, a summation function can be used to count the resources in the target cluster to obtain the number of resources to be deleted. Simultaneously, a summation function can also be used to count the remaining tokens in the preset token bucket to obtain the remaining token count. Next, the number of resources to be deleted is compared with a second threshold, and the remaining token count is compared with a first threshold. Further, if the number of resources to be deleted is less than or equal to the second threshold, and the remaining token count is greater than or equal to the first threshold, then the rate limiting verification is considered successful. If the number of resources to be deleted is greater than the second threshold, and / or the remaining token count is less than the first threshold, then the rate limiting verification is considered unsuccessful, and the resource deletion request is rejected.

[0280] In step 1320, if the rate limiting verification passes, the associated resource detection is performed on the target cluster resources to obtain the detection results.

[0281] The detection results are used to indicate whether the target cluster resources depend on other cluster resources of the target cluster.

[0282] In the specific implementation of this embodiment, if the rate limiting verification passes, it indicates that the execution of the resource deletion request will not lead to resource waste or system overload in the target cluster. However, considering that the target cluster resources may be interconnected with other cluster resources, and that other cluster resources may be unable to function properly without the target cluster resources, further detection of associated resources is required for the target cluster resources.

[0283] To save space, the specific process of performing associated resource detection on the target cluster resources in this embodiment of the present disclosure will be described in detail below, and will not be repeated here.

[0284] Furthermore, if the rate limiting verification fails, it indicates that executing the resource deletion request may lead to resource waste or system overload in the target cluster. In this case, the resource deletion request is rejected. Further, a retry mechanism can be set in this embodiment to resend the resource deletion request after a predetermined time, thereby re-verifying the rate limiting of the resource deletion request.

[0285] In step 1330, if the detection result indicates that the target cluster resources do not depend on other cluster resources, a resource deletion request is executed.

[0286] In this specific implementation, if the detection result indicates that the target cluster resource does not depend on other cluster resources, it means that no other cluster resources need to rely on the target cluster resource for use. Deleting the target cluster resource will not affect the normal use of other cluster resources. Based on this, a resource deletion request is executed, and the target cluster resource is deleted. Conversely, if the detection result indicates that the target cluster resource depends on other cluster resources, it means that other cluster resources need to rely on the target cluster resource for use. Deleting the target cluster resource will affect the normal use of other cluster resources. Based on this, the resource deletion request is rejected, and a message is displayed indicating that the target cluster resource is associated with other cluster resources, and direct deletion of the target cluster resource is not allowed.

[0287] The advantage of this embodiment is that it considers the potential for excessive traffic from requests to the target cluster, which could lead to wasted resources or system overload. Therefore, a rate-limiting verification mechanism is introduced into the resource deletion protection. Furthermore, considering that target cluster resources may be interdependent with resources in other clusters, and that some resources may be unusable without the target cluster's resources, a resource correlation detection step is also incorporated into the resource deletion protection. This approach achieves multiple layers of interception protection for critical resource deletion, improving the rationality of resource deletion requests to the target cluster, thereby enhancing cluster resource security and overall cluster operational security.

[0288] Please refer to Figure 14 In one embodiment, the specific process of performing associated resource detection on the target cluster resources may include, but is not limited to, the following steps 1410-1430:

[0289] Step 1410: Determine the resource hierarchy architecture of the target cluster;

[0290] Step 1420: Locate the target cluster resources in the resource hierarchy architecture and obtain the resource dependencies of the target cluster resources;

[0291] Step 1430: If, based on resource dependencies, it is determined that the target cluster resource does not have any lower-level cluster resources in the resource hierarchy architecture, then the detection result is determined to be that the target cluster resource does not depend on other cluster resources.

[0292] Steps 1410-1430 are described in detail below.

[0293] In step 1410, the resource hierarchy architecture of the target cluster is determined.

[0294] Resource hierarchy architecture is used to indicate the relationships and dependencies between various resources in the target cluster.

[0295] In this specific implementation, the resource hierarchy architecture is pre-constructed based on the interdependencies of different types of cluster resources in the target cluster and stored in a specific storage space. The resource hierarchy architecture is a tree structure, starting from the root node. The next level below the root node represents a subset of cluster resources that do not depend on any other cluster resource. The lower-level nodes of these independent cluster resources depend on those resources to function properly, and so on, with lower-level cluster resources depending on upper-level cluster resources. Based on this, with proper authorization, the resource hierarchy architecture stored in the specific storage space can be directly accessed.

[0296] like Figure 15 The diagram illustrates a simplified resource hierarchy architecture. Specifically, in the target cluster, cluster resources 1, 2, and 3 can be used without relying on other cluster resources. Cluster resources 4 and 5 depend on cluster resource 1 to function properly; cluster resource 6 depends on cluster resource 2. Based on this architecture, when a resource deletion request specifies one of cluster resources 4, 5, 6, or 3, the request can be executed directly if both the initial verification and rate limiting verification pass, because deleting cluster resources 4, 5, 6, or 3 will not affect other cluster resources. However, when a resource deletion request specifies either cluster resource 1 or 2, the request still cannot be executed directly even if both the initial verification and rate limiting verification pass. This is because changes to cluster resource 1 directly affect the normal use of cluster resources 4 and 5, and changes to cluster resource 2 directly affect the normal use of cluster resource 6.

[0297] In step 1420, the target cluster resources are located in the resource hierarchy architecture to obtain the resource dependencies of the target cluster resources.

[0298] Resource dependencies are used to indicate the operation and use of a target cluster resource, which other cluster resources it is associated with, and to indicate which higher-level cluster resources the target cluster resource belongs to, and which lower-level cluster resources belong to the target cluster resource.

[0299] In this specific implementation, since each node in the resource hierarchy corresponds to a cluster resource, the target cluster resource can be located by finding the node corresponding to it within the resource hierarchy. Next, based on the node containing the target cluster resource, the path extending upwards from that node to the root node is determined, as well as the path extending downwards from that node to the lowest level of the resource hierarchy. Further, based on the other cluster resources traversed along the path from that node to the root node and the path from that node to the lowest level of the resource hierarchy, the resource dependencies between the target cluster resource and other cluster resources are determined.

[0300] by Figure 15 For example, regarding cluster resource 1, this cluster resource has resource dependencies on cluster resource 4 and cluster resource 5, and cluster resource 4 and cluster resource 5 depend on cluster resource 1 for execution.

[0301] In step 1430, if it is determined based on resource dependencies that the target cluster resource does not have any lower-level cluster resources in the resource hierarchy architecture, then the detection result is determined to be that the target cluster resource does not depend on other cluster resources.

[0302] Lower-level cluster resources refer to cluster resources within the target cluster that belong to the target cluster. The operation and execution of lower-level cluster resources depend on the operation and execution of the target cluster resources; modifications or deletions of the target cluster resources will affect the normal operation and execution of lower-level cluster resources.

[0303] In the specific implementation of this embodiment, firstly, based on resource dependencies and the level of each cluster resource in the resource hierarchy, it is determined whether the target cluster resource has any lower-level cluster resources in the resource hierarchy. If the target cluster resource has other cluster resources directly or indirectly connected to it in the resource hierarchy, then those other cluster resources are the lower-level cluster resources. Further, if the target cluster resource does not have any lower-level cluster resources in the resource hierarchy, it indicates that no other cluster resources depend on the target cluster resource. The direct deletion of the target cluster resource will not affect the normal use of other cluster resources. Therefore, the detection result is determined to be that the target cluster resource does not depend on other cluster resources.

[0304] Furthermore, if the target cluster resource has lower-level cluster resources in the resource hierarchy architecture, and the target cluster resource is directly or indirectly connected to other cluster resources, it indicates that there are other cluster resources that depend on the target cluster resource. Directly deleting the target cluster resource will affect the normal use of other cluster resources. Therefore, the detection result is determined to be that the target cluster resource depends on other cluster resources.

[0305] The advantage of this embodiment is that, based on the pre-set resource hierarchy architecture, the target cluster resource can be located relatively quickly and accurately. Based on the path relationship in the resource hierarchy architecture, the resource dependency relationship of the target cluster resource can be found, thereby determining whether there are other cluster resources that depend on the target cluster resource. This allows the resource deletion request to be executed only without affecting the normal use of other cluster resources. It also enables the analysis of the association relationship between various cluster resources during resource deletion, thereby improving the security of deleting cluster resources.

[0306] The following is combined with Figures 16A to 16B An example is given of the process of intercepting and protecting resource deletion under normal circumstances based on a custom verification mechanism (Kubernetes Webhook).

[0307] like Figure 16A As shown, deletions based on command-line tools, control managers, and target applications are all resource deletions triggered under normal circumstances. For deletions based on command-line tools, control managers, and target applications, a deletion protection based on the first verification strategy (protector-Webhook) is employed. Specifically, when a resource deletion request is received, it is first forwarded to ValidatingAdmissionWebhook. ValidatingAdmissionWebhook verifies, according to a custom verification mechanism, whether the target cluster resource specified in the deletion request is allowed to be deleted, and generates a verification result. The specific process is similar to steps 810-830 above. Further, only after generating the verification result will the request enter the stage handled by the cluster API server kube-apiserver. At this point, only if the verification result shows that the target cluster resource is allowed to be deleted will the resource deletion request be actually executed, and the target cluster resource be deleted. In addition, the cluster API server will read / write / access the target storage space, and will restore the cluster resources reclaimed when executing resource deletion requests to the target storage space.

[0308] like Figure 16BAs shown, this embodiment of the disclosure can authorize the business containers (each container group) of the target cluster to allow normal deletion requests initiated by the business containers, while blocking other abnormal deletion requests. Specifically, when a resource deletion request is received, it first pre-checks whether the resource to be deleted has a valid resource tag or annotation to determine whether the resource to be deleted is allowed to be deleted directly. The pre-check process is similar to steps 810-830 above. Further, if the pre-check passes and the resource to be deleted is not allowed to be deleted directly, the resource deletion request is rate-limited, and the specific process is similar to step 1310 above. If the rate-limiting check fails, the resource deletion request is rate-limited, the resource deletion request is rejected, and the resource deletion request is resent after a period of time for retry. If the rate-limiting check passes, the process proceeds to the correlation detection of the resource during deletion to determine whether an error has occurred. An error in the detection means whether the target cluster resource to be deleted affects other cluster resources. If it affects other cluster resources, an error has occurred. At this point, if an error is detected, the entire process ends. If no error is detected, the deletion process requested by the resource deletion request is executed, and the specific process is similar to steps 1410-1430 above. Furthermore, if pre-verification passes and the resource to be deleted is allowed to be deleted directly, the deletion process requested by the resource deletion request is executed. To save space, this will not be elaborated further.

[0309] Step 340 will be described in detail below.

[0310] In step 340, if the resource deletion request is determined to be of the second type based on the target event information, a second verification is performed using the second verification strategy and the resource tag information so that the resource deletion request can be executed after the second verification is passed.

[0311] In this embodiment, the specific process of determining that the resource deletion request is of the second type based on the target event information is similar to steps 610-630 described above. The difference is that the event similarity greater than the first threshold described in steps 610-630 is based on... Figure 7 The specific content of the first to third lines (the various deletion types corresponding to normal circumstances) is used as reference event information. Step 340, however, requires... Figure 7 The specific content of lines four and five (various deletion types corresponding to abnormal situations) is used as reference event information. To save space, it will not be elaborated here.

[0312] Please refer to Figure 17 In one embodiment, the process of performing the second verification using the second verification strategy and resource tag information includes, but is not limited to, the following steps 1710-1730:

[0313] Step 1710: Based on the second verification strategy, call the preset protection component, and use the preset protection component to extract the tag value corresponding to the target cluster resource and the associated business of the target cluster resource from the resource tag information;

[0314] Step 1720: If the tag value is the first value, determine the initiating object of the resource deletion request and determine the list of authorized services of the initiating object;

[0315] Step 1730: If the associated business is in the list of authorized businesses, then the second verification is confirmed to be successful.

[0316] Steps 1710-1730 are described in detail below.

[0317] In step 1710, based on the second verification strategy, a preset protection component is invoked, and the preset protection component is used to extract the tag value corresponding to the target cluster resource and the associated business of the target cluster resource from the resource tag information.

[0318] The default protection component (protector-cri) refers to a pre-defined component used to implement resource deletion protection. This default protection component typically receives and processes authentication requests from the cluster API server (Kubernetes API Server) by probing the UDS. This default protection component is often probed through the default protection component interface protector.sock. The protector.sock interface is often configured as a configuration file so that the default protection component starts automatically when the target cluster starts.

[0319] Furthermore, the preset protection component can be deployed to the target cluster through a configuration file (systemdunit) defining the service; or it can be deployed to the target cluster through K8sDaemonSet, so that the preset protection component can automatically identify new nodes and install and deploy on the new nodes.

[0320] In this specific implementation, step 1710, which involves using a preset protection component to extract the tag value corresponding to the target cluster resource and the associated business of the target cluster resource from the resource tag information, is similar to step 1010 described above. The difference is that step 1010 is implemented by a custom verification mechanism Webhook, while step 1710 is implemented by the preset protection component protector.cri. For the sake of brevity, further details will not be provided.

[0321] In step 1720, if the tag value is the first value, the initiating object of the resource deletion request is determined, and the list of authorized services of the initiating object is determined.

[0322] In this specific implementation, step 1720 is similar to step 1020 described above. The difference is that step 1020 is implemented by a custom verification mechanism Webhook, while step 1720 is implemented by a preset protection component protector.sock. For the sake of brevity, further details will not be provided.

[0323] In step 1730, if the associated service is in the list of authorized services, then the second verification is confirmed to be successful.

[0324] In this specific implementation, step 1730 is similar to step 1030 described above. The difference is that step 1030 is implemented by a custom verification mechanism Webhook, while step 1730 is implemented by a preset protection component protector.sock. For the sake of brevity, further details will not be provided.

[0325] The following is combined with Figure 18 An example is given to describe the process of intercepting and protecting resources from deletion under abnormal conditions based on the preset protection component (protector.cri).

[0326] like Figure 18As shown, resource deletion triggered by node resource scarcity leading to Pod eviction and reconstruction, deletion of etcd resource paths, and upgrades to the cluster API server version are all resource deletions triggered by the node agent component under abnormal circumstances. When the node agent component detects node resource scarcity leading to Pod eviction and reconstruction, deletion of etcd resource paths, or upgrades to the cluster API server version, it automatically generates a resource deletion request. These resource deletion requests employ deletion protection based on a second verification strategy (protector-cri). Specifically, the core of this second verification strategy is the deployment of a preset protection component interface, protector.sock, between the node agent component kubelet and the container management socket interface cri.sock corresponding to the container management component CRI. When a node agent component sends a resource deletion request, the request is first forwarded through the default protection component interface `protector.sock` to the default protection component `protector-cri`. `protector-cri` then verifies the request. Only if the verification result allows the deletion, will `protector.sock` forward the request to the container management socket interface `cri.sock`. This `cri` then calls the process component (`containerd-shim-runc-v2`) within the image daemon `dockerd` / containerd, which manages the container lifecycle, to execute the resource deletion request. Further, this process component forwards the request to the lightweight component `runc`, which then starts / stops the target container group (container-x) within the `start` / `stop` container group.

[0327] The advantage of this embodiment is that a pre-deployed protection component is used in the target cluster; deletion requests triggered under abnormal conditions can be verified through this pre-deployed protection component. Currently, resource deletion requests triggered under abnormal conditions are directly sent to the container management component by the node proxy component for deletion execution. This solves, to some extent, the problem of intercepting and protecting resource deletions triggered under abnormal conditions, preventing resource deletion requests generated under abnormal conditions from being executed directly without verification. The interception and protection scheme of this embodiment can comprehensively cover resource deletions under various deletion types, thereby improving the reliability of deletion interception and protection for cluster resources and enhancing the security of cluster resources.

[0328] It should be noted that the preset protection component in this embodiment is used to implement the gRPC interface related to the standard K8s CRI interface, while other interfaces directly pass through and call the native socket interface (cri.sock) of the target cluster.

[0329] In this embodiment of the disclosure, the node agent component of the target cluster communicates with the container management component of the target cluster through a preset protection component. Figure 18 For example, the node proxy component (kubelet) connects to the default protection component (protector-cri) through the default protection component interface (protector.sock). Simultaneously, the node proxy component (kubelet) indirectly communicates with the image daemon dockerd / containerd (i.e., the container management component) through the connection between the default protection component interface (protector.sock) and the container management socket interface (cri.sock).

[0330] Please refer to Figure 19 In one embodiment, after invoking the preset protection component, the cluster resource processing method may include, but is not limited to, the following steps 1910-1920:

[0331] Step 1910: Detect the current status of the preset protection components;

[0332] Step 1920: If the current status indicates that the default protection component is deployed abnormally or that the default protection component is undergoing a version upgrade, then disconnect the communication connection between the default protection component and the node agent component and the container management component, and connect the node agent component and the container management component directly.

[0333] Steps 1910-1920 are described in detail below.

[0334] In step 1910, the current state of the preset protection component is detected.

[0335] The current status is used to indicate the deployment status of preset protection components, etc.

[0336] In this specific implementation, the current status of the preset protection components can be detected using the kubectl command-line tool, or the command `kubelet get pod -n xxx|grep protector` can be used to obtain the current status of the preset protection components in the Kubernetes cluster. Furthermore, since the log files of the preset protection components can also reflect the allowed status and potential problems, the log files of the preset protection components can also be retrieved, and their current status can be determined based on the specific content of the log files.

[0337] In step 1920, if the current status indicates that the preset protection component is deployed abnormally or that the preset protection component is undergoing a version upgrade, then the communication connection between the preset protection component and the node agent component and the container management component is cut off, and the node agent component and the container management component are directly connected.

[0338] In the specific implementation of this embodiment, if the current status indicates that the preset protection component is deployed abnormally or that the preset protection component is undergoing a version upgrade, the communication connection between the preset protection component and the node agent component and the container management component can be directly cut off, and the direct connection between the node agent component and the container management component can be re-established, so that the node agent component and the container management component can communicate and interact directly through the socket interface (cri.sock) in the container management component.

[0339] by Figure 18 For example, when the preset protection component experiences a deployment anomaly or is undergoing a version upgrade, the node agent component kubelet will directly connect to the socket interface cri.sock and communicate directly with the image daemon dockerd / containerd based on the socket interface cri.sock.

[0340] The advantage of this embodiment is that it considers periodically checking the current state of the preset protection component. If an abnormal deployment of the preset protection component is detected or if the preset protection component is undergoing a version upgrade, the communication method between the node agent component and the container management component is changed. This enables flexibility and timeliness in switching the communication methods of the node agent component and the container management component. This approach improves the security of communication between the node agent component and the container management component and reduces the adverse impact of changes in the state of the preset protection component on the normal communication between the node agent component and the container management component.

[0341] This disclosure provides an interception and protection scheme based on operation and maintenance end-user verification. This scheme protects against both resource deletion requests triggered under normal and abnormal circumstances. Resource deletion requests triggered under abnormal circumstances are often intercepted due to potential accidental deletion of critical resources. However, in special cases where environmental anomalies necessitate resource deletion, such requests may be incorrectly intercepted during the second verification process based on the second verification strategy, hindering the normal execution of the resource deletion request. Therefore, this disclosure provides an interception and protection scheme based on operation and maintenance end-user verification, which can reduce the probability of incorrectly intercepting resource deletion requests and ensure that all necessary resource deletion requests are executed normally.

[0342] Please refer to Figure 20In one embodiment, the process of performing the second verification using the second verification strategy and resource tag information includes, but is not limited to, the following steps 2010-2020:

[0343] Step 2010: In response to the failure of the second verification, the resource deletion request is sent to the operation and maintenance service platform so that when the operation and maintenance service platform determines that the resource deletion request meets the first condition, it adds the target field identifier to the resource deletion request, obtains the updated resource deletion request, and returns the updated resource deletion request.

[0344] Step 2020: Receive the updated resource deletion request and verify the request using the first verification strategy and the resource tag information of the updated resource deletion request, so as to execute the updated resource deletion request after the request verification is successful.

[0345] The following is a detailed description of steps 2010-2020.

[0346] In step 2010, in response to the failure of the second verification, a resource deletion request is sent to the operation and maintenance service platform so that when the operation and maintenance service platform determines that the resource deletion request meets the first condition, it adds a target field identifier to the resource deletion request, obtains the updated resource deletion request, and returns the updated resource deletion request.

[0347] An operations and maintenance service platform refers to a client used to run and maintain a target cluster.

[0348] The first condition is used to indicate the specific requirements that must be met for a resource deletion request triggered under abnormal circumstances to be executed normally.

[0349] The target field identifier is used to indicate that the resource deletion request should be allowed to proceed directly, and that the target cluster resource to be deleted by the request should be allowed to be deleted directly. For example, the target field identifier can be represented as an annotation (key=value) so that the target field identifier can be marked on the target cluster resource.

[0350] An updated resource deletion request refers to a deletion request initiated based on an abnormal situation but marked as still requiring execution. The request content of an updated resource deletion request is basically the same as that of a resource deletion request identified as type II, the difference being that the updated resource deletion request has an additional target field identifier.

[0351] In this specific implementation, if the second verification fails, it indicates that the abnormally triggered resource deletion request has been successfully intercepted. However, considering that the abnormally triggered resource deletion request may need to be executed, the resource deletion request is sent to the operation and maintenance service platform. At this time, the operation and maintenance service platform parses the resource deletion request and determines whether the resource deletion request meets the first condition based on the parsed request reason and request content. Further, when the operation and maintenance service platform determines that the resource deletion request meets the first condition, it adds a target field identifier to the resource deletion request to indicate that the resource deletion request can be directly executed, obtains the updated resource deletion request, and returns the updated resource deletion request so that the cluster API server can execute the updated resource deletion request.

[0352] In step 2020, an updated resource deletion request is received, and the request is verified using the first verification strategy and the resource tag information of the updated resource deletion request, so that the updated resource deletion request is executed after the request verification is successful.

[0353] In this specific implementation, step 2020 is similar to step 330 described above. The difference is that in step 330, the first verification and subsequent execution of the resource deletion request is a request initiated under normal circumstances; while in step 2020, the request verification and subsequent execution of the updated resource deletion request is a request initiated under abnormal circumstances but still needs to be executed. For the sake of brevity, this will not be elaborated further.

[0354] The advantage of this embodiment is that, for the second type of resource deletion request that genuinely requires resource deletion due to environmental anomalies, an interception and protection scheme based on operation and maintenance terminal verification is introduced. When the second verification fails, the resource deletion request is sent to the operation and maintenance service platform. The operation and maintenance service platform, after determining that the resource deletion request meets the first condition, adds a target field identifier to the resource deletion request to obtain an updated resource deletion request. This allows the updated resource deletion request with the target field identifier to directly pass verification and be executed normally. This method can reduce the probability of erroneously intercepting resource deletion requests to a certain extent, ensuring that all resource deletion requests that need to be executed can be executed normally, and improving the execution success rate of compliant resource deletion requests.

[0355] The apparatus and device according to embodiments of this disclosure will now be described.

[0356] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0357] It should be noted that in various specific embodiments of this application, when processing is required based on data related to the characteristics of the target object, such as target object attribute information or a set of attribute information, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require obtaining target object attribute information, separate permission or consent from the target object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of the embodiments of this application be obtained.

[0358] Figure 21 This is a schematic diagram of the structure of a cluster resource processing device 2100 provided in an embodiment of the present disclosure. The cluster resource processing device 2100 includes:

[0359] The first determining unit 2110 is used to determine the target cluster resource pointed to by the resource deletion request and the target event information associated with the resource deletion request in response to receiving a resource deletion request for the target cluster.

[0360] The second determining unit 2120 is used to determine the resource tag information of the target cluster resources;

[0361] The first verification unit 2130 is used to perform a first verification through a first verification strategy and resource tag information if the resource deletion request is determined to be of the first type based on the target event information, so as to execute the resource deletion request after the first verification is passed;

[0362] The second verification unit 2140 is used to perform a second verification using a second verification strategy and resource tag information if the resource deletion request is determined to be of the second type based on the target event information, so as to execute the resource deletion request after the second verification is passed.

[0363] Optionally, the first verification unit 2130 is used for:

[0364] Determine the reference event information corresponding to the first type;

[0365] The similarity between the target event information and the reference event information is calculated to obtain the event similarity.

[0366] If the event similarity is greater than the first threshold, the resource deletion request is determined to be of the first type.

[0367] Optionally, the first verification unit 2130 includes:

[0368] A determination module (not shown) is used to determine the object initiating the resource deletion request;

[0369] The first verification module (not shown) is used to perform the first sub-verification on the initiating object based on the first verification strategy;

[0370] The second verification module (not shown) is used to perform a second sub-verification of the resource tag information based on the first verification strategy.

[0371] Optionally, the first verification module (not shown) is used for:

[0372] Based on the first verification strategy, obtain the authorized operation objects and the baseline object information of the authorized operation objects in the target cluster;

[0373] Determine the target object information of the initiating object;

[0374] Based on the baseline object information and the target object information, the first sub-verification is performed on the initiating object.

[0375] Optionally, the second verification module (not shown) is used for:

[0376] Based on the first verification strategy, the tag value corresponding to the target cluster resource and the associated business of the target cluster resource are extracted from the resource tag information;

[0377] If the tag value is the first value, the initiating object of the resource deletion request is determined, and the list of authorized services of the initiating object is determined. The first value is used to indicate that the target cluster resource is deleted.

[0378] If the associated service is in the list of authorized services, then the second sub-verification is confirmed to be successful.

[0379] Optionally, the first verification module (not shown) is used for:

[0380] Based on the first verification strategy, the authorization certificate is extracted from the resource deletion request, and the validity period of the authorization certificate is determined;

[0381] If the baseline object information is consistent with the target object information, and the current time is within the validity period of the authorization certificate, then the first sub-verification is deemed successful.

[0382] Optionally,

[0383] The first verification unit 2130 is used for:

[0384] If the target cluster resource is a container group, the target cluster resource and the resources associated with the target cluster resource are unbound in order to reclaim the resources associated with the target cluster resource;

[0385] Stop the container group and delete the container group and its associated log information after the container group stops running.

[0386] Optionally, the first verification unit 2130 includes:

[0387] A rate limiting verification module (not shown) is used to perform rate limiting verification on resource deletion requests after the first verification passes.

[0388] The detection module (not shown) is used to perform associated resource detection on the target cluster resources if the rate limit verification passes, and obtain the detection results. The detection results are used to indicate whether the target cluster resources depend on other cluster resources of the target cluster.

[0389] An execution module (not shown) is used to execute a resource deletion request if the detection result indicates that the target cluster resource does not depend on other cluster resources.

[0390] Optionally, the speed limit verification module (not shown) is used for:

[0391] Determine the number of resources to be deleted in the target cluster and the number of remaining tokens in the preset token bucket;

[0392] If the number of resources to be deleted is less than or equal to the second threshold and the number of remaining tokens is greater than or equal to the first number, then the rate limit verification is considered successful.

[0393] Optionally, the detection module (not shown) is used for:

[0394] Determine the resource hierarchy architecture of the target cluster;

[0395] Locate the target cluster resources at the resource hierarchy level and obtain the resource dependencies of the target cluster resources;

[0396] If, based on resource dependencies, it is determined that the target cluster resource does not have any lower-level cluster resources in the resource hierarchy architecture, then the detection result is determined to mean that the target cluster resource does not depend on other cluster resources.

[0397] Optionally, the second verification unit 2140 is used for:

[0398] Based on the second verification strategy, the preset protection component is invoked, and the preset protection component is used to extract the tag value corresponding to the target cluster resource and the associated business of the target cluster resource from the resource tag information;

[0399] If the tag value is the first value, the initiating object of the resource deletion request is determined, and the list of authorized services of the initiating object is determined. The first value is used to indicate that the target cluster resource is deleted.

[0400] If the associated service is in the list of authorized services, then the second verification is considered successful.

[0401] Optionally, the node agent component of the target cluster communicates with the container management component of the target cluster through a preset protection component;

[0402] The cluster resource processing device 2100 also includes an update unit (not shown), which is used for:

[0403] Detect the current status of preset protection components;

[0404] If the current status indicates that the default protection component is deployed abnormally or that the default protection component is undergoing a version upgrade, then the communication connection between the default protection component and the node agent component and container management component will be cut off, and the node agent component and container management component will be directly connected.

[0405] Optionally, the cluster resource processing apparatus 2100 further includes a verification unit (not shown), which is used for:

[0406] In response to the failure of the second verification, the resource deletion request is sent to the operation and maintenance service platform so that when the operation and maintenance service platform determines that the resource deletion request meets the first condition, it adds a target field identifier to the resource deletion request, obtains the updated resource deletion request, and returns the updated resource deletion request.

[0407] The system receives an updated resource deletion request and verifies the request using a first verification strategy and the resource tag information of the updated resource deletion request, so that the updated resource deletion request can be executed after the request verification is successful.

[0408] Optionally, a resource deletion request may be generated in one of the following three ways:

[0409] Generate a resource deletion request based on command code written using a pre-defined command-line tool for the target object;

[0410] Generate a resource deletion request based on the resource deletion operation initiated in the target application by the development toolkit;

[0411] The control manager of the target cluster periodically probes the cluster status of the target cluster, and when it determines that the cluster status meets the second condition, the control manager generates a resource deletion request.

[0412] Optionally, the resource deletion request is generated by the node agent component in the target cluster using one of the following three methods:

[0413] In response to the detection of resource stress on a target node among multiple nodes in the target cluster, the node agent component identifies the target container group among multiple container groups associated with the target node and generates a resource deletion request for the target container group.

[0414] In response to the detection that the backend storage path associated with the target cluster resource of the target cluster has been deleted, the node agent component generates a resource deletion request for the target cluster resource.

[0415] In response to an incompatibility issue in the resource version field after an upgrade of the API server in the target cluster, the node agent component generates a resource deletion request for the target container group.

[0416] Reference Figure 22 , Figure 22 To implement the structural block diagram of the terminal portion of the cluster resource processing method according to the embodiments of this disclosure, the terminal includes: a radio frequency (RF) circuit 2210, a memory 2215, an input unit 2230, a display unit 2240, a sensor 2250, an audio circuit 2260, a wireless fidelity (WiFi) module 2270, a processor 2280, and a power supply 2290, etc. Those skilled in the art will understand that... Figure 22 The terminal structure shown does not constitute a limitation on mobile phones or computers and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0417] The RF circuit 2210 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 2280; in addition, it transmits uplink data to the base station.

[0418] The memory 2215 can be used to store software programs and modules. The processor 2280 executes various functional applications and data processing of the target terminal by running the software programs and modules stored in the memory 2215.

[0419] The input unit 2230 can be used to receive input numeric or character information, and to generate key signal inputs related to the settings and function control of the target terminal. Specifically, the input unit 2230 may include a touch panel 2231 and other input devices 2232.

[0420] Display unit 2240 can be used to display input or provided information, as well as various menus of the target terminal. Display unit 2240 may include display panel 2241.

[0421] Audio circuitry 2260, speaker 2261, and microphone 2262 provide an audio interface.

[0422] In this embodiment, the processor 2280 included in the terminal can execute the cluster resource processing method of the previous embodiment.

[0423] The terminals disclosed in this embodiment include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The embodiments of this invention can be applied to various scenarios, including but not limited to databases, cloud computing, data storage, and information technology.

[0424] Figure 23 This is a partial structural block diagram of a server for implementing the cluster resource processing method of this disclosure embodiment. The server can vary significantly due to different configurations or performance, and may include one or more Central Processing Units (CPUs) 2322 (e.g., one or more processors) and memory 2332, and one or more storage media 2130 (e.g., one or more mass storage devices) for storing application programs 2342 or data 2344. The memory 2332 and storage media 2330 can be temporary or persistent storage. The program stored in the storage media 2330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 2322 may be configured to communicate with the storage media 2330 and execute the series of instruction operations in the storage media 2330 on the server.

[0425] The server may also include one or more power supplies 2323, one or more wired or wireless network interfaces 2350, one or more input / output interfaces 2358, and / or one or more operating systems 2341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0426] The central processing unit 2322 in the server can be used to execute the cluster resource processing method of the present disclosure embodiments.

[0427] This disclosure also provides a computer-readable storage medium for storing program code for executing the cluster resource processing methods of the foregoing embodiments.

[0428] This disclosure also provides a computer program product comprising a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the cluster resource processing method described above.

[0429] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0430] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0431] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0432] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0433] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0434] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0435] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0436] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0437] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0438] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for processing cluster resources, characterized in that, The method includes: In response to receiving a resource deletion request for a target cluster, determine the target cluster resource to which the resource deletion request points and the target event information associated with the resource deletion request; Determine the resource tag information of the target cluster resources; If the resource deletion request is determined to be of the first type based on the target event information, then a first verification is performed using the first verification strategy and the resource tag information, so that the resource deletion request is executed after the first verification is passed; If the resource deletion request is determined to be of the second type based on the target event information, a second verification is performed using the second verification strategy and the resource tag information, so that the resource deletion request is executed after the second verification is passed.

2. The method according to claim 1, characterized in that, The step of determining that the resource deletion request is of the first type based on the event information includes: Determine the reference event information corresponding to the first type; The similarity between the target event information and the reference event information is calculated to obtain the event similarity. In response to the event similarity being greater than a first threshold, the resource deletion request is determined to be of the first type.

3. The method according to claim 1, characterized in that, The first verification, performed using the first verification strategy and the resource tag information, includes: Determine the object that initiated the resource deletion request; The first sub-verification is performed on the initiating object based on the first verification strategy; The resource tag information is subjected to a second sub-verification based on the first verification strategy.

4. The method according to claim 3, characterized in that, The second sub-verification of the resource tag information based on the first verification strategy includes: Based on the first verification strategy, the tag value corresponding to the target cluster resource and the associated business of the target cluster resource are extracted from the resource tag information; If the tag value is the first value, then the initiating object of the resource deletion request is determined, and the authorized service list of the initiating object is determined, wherein the first value is used to indicate that the target cluster resource is authorized to be deleted; If the associated service is within the list of authorized services, then the second sub-verification is deemed successful.

5. The method according to claim 3, characterized in that, The first sub-verification of the initiating object based on the first verification strategy includes: Based on the first verification strategy, obtain the authorized operation objects of the target cluster and the baseline object information of the authorized operation objects; Determine the target object information of the initiating object; Based on the baseline object information and the target object information, the initiating object undergoes a first sub-verification.

6. The method according to claim 5, characterized in that, The first sub-verification of the initiating object based on the baseline object information and the target object information includes: Based on the first verification strategy, the authorization certificate is extracted from the resource deletion request, and the validity period of the authorization certificate is determined; If the baseline object information is consistent with the target object information, and the current time is within the validity period of the authorization certificate, then the first sub-verification is determined to be successful.

7. The method according to claim 1, characterized in that, The step of executing the resource deletion request after the first verification is passed includes: If the target cluster resource is a container group, the target cluster resource and the resources associated with the target cluster resource are unbound in order to reclaim the resources associated with the target cluster resource; Stop the container group from running, and after the container group stops running, delete the container group and its associated log information.

8. The method according to claim 1, characterized in that, The step of executing the resource deletion request after the first verification is passed includes: After the first verification is passed, the resource deletion request is subject to rate limiting verification. If the rate limiting verification passes, the target cluster resources are subjected to associated resource detection to obtain the detection result, wherein the detection result is used to indicate whether the target cluster resources depend on other cluster resources of the target cluster; If the detection result indicates that the target cluster resource does not depend on the other cluster resources, then the resource deletion request is executed.

9. The method according to claim 8, characterized in that, The rate-limiting verification of the resource deletion request includes: Determine the number of resources to be deleted from the target cluster and the number of remaining tokens in the preset token bucket; If the number of resources to be deleted is less than or equal to the second threshold and the number of remaining tokens is greater than or equal to the first number, then the rate limit verification is determined to be successful.

10. The method according to claim 8, characterized in that, The process of performing associated resource detection on the target cluster resources to obtain detection results includes: Determine the resource hierarchy architecture of the target cluster; Locate the target cluster resources in the resource hierarchy architecture to obtain the resource dependencies of the target cluster resources; If, based on the resource dependency relationship, it is determined that the target cluster resource does not have any lower-level cluster resources in the resource hierarchy architecture, then the detection result is determined to mean that the target cluster resource does not depend on the other cluster resources.

11. The method according to claim 1, characterized in that, The second verification, performed using the second verification strategy and the resource tag information, includes: Based on the second verification strategy, a preset protection component is invoked, and the preset protection component is used to extract the tag value corresponding to the target cluster resource and the associated business of the target cluster resource from the resource tag information; If the tag value is the first value, then the initiating object of the resource deletion request is determined, and the authorized service list of the initiating object is determined, wherein the first value is used to indicate that the target cluster resource is authorized to be deleted; If the associated service is in the list of authorized services, then the second verification is deemed successful.

12. The method according to claim 11, characterized in that, The node agent component of the target cluster communicates with the container management component of the target cluster through the preset protection component; After invoking the preset protection component, the method further includes: Detect the current state of the preset protection component; If the current status indicates that the preset protection component is deployed abnormally or that the preset protection component is undergoing a version upgrade, then the communication connection between the preset protection component and the node agent component and the container management component is cut off, and the node agent component and the container management component are directly connected.

13. The method according to claim 1, characterized in that, After performing the second verification using the second verification strategy and the resource tag information, the method further includes: In response to the failure of the second verification, the resource deletion request is sent to the operation and maintenance service platform, so that when the operation and maintenance service platform determines that the resource deletion request meets the first condition, it adds a target field identifier to the resource deletion request to obtain an updated resource deletion request and returns the updated resource deletion request. The updated resource deletion request is received, and the request is verified using a first verification strategy and the resource tag information of the updated resource deletion request, so that the updated resource deletion request is executed after the request verification is successful.

14. The method according to claim 1, characterized in that, The resource deletion request is generated in one of the following three ways: The resource deletion request is generated based on the command code written by the target object using a preset command-line tool; The resource deletion request is generated based on the resource deletion operation initiated in the target application by the development toolkit; The control manager of the target cluster periodically probes the cluster status of the target cluster, and when it determines that the cluster status meets the second condition, the control manager generates the resource deletion request.

15. The method according to claim 1, characterized in that, The resource deletion request is generated by the node agent component in the target cluster using one of the following three methods: In response to the detection that a target node among multiple nodes in the target cluster is experiencing resource stress, the node proxy component determines the target container group among multiple container groups associated with the target node and generates a resource deletion request for the target container group. In response to the detection that the backend storage path associated with the target cluster resource of the target cluster has been deleted, the node proxy component generates a resource deletion request for the target cluster resource. In response to a resource version incompatibility issue arising after an upgrade of the API server in the target cluster, the node agent component generates a resource deletion request for the target container group.

16. A cluster resource device, characterized in that, The device includes: The first determining unit is configured to, in response to receiving a resource deletion request for a target cluster, determine the target cluster resource to which the resource deletion request points and the target event information associated with the resource deletion request; The second determining unit is used to determine the resource tag information of the target cluster resources; The first verification unit is configured to perform a first verification using a first verification strategy and the resource tag information if the resource deletion request is determined to be of the first type based on the target event information, so as to execute the resource deletion request after the first verification is passed. The second verification unit is configured to perform a second verification using a second verification strategy and the resource tag information if the resource deletion request is determined to be of the second type based on the target event information, so as to execute the resource deletion request after the second verification is passed.

17. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the cluster resource method according to any one of claims 1 to 15.

18. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the cluster resource method according to any one of claims 1 to 15.

19. A computer program product comprising a computer program that is read and executed by a processor of an electronic device, causing the electronic device to perform the cluster resource method according to any one of claims 1 to 15.

Images