Method, apparatus, and system for managing storage resources of a container cluster

Abstract

The present application provides a method and apparatus, as well as a system, for managing storage resources of a container cluster, and belongs to the field of computer technology. In the present application, storage resource description information is provided, and the storage resource requirements of the container cluster managed by the CCM are defined in the storage resource description information. The CCM initiates a resource allocation procedure to the VIM based on the storage resource description information to complete the configuration of the container cluster storage resources that meet the requirements of the description information. The container cluster storage resources are managed in a model description manner. Therefore, it is helpful to flexibly deploy the storage policy of the container cluster and perform differentiated deployment and configuration for the storage resources at different layers of the container cluster, so that the container cluster storage resources dynamically and flexibly adapt to different network service requirements.

Description

[Technical Field]

[0001] This application claims priority to Chinese Patent Application No. 202110745811.4, entitled "Method and Apparatus for Managing Storage Resources of a Container Cluster, and System," filed on July 1, 2021, which is incorporated herein by reference in its entirety.

[0002] The present application relates to the field of computer technology, and more particularly to a method and apparatus, and system for managing storage resources of a container cluster. [Background technology]

[0003] A container cluster is a system that includes processes that run one or more containers. In the European Telecommunications Standards Institute (ETSI) standard, the corresponding concept to a container cluster is a container infrastructure service cluster (CIS cluster).

[0004] Container cluster management (CCM) is a management plane network element in a network functions virtualization (NFV) management orchestration system. CCM is responsible for managing container clusters.

[0005] Container cluster management includes container cluster computing resource management, container cluster storage resource management, and container cluster network resource management. Currently, there is an urgent need to provide an effective method for more flexibly configuring container cluster storage resources. Summary of the Invention

[0006] The embodiments of the present application provide a method and apparatus, and a system for managing storage resources of a container cluster, to help configure the container cluster storage resources more flexibly. The technical solutions are as follows: [Means for solving the problem]

[0007] According to a first aspect, there is provided a method for managing storage resources of a container cluster, the method including: a container cluster management (CCM) obtaining storage resource description information, the storage resource description information being used to describe storage resource requirements of a container cluster object managed by the CCM, the container cluster object including at least one of a container cluster group, a container cluster, or a container cluster node; the CCM generating a resource allocation request based on the storage resource description information, the resource allocation request instructing a virtualization infrastructure manager (VIM) to allocate storage resources; and the CCM sending the resource allocation request to the VIM.

[0008] In the above-described method, storage resource description information is provided, and the storage resource requirements of a container cluster managed by the CCM are defined in the storage resource description information. The CCM initiates a resource allocation procedure to the VIM based on the storage resource description information to complete the configuration of container cluster storage resources that meet the requirements of the description information. The container cluster storage resources are managed in a model description manner. This helps flexibly deploy storage policies for the container cluster and perform differentiated deployment and configuration for storage resources at different layers of the container cluster, so that the container cluster storage resources dynamically and flexibly adapt to different network service requirements.

[0009] In a possible implementation, the storage resource description information includes a first attribute of the storage resource, where the first attribute indicates a tier of a container cluster object to which the storage resource is applicable.

[0010] In the above-described implementation, attributes for describing object layers are defined in storage resource description information, so that differentiated deployment configurations are performed for storage resources at different layers of a container cluster, and storage policies of the container cluster are deployed more flexibly.

[0011] In a possible implementation, the storage resource description information includes a second attribute of the storage resource. The second attribute indicates whether the storage resource is statically provided or dynamically allocated. Statically providing the storage resource means that the storage resource is allocated before the container cluster object is created. Dynamically allocating the storage resource means that the storage resource is allocated during the creation process of the container cluster object.

[0012] In the above-described implementation, since the second attribute is defined in the storage resource description information, a space for selection can be provided to the user in the deployment process of the container cluster object, specifically, when the storage resource is planned for deployment or when the storage resource is subsequently deployed in the creation process of the container cluster object. In this way, the solution is more flexible and more applicable to dynamically changing requirements.

[0013] In a possible implementation, the storage resource description information includes a third attribute of the storage resource, which is used to describe an affinity rule or an anti-affinity rule that the storage resource must satisfy.

[0014] In the aforementioned implementation, the affinity or anti-affinity rules that the storage resource must satisfy are defined in the storage resource description information, which therefore helps the physical scope of the storage resource meet the isolation requirements.

[0015] In a possible implementation, the third attribute of the storage resource is the same as the affinity rule or anti-affinity rule of the container cluster node bound to the storage resource. In other words, the container cluster node storage resource multiplexes the affinity rule or anti-affinity rule of the container cluster node.

[0016] In a possible implementation, the storage resource description information includes a fourth attribute of the storage resource, which is used to describe whether the storage resource is a persistent storage resource or a temporary storage resource.

[0017] The persistent storage resource is, for example, one or more hard disks or one or more hard disk arrays (for example, hard disk enclosures). The hard disks are, for example, solid state disks (SSDs), hard disk drives (HDDs), etc. The temporary storage resource is, for example, a memory resource.

[0018] The above implementations help to implement differentiated deployment of two types of storage resources: persistent storage resources and temporary storage resources, for improved flexibility.

[0019] In a possible implementation, the storage resource description information includes a fifth attribute of the storage resource. The fifth attribute is used to describe whether the storage resource is a dedicated storage resource or a shared storage resource. A dedicated storage resource is a storage resource that is permitted to be used only by a specific sub-object of a container cluster object. A shared storage resource is a storage resource that is permitted to be used by each sub-object of multiple sub-objects in the container cluster object.

[0020] A particular sub-object of a container cluster object may be, for example, one designated container cluster node of a container cluster, or one designated container cluster of a container cluster group.

[0021] According to the aforementioned implementations, to provide greater flexibility in managing storage resources, the storage resources can be configured as storage resources dedicated to a specified container cluster or container cluster node, or the storage resources can be configured as storage resources shared by multiple container clusters or storage resources shared by multiple container cluster nodes.

[0022] In a possible implementation, the storage resource description information includes a sixth attribute of the storage resource, and the sixth attribute includes at least one of a name of the storage resource, a specification of the storage resource, and description information of a virtual link connected to the storage resource.

[0023] In the above implementation, basic attributes, such as the specifications and names of storage resources, can be defined in the description information to improve the availability of the solution.

[0024] In a possible implementation, after the CCM sends a resource allocation request to the VIM, the method further includes the CCM receiving a resource allocation response from the VIM, the resource allocation response indicating storage resources already allocated by the VIM, and the CCM configuring the storage resources already allocated by the VIM for the container cluster object.

[0025] In a possible implementation, the method further includes the CCM receiving a creation request from a CCM consumer, the CCM consumer being an operations support system (OSS) / business support system (BSS) or a network function virtualization orchestrator (NFVO), the creation request instructing the CCM to create a container cluster object, the CCM creating the container cluster object, and the CCM sending a creation response to the CCM consumer.

[0026] In possible implementations, the creation request includes a container cluster group creation request and the creation response includes a container cluster group creation response, or the creation request includes a container cluster creation request and the creation response includes a container cluster creation response, or the creation request includes a container cluster node creation request and the creation response includes a container cluster node creation response.

[0027] In a possible implementation, the CCM obtaining the storage resource description information includes the CCM obtaining the storage resource description information from a descriptor file of a container cluster object, or the CCM obtaining the storage resource description information from a virtualization network function descriptor VNFD file.

[0028] In a possible implementation, before the CCM sends the resource allocation request to the VIM, the method further includes the CCM sending a lifecycle authorization request to the CCM consumer, the lifecycle authorization request being for requesting authorization for lifecycle management of the container cluster object, the CCM consumer being an OSS / BSS or NFVO, and the CCM receiving a lifecycle authorization response from the CCM consumer, the lifecycle authorization response including an authorization instruction, the authorization instruction indicating that the CCM is allowed to initiate a lifecycle management process for the container cluster object.

[0029] In the above implementation, the container cluster lifecycle management authorization process is integrated into the container cluster storage resource management process, and storage resource allocation is initiated only when the CCM consumer indicates authorization, improving security.

[0030] According to a second aspect, an apparatus for managing storage resources of a container cluster is provided. The apparatus for managing storage resources of a container cluster is disposed in a CCM, and the apparatus for managing storage resources of a container cluster has a function of implementing any one of the first aspect or any one of the optional schemes of the first aspect. The apparatus for managing storage resources of a container cluster includes at least one unit, where the at least one unit is configured to implement a method provided in any one of the first aspect or any one of the optional schemes of the first aspect. The apparatus for managing storage resources of a container cluster is optionally deployed in a core network and configured to provide a service for managing storage resources of a container cluster. In some embodiments, the unit of the apparatus for managing storage resources of a container cluster is implemented through software, and the unit of the apparatus for managing storage resources of a container cluster is a program module. In some other embodiments, the unit of the apparatus for managing storage resources of a container cluster is implemented by hardware or firmware. For specific details of the apparatus for managing storage resources of a container cluster provided in the second aspect, please refer to the first aspect or any one of the optional schemes of the first aspect. Details will not be repeated herein.

[0031] According to a third aspect, there is provided a computing device, the computing device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions to perform the method of the first aspect and possible implementations of the first aspect.

[0032] According to a fourth aspect, a (non-transitory) computer-readable storage medium is provided. The (non-transitory) computer-readable storage medium stores at least one instruction. When the instruction is executed on a computer, the computer is enabled to perform the method provided in the first aspect or any one of the optional manners of the first aspect. Types of storage media include, but are not limited to, volatile memory, such as random access memory, or non-volatile memory, such as flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0033] According to a fifth aspect, there is provided a computer program product, the computer program product comprising one or more computer program instructions which, when loaded and executed by a computer, enable the computer to perform the method provided in the first aspect or any one of the optional manners of the first aspect.

[0034] According to a sixth aspect, there is provided a chip including a memory and a processor, wherein the memory is configured to store computer instructions, and the processor is configured to retrieve and execute the computer instructions from the memory in order to perform the method of the first aspect or any one of the possible implementations of the first aspect.

[0035] According to a seventh aspect, there is provided a network system including the device for managing storage resources of a container cluster provided in the second aspect and a VIM. [Brief explanation of the drawings]

[0036] [Figure 1] FIG. 1 is an architecture diagram of an NFV system according to an embodiment of the present application. [Figure 2] 1 is a schematic diagram of a container cluster hierarchical storage resource according to an embodiment of the present application; [Figure 3] 1 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. [Figure 4] 1 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. [Figure 5] 1 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. [Figure 6] 1 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. [Figure 7] 1 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. [Figure 8] 1 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. [Figure 9] FIG. 1 is a schematic diagram of the structure of an apparatus for managing storage resources of a container cluster according to an embodiment of the present application; [Figure 10] 1 is a schematic diagram of the structure of a computing device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0037] To make the objectives, technical solutions and advantages of the present application clearer, the following further describes the implementation of the present application in detail with reference to the accompanying drawings.

[0038] In the following, some terms and concepts in the embodiments of the present application are explained and described.

[0039] (a) Container cluster object Container cluster objects are objects in different layers managed by the CCM, and include at least one of a container cluster group, a container cluster, or a container cluster node. The three concepts of a container cluster group, a container cluster, and a container cluster node are all related to a container cluster. The scope of the concepts has an inclusive relationship from large to small, that is, one container cluster group includes one or more container clusters, and one container cluster includes one or more container cluster nodes. Optionally, a container cluster group may provide the functionality of a container cluster service for a network service (NS), a container cluster may provide the functionality of a container cluster service for a virtualized network function (VNF), and a container cluster node may provide the functionality of a container cluster service for a virtualized network function component (VNFC).

[0040] (b) Container cluster group A container cluster group is a collection of a group of related container clusters, for example, those used by the same network service. A container cluster group is also called a container infrastructure service (CIS) cluster group.

[0041] The correspondence between different container clusters in a container cluster group may include multiple cases. For example, services supported by different container clusters in a container cluster group may be associated. For example, each container cluster in a container cluster group may be used for the same service or the same type of service. In another example, each container cluster in a container cluster group may be used by the same organization.

[0042] The container cluster group can be deployed in multiple modes. In a possible implementation, the deployment mode of the container cluster group is a distributed deployment, specifically, different container clusters of the container cluster group are deployed at different physical sites. In another possible implementation, the deployment mode of the container cluster group is a centralized deployment, specifically, different container clusters of the container cluster group are deployed at the same physical site.

[0043] (c) Container cluster A container cluster generally includes one or more monitoring management processes and one or more container cluster nodes. A container cluster is a dynamic system. Multiple containers can be deployed in the system. The system can monitor the status of these containers and the communication between them. In the ETSI standard, the concept corresponding to a container cluster is a container infrastructure service cluster (CIS Cluster).

[0044] Optionally, the same container cluster does not include a hybrid deployment scenario in which container cluster nodes based on virtual machines and container cluster nodes based on bare machines are deployed, in other words, each container cluster node of the same container cluster is a virtual machine or each container cluster node of the same container cluster is a bare machine, thereby ensuring ease of operation and maintenance management of the container cluster.

[0045] Optionally, the computing or storage resource specifications of all container cluster nodes in the same container cluster are the same. Alternatively, one container cluster includes container cluster nodes with different computing or storage resource specifications.

[0046] (d) Container cluster node A container cluster node is a computing node resource that can be used to deploy containers. One or more containers are deployed to one container cluster node.

[0047] A container cluster node may be a virtual node or a physical node, for example, a virtual machine, a bare metal device (also called a bare machine, computer hardware without an operating system), or another hardware device.

[0048] (e) Container Containers are an operating system-level virtualization technology. Different processes are isolated by using operating system isolation techniques, such as control groups (CGroups, a feature of the Linux kernel that is used to limit, control, and isolate resources for a group of processes) and namespaces in Linux. Container technology differs from hardware virtualization (hypervisor) technology. Containers typically do not have virtual hardware, and processes exist within containers, not operating systems. This key feature of container technology makes containers more lightweight and easier to manage than virtual machines. In the container execution state, a group of common management operations, such as starting, stopping, pausing, and deleting a container, is defined to manage the container lifecycle in a unified manner.

[0049] A container is also called a container object. In the ETSI standard, the concept corresponding to a container is a managed container infrastructure object (MCIO).

[0050] Currently, network transformation in the telecommunications sector is undergoing a process of evolution from Network Function Virtualization (NFV) to Cloud Native. Cloud Native is a new system implementation paradigm for building, running, and managing software in a cloud environment. It fully utilizes cloud infrastructure and platform services, adapts to the cloud environment, and incorporates architectural practices for key features such as (micro)service orientation, auto-scaling, distribution, high availability, multi-tenancy, and automation. In this transformation, the introduction of container management within the NFV management and orchestration (MANO) reference architecture is an important part of the multiple practices that transition NFV to Cloud Native.

[0051] The introduction of containers, or service architecture, in the cloud-native evolution process of telecommunications networks brings flexible restructuring to the development and operation (DevOps) of the telecommunications industry. Correspondingly, traditional large-scale monolithic network functions are gradually being decomposed and made service-based, and even microservice-based. Each service-based function is independently developed, delivered, and maintained, with more frequent version upgrades. However, the rapid increase in the number of containerized network functions does not lead to an exponential increase in the workload of interoperability testing. Stable API interface definitions ensure the consistency and reliability of interface function calls.

[0052] Currently, the most popular application in the container management and orchestration field is Google's Kubernetes (abbreviated as K8S), a container cluster management technology based on an open source platform. The core idea of ​​Kubernetes is "everything is service-centric and runs around services." Based on this idea, a container application system built on Kubernetes can run independently on physical machines, virtual machines, or corporate private clouds, and can also be hosted on public clouds. Another feature of Kubernetes is automation. Services can be automatically scaled out / in, diagnosed, and easily upgraded.

[0053] In the following, an example is used to explain the system architecture in the embodiment of the present application.

[0054] 1 is an architecture diagram of an NFV system according to an embodiment of the present application. The NFV system includes an NFV MANO system, VNFs, and a network functions virtualization infrastructure (NFVI). The NFV MANO system includes a CCM, a Container Infrastructure Service Management (CISM), a Network Functions Virtualization Orchestrator (NFVO), a Virtualized Network Function Manager (VNFM), and a Virtualized Infrastructure Manager (VIM).

[0055] The CCM is responsible for managing the container cluster. Specifically, the CCM is configured to create a container cluster node resource pool and perform capacity expansion and capacity reduction on the container cluster node resource pool. The container cluster node resource pool includes multiple container cluster nodes. For example, the container cluster node resource pool is resource pool 1 and resource pool 2 in FIG. 1. The container cluster nodes in resource pool 1 and resource pool 2 are all virtualized machines (VMs), and capacity expansion is, for example, allocating a new VM to resource pool 1 or resource pool 2. The CCM may optionally be a logical function or hardware (e.g., a computer) including a program.

[0056] CISM is also known as Container as a Service (CaaS) management. The open source prototype is Kubernetes. CISM is responsible for managing container objects called by containerized VNFs, including creating, updating, and deleting container objects, and scheduling the container objects to the corresponding node resources (compute, storage, and network) of the container cluster node resource pool managed by CISM.

[0057] The CISM and CCM provide management services to the NFVO or VNFM via northbound interfaces.

[0058] Specifically, a VNF is a containerized VNF. A VNF includes one or more container objects. A containerized VNF can be understood as a containerized workload that encapsulates NFVI resources, such as compute, storage, and network. A container object MCIO called by the workload is scheduled to a node in the container cluster for execution, and an image of a CISM instance or an image of a CIS instance is loaded onto the node in the container cluster. The CISM instance is used to provide CaaS management plane functions, such as a Kubernetes master node. The CIS instance is used to provide CaaS user plane functions, such as kubelet, kube-proxy, and docker, which are provided to Kubernetes worker nodes. The kubelet is a component on the worker node that runs Docker containers and maintains the pod lifecycle. The kube-proxy is another component on the worker node that is used for load balancing and inter-pod communication.

[0059] VIM is used at the infrastructure layer to manage (including reservation and allocation) virtual resources (including virtual compute, storage, and network resources), monitor the status of virtual resources, report faults, and provide a virtual resource pool to upper layer applications.

[0060] The VNFM is configured to perform lifecycle management, e.g., instantiation, updates, auto-scaling, and termination, for one or more VNFs. Optionally, the VNFM communicates with the VNFs to complete VNF lifecycle management and exchange configuration and status information.

[0061] NFVI is the infrastructure layer of NFV and includes hardware components, software components, or a combination of the two for establishing a virtualized environment and deploying, managing, and implementing VNFs. The hardware resources and virtualization layer are used to provide virtualized resources, such as virtual machines or other forms of virtual containers, to VNFs. The hardware resources include computing hardware, storage hardware, and network hardware. In one implementation, the computing hardware and storage hardware may be integrated together. The virtualization layer of NFVI may abstract the hardware resources and isolate the underlying physical network layer.

[0062] In some embodiments, the NFV system further includes Operation Support Systems (OSS) / Business Support Systems (BSS), which are not shown in FIG. 2. The OSS / BSS supports various end-to-end telecommunications services. Management functions supported by the OSS may include network configuration, service provisioning, fault management, etc. The BSS processes orders, payments, etc., and supports product management, order management, revenue management, and customer management.

[0063] The storage resources of a container cluster are hierarchical, including storage resources applicable to the entire container cluster, storage resources applicable to container cluster nodes, and storage resources applicable to container application MCIOs / Pods. Specifically, Figure 2 is a schematic diagram of a container cluster hierarchical storage resource. A container cluster includes the following types of hierarchical storage resources:

[0064] Cluster Persistent Volumes (PVs) 21 provide persistent storage capabilities in a container cluster, are independent of container cluster nodes, and have a lifecycle that aligns with the lifecycle of the container cluster. Persistent volumes can provide storage resources needed for cluster CISM instances running, for example, an operating system (OS) partition, a data partition, an image repository partition, etcd (a highly available key-value storage system), etc.

[0065] The ephemeral storage 22 dedicated to the cluster node coincides with the life cycle of the container cluster node and is dynamically created or deleted along with the creation / deletion of the container cluster node (computing resource). The ephemeral storage may provide storage resources, such as an OS partition, a data partition, etc., necessary to run the CIS instance of the container cluster node.

[0066] The cluster-shared temporary storage 23 is independent of the lifecycle of the container cluster node and shares common storage information between different container cluster nodes, for example, sharing storage resources required to run a CIS instance.

[0067] Referring to the above-mentioned storage resources of different types of hierarchical structures of container clusters, this embodiment provides container cluster storage resource description (CIS cluster storage description) information and implements the requirement for dynamically and elastically managing container cluster storage resources by using a model-driven method in the embodiment process. For details, please refer to the method shown in Figure 3.

[0068] In the following, an example is used to explain the method steps in one embodiment of the present application.

[0069] 3 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. The method shown in FIG. 3 includes the following steps S301 to S308.

[0070] The system architecture on which the method shown in Figure 3 is based is optionally shown in Figure 1. For example, with reference to Figure 1, the CCM in the method shown in Figure 3 is the CCM in Figure 1, and the VIM in the method shown in Figure 3 is the VIM in Figure 1.

[0071] Step S301: The CCM acquires storage resource description information.

[0072] The storage resource description information is also referred to as a storage resource template or a storage resource descriptor file. The storage resource description information is used to describe storage resource requirements for lifecycle management of container cluster objects managed by the CCM. The storage resource description information includes one or more attributes of a storage resource. Specifically, the storage resource description information is equivalent to a template used to configure a storage resource. The template defines the attributes that a storage resource must have during container cluster deployment and lifecycle management. In the process of deploying a container cluster or a storage resource, the storage resource description information can be used to indicate a management plane entity (e.g., the CCM in this embodiment) to perform configuration or management operations. Optionally, the storage resource description information is included in one file.

[0073] The source or generation method of the storage resource description information includes multiple cases. In a possible implementation, the storage resource description information is obtained by the container cluster provider through orchestration before the container cluster is deployed. Attributes included in the storage resource description information are set by the container cluster provider via a web interface, command line, or other method. Optionally, the storage resource description information is a file, and the container cluster provider sends the storage resource description information to the CCM via a file upload method.

[0074] The opportunity to acquire the storage resource description information includes multiple cases. Optionally, the storage resource description information is pre-configured in the CCM. Specifically, the CCM pre-stores the storage resource description information. When the CCM creates a container cluster object, the CCM acquires the pre-stored storage resource description information. Alternatively, the storage resource description information is acquired in the process of creating a container cluster object. Specifically, when the CCM needs to create a container cluster object, the CCM sends a request to acquire the storage resource description information, and the CCM receives a returned acquisition response for the storage resource description information, where the acquisition response includes the storage resource description information. The opportunity for the CCM to acquire the storage resource description information is not limited in this embodiment.

[0075]

[0033] Specific manners for acquiring storage resource description information include several cases. In a possible implementation, the CCM acquires the storage resource description information from the descriptor of a container cluster object. For example, in a process of creating a container cluster group, the CCM acquires the storage resource description information from the descriptor of the container cluster group to configure the storage resources required by the container cluster group. In a process of creating a container cluster, the CCM acquires the storage resource description information from the descriptor of the container cluster to configure the storage resources required by the container cluster. In a process of creating a container cluster node, the CCM acquires the storage resource description information from the descriptor file of the container cluster node to configure the storage resources required by the container cluster node. Alternatively, the CCM acquires the storage resource description information from a Virtualized Network Function Descriptor (VNFD) file. The specific manner for acquiring the storage resource description information is not limited in this embodiment.

[0076] Step S302: The CCM generates a resource allocation request based on the storage resource description information.

[0077] The resource allocation request instructs the VIM to allocate storage resources.

[0078] Optionally, the resource allocation request includes at least one attribute of the storage resource description information.

[0079] Step S303: The CCM sends a resource allocation request to the VIM.

[0080] Step S304: The VIM receives a resource allocation request from the CCM, and the VIM allocates storage resources in response to the resource allocation request.

[0081] In a possible implementation, the VIM obtains attributes of the storage resource from the resource allocation request, selects a storage resource from a resource pool that satisfies the attributes, and allocates the storage resource that satisfies the attributes.

[0082] Step S305: The VIM generates a resource allocation response based on the allocated storage resources.

[0083] Step S306: The VIM sends a resource allocation response to the CCM.

[0084] The resource allocation response indicates the storage resources that have already been allocated by the VIM. The resource allocation response includes identifiers of the storage resources that have already been allocated by the VIM.

[0085] Step S307: The CCM receives a resource allocation response from the VIM.

[0086] Step S308: The CCM configures the storage resources already allocated by the VIM for the container cluster object.

[0087] Specifically, the CCM obtains identifiers of storage resources already allocated by the VIM from the resource allocation response, and generates configuration information for the container cluster object based on the identifiers of the storage resources and storage resource description information in the resource allocation response to configure the storage resources to be used by the container cluster object.

[0088] The configuration information includes a correspondence relationship (also called an association relationship or a binding relationship) between the storage resource and the container cluster object. For example, the configuration information includes an identifier of the storage resource and an identifier of the container cluster object. Optionally, the configuration information further includes an attribute of the storage resource description information.

[0089] Optionally, after the CCM generates the configuration information, the CCM stores the configuration information for subsequently performing lifecycle management operations on the container cluster object based on the configuration information.

[0090] According to the method provided in this embodiment, storage resource description information is provided, and the storage resource requirements of a container cluster managed by the CCM are defined in the storage resource description information. The CCM initiates a resource allocation procedure to the VIM based on the storage resource description information to complete the configuration of container cluster storage resources that meet the requirements of the description information. The container cluster storage resources are managed in a model description manner. This helps flexibly deploy storage policies for the container cluster and perform differentiated deployment and configuration for storage resources at different layers of the container cluster, so that the container cluster storage resources dynamically and flexibly adapt to different network service requirements. In particular, the storage resource description information can be used as a template for creating and managing storage resources. The CCM can create a large number of storage resources in batches by copying attributes included in the same storage resource description information without determining corresponding attributes for each created storage resource. This saves time and effort.

[0091] The following describes attributes of a storage resource included in the storage resource description information. The storage resource description information optionally includes multiple attributes of the storage resource. To distinguish between different attributes, terms such as "first attribute," "second attribute," "third attribute," etc. are used to distinguish and describe multiple different attributes.

[0092] (1) The first attribute of the storage resource The first attribute indicates a tier of a container cluster object to which the storage resource is applicable. Optionally, the first attribute is an enumerated attribute. A value of the first attribute includes at least one of a container cluster group, a container cluster, or a container cluster node. Of the three container cluster objects, i.e., the container cluster group, the container cluster, and the container cluster node, the container cluster group has the highest tier, followed by the container cluster, and the container cluster node has the lowest tier.

[0093] Optionally, the value of the first attribute is one of a container cluster group, a container cluster, and a container cluster node. When the value of the first attribute of one storage resource is a container cluster group, it represents that the storage resource is applicable to a container cluster group. When the value of the first attribute of one storage resource is a container cluster, it represents that the storage resource is applicable to a container cluster. When the value of the first attribute of one storage resource is a container cluster node, it represents that the storage resource is applicable to a container cluster node. Alternatively, the value of the first attribute includes multiple items of a container cluster group, a container cluster, and a container cluster node.

[0094] Optionally, the first attribute is an attribute of a persistent storage resource. Alternatively, the first attribute is an attribute of a temporary storage resource.

[0095] When the storage resource description information includes the first attribute, the configuration information generated by the CCM in step S308 further indicates the tier of the container cluster object to which the storage resource already allocated by the VIM is applicable. The configuration information includes a binding relationship between the identifier of the storage resource and the container cluster object of the corresponding tier. Specifically, if the value of the first attribute of one storage resource is a container cluster group, the configuration information generated by the CCM includes a binding relationship between the storage resource and the container cluster group. Since the storage resource is bound to the container cluster group, the storage resource is permitted to be allocated to the container cluster group for subsequent use. Similarly, if the value of the first attribute of one storage resource is a container cluster, the configuration information generated by the CCM includes a binding relationship between the storage resource and the container cluster. If the value of the first attribute of one storage resource is a container cluster node, the configuration information generated by the CCM includes a binding relationship between the storage resource and the container cluster node.

[0096] In this embodiment, the first attribute is defined in the storage resource description information, which helps to implement differentiated deployment configurations for storage resources of different layers of the container cluster and to more flexibly deploy storage policies of the container cluster.

[0097] (2) Second attribute of storage resource The second attribute indicates whether the storage resource is statically provisioned or dynamically allocated. Optionally, the second attribute is a Boolean attribute. Specifically, the value of the second attribute has two types: one type represents static provisioning, and the other type represents dynamic allocation.

[0098] Statically providing a storage resource means that the storage resource is allocated before the container cluster object is created. When the value of the second attribute of a storage resource is static provisioning, the process of allocating the storage resource is separated from the process of creating the container cluster object, and the storage resource is allocated in advance before the container cluster object is created. Static provisioning, i.e., the value of the attribute, is defined in the description information to support planning and deployment of storage resources in advance.

[0099] Dynamically allocating a storage resource means allocating the storage resource in the process of creating a container cluster object. When the value of the second attribute of a storage resource is dynamic allocation, the storage resource is allocated in the lifecycle management of the container cluster object to which the storage resource belongs. For example, the storage resource is created when the container cluster object is created, and the storage resource is deleted when the container cluster object is deleted.

[0100] In some embodiments, the second attribute is used by the CCM to determine an opportunity to initiate a storage resource allocation procedure. Specifically, after the CCM obtains the storage resource description information, the CCM obtains the second attribute from the storage resource description information. The CCM determines whether the value of the second attribute is static provisioning or dynamic allocation. If the value of the second attribute is dynamic provisioning, upon receiving a container cluster object creation request, the CCM performs steps of generating and sending a resource allocation request to the VIM (in other words, steps S302 and S303). If the value of the second attribute is static allocation, the CCM is permitted to first create a storage resource and then initiate the container cluster object creation process. In the container cluster object creation process, the CCM allocates the storage resource to the container cluster for use.

[0101] Optionally, the second attribute is an attribute of a persistent storage resource.

[0102] In this embodiment, the second attribute is defined in the storage resource description information. In this way, a space for selection can be provided to the user in the deployment process of the container cluster object, specifically, when the storage resource is planned for deployment or when the storage resource is subsequently deployed in the creation process of the container cluster object, so that the solution is more flexible and more applicable to dynamically changing requirements.

[0103] (3) The third attribute of the storage resource The third attribute is equivalent to providing a constraint on the physical range in which the storage resource is located, and is used to support separation of storage resources with different physical ranges. The third attribute is used to describe affinity or anti-affinity rules that the storage resource must satisfy. An affinity rule specifies that the physical ranges in which different storage resources are located must be the same or similar, and an anti-affinity rule specifies that the physical ranges in which different storage resources are located must be different or dissimilar. Physical ranges include, but are not limited to, NFVI-PoP (site), zone, zone group, NFVI-node (host), etc.

[0104] For example, the physical range is a host. The storage resources to be created include a first storage resource and a second storage resource, where the first storage resource and the second storage resource are both memory resources. If the third attribute indicates that the first storage resource and the second storage resource must satisfy an affinity rule, two memory resources on the same host are selected and configured as the first storage resource and the second storage resource, respectively. If the third attribute indicates that the first storage resource and the second storage resource must satisfy an anti-affinity rule, two memory resources on different hosts are selected and configured as the first storage resource and the second storage resource, respectively.

[0105] Optionally, the third attribute of the storage resource is the same as the affinity rule or anti-affinity rule of the container cluster node bound to the storage resource. In other words, the container cluster node storage resource multiplexes the affinity rule or anti-affinity rule of the container cluster node. Specifically, when the third attribute is an affinity rule, the container cluster node storage resource and the container cluster node use the same affinity rule during deployment, or when the third attribute is an anti-affinity rule, the container cluster node storage resource and the container cluster node use the same anti-affinity rule during deployment. Optionally, the third attribute of the storage resource includes only container cluster resource isolation requirements within a physical range.

[0106] Optionally, the third attribute is an attribute of a temporary storage resource. Optionally, the temporary storage resource dedicated to the container cluster node multiplexes the affinity rule or anti-affinity rule of the container cluster node. In other words, the dedicated storage resource of the container cluster node has a third attribute, and the third attribute of the dedicated storage resource of the container cluster node is the same as the affinity rule or anti-affinity rule of the container cluster node. Alternatively, the third attribute is an attribute of a persistent storage resource.

[0107] The method for ensuring that the configured storage resource satisfies the third attribute includes several specific implementations. In the following, Implementation 1 and Implementation 2 are used as illustrative examples.

[0108] Implementation 1: The CCM determines the physical range of the storage resource based on the affinity rule or anti-affinity rule specified by the third attribute.

[0109] Specifically, when the CCM detects the physical range of each storage resource in the infrastructure layer, the CCM determines the physical range of the storage resource to be created based on the affinity or anti-affinity rule specified by the third attribute. The CCM includes the determined physical range in a resource allocation request, and then sends the resource allocation request including the physical range to the VIM. After the VIM receives the resource allocation request, the VIM obtains the physical range specified by the CCM from the resource allocation request. The VIM selects and allocates the storage resource in the physical range.

[0110] Implementation 2: The VIM determines the physical extent of the storage resource based on the affinity or anti-affinity rule specified by the third attribute.

[0111] Specifically, the CCM includes a third attribute in the resource allocation request, and the CCM sends the resource allocation request including the third attribute to the VIM. After the VIM receives the resource allocation request, the VIM retrieves the third attribute from the resource allocation request. The VIM determines the physical range of the storage resource to be created based on the affinity or anti-affinity rule specified by the third attribute. The VIM selects and allocates the storage resource in the physical range.

[0112] (4) The fourth attribute of storage resources The fourth attribute is used to describe whether the storage resource is a persistent storage resource or an ephemeral storage resource.

[0113] The persistent storage resource may be, for example, one or more hard disks or one or more hard disk arrays (e.g., hard disk enclosures). The hard disk may be, for example, a solid state disk (SSD), a hard disk drive (HDD), or the like.

[0114] A persistent storage resource is also called a persistent volume (PV). A persistent storage resource is used to provide persistent storage capabilities for container cluster objects. For example, a container cluster persistent storage resource (also called a cluster persistent volume) is used to provide persistent storage capabilities for a container cluster. A container cluster persistent storage resource is independent of a container cluster node, has a lifecycle that matches the lifecycle of the container cluster, and can provide storage resources required to run a cluster CISM instance, such as an OS partition, a data partition, an image repository partition, etcd, etc.

[0115] A tier of a container cluster object to which a persistent storage resource is applicable includes, but is not limited to, a container cluster group, a container cluster, or a container cluster node. In other words, a value of a first attribute of storage resource description information of a persistent storage resource is any one of a container cluster group, a container cluster, or a container cluster node. Optionally, the persistent storage resource and the container cluster object in the tier to which the persistent storage resource is applicable have the same life cycle. For example, the life cycle of a container cluster persistent storage resource is the same as the life cycle of a container cluster.

[0116] The layer of the container cluster object to which the temporary storage resource is applicable is the container cluster node. In other words, the value of the first attribute of the storage resource description information of the temporary storage resource is the container cluster node. The storage resource description information of the temporary storage resource is obtained from the description information of the container cluster node to which the temporary storage resource belongs. The life cycle of the temporary storage resource is the same as the life cycle of the container cluster node to which the temporary storage resource belongs.

[0117] A temporary storage resource is a dynamically allocated storage resource. In other words, the value of the second attribute of the storage resource description information of a temporary storage resource is dynamically allocated. A temporary storage resource is created when a container cluster node (computing resource) is created and deleted when the container cluster node is deleted. A temporary storage resource can provide storage required to run a container cluster node CIS instance, such as an OS partition, a data partition, etc. A temporary storage resource is also called a temporary volume.

[0118] The temporary storage resource is, for example, a memory resource. Generally, among the two types of storage resources, temporary storage resource and permanent storage resource, the physical range of the temporary storage resource is closer to the CPU, and the temporary storage resource is used to provide the CPU with a faster access speed than the permanent storage resource.

[0119] Optionally, in the three types of resources of the cloud infrastructure (in computing resources, storage resources, and network resources), persistent storage resources belong to storage resources, and temporary storage resources belong to computing resources. In the process of creating persistent storage resources, the resource allocation requests of steps S302 and S303 are specifically virtual storage resource allocation requests, and the resource allocation responses of steps S305 and S306 are specifically virtual storage resource allocation responses. In the process of creating temporary storage resources, the resource allocation requests of steps S302 and S303 are specifically virtual computing resource allocation requests, and the resource allocation responses of steps S305 and S306 are specifically virtual computing resource allocation responses.

[0120] (5) The fifth attribute of a storage resource The fifth attribute is used to describe whether the storage resource is a dedicated storage resource or a shared storage resource.

[0121] A dedicated storage resource is a storage resource that is permitted to be used only by a specific sub-object of a container cluster object. The specific sub-object is a designated sub-object of the container cluster object. For example, the container cluster object is a container cluster, and the specific sub-object is a designated container cluster node of the container cluster. In another example, the container cluster object is a container cluster group, and the specific sub-object is a designated container cluster of the container cluster group. When the fifth attribute of a storage resource is a specific storage resource, it indicates that the storage resource is dedicated to or exclusively occupied by a specific sub-object.

[0122] A shared storage resource is a storage resource that allows sharing among multiple sub-objects of a container cluster object. For example, a shared storage resource of a container cluster is a storage rule resource that allows sharing of shared storage required to run different container cluster nodes of the container cluster, such as CIS instances.

[0123] (6) The sixth attribute of the storage resource The sixth attribute includes at least one of a name of the storage resource, a specification of the storage resource (also referred to as the size or dimension of the resource), and description information of a virtual link connected to the storage resource. The sixth attribute is also referred to as a meta-data description or a basic attribute of the storage resource. Optionally, the name of the storage resource is specifically a type name of the storage resource. Optionally, the sixth attribute of the container cluster persistent storage resource further includes, for example, identification information or address information of a CISM image file and / or a CIS image file associated with the container cluster.

[0124] In some embodiments, the resource allocation request in the method shown in FIG. 3 includes a sixth attribute. Specifically, in the process of the CCM performing step S302, the CCM obtains the sixth attribute of the storage resource from the storage resource description information, and the CCM includes the sixth attribute of the storage resource in the resource allocation request. The CCM sends the resource allocation request including the sixth attribute to the VIM. In the process of performing step S304, the CIM allocates the storage resource based on the sixth attribute. For example, the sixth attribute is a storage resource specification. For example, the storage resource specification in the storage resource description information is 500 MB. When the CCM initiates a resource allocation procedure to the VIM based on the storage resource description information, the CCM includes 500 MB in the resource allocation request. After the VIM receives the resource allocation request, the VIM selects and allocates a storage resource having a specification of 500 MB from a resource pool.

[0125] The above lists various attributes that can be included in the storage resource description information. Optionally, the storage resource description information includes all of the attributes listed in (1) through (6). Alternatively, the storage resource description information includes some of the attributes in (1) through (6).

[0126] Optionally, the method shown in Fig. 3 is performed in a process of creating a container cluster object. Specifically, Fig. 4 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. In addition to all the steps in Fig. 3 , Fig. 4 further includes the following steps S311 to S315. For other steps other than steps S311 to S315 in Fig. 4 , please refer to the description of Fig. 3 . Optionally, steps S311 to S313 are performed before step S301, and steps S314 and S315 are performed after step S308. Optionally, the method shown in Fig. 4 is applicable to a case where the value of the second attribute of the storage resource description information is dynamic allocation.

[0127] Step S311: The CCM consumer sends a creation request to the CCM.

[0128] Consumption is the use of container cluster management services. Specifically, a CCM is a provider of container cluster management services, and a CCM consumer is a user of the container cluster management services provided by the CCM. A CCM consumer is, for example, an OSS / BSS or an NFVO.

[0129] The create request instructs the CCM to create a container cluster object. The create request sent by the CCM consumer is equivalent to a trigger condition for the CCM to initiate a resource allocation procedure to the VIM. Upon receiving the create request, the CCM configures storage resources for the created container cluster object according to the method shown in FIG. 3 based on the creation of the container cluster object.

[0130] Optionally, the creation request includes an identifier of description information of a container cluster object. The specific process of step S301 includes: the CCM obtaining an identifier of the description information of the container cluster object from the creation request; the CCM accessing the description information of the container cluster object based on the identifier of the description information of the container cluster object in the creation request; and the CCM obtaining storage resource description information from the description information of the container cluster object.

[0131] Step S312: The CCM receives a creation request from the CCM consumer.

[0132] Step S313: The CCM creates a container cluster object.

[0133] Step S314: The CCM generates a creation response.

[0134] Step S315: The CCM sends a creation response to the CCM consumer.

[0135] The create response includes a successful creation indication. The successful creation indication indicates that the container cluster object was successfully created. Optionally, the create response further includes an identifier of the container cluster object.

[0136] Optionally, the create request initiated by the CCM consumer is a container cluster group create request, which instructs the CCM to create a container cluster group. The container cluster object created by the CCM is specifically a container cluster group. The create response returned by the CCM is specifically a container cluster group create response. A create success indication in the container cluster group create response indicates that the container cluster group was created successfully.

[0137] Optionally, the create request initiated by the CCM consumer is a container cluster create request, which instructs the CCM to create a container cluster. The container cluster object created by the CCM is specifically a container cluster. The create response returned by the CCM is specifically a container cluster create response. A create success indication in the container cluster create response indicates that the container cluster was created successfully.

[0138] Optionally, the creation request initiated by the CCM consumer is a container cluster node creation request, which instructs the CCM to create a container cluster node. The container cluster object created by the CCM is specifically a container cluster node. The creation response returned by the CCM is specifically a container cluster node creation response. A creation success indication in the container cluster node creation response indicates that the container cluster node was successfully created.

[0139] Optionally, the method shown in Fig. 3 further includes a container cluster Life Cycle Management (LCM) authorization process. Specifically, Fig. 5 is a flowchart of a method for managing storage resources of a container cluster according to an embodiment of the present application. In addition to all the steps in Fig. 3, Fig. 5 further includes the following steps S321 to S326. For other steps other than steps S321 to S326 in Fig. 5, please refer to the description of Fig. 3. Optionally, steps S321 to S326 are performed before step S302.

[0140] Step S321: The CCM generates a life cycle authorization request.

[0141] The lifecycle authorization request is used to apply for authorization for lifecycle management of a container cluster object. Optionally, the lifecycle authorization request includes an identifier of storage resource description information.

[0142] Step S322: The CCM sends a life cycle authorization request to the CCM consumer.

[0143] Step S323: The CCM consumer receives a lifecycle authorization request from the CCM.

[0144] Step S324: The CCM consumer generates a lifecycle authorization response.

[0145] Specifically, the CCM consumer determines whether the CCM is authorized to start a lifecycle management process for a container cluster object. If the CCM consumer authorizes the CCM to start a lifecycle management process for a container cluster object, the CCM consumer includes an authorization indication in the lifecycle authorization response. If the CCM consumer denies the lifecycle management process for a container cluster object started by the CCM, the CCM consumer includes a denial indication in the lifecycle authorization response. The authorization indication indicates that the CCM is permitted to start a lifecycle management process for a container cluster object. The denial indication indicates that the CCM is denied to start a lifecycle management process for a container cluster object.

[0146] Step S325: The CCM consumer sends a life cycle authorization response to the CCM.

[0147] Step S326: The CCM receives a lifecycle authorization response from the CCM consumer.

[0148] The CCM determines whether the instruction carried in the lifecycle authorization response is an authorization instruction or a denial instruction. If the lifecycle authorization response carries an authorization instruction, which is equivalent to obtaining authentication of the CCM consumer, the CCM executes step S302 of the method shown in Figure 3 to initiate a procedure to request the VIM to allocate storage resources. If the lifecycle authorization response carries a denial instruction, the CCM cancels execution of the procedure to request the VIM to allocate storage resources.

[0149] In the following, we will use examples to explain the method shown in FIG. 3 with reference to three instances.

[0150] Instance 1 Instance 1 is for creating a container cluster group storage resource. The container cluster group in instance 1 is a container cluster object in the manner shown in Fig. 3. The value of the second attribute of the storage resource in instance 1 is dynamic allocation.

[0151] As shown in FIG. 6, instance 1 includes steps S401 to S406.

[0152] Step S401: A CCM consumer sends a container cluster group creation request to a CCM, which carries an identifier of description information of a container cluster group to be created.

[0153] Step S402: The CCM accesses the description information of the container cluster group according to the identifier of the description information of the container cluster group, and the CCM obtains the description information of the container cluster group storage resource from the description information of the container cluster group.

[0154] Step S403: The CCM sends a virtual storage resource allocation request to the VIM. The virtual storage resource allocation request carries information such as the name and resource specification / size of the storage resource to be created. The information carried in the virtual storage resource allocation request is from the description information of the container cluster group storage resource.

[0155] Step S404: The VIM allocates corresponding storage resources of the virtual storage resources managed by the VIM based on information such as the name and resource specifications / size of the resource to be created, and the VIM returns a virtual storage resource allocation response to the CCM.

[0156] Step S405: The CCM further locally configures container cluster group storage resource information, including information about container cluster group persistent storage resources, information about temporary storage resources shared by the container cluster group, and information about temporary storage resources dedicated to the container cluster.

[0157] Step S406: The CCM returns a container cluster group creation response to the CCM consumer, where the container cluster group creation response carries a creation success indication.

[0158] Instance 2 Instance 2 is for creating a container cluster storage resource. The container cluster in instance 2 is a container cluster object in the manner shown in Fig. 3. The value of the second attribute of the storage resource in instance 2 is dynamic allocation.

[0159] As shown in FIG. 7, instance 2 includes steps S501 to S506.

[0160] Step S501: A CCM consumer sends a container cluster creation request to a CCM, which carries an identifier of description information of the container cluster to be created.

[0161] Step S502: The CCM accesses the description information of the container cluster according to the identifier of the description information of the container cluster, and the CCM obtains the description information of the container cluster storage resource from the description information of the container cluster.

[0162] Step S503: The CCM sends a virtual storage resource allocation request to the VIM. The virtual storage resource allocation request carries information such as the name and resource specification / size of the storage resource to be created. The information carried in the virtual storage resource allocation request is from the description information of the container cluster storage resource.

[0163] Step S504: The VIM allocates corresponding storage resources of the virtual storage resources managed by the VIM based on information such as the name and resource specifications / size of the resource to be created, and the VIM returns a virtual storage resource allocation response to the CCM.

[0164] Step S505: The CCM further locally configures container cluster storage resource information, including information about container cluster persistent storage resources, information about temporary storage resources shared by the container cluster, and temporary storage resources, temporary volumes, dedicated to the container cluster nodes.

[0165] Step S506: The CCM returns a container cluster creation response to the CCM consumer, where the container cluster creation response carries a creation success indication.

[0166] Instance 3 Instance 3 adds a container cluster lifecycle management authorization process based on Instance 2, so that the container cluster lifecycle management authorization process is integrated into the container cluster storage resource management process. The procedure of Instance 3 is shown in Figure 8. The container cluster lifecycle management authorization process includes the following steps S603 and S604. For other steps other than steps S603 and S604 in Figure 8, please refer to the description of Instance 2.

[0167] Step S603: Before the CCM initiates a virtual storage resource allocation request to the VIM, the CCM first initiates a container cluster lifecycle management authorization request to the CCM consumer, and the lifecycle management authorization request carries an identifier of the description information of the container cluster storage resource to be created.

[0168] Step S604: The CCM consumer accesses the storage resource description of the container cluster description information based on the identifier of the container cluster storage resource description information, and determines whether the lifecycle management operation is authorized in the management domain of the CCM consumer. The CCM consumer returns a container cluster lifecycle management authorization response to the CCM, and the lifecycle management authorization response carries an instruction to allow or deny the lifecycle management operation.

[0169] 9 is a schematic diagram of the structure of an apparatus for managing storage resources of a container cluster according to an embodiment of the present application. The apparatus 800 shown in FIG. 9 is disposed in a CCM. The apparatus 800 includes an acquiring unit 801, a generating unit 802, and a sending unit 803. Optionally, the apparatus 800 further includes a receiving unit, a creating unit, and a configuring unit.

[0170] Optionally, referring to FIG. 1, the apparatus 800 shown in FIG. 9 is located in the CCM of FIG.

[0171] Optionally, referring to FIG. 2, the apparatus 800 shown in FIG. 9 is located in the CCM of FIG.

[0172] Optionally, referring to Fig. 3, the apparatus 800 shown in Fig. 9 is disposed in the CCM shown in Fig. 3. The obtaining unit 801 is configured to support the apparatus 800 in performing S301, the generating unit 802 is configured to support the apparatus 800 in performing S302, and the sending unit 803 is configured to support the apparatus 800 in performing S303. The receiving unit is configured to support the apparatus 800 in performing S307. The configuring unit is configured to support the apparatus 800 in performing S308.

[0173] Optionally, referring to Figure 4, the receiving unit is configured to support the device 800 in performing S312. The creating unit is configured to support the device 800 in performing S313.

[0174] Optionally, referring to Figure 5, the generating unit 802 is configured to support the device 800 in performing S321. The sending unit 803 is configured to support the device 800 in performing S322, and the receiving unit is configured to support the device 800 in performing S326.

[0175] Optionally, referring to Figure 6, the obtaining unit 801 is configured to support the device 800 in performing S402. The sending unit 803 is configured to support the device 800 in performing S403.

[0176] Optionally, referring to Figure 7, the obtaining unit 801 is configured to support the device 800 in performing S502. The sending unit 803 is configured to support the device 800 in performing S503.

[0177] Optionally, referring to Figure 8, the obtaining unit 801 is configured to support the device 800 in performing S602. The sending unit 803 is configured to support the device 800 in performing S603.

[0178] The device embodiment described in FIG. 9 is merely an example. For example, the division into units is merely a logical division of functions, and other divisions may be used in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. The functional units in the embodiments of the present application may be integrated into one processing unit, or each of the units may exist physically alone, or two or more units may be integrated into one unit.

[0179] All or some of the units of the device 800 may be implemented using software, hardware, firmware, or any combination thereof.

[0180] When software is used for implementation, for example, the generating unit 802 and the creating unit are implemented by software functional units that are generated after at least one processor 901 in FIG. 10 reads program code stored in memory 902.

[0181] When hardware is used for implementation, for example, the above units in Fig. 9 are separately implemented by different hardware of a computing device. For example, the generating unit 802 is implemented by some processing resources (e.g., one core or two cores of a multi-core processor) in the at least one processor 901 in Fig. 10, and the creating unit is implemented by the remaining processing resources (e.g., another core of the multi-core processor) in the at least one processor 901 in Fig. 10, or by using a programmable device such as an FPGA (field-programmable gate array), a coprocessor, etc. The acquiring unit 801 and the sending unit 803 are implemented by the network interface 903 in Fig. 10.

[0182] When a combination of software and hardware is used for implementation, for example, the generating unit 802 is implemented by a hardware programmable device, and the creating unit is a software functional unit that is generated after a CPU reads a program code stored in a memory.

[0183] In the following, an example is used to describe the basic hardware structure for implementing CCM.

[0184] 10 is a schematic diagram of the structure of a computing device according to one embodiment of the present application. The CCM is located on a computing device 900 shown in FIG.

[0185] Optionally, referring to FIG. 1, the computing device 900 shown in FIG. 10 is equipped with the CCM of FIG.

[0186] Optionally, referring to FIG. 2, the computing device 900 shown in FIG. 10 is equipped with the CCM of FIG.

[0187] Optionally, with reference to FIG. 3, the computing device 900 shown in FIG. 10 is configured to perform the steps that the CCM of FIG. 3 is responsible for performing.

[0188] The computing device 900 includes at least one processor 901 , memory 902 , and at least one network interface 903 .

[0189] The processor 901 may be, for example, a general-purpose central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits configured to implement the solutions of the present application. For example, the processor 901 may include an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or a combination thereof.

[0190] The memory 902 may be, for example, a read-only memory (ROM) or another type of static storage device capable of storing static information and instructions, a random access memory (RAM) or another type of dynamic storage device capable of storing information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other compact disc storage, an optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disc storage medium or other magnetic storage device, or any other medium capable of carrying or storing appropriate program code in the form of instruction structures or data structures and accessible by a computer. However, the memory 902 is not limited thereto. Optionally, the memory 902 exists independently and is connected to the processor 901 through an internal connection 904. Alternatively, optionally, the memory 902 and the processor 901 are integrated together.

[0191] The network interface 903 may be configured to communicate with another device or a communication network using any device, such as a transceiver. The network interface 903 may include, for example, at least one of a wired network interface and a wireless network interface. The wired network interface may be, for example, an Ethernet interface. The Ethernet interface may be, for example, an optical interface, an electrical interface, or a combination thereof. The wireless network interface may be, for example, a wireless local area network (WLAN) interface, a cellular network interface, or a combination thereof.

[0192] In some embodiments, processor 901 includes one or more CPUs, such as CPU 0 and CPU 1 shown in FIG.

[0193] Optionally, in some embodiments, computing device 900 includes multiple processors, such as processor 901 and processor 905 shown in Figure 10. Each of these processors may be, for example, a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Optionally, a processor herein is one or more devices, circuits, and / or processing cores configured to process data (e.g., computer program instructions).

[0194] In some embodiments, computing device 900 further includes an internal connection 904. Processor 901, memory 902, and at least one network interface 903 are connected through internal connection 904. Internal connection 904 includes a path for transferring information between the aforementioned components. Optionally, internal connection 904 is a board or bus. Optionally, internal connection 904 is categorized as an address bus, a data bus, a control bus, or the like.

[0195] In some embodiments, the computing device 900 further includes an input / output interface 906. The input / output interface 906 is connected to the internal connection 904.

[0196] Optionally, the processor 901 implements the method of the foregoing embodiment by reading the program code 910 stored in the memory 902, or the processor 901 implements the method of the foregoing embodiment by using the program code stored therein. When the processor 901 implements the method of the foregoing embodiment by reading the program code 910 stored in the memory 902, the memory 902 stores the program code for implementing the method for managing storage resources of a container cluster provided in the embodiment of the present application.

[0197] For more details about the processor 901 performing the aforementioned functions, please refer to the description of the aforementioned method embodiment, and the details will not be repeated here.

[0198] Each embodiment herein is described in an incremental manner, and references may be made to each other for the same or similar parts of each embodiment, with each embodiment focusing on the differences from other embodiments.

[0199] A refers to B, that is, A is the same as B or A is a simple variation of B.

[0200] In the description and claims of the embodiments of this application, terms such as "first," "second," etc. are intended to distinguish between different objects, but do not indicate a particular order of the objects, and cannot be understood as indicating or suggesting relative importance. For example, "first attribute" and "second attribute" are used to distinguish between different attributes, but are not used to describe a particular order of the attributes. The first attribute cannot be understood as being more important than the second attribute.

[0201] In the embodiments of this application, unless otherwise specified, "at least one" means one or more, and "plurality" means two or more. For example, a plurality of attributes means two or more attributes.

[0202] All or part of the above embodiments may be implemented using software, hardware, firmware, or any combination thereof. When software is used to implement the embodiments, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the procedures or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, or digital subscriber line (DSL)) or wireless (e.g., infrared, radio, or microwave) method. The computer-readable storage medium may be any available medium accessible by a computer, or may be a data storage device, such as a server or data center, incorporating one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a DVD), a semiconductor medium (e.g., a solid state disk (SSD)), or the like.

[0203] The foregoing embodiments are only for illustrating the technical solutions of the present application, and are not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that, without departing from the scope of the technical solutions of the embodiments of the present application, they may still make modifications to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some technical features thereof. [Explanation of symbols]

[0204] 21 Cluster Persistent Volumes 22 Temporary storage dedicated to cluster nodes 23 Temporary Storage 800 Device for managing storage resources of a container cluster 801 Acquisition Units 802 Generator Units 803 Transmitting Unit 900 computing devices 901 processor 902 memory 903 Network Interface 904 Internal Connection 905 processor 906 Input / Output Interface 910 Program Code

Claims

1. 1. A method for managing storage resources of a container cluster, the method comprising: obtaining storage resource description information by a container cluster management (CCM) that manages a container cluster including at least one or more container cluster nodes, the storage resource description information being used to describe storage resource requirements of a container cluster object managed by the CCM, the container cluster object being a container cluster; generating, by the CCM, a resource allocation request based on the storage resource description information, the resource allocation request instructing a Virtualization Infrastructure Manager (VIM) to allocate storage resources, the storage resources allocated by the VIM consisting of cluster persistent volumes (PVs) that provide storage resources necessary to run a Cluster Container Infrastructure Services Management (CISM) instance, the CISM instance being used to provide Container as a Service (CaaS) management plane functions; sending, by the CCM, the resource allocation request to the VIM; A method comprising:

2. 2. The method of claim 1, wherein the storage resource description information includes a third attribute of the storage resource, the third attribute being used to describe an affinity rule or an anti-affinity rule that the storage resource must satisfy.

3. The method of claim 1 , wherein the storage resource description information includes a first attribute of the storage resource, the first attribute indicating a tier of a container cluster object to which the storage resource is applicable.

4. 2. The method of claim 1, wherein the storage resource description information includes a second attribute of the storage resource, the second attribute indicating whether the storage resource is statically provided or dynamically allocated, wherein statically providing the storage resource means that the storage resource is allocated before the container cluster object is created, and dynamically allocating the storage resource means that the storage resource is allocated during the creation process of the container cluster object.

5. 2. The method of claim 1, wherein the storage resource description information includes a fourth attribute of the storage resource, the fourth attribute being used to describe whether the storage resource is a persistent storage resource or a temporary storage resource.

6. The method of claim 1, wherein the storage resource description information includes a fifth attribute of the storage resource, the fifth attribute being used to describe whether the storage resource is a dedicated storage resource or a shared storage resource, wherein the dedicated storage resource is a storage resource that is permitted to be used only by a particular sub-object of the container cluster object, and the shared storage resource is a storage resource that is permitted to be used by each sub-object of multiple sub-objects of the container cluster object.

7. 2. The method of claim 1, wherein the storage resource description information includes a sixth attribute of the storage resource, the sixth attribute including at least one of a name of the storage resource, a specification of the storage resource, and description information of a virtual link connected to the storage resource.

8. After the step of sending the resource allocation request to the VIM by the CCM, the method further comprises: receiving, by the CCM, a resource allocation response from the VIM, the resource allocation response indicating storage resources already allocated by the VIM; configuring, by the CCM, the storage resources already allocated by the VIM for the container cluster object; The method of claim 1 further comprising:

9. The method comprises: receiving, by the CCM, a creation request from a CCM consumer, the CCM consumer being an operations support system (OSS) / business support system (BSS) or a network function virtualization orchestrator (NFVO), the creation request instructing the CCM to create the container cluster object; creating, by the CCM, the container cluster object; sending, by the CCM, a create response to the CCM consumer; The method of claim 1 further comprising:

10. the creation request includes a container cluster group creation request, and the creation response includes a container cluster group creation response; the creation request comprises a container cluster creation request, and the creation response comprises a container cluster creation response; or the creation request includes a container cluster node creation request, and the creation response includes a container cluster node creation response; The method of claim 9.

11. The step of obtaining storage resource description information by the CCM includes: obtaining, by the CCM, the storage resource description information from a descriptor file of the container cluster object; or obtaining, by the CCM, the storage resource description information from a virtualized network function descriptor (VNFD) file; 2. The method of claim 1, comprising:

12. Before the step of sending the resource allocation request to the VIM by the CCM, the method further comprises: sending, by the CCM, a lifecycle authorization request to a CCM consumer, the lifecycle authorization request being for requesting authorization for lifecycle management of the container cluster object, the CCM consumer being an OSS / BSS or an NFVO; receiving, by the CCM, a lifecycle authorization response from the CCM consumer, the lifecycle authorization response including an authorization indication, the authorization indication indicating that the CCM is permitted to initiate a lifecycle management process for the container cluster object; The method of claim 1 further comprising:

13. An apparatus for managing storage resources of a container cluster, the apparatus being disposed in a container cluster management (CCM) that manages a container cluster including at least one or more container cluster nodes, the apparatus comprising: an acquiring unit configured to acquire storage resource description information, the storage resource description information being used to describe storage resource requirements of a container cluster object managed by the CCM, the container cluster object being a container cluster; and a generating unit configured to generate a resource allocation request based on the storage resource description information, the resource allocation request instructing a Virtualization Infrastructure Manager (VIM) to allocate storage resources, the storage resources allocated by the VIM consisting of cluster persistent volumes (PVs) that provide storage resources required to run a Cluster Container Infrastructure Services Management (CISM) instance, the CISM instance being used to provide Container as a Service (CaaS) management plane functions; a sending unit configured to send the resource allocation request to the VIM; An apparatus comprising:

14. 14. The apparatus of claim 13, wherein the storage resource description information includes a third attribute of the storage resource, the third attribute being used to describe an affinity rule or an anti-affinity rule that the storage resource must satisfy.

15. The apparatus of claim 13 , wherein the storage resource description information includes a first attribute of the storage resource, the first attribute indicating a tier of a container cluster object to which the storage resource is applicable.

16. The apparatus of claim 13, wherein the storage resource description information includes a second attribute of the storage resource, the second attribute indicating whether the storage resource is statically provided or dynamically allocated, wherein statically providing the storage resource means that the storage resource is allocated before the container cluster object is created, and dynamically allocating the storage resource means that the storage resource is allocated during the creation process of the container cluster object.

17. 14. The apparatus of claim 13, wherein the storage resource description information includes a fourth attribute of the storage resource, the fourth attribute being used to describe whether the storage resource is a persistent storage resource or a temporary storage resource.

18. The apparatus of claim 13, wherein the storage resource description information includes a fifth attribute of the storage resource, the fifth attribute being used to describe whether the storage resource is a dedicated storage resource or a shared storage resource, wherein the dedicated storage resource is a storage resource that is permitted to be used only by a particular sub-object of the container cluster object, and the shared storage resource is a storage resource that is permitted to be used by each sub-object of multiple sub-objects of the container cluster object.

19. 14. The apparatus of claim 13, wherein the storage resource description information includes a sixth attribute of the storage resource, the sixth attribute including at least one of a name of the storage resource, a specification of the storage resource, and description information of a virtual link connected to the storage resource.

20. The device, a receiving unit configured to receive a resource allocation response from the VIM, the resource allocation response indicating storage resources already allocated by the VIM; a configuration unit configured to configure the storage resources already allocated by the VIM for the container cluster object; The apparatus of claim 13 further comprising:

21. The device, a receiving unit configured to receive a creation request from a CCM consumer, the CCM consumer being an operation support system (OSS) / business support system (BSS) or a network function virtualization orchestrator (NFVO), the creation request instructing the container cluster object to be created; a creation unit configured to create the container cluster object; the sending unit further configured to send a creation response to the CCM consumer; The apparatus of claim 13 further comprising:

22. A computing device, the computing device comprising a processor and a memory; the memory stores computer instructions; The processor executes the computer instructions to perform the method of any one of claims 1 to 12. Computing devices.

23. A network system, said network system comprising an apparatus according to any one of claims 13 to 21 and an apparatus arranged in a VIM.

24. 13. A computer-readable storage medium storing at least one instruction that, when executed on a computer, enables the computer to perform the method of any one of claims 1 to 12.

Images