Storage resource scheduling method and device, computer device and storage medium

Abstract

Embodiments of the present application relate to a storage resource scheduling method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a virtual machine storage request, an idle storage resource amount of a candidate node, and a temporary reserved resource amount; adding the storage resource amount corresponding to the storage request and the temporary reserved resource amount of the candidate node to obtain a to-be-allocated storage resource amount; determining a target node from the candidate nodes whose idle storage resource amount meets the to-be-allocated storage resource amount; marking a persistent storage volume declaration of the virtual machine as the target node; and adding the storage resource amount corresponding to the storage request to the temporary reserved resource amount of the target node. Thus, in the virtual machine storage resource scheduling process, the storage request of the to-be-scheduled virtual machine and the existing temporary reserved resource amount of the candidate node are jointly included in the resource check, and the corresponding temporary reserved resource amount is recorded in time after the target node is determined, thereby avoiding repeated allocation of the same storage resource and reducing the risk of excessive allocation of storage resources in virtual machine concurrent scheduling.

Description

Technical Field

[0001] This application relates to cloud computing and virtualization resource management, and more particularly to a storage resource scheduling method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of container orchestration technology, virtual machines (VMs) can run in container clusters through virtualization management components, and the cluster scheduler selects the running node for the VM. VMs typically request storage resources through persistent storage volume declarations. The scheduler determines the target node to host the VM based on the storage resource status of each node and the VM's storage request. For storage methods that directly use the node's local disk, node storage resources also need to be classified and managed according to storage medium or storage purpose.

[0003] In the existing scheduling process, the determination of the target node, the selection of the node for persistent storage volume declaration, and the creation and binding of the persistent storage volume are usually executed asynchronously. There is a time interval between the scheduler determining the node and the storage resource completing the binding. During this time interval, storage resources selected by the virtual machine but not yet bound are not promptly included in the node's actual occupancy. Simultaneously, there is a delay in updating the node's storage status, which may cause the scheduler to make scheduling decisions based on outdated storage resource statuses when scheduling multiple virtual machines concurrently.

[0004] Therefore, existing technologies tend to repeatedly allocate the same portion of node storage resources to multiple virtual machines, leading to over-allocation of storage resources, storage binding failures, and virtual machines remaining in a waiting state for extended periods. The technical problem that needs to be solved is: how to avoid over-allocation of storage resources during concurrent virtual machine scheduling. Summary of the Invention

[0005] In view of this, in order to solve the above-mentioned technical problems or some of the technical problems, the embodiments of this application provide a storage resource scheduling method, apparatus, computer equipment and storage medium.

[0006] In a first aspect, embodiments of this application provide a storage resource scheduling method, including: The storage requests of virtual machines to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node are obtained. The amount of temporary reserved resources represents the amount of storage resources requested by virtual machines that have been allocated to candidate nodes but have not yet completed storage binding. For each candidate node, the amount of storage resources corresponding to the storage request is added to the amount of temporary reserved resources of the candidate node to obtain the amount of storage resources to be allocated, and the target node is determined from the candidate nodes whose free storage resources meet the amount of storage resources to be allocated. The persistent storage volume of the virtual machine to be scheduled is declared as the target node, and the amount of storage resources corresponding to the storage request is included in the temporary reserved resources of the target node.

[0007] In one possible implementation, after marking the persistent storage volume declaration as the target node, the method further includes: Based on the storage request, a persistent storage volume is created on the target node, and the persistent storage volume declaration is bound to the persistent storage volume; After the persistent storage volume declaration is completed and bound, the storage status information of the target node is updated according to the storage resources occupied by the persistent storage volume.

[0008] In one possible implementation, the step of including the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node includes: Listen for node selection information of the persistent storage volume declaration through a persistent storage volume declaration event listener; When a target node tag for indicating the target node is added to the node selection information, the virtual machine to be scheduled associated with the persistent storage volume declaration and the storage request of the virtual machine to be scheduled are obtained. Based on the target node marker, the amount of storage resources corresponding to the storage request is included in the temporary reserved resource amount of the target node.

[0009] In one possible implementation, the method further includes: The storage status information of the target node is monitored through a node event listener; When an update to the storage status information is detected, the binding status of each persistent storage volume declaration associated with the virtual machine to be scheduled is obtained; Once each persistent storage volume declaration has been bound, the temporary reserved resources corresponding to the virtual machine to be scheduled are deleted.

[0010] In one possible implementation, obtaining the amount of free storage resources for each candidate node and the amount of temporary reserved resources already available for each candidate node includes: The storage requests are divided according to disk category to obtain the requested storage resource amount of the virtual machine to be scheduled under each disk category; For each candidate node, obtain the amount of free storage resources and the amount of temporary reserved resources under each disk category; The step of determining the target node from candidate nodes whose free storage resources meet the required storage resource allocation includes: For each candidate node, the requested storage resource amount under the same disk category is added to the temporary reserved resource amount to obtain the corresponding storage resource amount to be allocated; Candidate nodes whose unallocated storage resource amount under each disk category is no greater than the corresponding free storage resource amount are determined as pre-selected nodes; Calculate the remaining resource ratio of each of the pre-selected nodes under each of the disk categories, take the minimum value of each remaining resource ratio as the storage matching score of the pre-selected node, and determine the pre-selected node with the highest storage matching score as the target node.

[0011] In one possible implementation, the step of including the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node includes: Generate a virtual machine identifier for the virtual machine to be scheduled based on the namespace and virtual machine name to which the virtual machine to be scheduled belongs; Query whether the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier; When the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier, the amount of storage resources corresponding to the storage request is used to replace the recorded amount of temporary reserved resources. When the target node does not record the amount of temporary reserved resources corresponding to the virtual machine identifier, the amount of storage resources corresponding to the storage request is associated with the virtual machine identifier and recorded to obtain the amount of temporary reserved resources corresponding to the virtual machine to be scheduled.

[0012] In one possible implementation, the method further includes: When the storage resource amount corresponding to the storage request is included in the temporary reserved resource amount of the target node, the corresponding generation time is recorded; According to the preset inspection cycle, the duration of the temporary reserved resource quantity is determined based on the current time and the generation time; When the duration reaches the preset survival time, the corresponding temporary reserved resources are deleted.

[0013] Secondly, embodiments of this application provide a storage resource scheduling apparatus, comprising: The acquisition module is used to acquire the storage request of the virtual machine to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node. The amount of temporary reserved resources represents the amount of storage resources requested by the virtual machine that has been allocated to the candidate node but has not yet completed storage binding. The first processing module is used to add the storage resource amount corresponding to the storage request to the temporary reserved resource amount of the candidate node for each candidate node to obtain the storage resource amount to be allocated, and to determine the target node from the candidate nodes whose free storage resource amount meets the storage resource amount to be allocated; The second processing module is used to mark the persistent storage volume declaration of the virtual machine to be scheduled as the target node, and to include the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node.

[0014] Thirdly, embodiments of this application provide a computer device, including: a processor and a memory, wherein the processor is configured to execute a storage resource scheduling program stored in the memory to implement the storage resource scheduling method described in any one of the first aspects above.

[0015] Fourthly, embodiments of this application provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the storage resource scheduling method described in any of the first aspects above.

[0016] The storage resource scheduling scheme provided in this application obtains the storage request of the virtual machine to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node. The temporary reserved resources represent the amount of storage resources requested by virtual machines that have been allocated to candidate nodes but have not yet completed storage binding. For each candidate node, the amount of storage resources corresponding to the storage request is added to the temporary reserved resources of the candidate node to obtain the amount of storage resources to be allocated. A target node is then determined from among the candidate nodes whose free storage resources meet the amount of storage resources to be allocated. The persistent storage volume declaration of the virtual machine to be scheduled is marked as the target node, and the amount of storage resources corresponding to the storage request is included in the temporary reserved resources of the target node. Therefore, by incorporating the storage request of the virtual machine to be scheduled and the amount of temporary reserved resources of the candidate nodes into resource verification during the virtual machine storage resource scheduling process, and by promptly recording the corresponding temporary reserved resources after determining the target node, subsequent scheduling can detect the occupation of resources that have not yet completed storage binding, thereby avoiding the duplicate allocation of the same storage resources and reducing the risk of over-allocation of storage resources in concurrent virtual machine scheduling. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a storage resource scheduling method provided in an embodiment of this application; Figure 2 A flowchart illustrating another storage resource scheduling method provided in an embodiment of this application; Figure 3This is a schematic diagram of the structure of a storage resource scheduling device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0020] This embodiment describes a specific implementation process for storage resource scheduling. Please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart illustrating a storage resource scheduling method provided in an embodiment of this application. Figure 1 As shown, the specific implementation steps for storage resource scheduling are as follows: Step S101: Obtain the storage request of the virtual machine to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node.

[0021] The storage resource scheduling method provided in this application can be specifically applied to virtual machine scheduling scenarios in container orchestration platforms, especially to scenarios where multiple virtual machines concurrently request local storage resources on nodes. The aforementioned application scenarios are merely exemplary and illustrative. In practical applications, this storage resource scheduling method can also be applied to other computing resource scheduling scenarios involving asynchronous storage allocation processes, and this application does not limit its application to these scenarios.

[0022] The storage resource scheduling method provided in this application can be executed by a scheduling controller in each step. The scheduling controller can be an extended scheduling component in a container orchestration platform, or a storage scheduling service that works in conjunction with a cluster scheduler. It is used to read the storage status of nodes, verify the storage requests of virtual machines, determine the target node, and maintain the amount of temporarily reserved resources. This application does not limit this.

[0023] Optionally, the container orchestration platform can be the Kubernetes open-source container orchestration platform, and the virtual machine management architecture can be the KubeVirt virtual machine management architecture. KubeVirt is used to run and manage virtual machines in a Kubernetes cluster. A PersistentVolumeClaim (PVC) is used to request storage resources; a PersistentVolume (PV) represents the actual storage resources provided to the virtual machine.

[0024] Temporarily reserved resources refer to the amount of storage resources requested by virtual machines during scheduling or storage binding that have not yet been included in the actual node usage. In one implementation, the amount of temporarily reserved resources can be recorded through InflightEntries, where Inflight indicates that the resource has entered the scheduling process but has not yet completed actual storage binding. InflightEntries can record the target node, virtual machine identifier, storage requests categorized by disk type, associated persistent storage volume declarations, and expiration times.

[0025] A virtual machine to be scheduled can be a virtual machine that has initiated a creation or run request but has not yet determined its final running node. A storage request can be storage resource information requested by the virtual machine to be scheduled for its system disk or data disk, including at least the requested storage resource amount, and may also include disk type, disk quantity, and associated persistent storage volume declaration. A candidate node can be a cluster node that, after verification of computing resources, memory resources, network resources, or basic operating conditions, has the potential to host the virtual machine to be scheduled. The amount of free storage resources can be the total storage capacity of the candidate nodes minus the actual amount of storage resources already occupied. The amount of temporarily reserved resources represents the amount of storage resources requested by virtual machines that have been allocated to candidate nodes but have not yet completed storage binding.

[0026] In this embodiment, after receiving the scheduling request corresponding to the virtual machine to be scheduled, the scheduling controller parses the persistent storage volume declaration associated with the virtual machine to be scheduled, obtains the requested capacity and disk type in each persistent storage volume declaration, and summarizes them to obtain the storage request of the virtual machine to be scheduled. The scheduling controller reads the storage status information of each candidate node, calculates the amount of free storage resources based on the total storage capacity of the node and the actual amount of storage resources already occupied; at the same time, it queries the scheduling entries belonging to each candidate node from the temporary reservation record set, and summarizes the storage resource amounts recorded in the scheduling entries to obtain the amount of temporary reserved resources already available for each candidate node.

[0027] For example, a virtual machine to be scheduled requests 80 GiB, where GiB is a unit of storage capacity measured in binary. Candidate node A has 200 GiB of free storage resources. Two other virtual machines on candidate node A that have not yet completed storage binding have reserved 20 GiB and 40 GiB respectively. Therefore, candidate node A has an existing temporary reserved resource amount of 60 GiB.

[0028] Optionally, node storage status information can be recorded in the annotations of the node object. Annotations are key-value pairs attached to resource objects and can be used to record the total capacity, used capacity, and free capacity of a node under different disk types and storage pools.

[0029] Step S102: For each candidate node, add the storage resource amount corresponding to the storage request to the temporary reserved resource amount of the candidate node to obtain the storage resource amount to be allocated, and determine the target node from the candidate nodes whose free storage resource amount meets the storage resource amount to be allocated.

[0030] The amount of storage resources to be allocated can be the total amount of current storage requests and existing temporary reserved resources that need to be jointly carried by the corresponding candidate nodes when performing resource verification on the virtual machine to be scheduled. The amount of free storage resources must satisfy the amount of storage resources to be allocated, meaning that the amount of free storage resources of the candidate nodes is not less than the amount of storage resources to be allocated. The target node can be a node that has passed storage resource verification and been determined to carry the virtual machine to be scheduled.

[0031] In this embodiment, the scheduling controller sequentially adds the requested storage resource amount of the virtual machine to be scheduled to the existing temporary reserved resource amount of the candidate node for each candidate node, forming a combined storage request, which is then used as the storage resource amount to be allocated. The scheduling controller compares the storage resource amount to be allocated with the free storage resource amount of the candidate node: if the storage resource amount to be allocated is greater than the free storage resource amount, it is determined that the candidate node cannot handle the current storage request; if the storage resource amount to be allocated is not greater than the free storage resource amount, it is determined that the candidate node passes the resource verification. The scheduling controller then determines the target node from the candidate nodes that have passed the resource verification.

[0032] For example, if a virtual machine to be scheduled requests 80 GiB of storage resources, and candidate node A has 200 GiB of free storage resources and 60 GiB of existing temporary reserved resources, then candidate node A's allocated storage resources are 140 GiB, and candidate node A passes the verification. Candidate node B, on the other hand, has 100 GiB of free storage resources and 40 GiB of existing temporary reserved resources, so candidate node B's allocated storage resources are 120 GiB, and candidate node B fails the verification. The scheduling controller can then determine candidate node A as the target node from the verified candidate nodes.

[0033] In Kubernetes extended scheduling scenarios, the above resource verification can be applied to the pre-selection verification stage and the optimization scoring stage, so that both stages can be aware of the amount of temporary reserved resources that have not yet been bound to storage.

[0034] Step S103: Mark the persistent storage volume of the virtual machine to be scheduled as the target node, and include the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node.

[0035] Marking a persistent storage volume declaration as a target node can be achieved by writing a target node marker in the node selection information of the persistent storage volume declaration. The target node marker indicates that subsequent storage provisioning operations should create or select a persistent storage volume on the target node. Including the storage resource amount corresponding to the storage request in the target node's temporary reserved resource amount can be done by recording the storage request as storage resource that has been selected and used by the target node but has not yet entered the actual occupancy statistics before the actual storage binding is completed.

[0036] In this embodiment, after determining the target node, the scheduling controller writes the target node's name into the node selection information of the persistent storage volume declaration associated with the virtual machine to be scheduled. In a Kubernetes scenario, this node selection information can be implemented using a selected-node annotation. After detecting the target node marker, the storage declaration event listener obtains the storage request of the virtual machine to be scheduled and generates a corresponding scheduling entry. This scheduling entry can record the target node name, virtual machine identifier, storage requests categorized by disk type, the list of associated persistent storage volume declarations, and the expiration time. The reservation management component, by aggregating the scheduling entries under the target node, includes the storage resource amount corresponding to this storage request in the target node's temporary reserved resource amount.

[0037] For example, after candidate node A is identified as the target node, the scheduling controller marks the persistent storage volume declaration of the data disk of the virtual machine to be scheduled as candidate node A, and records that the virtual machine temporarily reserves 80 GiB on candidate node A. At this time, the image data has not yet been actually written to candidate node A, but subsequent scheduling will include this 80 GiB in the existing temporary reserved resource amount of candidate node A.

[0038] As can be seen from the above, by including the storage requests of the virtual machines to be scheduled and the existing temporary reserved resources of the candidate nodes in the resource verification, and by increasing the corresponding temporary reserved resources in a timely manner after the target node is determined, the subsequent scheduling can detect the resource occupation that has not yet been completed in storage binding, thereby reducing the risk of repeated allocation of the same storage resources when multiple virtual machines are scheduled concurrently.

[0039] In one embodiment, after marking the persistent storage volume declaration as the target node, the following implementation steps are also included: Step S201: Based on the storage request, create a persistent storage volume on the target node and bind the persistent storage volume declaration to the persistent storage volume.

[0040] In this context, a persistent storage volume can be a resource object in the cluster that represents actual available storage resources. Creating a persistent storage volume involves allocating corresponding storage space from the storage resources of the target node and generating a persistent storage volume object based on the capacity, disk type, and access method requested in the persistent storage volume declaration. Binding establishes a one-to-one correspondence between persistent storage volume declarations and persistent storage volumes.

[0041] In this embodiment, the storage provisioning component detects the target node marker in the persistent storage volume declaration, reads the requested capacity and disk type corresponding to the persistent storage volume declaration, selects a storage space that meets the capacity requirements from the storage pool of the disk type corresponding to the target node, creates a persistent storage volume based on the selected storage space, and records the target node information in the persistent storage volume, and writes the name of the created persistent storage volume into the persistent storage volume declaration, so that the persistent storage volume declaration enters the bound state.

[0042] For example, if the target node has both a mechanical hard disk storage pool and a solid-state drive (SSD) storage pool, and the data disk request of the virtual machine to be scheduled belongs to the SSD category, then the storage provisioning component allocates the corresponding space from the SSD storage pool and creates a persistent storage volume. A storage pool can be a logical collection of one or more disks of the same disk category.

[0043] Step S202: After the persistent storage volume declaration is completed and bound, write the image data of the virtual machine to be scheduled into the persistent storage volume.

[0044] The image data can be the disk image content used to build or start the virtual machine to be scheduled, including operating system files, application files, or virtual machine initialization data. Writing image data to a persistent storage volume refers to writing the data in the image source to the actual storage space corresponding to the persistent storage volume on the target node.

[0045] In this embodiment, the image import component detects the binding status of the persistent storage volume declaration; after confirming that the persistent storage volume declaration has been bound to the corresponding persistent storage volume, it mounts the persistent storage volume to the image import component; it reads the image data of the virtual machine to be scheduled from the image repository, object storage, or other image sources, and writes the image data to the actual storage space of the target node according to the write path provided by the persistent storage volume; after the writing is completed, it records the image import success status.

[0046] For example, in a KubeVirt scenario, the virtual machine image importer (virt-importer) can perform the image writing. virt-importer is a container component used to import virtual machine images into persistent storage volumes, and can write images to persistent storage volumes corresponding to the system disk or data disk.

[0047] Step S203: Update the storage status information of the target node according to the storage resources occupied by the persistent storage volume.

[0048] Storage status information can be used to describe the total storage capacity, actual occupied storage resources, and free storage resources of the target node under different disk types and storage pools. Updating storage status information can include the storage resources occupied by newly created and bound persistent storage volumes in the target node's actual usage statistics.

[0049] In this embodiment, the node storage management component obtains the target node, disk type, storage pool, and occupied capacity corresponding to the newly created persistent storage volume; it adds the occupied capacity to the actual occupied storage resources of the corresponding disk type and storage pool; it recalculates the amount of free storage resources and writes the updated result to the storage status information of the target node. Subsequently, when the scheduling controller reads the storage status information of the target node, it can obtain the amount of free storage resources that already includes the actual storage allocation.

[0050] For example, when a virtual machine to be scheduled has one system disk persistent storage volume and two data disk persistent storage volumes, the disk type and capacity of the three persistent storage volumes can be obtained respectively, and the storage status information of the target node can be updated after being summarized by disk type.

[0051] As can be seen from the above, by creating and binding persistent storage volumes, writing virtual machine images, and updating node storage status on the target node, the temporary storage occupancy recorded during the scheduling phase can be converted into the actual storage occupancy on the target node, providing a basis for accurately cleaning up the amount of temporary reserved resources in the future.

[0052] In one embodiment, a specific implementation process is described that listens to node selection information for persistent storage volume declarations and records the amount of temporarily reserved resources when a target node tag is added. The specific implementation steps include the following: Step S301: Listen for the node selection information of the persistent storage volume declaration through the persistent storage volume declaration event listener.

[0053] The node selection information can be the information recorded in the persistent volume claim that records the node selected for the storage request. The storage claim event listener can be implemented using a persistent volume claim watcher (PVC Watcher) to monitor changes in the node selection information of the persistent volume claim.

[0054] In this embodiment, the reserved management component is a persistent storage volume declaration resource registration storage declaration event listener; the storage declaration event listener receives the creation event and update event of the persistent storage volume declaration; when an update event is received, the node selection information before and after the update is compared to determine whether a new target node tag has been added or whether the target node tag has changed.

[0055] For example, if the persistent storage volume declaration did not originally record the target node, after the scheduling controller determines node A, the selected-node annotation corresponding to node A is added to the node selection information, and the storage declaration event listener triggers temporary reservation processing accordingly.

[0056] Step S302: When a target node tag for indicating the target node is added to the node selection information, obtain the virtual machine to be scheduled associated with the persistent storage volume declaration and the storage request of the virtual machine to be scheduled.

[0057] The target node marker can be a marker used in the node selection information to indicate the selected node. The virtual machine to be scheduled associated with the persistent storage volume declaration can be a virtual machine that uses the persistent storage volume declaration as its system disk or data disk. The storage request can be determined based on the requested capacity and disk type in the associated persistent storage volume declaration.

[0058] In this embodiment, the storage claim event listener reads the namespace, name, owner information, or tag information of the persistent storage volume claim; based on the above information, it finds the virtual machine image import container group or virtual machine object associated with the persistent storage volume claim; it obtains the corresponding virtual machine name and all persistent storage volume claims associated with the virtual machine; and it reads the requested capacity and disk type in each persistent storage volume claim to form a storage request for the virtual machine to be scheduled.

[0059] For example, in a KubeVirt scenario, a container group can be called a Pod, and a Pod is the smallest scheduling unit in Kubernetes that hosts one or more containers. Storage claim event listeners can find the virt-importer Pod associated with a persistent storage volume claim and extract the virtual machine name, storage request, and list of persistent storage volume claims from that Pod and its associated resources.

[0060] Step S303: Based on the target node marker, the amount of storage resources corresponding to the storage request is included in the temporary reserved resource amount of the target node.

[0061] Specifically, the inclusion of resources based on the target node marker can be achieved by reading the node name indicated by the target node marker and assigning the corresponding storage request entry in the schedule to that target node. The inclusion of temporary reserved resources can be achieved by adding a new entry in the schedule and summarizing the storage resource amount in that entry when querying the temporary reserved resource amount of the target node.

[0062] In this embodiment, the reservation management component determines the target node based on the target node tag; organizes the storage requests of the virtual machines to be scheduled according to disk type; generates a scheduling entry including the target node name, virtual machine identifier, storage request, associated persistent storage volume declaration list, and expiration time; and writes the scheduling entry into the reservation record set in memory. When the scheduling controller subsequently queries the temporary reserved resource amount of the target node, the reservation management component summarizes all scheduling entries corresponding to the target node according to disk type.

[0063] For example, when the target node tag changes from null to node A, the storage declaration event listener determines that the persistent storage volume declaration belongs to virtual machine A, and obtains the 40GiB system disk and 100GiB data disk requested by virtual machine A; the reservation management component includes the requested capacity under the two disk categories into the temporary reserved resource amount of node A.

[0064] As can be seen from the above, by listening to the addition event of the target node marker in the persistent storage volume declaration, a temporary reserved record can be created in time before the storage resource has been actually bound, so that the scheduler can synchronously perceive the resource occupation during scheduling and binding.

[0065] In one embodiment, a specific implementation process for clearing temporary reserved resources after the storage state of the target node is updated is described, which includes the following implementation steps: Step S401: Listen to the storage status information of the target node through the node event listener.

[0066] In this context, the listener can be a scheduling controller or reserved management component that subscribes to change events of the target node's resource objects and receives event notifications when the target node's storage status information is updated. The node event listener (NodeWatcher) is used to detect changes in storage capacity or storage usage in node annotations.

[0067] In this embodiment, the reserved management component registers a node event listener for the target node; the node event listener continuously receives node resource addition or update events sent by the container orchestration platform; after receiving an update event, it compares the storage status information before and after the update, and when the information used to record the disk type, storage pool capacity or actual usage changes, it triggers the subsequent persistent storage volume declaration binding status check.

[0068] For example, when the actual usage of the solid-state drive storage pool of the target node is updated from 200GiB to 300GiB, the node event listener recognizes the change in storage status information and begins to check the persistent storage volume declaration associated with the temporary reservation record on the target node.

[0069] Step S402: When an update to the storage status information is detected, obtain the binding status of each persistent storage volume associated with the virtual machine to be scheduled.

[0070] The binding status indicates whether a persistent storage volume declaration has been bound to a corresponding persistent storage volume. Each persistent storage volume declaration associated with a virtual machine to be scheduled can include a persistent storage volume declaration corresponding to the system disk and persistent storage volume declarations corresponding to one or more data disks.

[0071] In this embodiment, the reservation management component reads the current status of each persistent storage volume declaration one by one according to the persistent storage volume declaration list stored in the scheduling entries; it determines whether the status stage of the persistent storage volume declaration is "bound" and whether the corresponding persistent storage volume name has been recorded in the persistent storage volume declaration. Only when the status stage is "bound" and the persistent storage volume name is not empty is the persistent storage volume declaration confirmed to be bound. If any persistent storage volume declaration has not yet been bound, the temporary reserved resource amount corresponding to the virtual machine to be scheduled continues to be reserved.

[0072] For example, if a virtual machine to be scheduled has three persistent storage volume declarations, two of which are already in the bound state and the other is still in the waiting state, the reservation management component will not delete the scheduling entry corresponding to the virtual machine to be scheduled.

[0073] Step S403: Once each persistent storage volume declaration has been bound, delete the temporary reserved resources corresponding to the virtual machine to be scheduled.

[0074] Deleting temporary reserved resources can be achieved by removing the scheduled entry of the virtual machine to be scheduled from the set of scheduled entries corresponding to the target node, and then no longer including the storage resource amount recorded in the scheduled entry when summarizing the temporary reserved resources of the target node in the subsequent calculation.

[0075] In this embodiment, after confirming that the storage status information of the target node has been updated and that each persistent storage volume declaration associated with the virtual machine to be scheduled has been bound, the reservation management component locates the entry in the schedule based on the scheduling object identifier or virtual machine identifier corresponding to the virtual machine to be scheduled; deletes the entry from the set of entries in the schedule in memory; and re-aggregates the remaining entries in the schedule of the target node to obtain the updated amount of temporary reserved resources.

[0076] For example, if a virtual machine to be scheduled temporarily reserves 140 GiB on the target node, after the corresponding persistent storage volumes are all bound and the actual usage on the target node has increased by 140 GiB, the reservation management component deletes the 140 GiB temporary reservation for the virtual machine to avoid the same part of resources being counted in both actual usage and temporary reservation at the same time.

[0077] Binding node storage status updates and the completion of each persistent storage volume declaration together as deletion conditions can prevent premature release of temporary reserved resources before the actual usage of the node has been updated.

[0078] As can be seen from the above, by sensing the storage status update of the target node through the node event listener, and deleting the amount of temporary reserved resources after each associated persistent storage volume declaration is bound, the temporary reserved resources can be accurately released after fulfilling the function of preventing duplicate allocation.

[0079] In the embodiments of steps 301 to 303 and 401 to 403, the persistent storage volume declaration event listener and the node event listener together constitute a dual-listener mechanism. The persistent storage volume declaration event listener uses the addition of the target node tag in the persistent storage volume declaration as the creation trigger condition for temporary reserved resources, so that storage resources that have selected target nodes but have not yet been included in the actual usage of the node can be recorded in a timely manner. The node event listener uses the update of the storage status information of the target node as the cleanup trigger condition, and deletes the corresponding temporary reserved entries after confirming that all persistent storage volume declarations associated with the temporary reserved entries have been bound. Thus, the start and end points of temporary reserved resources can be determined by the state changes of two different resource objects, avoiding the creation of temporary reserved records in a timely manner or their premature deletion before the actual storage status of the node is updated. This ensures that resource verification at any time during the scheduling process can consider the actual occupied resources or temporary reserved resources, thereby reducing the risk of duplicate allocation of storage resources.

[0080] In one embodiment, a specific implementation process for performing resource statistics, resource verification, and target node selection according to disk type is described, which includes the following implementation steps: Step S501: Divide storage requests according to disk category to obtain the requested storage resource amount of the virtual machine to be scheduled under each disk category.

[0081] Disk categories can be used to distinguish different storage media or storage resources for different purposes. In one implementation, a disk category can include a local mechanical hard disk category (local_hdd) for the system disk and a local solid-state hard disk category (local_ssd) for the data disk.

[0082] In this embodiment, the scheduling controller reads each persistent storage volume declaration associated with the virtual machine to be scheduled, and determines the corresponding disk category and requested capacity from each persistent storage volume declaration; the requested capacities of the same disk category are aggregated to obtain the requested storage resource amount of the virtual machine to be scheduled under each disk category.

[0083] For example, a virtual machine to be scheduled includes a 40 GiB system disk and two data disks of 60 GiB and 100 GiB respectively. The system disk belongs to the local_hdd category, and the two data disks belong to the local_ssd category. Then, the requested storage resource amount for the local_hdd category is 40 GiB, and the requested storage resource amount for the local_ssd category is 160 GiB.

[0084] Step S502: For each candidate node, obtain the amount of free storage resources and temporary reserved resources for each disk category.

[0085] The amount of free storage resources under each disk category can be the capacity of one or more storage pools corresponding to that disk category that the candidate node has not yet been actually used. The amount of temporary reserved resources under each disk category can be the sum of the reserved capacity of virtual machines on the candidate node that have not yet completed storage binding under that disk category.

[0086] In this embodiment, the scheduling controller parses the storage status information of candidate nodes according to disk category, summarizes the amount of free storage resources of each storage pool under the same disk category, queries all the scheduled entries corresponding to the candidate nodes, and groups and sums the storage resources according to the disk category in the scheduled entries to obtain the amount of temporary reserved resources under each disk category.

[0087] For example, candidate node A has two local_ssd storage pools with free capacities of 120 GiB and 180 GiB respectively, so the free storage resource amount for the local_ssd category is 300 GiB; the node already has two local_ssd entries in the schedule, with 30 GiB and 50 GiB reserved respectively, so the temporary reserved resource amount for the local_ssd category is 80 GiB.

[0088] Step S503: For each candidate node, add the requested storage resource amount and the temporary reserved resource amount under the same disk category to obtain the corresponding storage resource amount to be allocated.

[0089] The amount of storage resources to be allocated under the same disk category can be the sum of the requested storage resources of the virtual machine to be scheduled under that disk category and the amount of temporary reserved resources that the candidate node already has under that disk category.

[0090] In this embodiment, the scheduling controller uses disk category as the correspondence to merge the current storage request and the storage resource amount in the existing scheduling entries of the candidate node into the same category; for each disk category, the requested storage resource amount is added to the temporary reserved resource amount to generate the category of storage resource amount to be allocated for that candidate node.

[0091] For example, if candidate node A has a local_hdd temporary reserved resource size of 20 GiB and a local_ssd temporary reserved resource size of 80 GiB, and the current requests are 40 GiB and 160 GiB in the two categories respectively, then the storage resources to be allocated for candidate node A in the two categories are 60 GiB and 240 GiB respectively.

[0092] Step S504: Determine the candidate nodes whose unallocated storage resource amount under each disk category is no greater than the corresponding free storage resource amount as pre-selected nodes.

[0093] The pre-selected node can be a candidate node that can meet the storage resource requirements of each disk category involved in the virtual machine to be scheduled. If the amount of storage resources to be allocated in any disk category is greater than the corresponding amount of free storage resources, the candidate node cannot be determined as a pre-selected node.

[0094] In this embodiment, the scheduling controller compares the amount of storage resources to be allocated and the corresponding amount of free storage resources for each disk category for a candidate node. If each comparison result shows that the amount of storage resources to be allocated is not greater than the amount of free storage resources, the candidate node is added to the pre-selected node set. If at least one disk category does not meet the requirements, the candidate node is excluded from the subsequent scoring range.

[0095] For example, if candidate node A has 100 GiB of free storage resources in the local_hdd and local_ssd categories and 300 GiB of free storage resources in the local_hdd and local_ssd categories, respectively, and the corresponding storage resources to be allocated are 60 GiB and 240 GiB, then candidate node A can be selected as a pre-selected node; candidate node B has 220 GiB of free local_ssd storage resources, which is less than the 240 GiB of storage resources to be allocated in this category, then candidate node B cannot be selected as a pre-selected node.

[0096] Step S505: Calculate the remaining resource ratio of each pre-selected node under each disk category, take the minimum value of each remaining resource ratio as the storage matching score of the pre-selected node, and determine the pre-selected node with the highest storage matching score as the target node.

[0097] The remaining resource ratio represents the proportion of storage resources that can still be reserved for a given disk category after the current allocation is completed and existing temporary reservations are taken into account, relative to the total free storage resources of that disk category. The storage matching score can be the minimum of the remaining resource ratios for each disk category within the same pre-selected node, reflecting the degree of remaining resources for the most resource-constrained disk category within that pre-selected node.

[0098] In this embodiment, for each pre-selected node and each disk category, the scheduling controller calculates the difference between the amount of free storage resources and the amount of storage resources to be allocated, and determines the ratio of this difference to the corresponding amount of free storage resources as the remaining resource ratio. The minimum remaining resource ratio is selected from multiple remaining resource ratios corresponding to the same pre-selected node as the storage matching score for that pre-selected node. The storage matching scores of multiple pre-selected nodes are compared, and the pre-selected node with the highest score is determined as the target node. If multiple pre-selected nodes have the same highest score, the target node can be determined according to the overall node load, a preset node order, or existing priority rules of the scheduler.

[0099] For example, if the remaining resource ratios of pre-selected node A in the local_hdd and local_ssd categories are 0.40 and 0.20 respectively, then node A's storage matching score is 0.20; if the remaining resource ratios of pre-selected node C in the two categories are 0.50 and 0.35 respectively, then node C's storage matching score is 0.35. Since node C has a higher storage matching score, node C can be selected as the target node.

[0100] As can be seen from the above, by separately counting requests, idle resources, and temporary reserved resources according to disk category, we can avoid the substitution of storage resources of different media or different purposes; by scoring the pre-selected nodes that meet the capacity conditions of each disk category, we can determine the target node with a more suitable resource structure from multiple pre-selected nodes.

[0101] In one embodiment, a specific implementation process for deduplicating the recording of temporary reserved resource quantities based on virtual machine identifiers is described, which includes the following implementation steps: Step S601: Generate the virtual machine identifier of the virtual machine to be scheduled based on the namespace and virtual machine name of the virtual machine to be scheduled.

[0102] The namespace can be a scope identifier used in a container orchestration platform for logical isolation of resources. The virtual machine name can be a name used to identify a virtual machine within the corresponding namespace. The virtual machine identifier can be a deduplicated identifier generated jointly by the namespace and the virtual machine name to identify the same virtual machine. In one implementation, the virtual machine identifier can be represented by a Virtual Machine Information Key (VmInfoKey).

[0103] In this embodiment, the reserved management component reads the namespace and virtual machine name of the virtual machine to be scheduled, and combines them into a virtual machine identifier according to a preset combination rule. For example, the virtual machine identifier can be generated in the form of "namespace / virtual machine name". Compared to the unique identifier of a single scheduling object, this virtual machine identifier can remain unchanged when the virtual machine scheduling is retried or when the image is imported into the container group and recreated.

[0104] For example, if virtual machine A belongs to the namespace tenant-a and the virtual machine name is vm-01, then the virtual machine identifier tenant-a / vm-01 can be generated.

[0105] Step S602: Query whether the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier.

[0106] One method to check whether a target node has been recorded is to search for a scheduling entry in the set of entries corresponding to the target node whose virtual machine identifier is the same as the virtual machine identifier of the virtual machine to be scheduled.

[0107] In this embodiment, the reservation management component first filters the scheduled entries according to the target node name, then traverses or indexes the filtered scheduled entries and compares the virtual machine identifier in each scheduled entry with the current virtual machine identifier. If the same virtual machine identifier exists, it is determined that the target node has recorded the corresponding temporary reserved resource amount; if the same virtual machine identifier does not exist, it is determined that the target node has not recorded the corresponding temporary reserved resource amount. Optionally, the combination of the target node name and the virtual machine identifier can be used as the index key of the memory mapping to directly locate the corresponding entry.

[0108] For example, if the virtual machine identifier tenant-a / vm-01 already exists in the scheduling entry set of node A, then when the same virtual machine's reservation event for node A is received again, it is determined that node A has recorded the corresponding temporary reserved resource amount.

[0109] Step S603: When the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier, replace the recorded amount of temporary reserved resources with the amount of storage resources corresponding to the storage request.

[0110] Replacement can involve deleting or overwriting the original scheduled entry corresponding to the same virtual machine identifier in the same target node, and then forming a new scheduled entry based on the current storage request, rather than continuing to accumulate the current storage resource amount to the original temporary reserved resource amount.

[0111] In this embodiment, the reserved management component locates the original schedule entry corresponding to the same target node and the same virtual machine identifier; removes the original schedule entry or clears the storage requests recorded therein; re-records the storage resource quantity of each disk type according to the latest storage request obtained in this scheduling, and synchronously updates the associated persistent storage volume declaration list and expiration time.

[0112] For example, when virtual machine A is first scheduled to node A, 100 GiB has already been reserved. Subsequently, due to scheduling retry, the same reservation event on node A is triggered again, and the latest storage request is still 100 GiB. If directly accumulated, it will incorrectly form a temporary reservation of 200 GiB; after using the replacement process, the temporary reserved resource amount corresponding to virtual machine A in node A is still 100 GiB.

[0113] Step S604: When the target node does not record the amount of temporary reserved resources corresponding to the virtual machine identifier, associate the amount of storage resources corresponding to the storage request with the virtual machine identifier and record it to obtain the amount of temporary reserved resources corresponding to the virtual machine to be scheduled.

[0114] The associated record can be a record that simultaneously stores the virtual machine identifier and the corresponding storage resource amount in the scheduling entry, enabling subsequent queries, replacements, and deletions based on the virtual machine identifier. The temporary reserved resource amount corresponding to the virtual machine to be scheduled can include the storage resource amount under one or more disk categories.

[0115] In this embodiment, the reserved management component creates a new scheduled entry; writes the target node name and virtual machine identifier into the scheduled entry; writes the storage resource amount corresponding to the storage request according to the disk type; writes the associated persistent storage volume declaration list and expiration time; and adds the new scheduled entry to the scheduled entry set corresponding to the target node.

[0116] For example, if there is no scheduled entry for the virtual machine identifier tenant-a / vm-01 in node A, the reserved management component creates a new record and associates the requested capacity of the virtual machine under the local_hdd and local_ssd categories with tenant-a / vm-01 respectively.

[0117] As can be seen from the above, by generating a stable virtual machine identifier using namespace and virtual machine name, and by replacing or adding temporary reserved resources for the same virtual machine within the same target node, duplicate reservations caused by virtual machine scheduling retries can be avoided.

[0118] In one embodiment, a specific implementation process for performing timeout cleanup based on the duration of temporary reserved resources is described, which includes the following implementation steps: Step S701: When the storage resource amount corresponding to the storage request is included in the temporary reserved resource amount of the target node, the corresponding generation time is recorded.

[0119] The generation time can be the time when an entry in the schedule is created or updated due to a schedule retry. The generation time determines how long an entry in the schedule has been continuously stored. The lifetime time limits the maximum duration an entry in the schedule can be continuously stored.

[0120] In this embodiment, the reservation management component reads the current system time when creating a scheduled entry and writes it as the generation time into the scheduled entry. It then calculates the expiration time based on the generation time and a preset lifespan, and writes the expiration time into the scheduled entry as well. When an existing scheduled entry in the same virtual machine is replaced by a new storage request, the generation time and expiration time can be updated synchronously.

[0121] For example, the preset liveness time can be set to 5 minutes, allowing normal storage creation and binding processes to complete within this timeframe, while setting automatic reclamation boundaries for abnormally residual entries in the schedule. The aforementioned 5-minute timeframe is merely an example and does not constitute a limitation.

[0122] Step S702: Determine the duration of the temporary reserved resources based on the current time and the generation time according to the preset inspection cycle.

[0123] The preset check cycle can be a time interval reserved for the management component to check whether entries in the schedule have expired. The duration can be the time from the creation time of the entry in the schedule to the current time.

[0124] In this embodiment, the reserved management component starts a background periodic check task; each time the periodic check task is triggered, it iterates through the currently saved entries in the schedule; it reads the generation time of each entry in the schedule, calculates the time difference between the current time and the generation time, and obtains the duration; or it directly compares the current time with the expiration time in the entry in the schedule.

[0125] For example, the preset check cycle can be set to 10 seconds, reserving the management component to check all scheduled entries every 10 seconds. A preset check cycle shorter than the preset lifespan ensures that expired entries are detected promptly after reaching the preset lifespan. The aforementioned 10 seconds is merely an example and does not constitute a limitation.

[0126] Step S703: When the duration of existence reaches the preset survival duration, delete the corresponding temporary reserved resources.

[0127] Among them, the survival time reaching the preset survival time indicates that the item in the schedule has not been deleted through the normal cleanup process outside of the reasonable storage supply and binding time. The item in the schedule may correspond to abnormal situations such as scheduling failure, abnormal storage supply, event loss or deletion of associated resources.

[0128] In this embodiment, the periodic check task compares the duration of each scheduled entry with the preset survival time; when the duration is less than the preset survival time, the corresponding scheduled entry is retained; when the duration reaches or exceeds the preset survival time, the entry is deleted according to the target node name and virtual machine identifier in the scheduled entry, and the temporary reserved resource amount of the target node is re-aggregated.

[0129] For example, if the entry in the schedule corresponding to virtual machine A is not deleted normally due to node status update and storage volume binding completion within 5 minutes after its generation, the background periodic check task will delete the entry in the schedule and release the corresponding temporary reserved resources.

[0130] Optionally, before deleting a timeout entry, you can also read the associated persistent storage volume declaration and virtual machine object again to confirm whether the corresponding scheduling process has been terminated or whether the entry has expired.

[0131] As can be seen from the above, by setting the generation time and preset lifespan of entries in the schedule, and performing expiration checks according to the preset check cycle, the amount of temporary reserved resources can be automatically reclaimed in the case of abnormal events that have not been executed during normal event-driven cleanup, thus avoiding abnormal residues from occupying the schedulable storage capacity of the node for a long time.

[0132] In summary, this application embodiment records the amount of temporarily reserved resources before the actual binding of storage resources is completed, and verifies the current storage request together with the existing amount of temporarily reserved resources, so that the scheduling controller can perceive the resource occupation during asynchronous binding; by judging capacity and scoring nodes according to disk type, and combining event-triggered reservation, virtual machine identifier deduplication, normal cleanup and timeout cleanup, lifecycle management of temporarily reserved resources is realized, thereby reducing the risk of over-allocation of storage resources and storage binding failure in virtual machine concurrent scheduling.

[0133] As an example, such as Figure 2 The diagram shown is a flowchart of another storage resource scheduling method provided in this application embodiment. It is applied to the virtual machine storage resource scheduling scenario based on the container orchestration platform Kubernetes and the extension framework KubeVirt that runs virtual machines on Kubernetes, and is used to solve the problem of resource overselling caused by asynchronous binding of storage resources during virtual machine scheduling.

[0134] Figure 2The `Persistent Volume Claim` is used to represent a virtual machine's request for persistent storage resources. `selected-node` indicates the target node chosen by the persistent volume claim. `Annotation` is a key-value description appended to a resource object. `Annotated Added` indicates that an annotation has been added to the persistent volume claim to indicate the target node. `PVC Watcher` is a persistent volume claim listener used to monitor changes to the persistent volume claim and its node selection information. `Watcher` is a listener used to monitor the creation, update, or deletion of resource objects. `virt-importer Pod` is a virtual machine image import runtime instance used to import virtual machine image data into a persistent storage volume. `Pod` is the smallest unit of workload execution in a container orchestration platform, which can contain one or more containers. `VM` is a Virtual Machine. `Inflight` is a temporary reserved state in the schedule, indicating that the storage resource has been selected for use by a virtual machine, but storage binding has not yet been completed and included in the node's actual occupancy. Inflight Entry: A temporary reservation entry in the schedule, used to record information such as the virtual machine, target node, storage request, associated persistent storage volume declaration, and expiration time. InflightManager: The temporary reservation manager in the schedule, used to add, query, and delete temporary reservation entries. Add: The add operation, used to write a temporary reservation entry in the schedule to the temporary reservation manager. nodeName: The node name, used to identify the target node corresponding to the temporary reservation entry in the schedule. podUID: Pod Unique Identifier, a unique identifier for the running instance, used to uniquely identify the virtual machine image imported into the running instance. vmKey: The virtual machine identifier key, used to distinguish different virtual machines and to deduplicate temporary reservation entries corresponding to the same virtual machine. TTL: Time To Live, used to limit the validity period of the temporary reservation entry in the schedule. Predicate: The filtering phase, used to determine whether the storage resources of candidate nodes meet the storage requests of the virtual machine. Prioritize: The prioritization phase, used to score or rank the candidate nodes that pass the filtering. PodRequest: The storage request for the running instance, representing the storage resources requested by the virtual machine currently to be scheduled. InflightRequest: A temporary reserved request in the schedule, representing a storage resource request that has been recorded on the target node but has not yet been bound to storage. CombinedRequest: A combined storage request, obtained by merging the storage request of the currently scheduled virtual machine with the existing temporary reserved requests in the schedule on the target node.Bound: Bound status, indicating that the persistent storage volume declaration has been bound to the corresponding persistent storage volume. Node: A node is a cluster device that provides compute, memory, and storage resources for virtual machines or containers. Node Annotation Updated: The node annotation has been updated, indicating that the actual storage resource usage information recorded by the node has been updated. Node Watcher: A node watcher used to monitor changes in node storage status information or node annotations. RemoveByPodUID: Performs deletion based on the unique identifier of the running instance, used to delete the temporary reserved entry in the schedule corresponding to the running instance imported from the specified virtual machine image.

[0135] In this implementation, when a PVC selected-node Annotation Added event occurs (i.e., a new selected-node annotation is added to the PVC), the PVC Watcher detects the selected-node and locates the corresponding virt-importer Pod based on the PVC. After finding the corresponding virt-importer Pod, it extracts the storage request and PVC list from the virt-importer Pod, and extracts the VM name. The VM name is used to generate the virtual machine identifier required for VM deduplication, to avoid generating multiple duplicate Inflight entries for the same VM during repeated scheduling.

[0136] Then, InflightManager.Add(nodeName, podUID, vmKey, ...) is called to add an Inflight entry to the Inflight manager. Here, nodeName represents the target node name, podUID represents the unique identifier of the virt-importerPod, vmKey represents the deduplicated identifier of the virtual machine generated based on the VM name, and the ellipsis indicates that information such as storage requests, PVC lists, and expiration times can also be included.

[0137] When adding an Inflight entry, a TTL is configured for that entry, for example, the default TTL is 5 minutes. The Inflight entry records information such as the target node, virt-importer Pod, VM, storage requests, and associated PVCs to represent temporary storage resource usage that has not yet been bound. In the subsequent Predicate / Prioritize phase, the total Inflight count of the current candidate nodes is obtained. Specifically, the storage requests corresponding to each existing Inflight entry on the candidate node are summarized to obtain an InflightRequest. The PodRequest corresponding to the currently scheduled Pod is added to the InflightRequest to obtain a CombinedRequest, i.e.: CombinedRequest=PodRequest+InflightRequest.

[0138] The CombinedRequest is used for verification and scoring. In the Predicate phase, the CombinedRequest is used to determine whether the idle storage resources of the candidate nodes meet the storage resource requirements of the current Pod to be scheduled and the existing Inflight entries. In the Prioritize phase, the candidate nodes that meet the resource requirements are scored, and the target node is determined based on the scoring results.

[0139] After a PVC is bound and the Node Annotation is updated (i.e., the PVC is bound and the stored state information in the node annotation is updated), the Node Watcher detects the annotation change. Based on the node with the annotation change, the Node Watcher checks the corresponding Inflight entry for that node and obtains the list of PVCs associated with each Inflight entry. For each Inflight entry, it verifies whether all PVCs are bound.

[0140] After verifying that all PVCs are bound, InflightManager.RemoveByPodUID(podUID) is called to delete the corresponding Inflight entry according to the podUID, thus completing the Inflight entry cleanup. By cleaning up Inflight entries only after the node storage state information has been updated and all PVCs are bound, premature release of Inflight resources can be avoided.

[0141] In addition, the process also includes TTL protection. TTL protection checks Inflight entries according to a preset check cycle, such as once every 10 seconds. When the duration of an Inflight entry reaches the default TTL, such as 5 minutes, expired entries are automatically cleaned up to prevent Inflight entry residue from occurring in case of abnormalities such as PVC binding failure, loss of listening events, or interruption of the scheduling process.

[0142] As described above, by creating Inflight entries through PVC Watcher, using CombinedRequest for verification and scoring during the Predicate / Prioritize phase, and having Node Watcher clean up Inflight entries after PVC Bound and Node Annotation Updated, while automatically cleaning up expired entries through TTL protection, the scheduler can be aware of temporary resource occupation during the storage binding process, avoid over-allocation of storage resources, and ensure that Inflight entries can be reclaimed in a timely and accurate manner.

[0143] Figure 3 This is a schematic diagram of a storage resource scheduling device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the device specifically includes: The acquisition module 31 is used to acquire the storage request of the virtual machine to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node, wherein the amount of temporary reserved resources represents the amount of storage resources requested by the virtual machine that has been allocated to the candidate node but has not yet completed storage binding. The first processing module 32 is used to add the amount of storage resources corresponding to the storage request to the amount of temporary reserved resources of the candidate node for each candidate node to obtain the amount of storage resources to be allocated, and to determine the target node from the candidate nodes whose free storage resources meet the amount of storage resources to be allocated. The second processing module 33 is used to mark the persistent storage volume declaration of the virtual machine to be scheduled as the target node, and to include the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node.

[0144] In one possible implementation, the second processing module is further configured to create a persistent storage volume on the target node according to the storage request, and bind the persistent storage volume declaration to the persistent storage volume; After the persistent storage volume declaration is completed and bound, the storage status information of the target node is updated according to the storage resources occupied by the persistent storage volume.

[0145] In one possible implementation, the second processing module is specifically configured to listen for node selection information of the persistent storage volume declaration through a persistent storage volume declaration event listener; When a target node tag for indicating the target node is added to the node selection information, the virtual machine to be scheduled associated with the persistent storage volume declaration and the storage request of the virtual machine to be scheduled are obtained. Based on the target node marker, the amount of storage resources corresponding to the storage request is included in the temporary reserved resource amount of the target node.

[0146] In one possible implementation, the second processing module is further configured to listen to the storage state information of the target node through a node event listener; When an update to the storage status information is detected, the binding status of each persistent storage volume declaration associated with the virtual machine to be scheduled is obtained; Once each persistent storage volume declaration has been bound, the temporary reserved resource amount corresponding to the virtual machine to be scheduled is deleted.

[0147] In one possible implementation, the acquisition module is specifically used to divide the storage requests according to disk category and obtain the requested storage resource amount of the virtual machine to be scheduled under each disk category; For each candidate node, obtain the amount of free storage resources and the amount of temporary reserved resources under each disk category; The step of determining the target node from candidate nodes whose free storage resources meet the required storage resource allocation includes: For each candidate node, the requested storage resource amount under the same disk category is added to the temporary reserved resource amount to obtain the corresponding storage resource amount to be allocated; Candidate nodes whose unallocated storage resource amount under each disk category is no greater than the corresponding free storage resource amount are determined as pre-selected nodes; Calculate the remaining resource ratio of each of the pre-selected nodes under each of the disk categories, take the minimum value of each remaining resource ratio as the storage matching score of the pre-selected node, and determine the pre-selected node with the highest storage matching score as the target node.

[0148] In one possible implementation, the second processing module is further configured to generate a virtual machine identifier for the virtual machine to be scheduled based on the namespace to which the virtual machine to be scheduled belongs and the virtual machine name. Query whether the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier; When the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier, the amount of storage resources corresponding to the storage request is used to replace the recorded amount of temporary reserved resources. When the target node does not record the amount of temporary reserved resources corresponding to the virtual machine identifier, the amount of storage resources corresponding to the storage request is associated with the virtual machine identifier and recorded to obtain the amount of temporary reserved resources corresponding to the virtual machine to be scheduled.

[0149] In one possible implementation, the second processing module is further configured to record the corresponding generation time when the amount of storage resources corresponding to the storage request is included in the temporary reserved resource amount of the target node; According to the preset inspection cycle, the duration of the temporary reserved resource quantity is determined based on the current time and the generation time; When the duration reaches the preset survival time, the corresponding temporary reserved resources are deleted.

[0150] The storage resource scheduling device provided in this embodiment can be as follows: Figure 3 The apparatus shown can perform, for example Figure 1 All steps of the method, thus achieving Figure 1 For details on the technical effects of the method shown, please refer to [link / reference]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0151] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 4 The computer device 400 shown includes at least one processor 401, a memory 402, at least one network interface 404, and other user interfaces 403. The various components in the computer device 400 are coupled together via a bus system 405. It is understood that the bus system 405 is used to implement communication between these components. In addition to a data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general designated all buses as Bus System 405.

[0152] The user interface 403 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0153] It is understood that the memory 402 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 402 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0154] In some implementations, memory 402 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 4021 and application program 4022.

[0155] The operating system 4021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 4022 includes various applications, such as a media player and a browser, used to implement various application functions. Programs implementing the methods of this application embodiment can be included in application program 4022.

[0156] In this embodiment, by calling the program or instructions stored in memory 402, specifically the program or instructions stored in application program 4022, processor 401 executes the method steps provided in each method embodiment, including, for example: The storage requests of virtual machines to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node are obtained. The amount of temporary reserved resources represents the amount of storage resources requested by virtual machines that have been allocated to candidate nodes but have not yet completed storage binding. For each candidate node, the amount of storage resources corresponding to the storage request is added to the amount of temporary reserved resources of the candidate node to obtain the amount of storage resources to be allocated, and the target node is determined from the candidate nodes whose free storage resources meet the amount of storage resources to be allocated. The persistent storage volume of the virtual machine to be scheduled is declared as the target node, and the amount of storage resources corresponding to the storage request is included in the temporary reserved resources of the target node.

[0157] The methods disclosed in the embodiments of this application can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 402. Processor 401 reads the information in memory 402 and, in conjunction with its hardware, completes the steps of the above method.

[0158] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0159] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0160] The computer device provided in this embodiment may be as follows: Figure 4 The device shown can perform, for example Figure 1 All steps of the method, thus achieving Figure 1 For details on the technical effects of the method shown, please refer to [link / reference]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0161] This application also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0162] When one or more programs in the storage medium can be executed by one or more processors, the storage resource scheduling method described above, which is executed on the device side, can be implemented.

[0163] The processor is used to execute a storage resource scheduler stored in memory to implement the following steps of a storage resource scheduling method executed on the device side: The storage requests of virtual machines to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node are obtained. The amount of temporary reserved resources represents the amount of storage resources requested by virtual machines that have been allocated to candidate nodes but have not yet completed storage binding. For each candidate node, the amount of storage resources corresponding to the storage request is added to the amount of temporary reserved resources of the candidate node to obtain the amount of storage resources to be allocated, and the target node is determined from the candidate nodes whose free storage resources meet the amount of storage resources to be allocated. The persistent storage volume of the virtual machine to be scheduled is declared as the target node, and the amount of storage resources corresponding to the storage request is included in the temporary reserved resources of the target node.

[0164] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0166] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A storage resource scheduling method, characterized in that, include: The storage requests of virtual machines to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node are obtained. The amount of temporary reserved resources represents the amount of storage resources requested by virtual machines that have been allocated to candidate nodes but have not yet completed storage binding. For each candidate node, the amount of storage resources corresponding to the storage request is added to the amount of temporary reserved resources of the candidate node to obtain the amount of storage resources to be allocated, and the target node is determined from the candidate nodes whose free storage resources meet the amount of storage resources to be allocated. The persistent storage volume of the virtual machine to be scheduled is declared as the target node, and the amount of storage resources corresponding to the storage request is included in the temporary reserved resources of the target node.

2. The method according to claim 1, characterized in that, After marking the persistent storage volume declaration as the target node, the method further includes: Based on the storage request, a persistent storage volume is created on the target node, and the persistent storage volume declaration is bound to the persistent storage volume; After the persistent storage volume declaration is completed and bound, the storage status information of the target node is updated according to the storage resources occupied by the persistent storage volume.

3. The method according to claim 1, characterized in that, The step of including the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node includes: Listen for node selection information of the persistent storage volume declaration through a persistent storage volume declaration event listener; When a target node tag for indicating the target node is added to the node selection information, the virtual machine to be scheduled associated with the persistent storage volume declaration and the storage request of the virtual machine to be scheduled are obtained. Based on the target node marker, the amount of storage resources corresponding to the storage request is included in the temporary reserved resource amount of the target node.

4. The method according to claim 1, characterized in that, The method further includes: The storage status information of the target node is monitored through a node event listener; When an update to the storage status information is detected, the binding status of each persistent storage volume declaration associated with the virtual machine to be scheduled is obtained; Once each persistent storage volume declaration has been bound, the temporary reserved resources corresponding to the virtual machine to be scheduled are deleted.

5. The method according to claim 1, characterized in that, Obtain the amount of free storage resources for each candidate node and the amount of temporary reserved resources already available for each candidate node, including: The storage requests are divided according to disk category to obtain the requested storage resource amount of the virtual machine to be scheduled under each disk category; For each candidate node, obtain the amount of free storage resources and the amount of temporary reserved resources under each disk category; The step of determining the target node from candidate nodes whose free storage resources meet the required storage resource allocation includes: For each candidate node, the requested storage resource amount under the same disk category is added to the temporary reserved resource amount to obtain the corresponding storage resource amount to be allocated; Candidate nodes whose unallocated storage resource amount under each disk category is no greater than the corresponding free storage resource amount are determined as pre-selected nodes; Calculate the remaining resource ratio of each of the pre-selected nodes under each of the disk categories, take the minimum value of each remaining resource ratio as the storage matching score of the pre-selected node, and determine the pre-selected node with the highest storage matching score as the target node.

6. The method according to claim 3, characterized in that, The step of including the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node includes: Generate a virtual machine identifier for the virtual machine to be scheduled based on the namespace and virtual machine name to which the virtual machine to be scheduled belongs; Query whether the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier; When the target node has recorded the amount of temporary reserved resources corresponding to the virtual machine identifier, the amount of storage resources corresponding to the storage request is used to replace the recorded amount of temporary reserved resources. When the target node does not record the amount of temporary reserved resources corresponding to the virtual machine identifier, the amount of storage resources corresponding to the storage request is associated with the virtual machine identifier and recorded to obtain the amount of temporary reserved resources corresponding to the virtual machine to be scheduled.

7. The method according to claim 1, characterized in that, The method further includes: When the storage resource amount corresponding to the storage request is included in the temporary reserved resource amount of the target node, the corresponding generation time is recorded; According to the preset inspection cycle, the duration of the temporary reserved resource quantity is determined based on the current time and the generation time; When the duration reaches the preset survival time, the corresponding temporary reserved resources are deleted.

8. A storage resource scheduling device, characterized in that, include: The acquisition module is used to acquire the storage request of the virtual machine to be scheduled, the amount of free storage resources of each candidate node, and the amount of temporary reserved resources of each candidate node. The amount of temporary reserved resources represents the amount of storage resources requested by the virtual machine that has been allocated to the candidate node but has not yet completed storage binding. The first processing module is used to add the storage resource amount corresponding to the storage request to the temporary reserved resource amount of the candidate node for each candidate node to obtain the storage resource amount to be allocated, and to determine the target node from the candidate nodes whose free storage resource amount meets the storage resource amount to be allocated; The second processing module is used to mark the persistent storage volume declaration of the virtual machine to be scheduled as the target node, and to include the storage resource amount corresponding to the storage request in the temporary reserved resource amount of the target node.

9. A computer device, characterized in that, include: A processor and a memory, the processor being configured to execute a storage resource scheduling program stored in the memory to implement the storage resource scheduling method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the storage resource scheduling method according to any one of claims 1 to 7.

Images