GPU computing power resource scheduling method and device based on load awareness and medium

Abstract

The invention discloses a GPU computing power resource scheduling method and device based on load awareness and a medium, and relates to the technical field of computing power scheduling. The method comprises the steps that according to the video memory capacity and the number of calculation cores of the GPU, the video memory capacity is divided into a plurality of continuous fragments, the calculation cores are divided into a plurality of logic calculation groups, and sub-resource units are obtained; recording occupation states and load indexes of the sub-resource units in real time to obtain a resource pool; analyzing a job submitted by a user, and collecting an execution period of a kernel function, a video memory access mode and an instruction pipeline blocking rate when the job runs; and on the basis of a reinforcement learning algorithm, predicting a resource demand inflection point of the job according to the load fingerprint model, performing hierarchical scheduling on the sub-resource units based on virtualization isolation and multi-dimensional resource quantitative evaluation, and generating a preemptive allocation strategy of the sub-resource units. According to the method, efficient utilization and secure sharing of GPU resources are realized through collaborative design of dynamic granularity segmentation, load aware scheduling and hardware-level security isolation.

Description

Technical Field

[0001] The present application relates to the field of computing power scheduling technology, and in particular to a method, device, and medium for scheduling GPU computing power resources based on load perception. Background Art

[0002] With the rapid development of applications such as artificial intelligence and big data analysis, GPUs (Graphics Processing Units) are becoming increasingly important as a key hardware resource for high-performance computing in various cloud platforms and data centers. However, existing GPU resource scheduling methods generally suffer from the following problems:

[0003] There is a serious waste of resources. GPUs are typically allocated at the granularity of the entire card, resulting in most resources being idle when used by small jobs. Sharing and isolation are insufficient. The traditional sharing method, MPS (Multi-Process Service), cannot completely isolate jobs from different tenants, resulting in performance interference and security risks. It also lacks dynamic scheduling capabilities. Current resource scheduling is typically based on static demand configuration and cannot be flexibly adjusted based on actual job load changes. Heterogeneous environments are poorly compatible. In clusters with different GPU models, resource scheduling lacks a unified standard, resulting in low utilization.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows:

[0005] The existing GPU resource scheduling technology suffers from resource waste, insufficient shared isolation, lack of dynamic adjustment capabilities, and poor compatibility with heterogeneous environments. Summary of the Invention

[0006] The embodiments of the present application provide a load-aware GPU computing power resource scheduling method, device, and medium, which can solve the problems of resource waste, insufficient shared isolation, lack of dynamic adjustment capabilities, and poor compatibility with heterogeneous environments in GPU resource scheduling in the prior art.

[0007] In the first aspect, an embodiment of the present application provides a load-aware GPU computing power resource scheduling method, the method comprising: dividing the GPU memory capacity into multiple continuous segments and the computing cores into multiple logical computing groups according to the GPU's memory capacity and the number of computing cores, to obtain sub-resource units; registering the sub-resource units to the scheduling system, and recording the occupancy status and load indicators of the sub-resource units in real time to obtain a resource pool; parsing the jobs submitted by the user, and collecting the execution cycle of the kernel function, memory access mode and instruction pipeline blocking rate of the job at runtime; based on the reinforcement learning algorithm, predicting the resource demand inflection point of the job according to the load fingerprint model, and based on virtualization isolation and multi-dimensional resource quantitative evaluation, hierarchically scheduling the sub-resource units to generate a preemptive allocation strategy for the sub-resource units.

[0008] In one implementation of the present application, the job submitted by the user is parsed, and the execution cycle of the kernel function, the memory access pattern and the instruction pipeline blocking rate of the job during runtime are collected, specifically including: inserting probe points in the call chain of the kernel function to capture monitoring data, the monitoring data including the instruction-level parallelism and branch prediction failure rate of the computing core; based on the monitoring data, analyzing the spatiotemporal locality characteristics of the memory access and the probability distribution diagram of the memory access to obtain the memory access pattern; based on the memory access pattern, classifying the blocking events into compute-limited, storage-limited and synchronous wait types, and generating load fingerprints for each type.

[0009] In one implementation of the present application, based on a reinforcement learning algorithm, the resource demand inflection point of a job is predicted according to the load fingerprint, specifically including: taking resource utilization as a reward function, calculating the priority mapping relationship between historical load fingerprints and resource utilization through a Q-learning model; when it is detected that the resource utilization reaches a threshold, allocating sub-resource units according to the priority mapping relationship; introducing a resource allocation regret value mechanism, and rolling back to the historical optimal strategy when the prediction deviation exceeds the threshold.

[0010] In one implementation of the present application, based on virtualization isolation and multi-dimensional resource quantitative evaluation, sub-resource units are hierarchically scheduled to generate a preemptive allocation strategy for sub-resource units, specifically including: for the same GPU, through virtualization technology and address space isolation mechanism, independent contexts are allocated to tenant jobs sharing the same GPU; for heterogeneous GPU clusters, a normalized score of the heterogeneous GPU cluster is calculated, and the normalized score comprehensively calculates the number of cores, memory bandwidth and floating-point computing power; based on the normalized score and cold migration cost, the allocation priority of the sub-resource unit is generated.

[0011] In one implementation of the present application, for a heterogeneous GPU cluster, a normalized score of the heterogeneous GPU cluster is calculated, specifically including: constructing a topology graph for cross-node scheduling, wherein the edge weights of the topology graph include network latency, remaining PCIe bandwidth, and migration failure probability; using an ant colony optimization algorithm to search for a Pareto optimal path that meets SLA constraints in the topology graph; reserving a backup path for jobs that are longer than a preset period, and triggering migration when a node on the optimal path fails.

[0012] In one implementation of the present application, after registering the sub-resource unit to the scheduling system and recording the occupancy status and load indicators of the sub-resource unit in real time to obtain the resource pool, the method also includes: scanning the sub-resource units in the resource pool according to a preset period to scan whether there are fragmented units; matching jobs shorter than the preset period with the fragmented units to generate a filling scheduling strategy.

[0013] In one implementation of the present application, jobs shorter than a preset period are matched with fragmentation units to generate a filling scheduling strategy, which specifically includes: recording the physical distribution of the fragmentation units and calculating the continuity gain after the reorganization; freezing the access to the resources to be migrated, and atomically completing the resource block reorganization, injecting a virtual resource view into the low-priority jobs to be moved, and shielding the underlying physical resource changes; after the reorganization is completed, updating the resource pool and triggering the address space remapping of the current job.

[0014] In one implementation of the present application, the video memory capacity is divided into multiple continuous segments and the computing cores are divided into multiple logical computing groups according to the video memory capacity and the number of computing cores of the GPU to obtain sub-resource units, which specifically include: dividing the video memory segments according to the load pressure of the GPU; and dividing adjacent computing cores into the same logical computing group based on the physical location of the computing cores.

[0015] In the second aspect, an embodiment of the present application also provides a load-aware GPU computing resource scheduling device, the device including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: divide the video memory capacity into multiple continuous segments and divide the computing cores into multiple logical computing groups according to the video memory capacity and the number of computing cores of the GPU to obtain sub-resource units; register the sub-resource units to the scheduling system, and record the occupancy status and load indicators of the sub-resource units in real time to obtain a resource pool; parse the jobs submitted by the user, and collect the execution cycle of the kernel function, video memory access mode and instruction pipeline blocking rate of the job at runtime; based on the reinforcement learning algorithm, predict the resource demand inflection point of the job according to the load fingerprint model, and based on virtualization isolation and multi-dimensional resource quantitative evaluation, hierarchically schedule the sub-resource units to generate a preemptive allocation strategy for the sub-resource units.

[0016] On the third aspect, the embodiment of the present application also provides a non-volatile computer storage medium for GPU computing power resource scheduling based on load perception, which stores computer executable instructions, and the computer executable instructions are set to: divide the video memory capacity into multiple continuous segments and divide the computing cores into multiple logical computing groups according to the video memory capacity and the number of computing cores of the GPU to obtain sub-resource units; register the sub-resource units to the scheduling system, and record the occupancy status and load indicators of the sub-resource units in real time to obtain a resource pool; parse the jobs submitted by the user, and collect the execution cycle of the kernel function, video memory access mode and instruction pipeline blocking rate of the job at runtime; based on the reinforcement learning algorithm, predict the resource demand inflection point of the job according to the load fingerprint model, and based on virtualization isolation and multi-dimensional resource quantitative evaluation, hierarchically schedule the sub-resource units to generate a preemptive allocation strategy for the sub-resource units.

[0017] The embodiments of the present application provide a method, device and medium for GPU computing resource scheduling based on load perception, which realizes efficient utilization and safe sharing of GPU resources through the collaborative design of dynamic granularity segmentation, load-aware scheduling and hardware-level security isolation. Based on the decoupling division and elastic scaling mechanism of video memory and computing core, physical GPU resources are converted into dynamically combinable sub-units, so that small tasks can accurately match resource requirements, while supporting on-demand expansion of large tasks, and reducing waste of video memory fragmentation; through multi-dimensional load fingerprint modeling and reinforcement learning-driven preemptive allocation strategy, the inflection point of job resource demand is predicted in real time, resource supply is adjusted dynamically, and the overall utilization rate of the cluster is improved; an active reorganization mechanism for fragmented resources is designed, and through atomic migration and logical continuity optimization, the resource degradation problem caused by long-term operation is significantly reduced. In addition, the introduction of multi-version resource configuration templates and online hot-swappable interfaces further enhances the system's elastic adaptability in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A flowchart of a load-aware GPU computing resource scheduling method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the internal structure of a load-aware GPU computing resource scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] The embodiments of the present application provide a load-aware GPU computing power resource scheduling method, device, and medium, which solve the problems of resource waste, insufficient shared isolation, lack of dynamic adjustment capabilities, and poor compatibility with heterogeneous environments in GPU resource scheduling in the prior art.

[0023] The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1This is a flow chart of a GPU computing resource scheduling method based on load perception provided by an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a load-aware GPU computing resource scheduling method, which specifically includes the following steps:

[0025] Step 10: Based on the GPU memory capacity and the number of computing cores, the memory capacity is divided into multiple continuous segments, and the computing cores are divided into multiple logical computing groups to obtain sub-resource units;

[0026] In this step, the video memory is divided into multiple continuous logical segments based on the current GPU's total video memory capacity and the preset partitioning strategy. During the partitioning process, it is necessary to ensure that the starting address of each segment is aligned to avoid performance loss caused by cross-page access. Based on the physical topology of the GPU chip, adjacent streaming multiprocessors are divided into the same logical computing group to minimize cross-group communication delays.

[0027] If it is detected that the overall GPU load pressure is high, the memory segment granularity will be automatically reduced to improve resource utilization; otherwise, the granularity will be increased to reduce management overhead.

[0028] Step 20: Register the sub-resource unit to the scheduling system and record the occupancy status and load indicators of the sub-resource unit in real time to obtain the resource pool;

[0029] In this step, a unique identifier is assigned to each sub-resource unit, and hardware characteristics are recorded: memory bandwidth, SM architecture version, and power consumption limit. The occupancy status of each sub-unit is collected in real time through the GPU driver layer interface to determine whether it is allocated, memory read and write rates, and SM instruction throughput.

[0030] Step 30: Analyze the job submitted by the user and collect the execution cycle of the kernel function, the memory access mode, and the instruction pipeline blocking rate during the job runtime;

[0031] As an optional embodiment, the job submitted by the user is parsed to collect the execution cycle of the kernel function, the memory access mode, and the instruction pipeline blocking rate during the job runtime. Specifically, the following information may be collected:

[0032] Step 301: inserting probe points into the call chain of the kernel function to capture monitoring data, including the instruction-level parallelism and branch prediction failure rate of the computing core;

[0033] In this step, lightweight monitoring code is inserted into the GPU kernel function or CPU key code path to collect the hardware operation status in real time. The probe must meet the requirements of low intrusion and high accuracy. The ability of the computing unit to execute multiple instructions in the same clock cycle reflects the utilization of hardware resources; the proportion of processors predicting branch directions incorrectly. A high failure rate will lead to pipeline flushing and performance degradation.

[0034] Step 302: Analyze the spatiotemporal locality characteristics of video memory access and the probability distribution diagram of video memory access based on the monitoring data to obtain the video memory access pattern;

[0035] In this step, we count the number of times the same memory address is repeatedly accessed within a short period of time, calculate the reuse distance or reuse rate, analyze the continuity of accesses to adjacent addresses, calculate the stride size and the proportion of consecutive accesses, and use a histogram to calculate the access frequency of memory addresses and identify hot and cold data areas. In other words, based on the memory access log, we calculate temporal locality (the frequency of repeated accesses to the same data) and spatial locality (the probability of consecutive accesses to data at adjacent addresses) to generate a memory access probability distribution map.

[0036] Step 303: Classify the blocking events into computation-limited, storage-limited, and synchronization-wait types according to the video memory access mode, and generate load fingerprints for each type.

[0037] In this step, compute-limited pipeline blocking is caused by insufficient ALU / FPU resources or data dependencies, resulting in high instruction execution counts, low memory access frequency, and a low to medium percentage of blocked cycles. Memory-limited blocking is caused by memory access latency or memory controller contention, resulting in high memory access volume, low instruction execution efficiency, and pipeline stalls waiting for data. Synchronous wait blocking is caused by thread synchronization or cross-device communication. Synchronization points are dense, and kernel execution cycles exhibit significant segmented stalls. Features are then represented using multidimensional vectors, or compact fingerprint codes are generated using machine learning model clustering algorithms.

[0038] Step 40: Based on the reinforcement learning algorithm, the resource demand inflection point of the job is predicted according to the load fingerprint model. Based on virtualization isolation and multi-dimensional resource quantitative evaluation, the sub-resource units are hierarchically scheduled and a preemptive allocation strategy for the sub-resource units is generated.

[0039] As an optional embodiment, based on a reinforcement learning algorithm, the resource demand inflection point of a job is predicted according to the load fingerprint, which may specifically include:

[0040] Step 401: Using resource utilization as a reward function, a Q-learning model is used to calculate a priority mapping relationship between historical load fingerprints and resource utilization.

[0041] In this step, the state space is collected, including the current resource pool status, job load fingerprint, and historical utilization trends; the action space includes expanding / contracting sub-resource units and migrating jobs across nodes. Then, with the overall resource utilization of the cluster as the core indicator, taking into account the energy efficiency ratio, characteristics and priority mapping, the system can convert the multi-dimensional load fingerprint into a specific priority value or label.

[0042] Step 402: when it is detected that the resource utilization reaches a threshold, sub-resource units are allocated according to the priority mapping relationship;

[0043] In this step, when a high-priority job requests resources, if the resource pool is insufficient, the subunit of the low-priority task is preempted based on the Q-value prediction, and the regret value, which is the difference between the actual benefit and the predicted benefit, is recorded.

[0044] Step 403: Introduce a resource allocation regret value mechanism, and roll back to the historical optimal strategy when the prediction deviation exceeds a threshold.

[0045] In this step, if the regret value exceeds the threshold, roll back to the historical optimal strategy.

[0046] As an optional embodiment, based on virtualization isolation and multi-dimensional resource quantitative evaluation, sub-resource units are hierarchically scheduled to generate a preemptive allocation strategy for the sub-resource units.

[0047] As an optional embodiment, based on virtualization isolation and multi-dimensional resource quantitative evaluation, hierarchical scheduling of sub-resource units is performed to generate a preemptive allocation strategy for the sub-resource units, which may specifically include:

[0048] Step 404: For the same GPU, using virtualization technology and address space isolation mechanism, allocate independent contexts for tenant jobs sharing the same GPU;

[0049] In this step, each tenant is allocated a separate GPU context container, isolating the SM group and memory segments through the virtualization layer. Optionally, memory encryption implements dynamic address remapping in the GPU MMU and uses the AES encryption algorithm to encrypt tenant memory data in real time. The key is bound to the hardware PUF to prevent unauthorized access.

[0050] Step 405: Calculate a normalized score of the heterogeneous GPU cluster, where the normalized score comprehensively considers the number of cores, memory bandwidth, and floating-point computing capability.

[0051] Step 406: Generate allocation priorities of sub-resource units based on the normalized scores and the cold migration costs.

[0052] As an optional embodiment, for a heterogeneous GPU cluster, calculating a normalized score of the heterogeneous GPU cluster may specifically include:

[0053] Step 4051: Construct a topology graph for cross-node scheduling, where the edge weights of the topology graph include network latency, remaining PCIe bandwidth, and migration failure probability;

[0054] Step 4052: Use the ant colony optimization algorithm to search for a Pareto optimal path that meets the SLA constraints in the topology graph;

[0055] In this step, the SLA constraint, the service level agreement, means that the ant can only choose a route that meets this time limit. There is no route that can shorten the time and reduce the cost at the same time. It can only find an optimal compromise between the two.

[0056] Step 4053: Reserve a backup path for jobs that are longer than a preset period, and trigger migration when a node on the optimal path fails.

[0057] As an optional embodiment, after registering the sub-resource unit to the scheduling system and recording the occupancy status and load indicators of the sub-resource unit in real time to obtain the resource pool, the method may also include: scanning the sub-resource units in the resource pool according to a preset period to scan whether there are fragmented units; matching jobs shorter than the preset period with the fragmented units to generate a filling scheduling strategy.

[0058] In this step, the resource pool is scanned periodically to identify non-contiguously occupied memory segments or idle computing groups, and mark them as fragment units.

[0059] As an optional embodiment, jobs shorter than a preset period are matched with fragmentation units to generate a filling scheduling strategy, which may specifically include: recording the physical distribution of fragmentation units and calculating the continuity gain after reorganization; freezing access to resources to be migrated, and atomically completing resource block reorganization, injecting virtual resource views into the low-priority jobs to be moved, and shielding the underlying physical resource changes; after the reorganization is completed, updating the resource pool and triggering the address space remapping of the current job.

[0060] In this step, through defragmentation, small free blocks are pieced together into large continuous blocks to meet the needs of larger tasks; freezing and atomic operations ensure that running tasks are not affected during resource migration, so that users only need to focus on logical resources and do not need to worry about changes in the underlying physical location. In other words, by "deceiving" users into thinking that the resource location has not changed, program crashes are avoided. Finally, the underlying records are updated to let the system know that "the actual resource location has changed."

[0061] As an optional embodiment, based on the GPU's video memory capacity and the number of computing cores, the video memory capacity is divided into multiple continuous segments, and the computing cores are divided into multiple logical computing groups to obtain sub-resource units. Specifically, this may include: dividing the video memory segments according to the GPU's load pressure; and dividing adjacent computing cores into the same logical computing group based on the physical location of the computing cores.

[0062] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, this application embodiment also provides a GPU computing resource scheduling device based on load perception, whose structure is as follows: Figure 2 shown.

[0063] Figure 2 This is a schematic diagram of the internal structure of a GPU computing resource scheduling device based on load perception provided in an embodiment of the present application. Figure 2 As shown, the equipment includes:

[0064] at least one processor 201;

[0065] and, a memory 202 communicatively coupled to the at least one processor;

[0066] Among them, the memory 202 stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor 201 so that the at least one processor 201 can: divide the video memory capacity into multiple continuous segments and divide the computing cores into multiple logical computing groups according to the video memory capacity and the number of computing cores of the GPU to obtain sub-resource units; register the sub-resource units with the scheduling system, and record the occupancy status and load indicators of the sub-resource units in real time to obtain a resource pool; parse the jobs submitted by the user, and collect the execution cycle of the kernel function, video memory access mode and instruction pipeline blocking rate of the job during runtime; based on the reinforcement learning algorithm, predict the resource demand inflection point of the job according to the load fingerprint model, and based on virtualization isolation and multi-dimensional resource quantitative evaluation, hierarchically schedule the sub-resource units to generate a preemptive allocation strategy for the sub-resource units.

[0067] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for load-aware GPU computing resource scheduling stores computer executable instructions, which are configured to: divide the GPU memory capacity into multiple continuous segments and the computing cores into multiple logical computing groups according to the GPU's memory capacity and the number of computing cores, thereby obtaining sub-resource units; register the sub-resource units with the scheduling system, and record the occupancy status and load indicators of the sub-resource units in real time to obtain a resource pool; parse user-submitted jobs, and collect the execution cycle of kernel functions, memory access patterns, and instruction pipeline blocking rates of the jobs during runtime; predict the resource demand inflection point of the job based on a reinforcement learning algorithm and a load fingerprint model; and schedule the sub-resource units in layers based on virtualization isolation and multi-dimensional resource quantitative evaluation, thereby generating a preemptive allocation strategy for the sub-resource units.

[0068] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the IoT device and media embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0069] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0070] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0071] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0074] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0075] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0076] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0077] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0078] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A GPU computing resource scheduling method based on load perception, characterized in that: The method comprises: According to the memory capacity and the number of computing cores of the GPU, the memory capacity is divided into a plurality of continuous segments, and the computing cores are divided into a plurality of logical computing groups to obtain sub-resource units; Registering the sub-resource unit to the scheduling system, and recording the occupancy status and load index of the sub-resource unit in real time to obtain a resource pool; Parse the job submitted by the user, collect the execution cycle of the kernel function, video memory access mode and instruction pipeline blocking rate of the job during runtime, and generate a load fingerprint; Based on a reinforcement learning algorithm, the resource demand inflection point of the job is predicted according to the load fingerprint model, and based on virtualization isolation and multi-dimensional resource quantitative evaluation, the sub-resource units are hierarchically scheduled to generate a preemptive allocation strategy for the sub-resource units.

2. A GPU computing resource scheduling method based on load perception according to claim 1, characterized in that: The job submitted by the user is parsed, and the execution cycle of the kernel function, the memory access mode, and the instruction pipeline blocking rate of the job are collected during runtime, specifically including: Inserting probe points in the call chain of the kernel function to capture monitoring data, wherein the monitoring data includes the instruction-level parallelism and branch prediction failure rate of the computing core; Analyzing the spatiotemporal locality characteristics of the video memory access and the probability distribution graph of the video memory access according to the monitoring data to obtain a video memory access pattern; According to the video memory access mode, the blocking events are classified into a computation-limited type, a storage-limited type, and a synchronization-wait type, and load fingerprints are generated for each type.

3. The method for scheduling GPU computing power resources based on load perception according to claim 2, characterized in that: The prediction of the resource demand inflection point of the job according to the load fingerprint based on the reinforcement learning algorithm specifically includes: Using resource utilization as the reward function, the Q-learning model is used to calculate the priority mapping relationship between historical load fingerprints and resource utilization; When detecting that the resource utilization reaches a threshold, allocating the sub-resource unit according to the priority mapping relationship; A resource allocation regret value mechanism is introduced to roll back to the historical optimal strategy when the prediction deviation exceeds the threshold.

4. The method for scheduling GPU computing resources based on load perception according to claim 1, characterized in that: Based on virtualization isolation and multi-dimensional resource quantitative evaluation, the sub-resource units are hierarchically scheduled to generate a preemptive allocation strategy for the sub-resource units, specifically including: For the same GPU, using virtualization technology and address space isolation mechanism, independent contexts are allocated to tenant jobs sharing the same GPU; For a heterogeneous GPU cluster, calculating a normalized score of the heterogeneous GPU cluster, the normalized score comprehensively considering the number of computing cores, video memory bandwidth, and floating-point computing capability; An allocation strategy for the sub-resource unit is generated based on the normalized score and the cold migration cost.

5. The method for scheduling GPU computing resources based on load perception according to claim 4, characterized in that: Calculating the normalized score of the heterogeneous GPU cluster specifically includes: Constructing a topology graph for cross-node scheduling, wherein edge weights of the topology graph include network latency, remaining PCIe bandwidth, and migration failure probability; An ant colony optimization algorithm is used to search for a Pareto optimal path that meets the SLA constraints in the topology graph; A backup path is reserved for jobs that are longer than a preset period, and migration is triggered when a node on the optimal path fails.

6. The method for scheduling GPU computing resources based on load perception according to claim 1, characterized in that: After registering the sub-resource unit with the scheduling system and recording the occupancy status and load index of the sub-resource unit in real time to obtain a resource pool, the method further includes: Scan the sub-resource units in the resource pool according to a preset period to check whether there are fragmented units; The jobs shorter than a preset period are matched with the fragment units to generate a filling scheduling strategy.

7. The method for scheduling GPU computing resources based on load perception according to claim 6, characterized in that: The matching of jobs shorter than a preset period with fragment units to generate a filling scheduling strategy specifically includes: Recording the physical distribution of the fragmented units and calculating the continuity gain after reorganization; Freeze access to resources to be migrated and atomically reorganize resource blocks, injecting virtual resource views into the relocated low-priority jobs while shielding changes to underlying physical resources. After the reorganization is completed, the resource pool is updated and the address space remapping of the current job is triggered.

8. The method for GPU computing resource scheduling based on load perception according to claim 1, characterized in that: According to the GPU memory capacity and the number of computing cores, the memory capacity is divided into multiple continuous segments, and the computing cores are divided into multiple logical computing groups to obtain sub-resource units, which specifically includes: Dividing the video memory segments according to the load pressure of the GPU; Based on the physical locations of the computing cores, adjacent computing cores are divided into the same logical computing group.

9. A GPU computing resource scheduling device based on load perception, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: According to the memory capacity and the number of computing cores of the GPU, the memory capacity is divided into a plurality of continuous segments, and the computing cores are divided into a plurality of logical computing groups to obtain sub-resource units; Registering the sub-resource unit to the scheduling system, and recording the occupancy status and load index of the sub-resource unit in real time to obtain a resource pool; Parse the job submitted by the user and collect the execution cycle of the kernel function, video memory access mode and instruction pipeline blocking rate of the job during runtime; Based on a reinforcement learning algorithm, the resource demand inflection point of the job is predicted according to the load fingerprint model, and based on virtualization isolation and multi-dimensional resource quantitative evaluation, the sub-resource units are hierarchically scheduled to generate a preemptive allocation strategy for the sub-resource units.

10. A non-volatile computer storage medium for load-aware GPU computing resource scheduling, storing computer-executable instructions, characterized in that: The computer executable instructions are configured to: According to the memory capacity and the number of computing cores of the GPU, the memory capacity is divided into a plurality of continuous segments, and the computing cores are divided into a plurality of logical computing groups to obtain sub-resource units; Registering the sub-resource unit to the scheduling system, and recording the occupancy status and load index of the sub-resource unit in real time to obtain a resource pool; Parse the job submitted by the user and collect the execution cycle of the kernel function, video memory access mode and instruction pipeline blocking rate of the job during runtime; Based on a reinforcement learning algorithm, the resource demand inflection point of the job is predicted according to the load fingerprint model, and based on virtualization isolation and multi-dimensional resource quantitative evaluation, the sub-resource units are hierarchically scheduled to generate a preemptive allocation strategy for the sub-resource units.

Images