Heterogeneous GPU (Graphics Processing Unit) pooling straight-through scheduling method, device, equipment and medium
By constructing a unified resource pool and utilizing pass-through virtualization drivers and resource grid scheduling algorithms, the heterogeneous compatibility and dynamic response issues of heterogeneous GPU devices were resolved, achieving efficient resource management and business continuity.
Patent Information
- Application Number
- CN202511112551.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-09
- Publication Date
- 2025-11-18
AI Technical Summary
In existing technologies, in heterogeneous computing environments, it is difficult for x86/ARM hybrid architecture servers and GPU devices of different generations to achieve fine-grained collaborative scheduling, resulting in low resource utilization, difficulty in balancing virtualization performance and resource isolation, and a lack of dynamic response mechanisms, which leads to impaired business continuity.
A unified resource pool is constructed, including multi-architecture servers with x86/ARM architecture and various types of GPU devices. Virtual GPU resource units are generated using a pass-through virtualization driver model. User requests are responded to through a resource grid pooling scheduling algorithm, target resources are dynamically matched, and resources are reconstructed and replaced under change events.
It achieves unified management of hardware resources across architectures, improves resource utilization, maintains high performance and elastic scaling, ensures business continuity, and solves the problems of heterogeneous compatibility and dynamic response.
Smart Images
Figure CN120973535A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of computer system architecture and resource scheduling, and particularly relates to a heterogeneous GPU pooling pass-through scheduling method, device, equipment and medium. BACKGROUND
[0002] With the development of heterogeneous computing and cloud native technology, there is a large-scale GPU resource pooling scheduling demand. This technology needs to realize unified management and low-latency resource supply across architecture servers and multiple GPU devices. In traditional technology, two resource scheduling methods are usually used: a GPU resource segmentation scheme based on full virtualization technology, which realizes resource reuse by deploying a complete virtualization layer on a physical GPU, but introduces significant performance loss; and a pass-through mode using physical device static binding, which guarantees computing performance but leads to low resource utilization and cannot dynamically adapt to business needs. The current scheduling method has the following key defects: insufficient heterogeneous compatibility, x86 / ARM mixed architecture servers and different generation GPU devices are difficult to implement fine-grained collaborative scheduling in a unified resource pool; virtualization performance and resource isolation are difficult to balance, the full virtualization model has more than 20% performance decay due to instruction conversion, and the physical pass-through mode lacks flexibility and scalability; low resource matching accuracy, traditional scheduling algorithms only rely on basic computing power indicators and do not integrate SLA levels, physical topology locations and architecture characteristics and other multi-dimensional constraints; and lack of dynamic response mechanism, when the physical device state changes or the load exceeds the threshold, there is a lack of real-time reconstruction of resource units and fast migration mechanism of failed resources, resulting in damage to business continuity. SUMMARY
[0003] Therefore, it is necessary to provide a heterogeneous GPU pooling pass-through scheduling method, device, equipment and medium which can solve the above problems.
[0004] In a first aspect, the present application provides a heterogeneous GPU pooling pass-through scheduling method, comprising:
[0005] constructing a unified resource pool containing multi-architecture servers of x86 / ARM architecture and multiple types of GPU devices;
[0006] virtualizing the unified resource pool by using a pass-through virtualization driver model to generate virtual GPU resource units;
[0007] allocating target resources from the virtual GPU resource units through a resource grid pooling scheduling algorithm in response to user resource requests;
[0008] mounting the target resources to the user computing environment.
[0009] In one embodiment, constructing a unified resource pool containing multi-architecture servers of x86 / ARM architecture and multiple types of GPU devices comprises:
[0010] Obtaining hardware parameters of a multi-architecture server of x86 / ARM architecture and a plurality of types of GPU devices, and generating a resource directory;
[0011] According to a preset GPU resource division strategy, the hardware resources in the resource directory are isolated according to computing capabilities to form a plurality of schedulable resource units;
[0012] According to a preset business scenario rule, SLA level labels are marked for the resource units to form a unified resource pool, wherein the SLA level labels contain level information that is numerical or quantifiable.
[0013] In one embodiment, the method further comprises:
[0014] Monitoring a change event of the unified resource pool, wherein the change event includes a physical state change of the multi-architecture server or the plurality of types of GPU devices or a real-time load exceeding a preset threshold;
[0015] In response to monitoring the change event, the following steps are performed:
[0016] Re-obtaining hardware parameters of a multi-architecture server of x86 / ARM architecture and a plurality of types of GPU devices, and updating the resource directory;
[0017] Based on the updated resource directory, according to a preset GPU resource division strategy, the schedulable resource units are re-generated and the original resource units are replaced;
[0018] According to a preset business scenario rule, SLA level labels are marked for the replaced resource units to form a new unified resource pool.
[0019] In one embodiment, the unified resource pool is virtualized by using a pass-through virtualization drive model to generate virtual GPU resource units, including:
[0020] Based on the computing capability isolation attribute and the SLA level label of the resource unit, corresponding pass-through virtualization drive parameters are configured;
[0021] Direct hardware access mapping of the physical GPU device is performed through the pass-through virtualization drive model to generate a virtual GPU resource unit with an independent device address space; wherein the virtual GPU resource unit inherits the computing architecture characteristics and the SLA level label of the corresponding physical GPU device, and a virtual GPU instruction set simulation layer is configured to achieve compatibility.
[0022] In one embodiment, in response to a user resource request, a target resource is allocated from the virtual GPU resource units through a resource grid pooling scheduling algorithm, including:
[0023] Receiving a user resource request, parsing the request parameters to obtain the target computing architecture type, SLA level requirement and computing power specification requirement;
[0024] Based on the target computing architecture type, SLA level requirement and computing power specification requirement, traversing the virtual GPU resource unit and performing the following matching steps:
[0025] Verify whether the architecture compatibility identifier is consistent with the target computing architecture type;
[0026] Detect the independent device address space isolation state;
[0027] Calculate the numerical ratio of its SLA level label and SLA level requirement;
[0028] Obtain the physical topology position of its corresponding physical GPU device;
[0029] Based on the matching step result, the virtual GPU resource unit is scored and scheduled, and the virtual GPU resource unit with the highest scheduling score is selected as the target resource.
[0030] In one embodiment, based on the matching step result, the virtual GPU resource unit is scored and scheduled, and the following formula is used:
[0031]
[0032] Wherein, v j is the candidate virtual GPU resource unit, R is the user resource request parameter set, f1 is the architecture compatibility evaluation function, when the resource unit architecture identifier arch V matches the request architecture arch R , output 1, otherwise output 0, f2 is the address space isolation state indication function, output 1 when the resource unit is in idle isolation state, otherwise output 0, f3 is the SLA level matching function, calculating the ratio of resource unit labeled SLA level SLA V and request level SLA R , and f3≤1, f4 is the physical topology distance attenuation function, based on the physical location distance d(·) between the resource unit and the user computing environment to calculate the network transmission cost, w k is the preset dynamic weight coefficient of each dimension, which is dynamically adjusted according to the real-time load state according to the preset strategy and meets the normalization condition, S(·) is the scheduling score of the virtual GPU resource unit.
[0033] In one embodiment, the method further comprises:
[0034] When the change event causes the allocated target resource to be invalid, based on the new unified resource pool, using the pass-through virtualization driver model to perform virtualization processing to form a new virtual GPU resource unit;
[0035] allocating a target resource from the virtual GPU resource units by a resource grid pool scheduling algorithm;
[0036] unmounting the unallocated target resource from the user computing environment and mounting the replacement resource to the user computing environment.
[0037] In a second aspect, the present application further provides a heterogeneous GPU pool pass-through scheduling apparatus, comprising:
[0038] a resource pool construction module configured to construct a unified resource pool comprising multi-architecture servers of x86 / ARM architecture and multiple types of GPU devices;
[0039] a virtualization driving module configured to perform virtualization processing on the unified resource pool by using a pass-through virtualization driving model to generate virtual GPU resource units;
[0040] a dynamic scheduling module configured to allocate a target resource from the virtual GPU resource units by a resource grid pool scheduling algorithm in response to a user resource request;
[0041] a resource mounting execution module configured to mount the target resource to a user computing environment.
[0042] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned heterogeneous GPU pool pass-through scheduling method when executing the computer program.
[0043] In a fourth aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above-mentioned heterogeneous GPU pool pass-through scheduling method.
[0044] The above-mentioned heterogeneous GPU pool pass-through scheduling method, apparatus, computer device and storage medium construct a unified resource pool comprising multi-architecture servers of x86 / ARM architecture and multiple types of GPU devices, realize unified management of cross-architecture hardware resources, and solve the heterogeneous compatibility problem that x86 / ARM hybrid architecture and different generations of GPU devices are difficult to be cooperatively scheduled. The unified resource pool is processed by using a pass-through virtualization driving model to generate virtual GPU resource units, direct hardware access mapping is used to avoid performance degradation caused by traditional full virtualization, while ensuring resource isolation, and high performance and elastic scalability are taken into account. A target resource is allocated in response to a user resource request by a resource grid pool scheduling algorithm, the algorithm integrates multi-dimensional constraints to improve resource matching accuracy. The target resource is mounted to a user computing environment to realize low-latency resource supply and dynamic response capability, and to eliminate the problem of low resource utilization caused by traditional static binding. BRIEF DESCRIPTION OF DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiment or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0046] Figure 1 A flow chart of the heterogeneous GPU pooling pass-through scheduling method of the present application;
[0047] Figure 2 A structure diagram of the heterogeneous GPU pooling pass-through scheduling device of the present application. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.
[0049] In one embodiment, as shown in Figure 1 A heterogeneous GPU pooling pass-through scheduling method is provided, and the implementation environment is based on a multi-architecture server cluster containing x86 / ARM architecture (such as domestic Feiteng / Kunpeng server and Intel / AMD server mixed deployment) and multiple types of GPU devices (including domestic Pangu architecture GPU, Nvidia T4 / A100 series, AMD V series, etc. Professional computing card), through hyper-converged infrastructure to organize computing, storage and network resources, and build a unified resource pool supporting PB-level storage capacity. It can be understood that this method can also be applied to terminals, and can also be applied to systems including terminals and server clusters, and is realized through the interaction of terminals and server clusters. Its application scenarios cover AI model training, real-time rendering and high-security computing power scheduling needs, and the specific performance is as follows: when the terminal user initiates a resource request containing target architecture type (such as ARM), SLA level (such as military-grade BM30) and computing power specification (such as FP3220TFLOPS), the server cluster maps the physical GPU resources to virtual GPU units with independent address space (inherits the physical device computing architecture characteristics) through the pass-through virtualization driver model, and dynamically matches and allocates resources by the resource grid pooling scheduling algorithm; automatically perform physical topology distance optimization calculation, and mount the target resource to the user computing environment (such as a secure isolated AI training container). In this embodiment, the method includes the following steps:
[0050] S01, a unified resource pool containing multi-architecture servers of x86 / ARM and multiple types of GPU devices is constructed.
[0051] Among them, the unified resource pool (a set of schedulable resources formed by logically integrating physical heterogeneous computing resources through a software abstraction layer) can be constructed by collecting hardware parameters (such as the number of computing cores, the capacity of video memory, and the PCIe bandwidth) of multi-architecture servers (including Feiteng, Kunpeng, and other domestic servers covering x86 / ARM architecture, and Intel / AMD servers) and multi-element GPU devices (including domestic Pangu architecture GPU, Nvidia T4 / A100 series, AMD V series, etc.) in real time through a bottom-layer monitoring agent, and generating a structured resource directory. Based on a preset GPU resource division strategy, the physical hardware is divided into multiple independently schedulable resource units, and according to preset business scenario rules, each resource unit is labeled with a numerical SLA level tag to form a unified resource pool with quantitative scheduling attributes. Heterogeneous resources can be dynamically organized through elastic Hash algorithm, so that domestic and non-domestic GPUs can be unified and managed under the architecture of one cloud and multiple cores, laying a physical resource foundation for subsequent virtualization scheduling.
[0052] S02, using a pass-through virtualization drive model to virtualize the unified resource pool to generate a virtual GPU resource unit.
[0053] Among them, the pass-through virtualization drive model (uses hardware-assisted virtualization technology to realize a direct hardware access mapping mechanism for physical GPU resources). When implemented, the memory mapping strategy and interrupt redirection parameters of the pass-through virtualization drive can be configured based on the computing capability isolation attributes and SLA level tags of the resource units in the unified resource pool; the memory space, PCIe channel, and computing core of the physical GPU device are directly mapped to generate a virtual GPU resource unit with an independent device address space through the model, which inherits the computing architecture characteristics and SLA tags of the physical device, and at the same time configures a virtual GPU instruction set simulation layer to solve the cross-architecture compatibility problem. At the same time, through a deep optimization of the device address space isolation mechanism, more than 95% of the native computing performance is preserved under the premise of ensuring a <100μs delay, and the hardware encoding and decoding capabilities (H.265 / HEVC, AV1, etc.) and API interfaces (OpenGL 4.6 / Vulkan 1.3) of domestic GPUs are fully transparently supported, providing a virtualization resource carrier with near-physical-level performance for heterogeneous computing power.
[0054] S03, in response to a user resource request, allocating target resources from the virtual GPU resource unit through a resource grid pooling scheduling algorithm.
[0055] Wherein, when the user resource request (such as AI training task needs ARM architecture, SLA level BM30 and 20TFLOPS computing power) triggers, the dynamic scheduling module parses the request parameters and traverses the virtual GPU resource unit, executes four-dimensional matching verification, verifies the architecture compatibility identifier of the candidate unit and the consistency of the request type, and outputs the Boolean quantity f1; secondly, detect its independent device address space isolation state (idle state output f2=1), ensure the resource exclusivity; calculate the SLA level matching degree Wherein, SlA V is the SLA level of the resource unit, SlA R is the request level (such as military level BM30 unit for civilian level request output f3=0.8); at the same time, the physical topology position is obtained and the network transmission cost (based on the physical position distance d of the super-converged architecture), according to the dynamic weight coefficient w k (adjust according to the real-time load according to the preset strategy) quantitatively score the candidate unit through the scoring formula, and select the highest score unit as the target resource. This algorithm can rely on the elastic Hash capability of the centerless symmetrical architecture to realize millisecond-level response in the PB-level resource pool, and at the same time, through the GPU pooling scheduling and dynamic roaming technology, support the online migration of domestic and non-domestic heterogeneous GPUs across storage resources (such as SAN / super-converged storage), ensure that the target resource allocation meets the requirements of architecture adaptability, computing power accuracy and physical topology optimization.
[0056] S04, mount the target resource to the user computing environment.
[0057] Wherein, mounting (through the pass-through virtualization driver model, the allocated target resource, that is, the virtual GPU resource unit is dynamically connected to the user computing environment, such as a secure isolated AI training container, a virtual machine or a cloud host, to realize the physical binding of resources and computing tasks), when implemented, based on the independent device address space attribute (inherited from the physical GPU memory and PCIe channel mapping) and the SLA level label (such as military level BM30) of the target resource, the compatibility interface of the virtual device address and the user computing environment (for example, through the instruction set simulation layer to convert CUDA to OpenCL instructions), execute direct hardware access mapping to establish a low-latency (<100μs) data transmission channel, realize heterogeneous GPU pooling pass-through scheduling.
[0058] The heterogeneous GPU pooling pass-through scheduling method constructs a unified resource pool containing x86 / ARM architecture multi-architecture servers and various types of GPU devices, uses a pass-through virtualization driver model to virtualize the resource pool, generates a virtual GPU resource unit that inherits the computing architecture characteristics and SLA tags of the physical device, and responds to user resource requests by allocating the optimal target resource through a resource grid pooling scheduling algorithm and mounting it to the user computing environment. The unified resource pool solves the heterogeneous compatibility problem of x86 / ARM hybrid architecture and different generations of GPU devices; the pass-through virtualization model takes into account performance and resource isolation, eliminating the 20% performance degradation of full virtualization and the lack of flexibility of static binding; the resource grid scheduling algorithm integrates multi-dimensional constraints such as architecture type, SLA level, and physical topology to improve matching accuracy; the mounting mechanism supports cross-storage online migration in combination with the hyper-converged architecture, realizes dynamic response and efficient resource reuse, and achieves unified management, high-performance supply, and business continuity protection of heterogeneous GPU resources.
[0059] In one embodiment, a unified resource pool containing x86 / ARM architecture multi-architecture servers and various types of GPU devices is constructed, including:
[0060] S11, obtaining the hardware parameters of the x86 / ARM architecture multi-architecture servers and various types of GPU devices, and generating a resource directory;
[0061] S12, according to the preset GPU resource division strategy, the hardware resources in the resource directory are isolated according to the computing power to form a plurality of schedulable resource units;
[0062] S13, according to the preset business scenario rule, the resource unit is labeled with an SLA level tag to form a unified resource pool, wherein the SLA level tag contains numerical or quantifiable comparison level information.
[0063] Specifically, the hardware parameters (including the number of computing cores, the memory capacity, the PCIe bandwidth, and the encoding and decoding capabilities) of the multi-architecture servers and the multi-type GPU devices can be collected in real time by the underlying monitoring agent to generate a structured resource directory; based on a preset GPU resource division strategy (such as the computing capability isolation principle), the physical hardware resources are isolated into multiple independently schedulable resource units according to the computing power granularity (for example, a single A100 GPU is divided into 10 20TFLOPS computing power units); according to preset business scenario rules (such as military BM30 or civilian standard), each resource unit is labeled with a numerical SLA level label (delay-sensitive tasks are labeled with SLA = 9.5, and batch computing tasks are labeled with SLA = 7.0), forming a unified resource pool with quantitative scheduling attributes. In implementation, relying on the centerless symmetrical design of the hyper-converged infrastructure, the heterogeneous resources can be dynamically organized through the elastic Hash algorithm, and the 2+1 replica mechanism is used to ensure data high availability, so as to realize the unified management of the domestic and non-domestic heterogeneous GPU resources under the one-cloud multi-core architecture.
[0064] In one of the embodiments, the method further comprises:
[0065] S21, monitoring a change event of the unified resource pool, wherein the change event includes a physical state change of the multi-architecture servers or the multi-type GPU devices or a real-time load exceeding a preset threshold;
[0066] S22, in response to monitoring the change event, performing the following steps:
[0067] S22.1, reacquiring the hardware parameters of the multi-architecture servers of x86 / ARM architecture and the multi-type GPU devices, and updating the resource directory;
[0068] S22.2, based on the updated resource directory, re-generating the schedulable resource units according to the preset GPU resource division strategy, and replacing the original resource units;
[0069] S23, labeling the replaced resource units with SLA level labels according to the preset business scenario rules, and forming a new unified resource pool.
[0070] Exemplarily, a change event (an event of a physical state anomaly of a multi-architecture server or a GPU device (such as device failure, power-off) or real-time load exceeding a preset threshold (such as GPU utilization > 90%)); when implemented, the resource pool state can be monitored in real time by a bottom monitoring agent, when a change event is triggered, the heterogeneous hardware parameters are immediately re-collected, and the structured resource directory is updated; based on the updated directory, the schedulable resource units are regenerated according to a preset GPU resource division strategy (according to the principle of computing power isolation) (for example, the computing power of a failed GPU is dynamically migrated to an idle device and divided into 20TFLOPS units), replacing the original units; according to the preset business scenario rules, the replaced units are labeled with numerical SLA tags (such as a high-availability unit is labeled with SLA = 9.8), forming a new unified resource pool, ensuring that the business is not interrupted when the resource pool is dynamically reconstructed, and through the real-time resource unit replacement and SLA tag relabeling mechanism, the problem of business continuity being damaged due to the change of the physical device state is solved.
[0071] In one of the embodiments, the unified resource pool is processed by using a pass-through virtualization drive model to generate a virtual GPU resource unit, including:
[0072] S31, based on the computing power isolation attribute and the SLA level label of the resource unit, configuring corresponding pass-through virtualization drive parameters;
[0073] S32, performing direct hardware access mapping of the physical GPU device through the pass-through virtualization drive model to generate a virtual GPU resource unit with an independent device address space; wherein the virtual GPU resource unit inherits the computing architecture characteristics and the SLA level label of the corresponding physical GPU device, and configures the instruction set simulation layer of the virtual GPU to realize compatibility.
[0074] Specifically, the pass-through virtualization driver model (which implements a direct hardware access mapping mechanism for physical GPU resources using hardware-assisted virtualization technology (such as IOMMU / SRIOV)), when implemented, can dynamically configure the memory mapping strategy and interrupt redirection parameters of the pass-through virtualization driver based on the computing power isolation attributes of the resource unit (such as a single resource unit of 20 TFLOPS computing power) and the SLA level label (such as military-grade BM30). Through this model, direct hardware access mapping of physical GPU devices is performed to map the physical GPU's video memory space (such as a maximum of 16GB GDDR6), PCIe channel (such as 4.0x16), and computing core (such as 2048) to generate a virtual GPU resource unit with an independent device address space. This unit inherits the computing architecture characteristics (such as the ARM architecture instruction set) and SLA level label of the corresponding physical device, while configuring a virtual GPU instruction set simulation layer (such as a real-time instruction conversion engine from CUDA to OpenCL) to solve cross-architecture compatibility problems. By deeply optimizing the device address space isolation mechanism, more than 95% of the original computing performance can be preserved under the premise of ensuring a delay of <100μs, providing a virtualization resource carrier with near-physical-level performance and elastic scalability for heterogeneous computing power.
[0075] In one of the embodiments, in response to a user resource request, a target resource is allocated from the virtual GPU resource unit through a resource grid pooling scheduling algorithm, including:
[0076] S41, receiving a user resource request, parsing the request parameters to obtain the target computing architecture type, SLA level requirement, and computing power specification requirement;
[0077] S42, based on the target computing architecture type, SLA level requirement, and computing power specification requirement, traversing the virtual GPU resource unit and performing the following matching steps:
[0078] S42.1, verifying whether the architecture compatibility identifier is consistent with the target computing architecture type;
[0079] S42.2, detecting the independent device address space isolation state;
[0080] S42.3, calculating the numerical ratio of the SLA level label and the SLA level requirement;
[0081] S42.4, obtaining the physical topology position of the corresponding physical GPU device;
[0082] S43, based on the matching step result, scheduling and scoring the virtual GPU resource unit, and selecting the virtual GPU resource unit with the highest scheduling score as the target resource.
[0083] Exemplarily, the resource grid pooling scheduling algorithm realizes the dynamic matching and allocation mechanism of virtual GPU resources through a multi-dimensional constraint optimization model. When a user resource request (such as an AI training task requiring an ARM architecture, an SLA level BM30, and a 20TFLOPS computing power) is triggered, the dynamic scheduling module receives the request and parses the parameters to obtain the target computing architecture type, the SLA level requirement, and the computing power specification requirement. Based on the requirement, the virtual GPU resource unit (such as a unit inheriting the characteristics of a domestic Pangu GPU or a Nvidia A100 architecture) is traversed, and a four-dimensional matching step is performed to verify the consistency of the candidate unit architecture compatibility identifier (such as the arch identifier) and the request type (output Boolean quantity f1), detect the independent device address space isolation state (idle state output f2 = 1), calculate the numerical ratio of the SLA level label and the request level wherein, SLA V is the SLA level of the resource unit, SLA R is the request level (such as a military level BM30 unit outputting f3 = 0.8 for a civilian level request), and the physical topology position of the corresponding physical GPU device is obtained (based on the super-converged architecture computing network transmission cost wherein d is the physical distance from the user computing environment); based on the matching result, the candidate unit is quantitatively scored, and the highest score unit is selected as the target resource. Through the fusion of architecture compatibility, resource isolation, SLA matching degree, and physical topology, the problem of low resource matching accuracy of traditional scheduling algorithms is solved, and more than 95% resource utilization and business continuity are achieved.
[0084] In one embodiment, S51, the virtual GPU resource unit is scored based on the matching step result, using the following formula:
[0085]
[0086] wherein, v j is the candidate virtual GPU resource unit, R is the user resource request parameter set, f1 is the architecture compatibility evaluation function, which outputs 1 when the resource unit architecture identifier arch V matches the request architecture arch R , and outputs 0 otherwise, f2 is the address space isolation state indication function, which outputs 1 when the resource unit is in an idle isolation state, and outputs 0 otherwise, f3 is the SLA level matching function, which calculates the ratio of the resource unit labeled SLA level SLA V to the request level SLA R , and f3 ≤ 1, f4 is the physical topology distance attenuation function, which calculates the network transmission cost based on the physical location distance d(·) of the resource unit and the user computing environment, w kThe dynamic weight coefficients for each dimension are preset and dynamically adjusted according to the real-time load status and the normalization condition is met. S(·) is the scheduling score of the virtual GPU resource unit.
[0087] Specifically, the scheduling scoring formula refers to a mechanism that uses a multi-dimensional constrained optimization model to quantitatively evaluate virtual GPU resource units. Its specific implementation is the formula... Where v j Let R be the candidate virtual GPU resource unit, R be the user resource request parameter set (including the target computing architecture type, SLA level requirements, and computing power specifications), and f1 be the architecture compatibility evaluation function (when the resource unit architecture identifier arch...). V With request architecture arch R f2 is the address space isolation status indicator function, which outputs 1 when the resource unit is in idle isolation state and 0 otherwise. f3 is the SLA level matching function (calculates the SLA level of the resource unit). V With Request Level SLA R ratio And f3≤1), f4 is the physical topological distance decay function. (The network transmission cost is calculated based on the physical distance d(·) between the resource unit and the user's computing environment, such as measuring the physical topology distance d through a hyperconverged architecture and outputting the cost value), w k The system pre-sets dynamic weight coefficients for each dimension (which are dynamically adjusted according to the real-time load status and meet the normalization conditions). During implementation, the system can use this formula to score candidate units based on the results of the matching steps (including architecture compatibility verification, address space isolation status detection, SLA level ratio calculation, and physical topology location acquisition), and select the unit with the highest score as the target resource. This integrates constraints from dimensions such as architecture compatibility, resource isolation, SLA matching degree, and physical topology to solve the problem of low resource matching accuracy in traditional scheduling algorithms.
[0088] In one embodiment, the method further includes:
[0089] S61, When a change event causes the allocated target resources to become invalid, virtualization processing is performed based on the new unified resource pool using the pass-through virtualization driver model to form a new virtual GPU resource unit.
[0090] S62 allocates alternative resources from new virtual GPU resource units through a resource grid pooling scheduling algorithm;
[0091] S63, unmount the failed allocated target resource from the user computing environment and mount the replacement resource to the user computing environment.
[0092] Exemplarily, when a change event (such as GPU device failure or load exceeding threshold) causes the allocated target resource to be invalid, the business continuity can be ensured by dynamic resource reconstruction and mounting switching. When the new unified resource pool (generated by S23 step) is used to perform direct hardware access mapping on the reconstructed physical resource by using the pass-through virtualization drive model (through IOMMU / SRIOV technology), a new virtual GPU resource unit is formed, which inherits the computing architecture characteristics and SLA label. Through the resource grid pooling scheduling algorithm (using a multi-dimensional scoring formula The replacement resource is dynamically matched from the new resource unit (preferably selected from the units that are compatible with the architecture, idle, isolated, and have high SLA matching degree). The invalid target resource and the user computing environment (such as an AI training container) are unmounted, and the remaining information protection mechanism (through memory zeroing and disk erasing to prevent residual data leakage) is activated. The replacement resource is mounted to the original user environment, a data transmission channel is established, and the business interruption caused by the change of the physical device state is eliminated.
[0093] The above heterogeneous GPU pooling pass-through scheduling method realizes the unified management of cross-architecture hardware resources by constructing a unified resource pool containing x86 / ARM architecture multi-architecture servers and various types of GPU devices, solves the heterogeneous compatibility problem that x86 / ARM mixed architecture and different generations of GPU devices are difficult to be cooperatively scheduled in the background technology. Specifically, the resource directory is generated by collecting hardware parameters in real time through the bottom monitoring agent, and the hardware resources are isolated into multiple schedulable resource units according to the preset GPU resource division strategy according to the computing power. At the same time, the resource units are labeled with numerical SLA level tags based on the preset business scenario rules, forming a resource pool with quantitative attributes, thereby eliminating resource islands and improving resource utilization. The virtual GPU resource unit is generated by using the pass-through virtualization drive model to virtualize the unified resource pool. The model configures the drive parameters based on the computing power isolation attribute and the SLA level label of the resource unit, performs direct hardware access mapping of the physical GPU device, generates a virtual GPU resource unit with independent device address space, and configures the instruction set simulation layer of the virtual GPU (such as real-time conversion from CUDA to OpenCL) to realize compatibility, avoiding more than 20% performance degradation caused by traditional full virtualization model. At the same time, the address space isolation mechanism is used to ensure resource exclusivity, taking into account high performance and elastic scaling, solving the problem that virtualization performance and resource isolation are difficult to be considered. In response to a user resource request, the target resource is allocated from the virtual GPU resource unit by using the resource grid pooling scheduling algorithm. After analyzing the request parameters, the algorithm traverses the resource units and performs four-dimensional matching steps: verifying the consistency of architecture compatibility identification, detecting address space isolation state, calculating SLA level ratio and obtaining the physical topology position. Based on the matching result, the scoring formula The candidate unit is quantitatively scored, the highest-scored unit is selected as a target resource, and multi-dimensional constraints such as architecture types, SLA levels, and physical topologies are fused to improve resource matching accuracy and solve the defects of traditional scheduling algorithms that only rely on basic computing power indicators. The target resource is mounted to a user computing environment (such as a secure isolated AI training container), a low-latency data transmission channel is established through a pass-through virtualization drive model, dynamic resource supply is realized; at the same time, the change events (such as device failure or load threshold) of the unified resource pool are monitored, the resource directory is updated, the resource unit is reconstructed and the SLA label is marked when responding, and when the allocated resource fails, the replacement resource is automatically allocated and mounted, ensuring business continuity, solving the problem of lack of dynamic response mechanism, achieving unified management of heterogeneous GPU resources, high-performance supply (more than 95% of the original performance is reserved), resource utilization improvement, low-latency switching when business interruption caused by device failure, effectively eliminating the defects of low resource utilization, insufficient flexibility and damaged business continuity.
[0094] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.
[0095] Based on the same inventive concept, the embodiments of the present application also provide a heterogeneous GPU pooling pass-through scheduling device for implementing the above-mentioned heterogeneous GPU pooling pass-through scheduling method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more heterogeneous GPU pooling pass-through scheduling device embodiments provided below can refer to the limitations of the heterogeneous GPU pooling pass-through scheduling method in the above text, which will not be repeated here.
[0096] In one exemplary embodiment, as shown in Figure 2 a heterogeneous GPU pooling pass-through scheduling device is provided, comprising:
[0097] The resource pool construction module 101 is configured to construct a unified resource pool comprising a plurality of multi-architecture servers of x86 / ARM architecture and a plurality of types of GPU devices;
[0098] The virtualization driving module 102 is configured to utilize a pass-through virtualization driving model to perform virtualization processing on the unified resource pool, and generate a virtual GPU resource unit;
[0099] The dynamic scheduling module 103 is configured to allocate target resources from the virtual GPU resource unit by a resource grid pooling scheduling algorithm in response to a user resource request;
[0100] The resource mounting execution module 104 is configured to mount the target resources to a user computing environment.
[0101] In one embodiment, the resource pool construction module 101 is further configured to:
[0102] Obtain hardware parameters of the multi-architecture server of the x86 / ARM architecture and the GPU devices of multiple types, and generate a resource directory;
[0103] According to a preset GPU resource division strategy, the hardware resources in the resource directory are isolated according to computing capabilities to form multiple schedulable resource units;
[0104] According to a preset business scenario rule, SLA level labels are marked for the resource units to form the unified resource pool, wherein the SLA level labels contain level information that is numerical or quantifiable.
[0105] In one embodiment, the resource pool construction module 101 is further configured to:
[0106] Monitor change events of the unified resource pool, wherein the change events include physical state changes of the multi-architecture server of the x86 / ARM architecture and the GPU devices of multiple types or real-time loads exceeding a preset threshold;
[0107] In response to monitoring the change events, the following steps are performed:
[0108] Re-obtain hardware parameters of the multi-architecture server of the x86 / ARM architecture and the GPU devices of multiple types, and update the resource directory;
[0109] Based on the updated resource directory, according to the preset GPU resource division strategy, the schedulable resource units are re-generated, and the original resource units are replaced;
[0110] According to the preset business scenario rule, SLA level labels are marked for the replaced resource units to form a new unified resource pool.
[0111] In one embodiment, the virtualization driving module 102 is further configured to:
[0112] Based on the computing capability isolation attribute and the SLA level label of the resource unit, corresponding pass-through virtualization driving parameters are configured;
[0113] The direct hardware access mapping of the physical GPU device is performed through a pass-through virtualization drive model to generate a virtual GPU resource unit with an independent device address space; wherein the virtual GPU resource unit inherits the computing architecture characteristics and SLA level label of the corresponding physical GPU device, and configures the instruction set simulation layer of the virtual GPU to realize compatibility.
[0114] In one of the embodiments, the dynamic scheduling module 103 is further configured to:
[0115] receive a user resource request, parse the request parameters to obtain a target computing architecture type, SLA level requirement and computing power specification requirement;
[0116] based on the target computing architecture type, SLA level requirement and computing power specification requirement, traverse the virtual GPU resource unit and perform the following matching steps:
[0117] verify whether the architecture compatibility identifier is consistent with the target computing architecture type;
[0118] detect the independent device address space isolation state thereof;
[0119] calculate the numerical ratio of the SLA level label and the SLA level requirement;
[0120] obtain the physical topology position of the corresponding physical GPU device thereof;
[0121] based on the matching step result, score the virtual GPU resource unit, and select the virtual GPU resource unit with the highest score as the target resource.
[0122] In one of the embodiments, the dynamic scheduling module 103 is further configured to score the virtual GPU resource unit based on the matching step result using the following formula:
[0123]
[0124] wherein, v j is the candidate virtual GPU resource unit, R is the user resource request parameter set, f1 is the architecture compatibility evaluation function, and when the resource unit architecture identifier arch V matches the request architecture arch R , 1 is output, otherwise 0 is output, f2 is the address space isolation state indication function, and 1 is output when the resource unit is in an idle isolation state, otherwise 0 is output, f3 is the SLA level matching function, which calculates the ratio of the resource unit labeled SLA level SLA V and the request level SLA R , and f3≤1, f4 is the physical topology distance attenuation function, which calculates the network transmission cost based on the physical location distance d(·) between the resource unit and the user computing environment, and w kThe dynamic weight coefficient preset for each dimension is dynamically adjusted according to a real-time load state according to a preset strategy and satisfies a normalization condition, and S(·) is a scheduling score of the virtual GPU resource unit.
[0125] In one of the embodiments, the virtualization driving module 102 is further configured to, when the change event causes the allocated target resource to be invalid, perform virtualization processing based on the new unified resource pool by using the pass-through virtualization driving model to form a new virtual GPU resource unit;
[0126] The dynamic scheduling module 103 is further configured to allocate a replacement resource from the new virtual GPU resource unit by using a resource grid pooling scheduling algorithm.
[0127] The resource mounting execution module 104 is further configured to unmount the invalid allocated target resource from the user computing environment, and mount the replacement resource to the user computing environment.
[0128] In one embodiment, a computer device is provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the heterogeneous GPU pooling pass-through scheduling method as described above when executing the computer program.
[0129] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned method embodiments.
[0130] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiments described above are only schematic and the components shown as separate components can or can not be physically separate and the components shown as a unit can or can not be physical unit, that is, can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure. Those skilled in the art can understand and implement it without creative labor.
[0131] The above-described embodiments only express several implementation manners of the present application, the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application.
Claims
1. A heterogeneous GPU pool pass-through scheduling method, characterized in that, The method comprises: Building a unified resource pool comprising a multi-architecture server of x86 / ARM architecture and multiple types of GPU devices; Virtualizing the unified resource pool by using a pass-through virtualization drive model to generate a virtual GPU resource unit; In response to a user resource request, allocating a target resource from the virtual GPU resource unit by using a resource grid pooling scheduling algorithm; Mounting the target resource to a user computing environment.
2. The method of claim 1, wherein, The building of the unified resource pool comprising a multi-architecture server of x86 / ARM architecture and multiple types of GPU devices comprises: Obtaining hardware parameters of a multi-architecture server of x86 / ARM architecture and multiple types of GPU devices to generate a resource directory; According to a preset GPU resource division strategy, isolating hardware resources in the resource directory according to computing power to form multiple schedulable resource units; According to a preset business scenario rule, labeling the resource units with SLA level tags to form the unified resource pool, wherein the SLA level tags contain numerical or quantifiable comparison level information.
3. The method of claim 2, wherein, The method further comprises: Monitoring change events of the unified resource pool, wherein the change events include physical state changes of the multi-architecture server or multiple types of GPU devices or real-time load exceeding a preset threshold; In response to monitoring a change event, performing the following steps: Re-obtaining hardware parameters of a multi-architecture server of x86 / ARM architecture and multiple types of GPU devices to update the resource directory; Based on the updated resource directory, re-generating schedulable resource units according to the preset GPU resource division strategy and replacing the original resource units; According to the preset business scenario rule, labeling the replaced resource units with SLA level tags to form a new unified resource pool.
4. The method of claim 2, wherein, The virtualizing of the unified resource pool by using a pass-through virtualization drive model to generate a virtual GPU resource unit comprises: Based on the computing power isolation attribute and the SLA level tag of the resource unit, configuring corresponding pass-through virtualization drive parameters; Performing direct hardware access mapping of the physical GPU device by using the pass-through virtualization drive model to generate a virtual GPU resource unit with an independent device address space; wherein the virtual GPU resource unit inherits the computing architecture characteristics and the SLA level tag of the corresponding physical GPU device, and a virtual GPU instruction set simulation layer is configured to achieve compatibility.
5. The method of claim 4, wherein, The allocating of a target resource from the virtual GPU resource unit by using a resource grid pooling scheduling algorithm in response to a user resource request comprises: Receiving a user resource request, analyzing the request parameters to obtain a target computing architecture type, SLA level demand, and computing power specification requirement; Based on the target computing architecture type, SLA level demand, and computing power specification requirement, traversing the virtual GPU resource unit and performing the following matching steps: Verifying whether the architecture compatibility identifier is consistent with the target computing architecture type; Detecting the independent device address space isolation state; Calculating the numerical ratio of the SLA level tag to the SLA level demand; Obtaining the physical topology position of the corresponding physical GPU device; Scheduling score the virtual GPU resource units based on the matching step result, and select the virtual GPU resource unit with the highest scheduling score as the target resource.
6. The method of claim 5, wherein, The scheduling score of the virtual GPU resource units based on the matching step result uses the following formula: wherein, v j is a candidate virtual GPU resource unit, R is a user resource request parameter set, f1 is an architecture compatibility evaluation function, and when the resource unit architecture identifier arch V matches the request architecture arch R , output 1, otherwise output 0, f2 is an address space isolation state indication function, and when the resource unit is in an idle isolation state, output 1, otherwise output 0, f3 is an SLA level matching function, and the ratio of the resource unit annotated SLA level SLA V to the request level SLA R is calculated, and f3≤1, f4 is a physical topology distance attenuation function, and the network transmission cost is calculated based on the physical location distance d(·) between the resource unit and the user computing environment, w k is the preset dynamic weight coefficient of each dimension, which is dynamically adjusted according to the real-time load state according to the preset strategy and satisfies the normalization condition, and S(·) is the scheduling score of the virtual GPU resource unit.
7. The method of claim 3, wherein, The method further comprises: When the change event causes the assigned target resource to be invalid, based on the new unified resource pool, virtualization processing is performed using a pass-through virtualization drive model to form a new virtual GPU resource unit; Allocating a replacement resource from the new virtual GPU resource unit through the resource grid pooling scheduling algorithm; Unmounting the invalid assigned target resource from the user computing environment and mounting the replacement resource to the user computing environment.
8. A heterogeneous GPU pool pass-through scheduling apparatus, comprising: The device comprises: A resource pool construction module for constructing a unified resource pool containing multi-architecture servers of x86 / ARM architecture and multiple types of GPU devices; A virtualization drive module for performing virtualization processing on the unified resource pool using a pass-through virtualization drive model to generate a virtual GPU resource unit; A dynamic scheduling module for allocating a target resource from the virtual GPU resource unit through a resource grid pooling scheduling algorithm in response to a user resource request; A resource mounting execution module for mounting the target resource to a user computing environment. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.