Isomeric computing power fusion driven multi-architecture compilation intermediate representation generation method and system

Abstract

The application discloses a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method and system, relates to the GPU computing power pooling scheduling technical field, and the method comprises the following steps: by constructing a unified operation vector space, projecting a device basic attribute vector and carrying out similarity matching with an operator operation vector, combining an operator dependency relationship to divide an operation segment with consistent resource requirements. Further, an intermediate representation carrying an operator sequence, resource requirements and an architecture type is generated for each operation segment, and device matching score calculation is carried out in combination with the real-time resource state of the device, and after the optimal target device is selected, architecture-adapted executable code for the operation segment is generated and submitted for execution. Therefore, the scheduling accuracy, resource utilization and execution stability in the heterogeneous environment are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the GPU (Graphics Processing Unit) computing power pooling scheduling technical field, and particularly relates to a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method and system. BACKGROUND

[0002] With the rapid growth of deep learning training and inference model size, the demand for computing power resources of enterprises and research institutions is growing exponentially. Because high-end GPU devices are expensive and prone to be idle, people begin to explore the "computing power pooling" way of unified management and on-demand allocation of multiple GPU resources. Through pooling scheduling, users can dynamically obtain computing resources from a shared pool, significantly improving overall utilization. However, pooling scheduling involves virtualizing physical devices and allocating them to multiple tasks, which can cause scheduling delays, resource fragmentation, and unpredictable performance. Especially in the case of multiple hardware vendors coexisting, the hardware ecology is different, and the scheduling is more difficult.

[0003] Compared with mature CPU virtualization technology, the parallel structure of GPU is more complex. A GPU is composed of a large number of parallel stream processors, adopts a SIMD (Single Instruction, Multiple Data) or SIMT (Single Instruction, Multiple Threads) execution model, and usually has to wait for the kernel to complete before switching contexts. GPU context switching is much more complex than CPU. Before switching, GPU needs to complete the execution of the kernel and save a large number of registers and cache states. GPU also has a fixed capacity of VRAM, which is managed differently from the page-based memory management of CPU. Therefore, when switching between multiple processes, GPU not only needs to switch the computing context, but also needs to handle VRAM allocation and recovery, which makes it more difficult to achieve resource sharing. SUMMARY

[0004] The present application provides a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method and system to at least solve the problems of inaccurate heterogeneous resource adaptation, inefficient fragmentation scheduling and lagging response of cross-architecture execution in the related art.

[0005] In a first aspect, the embodiments of the present application provide a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method. The method is directed to a computing power pool composed of computing devices of multiple architectures. The method comprises: constructing a device basic attribute vector corresponding to each computing device according to the device resource basic attribute of each computing device in the computing power pool; collecting an operator operation list of each to-be-scheduled task and operation resource requirements of each operator operation, and generating a corresponding directed acyclic graph for each to-be-scheduled task; the node of the directed acyclic graph represents an operator operation vector, and the operator operation vector is defined by the operation resource requirements of the corresponding operator operation; a directed edge is defined by the operator dependency relationship of the connected nodes; projecting each device basic attribute vector to the operation vector space to calculate the similarity score distribution of the operator operation vector for each device basic attribute vector, dividing each operator operation vector into an operation segment according to the similarity score distribution and the operator dependency relationship, and generating a corresponding segment intermediate representation carrying resource requirement metadata; the operation vector space is a feature space constructed according to the operator operation vectors corresponding to each node in all directed acyclic graphs; the resource requirement metadata includes an operator code sequence in the operation segment, an operator resource requirement total amount, and a target device architecture type; for each segment intermediate representation, collecting device resource real-time indicators of each computing device to construct real-time residual resource vectors of each computing device, calculating device matching scores between each real-time residual resource vector and the resource requirement metadata corresponding to the segment intermediate representation based on the resource requirement metadata, determining a target computing device corresponding to the segment intermediate representation according to the device matching scores, compiling the segment intermediate representation into executable code according to the device architecture type of the target computing device, and submitting the executable code to the target computing device for execution.

[0006] In a second aspect, an embodiment of the present application provides a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation system, which is directed to a computing power pool composed of computing devices of multiple architectures, and includes: a device basic attribute modeling unit, configured to construct a device basic attribute vector corresponding to each computing device in the computing power pool according to device resource basic attributes of each computing device; a task graph modeling unit, configured to collect an operator operation list of each to-be-scheduled task and operation resource requirements of each operator operation, and generate a corresponding directed acyclic graph for each to-be-scheduled task; a node of the directed acyclic graph represents an operator operation vector, and the operator operation vector is defined by operation resource requirements of a corresponding operator operation, and a directed edge is defined by an operator dependency relationship of connected nodes; an intermediate representation generation unit, configured to project each device basic attribute vector to the operation vector space to calculate a similarity score distribution of the operator operation vector for each device basic attribute vector, divide each operator operation vector into an operation segment according to the similarity score distribution and the operator dependency relationship, and generate a corresponding segment intermediate representation carrying resource requirement metadata; the operation vector space is a feature space constructed according to operator operation vectors corresponding to each node in all directed acyclic graphs; the resource requirement metadata includes an operator code sequence in the operation segment, an operator resource requirement total amount, and a target device architecture type; and a scheduling and compilation execution unit, configured to, for each segment intermediate representation, collect device resource real-time indexes of each computing device to construct real-time residual resource vectors of each computing device, calculate device matching scores between the segment intermediate representation and each real-time residual resource vector based on resource requirement metadata corresponding to the segment intermediate representation, determine a target computing device corresponding to the segment intermediate representation according to the device matching scores, compile the segment intermediate representation into executable code according to a device architecture type of the target computing device, and submit the executable code to the target computing device for execution.

[0007] In a third aspect, an electronic device is provided, which includes at least one processor, and a memory connected with the at least one processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform steps of the heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method of any embodiment of the present application.

[0008] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements steps of the heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method of any embodiment of the present application.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer programs / instructions, which, when executed by a processor, implement the steps of the method for generating a multi-architecture compilation intermediate representation driven by heterogeneous computing power fusion according to any of the embodiments of the present application.

[0010] The method and system for generating a multi-architecture compilation intermediate representation driven by heterogeneous computing power fusion provided by the present application can achieve at least the following technical effects:

[0011] (1) By uniformly expressing device basic attributes and task operation resource demand in vector form and mapping both to the same feature space, the bridging of task demand and device resource capability in vector space is achieved, so that fine-grained similarity evaluation in vector space is achieved in the scheduling process instead of relying on coarse-grained label matching, improving the accuracy and robustness of scheduling matching in a heterogeneous resource scenario.

[0012] (2) By combining operator dependency relationships and device similarity distribution for operation segment division, and generating an intermediate representation containing resource demand metadata for each segment, fine-grained segmentation at the operator level and intermediate representation metadata-driven scheduling mode are achieved. Thus, by adaptively segmenting the task and independently scheduling and allocating each operator operation segment, different segments of the same task can run on multiple devices of different architectures, thereby maximizing the use of computing resources of all heterogeneous devices in the pool and improving overall throughput and resource utilization.

[0013] (3) In the scheduling stage, real-time acquisition of device residual resource indicators is further introduced, and by performing device matching scoring on the resource demand of the segment intermediate representation and the real-time residual resource vector, the optimal target computing device is dynamically determined, and automatic compilation and code distribution for different architecture types are supported. Thus, high linkage between scheduling decisions and device load states is achieved, avoiding the static rigidity problem in resource allocation, ensuring the timeliness and rationality of scheduling decisions, and effectively improving scheduling fairness and execution efficiency under multi-task concurrency.

[0014] By the technical solution, the operation segment intermediate representation of the task is dynamically mapped to the heterogeneous device resource capability, and a unified operation vector space and real-time resource evaluation mechanism are used to achieve high adaptability, strong scalability and intelligent scheduling capability in a heterogeneous computing power pool environment, improving task execution performance and resource utilization in a complex computing power environment, and providing efficient computing power support for deep learning and high-performance computing scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0016] Figure 1 A flow chart of an example of a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method according to an embodiment of the present application is shown.

[0017] Figure 2 An operation flow chart of an example of dividing operation segments from an operation vector space according to an embodiment of the present application is shown.

[0018] Figure 3 An operation flow chart of an example of generating segment intermediate representations corresponding to operation segments according to an embodiment of the present application is shown.

[0019] Figure 4 An operation flow chart of an example of assigning operation segments to target computing devices according to an embodiment of the present application is shown.

[0020] Figure 5 A distribution histogram of an example of all task waiting times under a baseline polling scheduling algorithm and the adaptive scheduling algorithm herein is shown.

[0021] Figure 6 A simulation effect schematic diagram of an example of the allocation relationship between the computing requirements of task operation segments and devices in the adaptive scheduling algorithm herein is shown.

[0022] Figure 7 A structural block diagram of an example of a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0024] It should be noted that in the prior art, some experts and scholars have proposed some novel directions for GPU computing power pooling resource sharing. Specifically, in order to improve utilization, some experts and scholars have proposed virtual GPU (vGPU), Multi-Process Service (MPS), Multi-Instance GPU (MIG) and other technical solutions.

[0025] Although these solutions can achieve resource sharing to some extent, they still have limitations:

[0026] High switching delay. Complete GPU context switching takes about 25-50 microseconds, and thread block switching still takes more than 100 cycles. Even though MPS can insert multiple process instructions at the micro level, it still cannot completely avoid memory conflicts and error propagation.

[0027] Insufficient memory isolation. Many methods rely on software-level memory allocation, without hardware isolation, and memory leaks can affect other tasks.

[0028] Ecological fragmentation. The current GPU field is fragmented by CUDA (Compute Unified Device Architecture), ROCm (Radeon Open Compute), and other proprietary ecosystems. Different GPU instruction sets and runtimes are incompatible, and moving tasks between heterogeneous devices requires recompilation.

[0029] Lack of unified intermediate representation. The coexistence of multiple architectures makes it difficult for compilers to optimize uniformly. LLVM (Low Level Virtual Machine) was originally a virtual machine project, and has now developed into a general-purpose compilation infrastructure that supports multiple languages and multiple platforms. The MLIR (Multi-Level Intermediate Representation) framework proposed by LLVM provides an extensible multi-level intermediate representation that supports representing data flow graphs, loop optimization, and can contain high-level operations for specific accelerators, aiming to solve software stack fragmentation and improve compilation efficiency on heterogeneous hardware. However, MLIR itself only provides a basic framework and lacks a unified description for heterogeneous scheduling.

[0030] More specifically, in vGPU, a physical GPU is divided into multiple virtual GPU instances by splitting it in kernel or user space, and assigned to different virtual machines or containers. Official solutions like NVIDIA vGPU and AMD MxGPU provide mature hardware and driver support, with the advantage of flexible allocation of computing units and memory, and the ability to run multiple containers or VMs on one GPU. However, these solutions usually only support virtual machines, not containers, and have high licensing fees; open-source alternatives like vCUDA and cGPU have cumbersome adaptation to different CUDA versions and weak security isolation.

[0031] MPS is a "context merging" method provided by NVIDIA, where multiple processes share the context of an MPS daemon process as MPS clients, and the hardware schedules different processes' kernels at a micro level. This solution has the advantages of better performance, reducing frequent context switching, and being suitable for small-scale inference or multi-process training within the same framework. However, it has the disadvantage of poor fault isolation: a daemon process or a task crash can affect all shared processes, and lacks hard memory isolation, requiring upper-level scheduling to limit memory allocation.

[0032] The context switching cost of GPU and CPU is very different. A complete GPU context switch takes 25-50 microseconds, and thread block launch takes hundreds of cycles. Although single warp switching can occur every cycle, it still involves register bank selection when the working set crosses different SMs. At the same time, the number of GPU registers is huge (the register file of each SM contains 65536 4-byte registers), so switching does not require moving registers out of the chip, but requires modifying register indices, which is still a critical path for hardware scheduling.

[0033] The MLIR framework proposed by the LLVM Foundation provides an extensible, multi-level intermediate representation that aims to solve software fragmentation and improve compilation efficiency for heterogeneous hardware. The project page states that MLIR can represent data flow graphs and support memory layout conversion, vectorization, and other optimizations, and can also carry high-level operations for specific accelerators. MLIR emphasizes that it does not directly support low-level machine code generation, but rather translates through a series of reductions to LLVM IR and then to target architecture instructions. However, MLIR lacks a unified mechanism to describe the behavior of heterogeneous devices, and developers need to write dialects for each hardware.

[0034] In recent years, researchers have proposed the hyper dialect to unify the compilation process on heterogeneous hardware. This dialect is based on MLIR and abstracts data management and parallel computing functions to form a unified compilation framework for heterogeneous devices. Studies have shown that this dialect can improve performance by an average of 1.93x, 1.18x, and 1.12x in the heterogeneous execution of SHA1, MD5, and SM3 hash algorithms. This shows that using a unified dialect to manage heterogeneous scheduling can indeed bring performance improvements. However, this research focuses on cryptography scenarios and lacks task decomposition and resource mapping algorithms for power pool scheduling. It also does not fully consider GPU memory fragmentation and dynamic task granularity division issues.

[0035] In summary, existing technologies either rely on virtualization mechanisms provided by underlying manufacturers or provide a unified intermediate representation at the compilation level, but still do not solve the problems of dynamic scheduling of heterogeneous computing power pools, resource fragmentation, and intermediate representation generation.

[0036] In view of the above problems, an innovative solution is needed that can both utilize existing GPU virtualization technologies and break through their limitations. By using an intermediate representation for heterogeneous environments to drive pool scheduling, resource utilization and scheduling efficiency can be improved from the root cause.

[0037] It should be understood that the above description of the prior art is intended only to facilitate a better understanding of the inventive spirit and motivation of the present application, and is not considered a limitation on the present application. In addition, the technical solutions described in the above prior art may not be prior art, but may also be undisclosed technical solutions, such as solutions under research or in the laboratory stage.

[0038] In the technical solutions of the present application, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information, etc. comply with relevant laws and regulations and do not violate public order and good customs.

[0039] Figure 1 A flowchart showing an example of a heterogeneous computing power fusion-driven multi-architecture compilation intermediate representation generation method according to an embodiment of the present application is shown.

[0040] Regarding the execution subject of the method of the present application, it can be any controller or processor with computing or processing capability, such as a computing power scheduling platform controller. In some examples, it can be integrated and configured in an electronic device or terminal through software, hardware, or a combination of software and hardware, and the type of terminal or electronic device can be diverse.

[0041] As the scale of tasks in fields such as deep learning, scientific computing, and data analysis continues to expand, single-type computing devices can hardly meet the dual demands of high performance and high throughput. Heterogeneous computing power pools, as an architecture that integrates and uniformly schedules the resources of various computing devices (such as GPUs, CPUs, FPGAs, etc. from different manufacturers or models), are becoming the mainstream direction of next-generation computing infrastructure.

[0042] like Figure 1 As shown, in step S110, a basic attribute vector corresponding to each computing device is constructed based on the basic attributes of the device resources of each computing device in the computing power pool.

[0043] It should be noted that, in the context of computing power pool scheduling, since each computing device (such as GPUs, NPUs, FPGAs, etc. from different manufacturers) has differences in computing power, memory structure, parallelism, kernel scheduling methods, etc., it is impossible to directly express its features with a fixed dimension or a unified structure. Instead, the hardware and operating structure of each device are standardized and modeled through the device basic attribute vector mechanism.

[0044] Specifically, for each computing device in the computing power pool, its core hardware parameters are collected and standardized, including but not limited to: device type (such as GPU model, CPU architecture, FPGA series), number of computing cores (such as number of CUDA cores, number of threads, etc.), clock speed, memory capacity and bandwidth, number of supported concurrent streams, etc.

[0045] The vector is encoded into a multi-dimensional vector using a structured method, with each dimension standardized or normalized to ensure that attributes across different devices can be compared and computed. Furthermore, this vector can be compressed and stored using vector embedding or sparse coding methods to adapt to the scheduling system of large-scale computing pools.

[0046] In some examples of embodiments of this application, each computing device is considered within the device set of the computing power pool. , computing devices Device architecture type coding Equipment computing power and device memory capacity After normalization, the basic attribute vector of the device is formed. .

[0047] Equation (1)

[0048] in, and These are respectively for the device's computing power and device memory capacity The normalization function.

[0049] In some embodiments, each device first needs a structure type identifier to express the hardware architecture category it belongs to, for example represents NVIDIA GPU (based on CUDA architecture), represents AMD GPU (based on ROCm architecture), represents AI accelerator (such as Huawei Ascend, Baidu Kunlun).

[0050] Computing power is one of the most important indicators in the scheduling strategy. The original parameters may include core number (such as CUDA core number, stream processing unit number), frequency (GHz), maximum thread number that can be supported, etc. Due to the different definitions of these indicators by different device manufacturers, in order to achieve comparability, a normalization function is introduced to map the original computing power value to a continuous, dimensionless interval (such as [0, 1]). For multi-dimensional computing indicators, a multi-dimensional embedding method can be introduced to combine multiple indicators into a single computing power vector, and principal component dimension reduction or weight weighted fusion can be performed to finally form a single dimension of normalized value .

[0051] Memory capacity is an important parameter in task scheduling to determine whether the task can be loaded and whether it can be parallel. The design of the memory normalization function is similar to the computing power normalization. Specifically, the upper limit value of the memory capacity of all devices in the current computing pool is collected, and then the maximum and minimum values are used as the normalization boundary to linearly map the memory value.

[0052] Finally, after aggregating the above dimensions, a unified vector is formed, and each dimension represents a specific resource characteristic. Through vector expression, the heterogeneous capabilities of devices are projected into the same metric space, so that the scheduling system can quantitatively understand the resources of each device.

[0053] In step S120, the operator operation list of each to-be-scheduled task and the operation resource requirement of each operator operation are collected, and a corresponding directed acyclic graph is generated for each to-be-scheduled task.

[0054] Here, the nodes of the directed acyclic graph represent operator operation vectors, which are defined by the operation resource requirements of the corresponding operator operations, and the directed edges are defined by the operator dependency relationship of the connected nodes.

[0055] It should be noted that deep learning and scientific computing tasks are often composed of multiple operator operations, and there are some computing dependency paths and resource consumption characteristics between these operations. In the embodiment, in order to finely manage the resource requirements inside the task, the task is disassembled to the operator level, the resource requirements of each operator in the task are analyzed, and the data / control dependency between operators is constructed, so as to effectively depict the operation topology structure and execution path inside the task, and to characterize and abstract the operator resource consumption.

[0056] Specifically, for each task to be scheduled, all operator operations to be executed in the task are collected. Taking deep learning model inference as an example, common operators include convolution (Conv), full connection (FC), normalization (BatchNorm), activation (ReLU), pooling (Pooling), Softmax, etc.

[0057] For each operator, the specific operation resource requirements are analyzed, including but not limited to: computation intensity (such as FLOPS number), memory occupation (such as input / output tensor size), memory bandwidth requirement, whether special hardware support is needed (such as Tensor Core acceleration). Then, the task to be scheduled is disassembled into a directed acyclic graph (DAG), the node is an operator operation vector, and the edge is the data dependency relationship between operators, wherein the node attribute contains the operation resource requirement vector of the corresponding operator, and the edge describes the operator dependency sequence and data flow direction.

[0058] In some examples of the embodiments of the present application, the nodes in the corresponding directed acyclic graph are respectively defined based on each operator in the task to be scheduled, and the operation resource requirement vector of each node is calculated based on the operation resource requirement of the corresponding operator. The corresponding operator operation vector is constructed .

[0059] , formula (2)

[0060] In the formula, is the operator type code corresponding to the node , is the original calculation requirement of the node , which reflects the basic operation amount required to complete the operator operation; is the normalized value of the calculation requirement corresponding to the node , is the original memory requirement of the node , which reflects the memory resource consumption required by the operator during operation; is the normalized value of the memory requirement corresponding to the node , is the parallelism degree of the node . ​​​

[0061] Here, the operator operation vector For each node in the task DAG The structured representation of the resource requirements and scheduling characteristics of the corresponding operators.

[0062] Specifically, field operator type encoding Used to characterize nodes The specific operator type corresponding to it, for example Represents convolution. Indicates a fully connected structure. Representing pooling and other operations can effectively express operator types and help guide device adaptability estimation. For example, some hardware may have hard acceleration units for specific operators (such as TensorCore's support for convolution).

[0063] Calculate the demand This represents the basic computational intensity required to complete the operator; the original value. FLOPs (floating-point operations per second) or the number of operations performed are typically used as a metric. The calculation method depends on the specific operator type and parameters; for example, convolution operators and matrix multiplication are calculated differently. This is achieved through a normalization function. Statistics of all nodes in the current task The maximum and minimum values ​​are determined, and linear normalization is used to ensure that the computational requirements fall within the [0,1] interval, facilitating spatial mapping with the device's computational capability vector components.

[0064] Video memory requirements This is used to express the amount of video memory used during the execution of the operator. For example, it can be calculated by input tensor video memory requirements, output tensor video memory requirements, and intermediate buffer requirements (such as temporary workspace). The normalized video memory requirements can be directly used in device resource matching calculations to facilitate determining whether the computing device currently has enough video memory to support the operator.

[0065] Parallelism The maximum degree of parallelism theoretically supported by an operator can be estimated through analysis of its internal structure, thread partitioning strategies, or provided by the framework analyzer. For example, for convolution operators, the degree of parallelism is roughly equal to the number of pixels in the output feature map; for matrix multiplication, the degree of parallelism is the larger of the number of rows or columns of the matrix; for scalar operators (such as activation), the degree of parallelism may be the total number of elements in the input tensor. The magnitude of the degree of parallelism directly affects the scheduling system's preference for devices with strong parallel execution capabilities (such as GPUs with a large number of SM (Streaming Multiprocessor) units), and is also a reference factor in determining whether the granularity of an operator is suitable for inclusion in a certain segment.

[0066] Finally, the above-mentioned various feature encodings are spliced to obtain the final operator operation vector of each node, realizing the structured expression of the resource requirement of the operator granularity in the task, no longer relying on the original parameters or framework-specific meta-information in the task description file, solidifying the operator requirement in the form of a standard vector, making each node in the task graph have clear and measurable resource requirement features, and greatly enhancing the automatic analysis capability of the scheduling system.

[0067] In step S130, each device basic attribute vector is projected into the operation vector space to calculate the similarity score distribution of the operator operation vector for each device basic attribute vector, and each operator operation vector is divided into operation segments according to the similarity score distribution and the operator dependency relationship, and a corresponding segment intermediate representation carrying resource requirement metadata is generated.

[0068] Here, the operation vector space is a feature space constructed according to the operator operation vectors corresponding to each node in all directed acyclic graphs; the resource requirement metadata includes the operator code sequence within the operation segment, the total amount of operator resource requirement, and the target device architecture type.

[0069] In the present embodiment, the device basic attribute vector is projected into the operation vector space to establish the spatial similarity relationship between the device capability and the task requirement, so as to measure the "affinity" of the device to a certain type of operator. On this basis, in combination with the structure of the task graph, the operator cluster is identified, and the operation segment that can be independently scheduled and executed is divided, and then the intermediate representation is generated.

[0070] In some embodiments, by analyzing the operator operation vectors of all task graphs, the principal components or feature dimensions thereof are extracted to form a high-dimensional representation space. Then, each device basic attribute vector is projected into the space, for example, based on cosine similarity, Euclidean distance or learning-based metric space mapping method, the matching score distribution between the device and the operator is calculated. Finally, the high-similarity continuous operator path is found in the directed graph, and the operation segment is divided according to the continuity of the dependency structure between the operators and the device score. An intermediate representation (IR) is generated for each segment, which includes: the code sequence of the operators within the segment (operation type and parameters), the total resource requirement vector (aggregated memory, computation, thread, etc. indicators), and the initial matching target device type (as a compilation and scheduling reference).

[0071] It should be noted that the operators are divided into several operation segments according to resource adaptation, dependency order and scheduling algorithm. Each segment contains a group of operators that can be continuously executed on the same device (may belong to the same task or different tasks depending on the scheduling result). Therefore, instead of allocating a complete task to a computing device, each segment is allocated to the most suitable device. Because the task is segmented, each operator operation segment can be independently scheduled and allocated in a more fine-grained manner. Different operator operation segments of the same task (for example, operator operation segments without direct data dependency) are also likely to be distributed on multiple devices with different architectures for running, maximizing the use of computing / storage resources of all heterogeneous devices in the pool and improving overall throughput and utilization.

[0072] In step S140, for each segment intermediate representation, the device resource real-time indicators of each computing device are collected to construct the real-time residual resource vector of each computing device, and the device matching score between each real-time residual resource vector is calculated based on the resource requirement metadata corresponding to the segment intermediate representation. The target computing device corresponding to the segment intermediate representation is determined according to the device matching score, and the segment intermediate representation is compiled into executable code according to the device architecture type of the target computing device and submitted to the target computing device for execution.

[0073] It should be noted that the device resource state of the heterogeneous computing power pool may change dynamically at any time (such as task scheduling, task release, device health fluctuation, etc.). Therefore, the final scheduling and allocation of the operation segment need to be based on the real-time resource state of the device to maximize the use of the remaining capacity of the computing power pool and achieve the best "resource-task" matching.

[0074] More specifically, the runtime indicators of each device can be collected periodically or event-triggered to construct the current real-time residual resource vector, such as the current free video memory capacity, the number of available threads / computing unit load, the task queue length or the expected delay, etc. The corresponding real-time residual resource vector is obtained by modeling the device resource vector. Then, for each segment intermediate representation to be scheduled, read its resource requirement metadata and match it with the above real-time vector to calculate the device matching score, for example, by weighted similarity or scheduling score function, which should not be limited here. Finally, the device with the highest matching degree is selected as the target computing device, and the corresponding compilation tool chain (such as NVCC, ROCm, Ascend CCE, etc.) is called to generate device-specific executable code for the operator operation segment according to the architecture type annotated by the segment. After compilation, the code segment is submitted to the target device queue for execution scheduling of the corresponding operator operation segment.

[0075] In this embodiment, by dynamically sensing the resource status of devices, task operator fragments are allocated "on-demand," ensuring strong real-time scheduling decisions and accurate resource matching, effectively avoiding the failure of static matching strategies in dynamic environments. Therefore, elastic allocation is performed based on the current resource status during scheduling, thereby improving the utilization, throughput, and response capability of the entire computing pool. Simultaneously, dedicated compilation paths based on architecture types avoid wasting compilation resources, support automated heterogeneous compilation, improve code generation efficiency and execution performance, and ensure the availability and adaptability of devices with different architectures.

[0076] Figure 2 An example operation flowchart of dividing operation segments from the operation vector space according to an embodiment of this application is shown.

[0077] like Figure 2 As shown, in step S210, for each device basic attribute vector, a nonlinear embedding function is used to map it to the operation vector space to obtain the device embedding vector.

[0078] Here, to calculate the vector space similarity between the operator and the device, it is first necessary to convert the device's basic attribute vector... Mapped into the operation vector space so as to correspond with the operator operation vectors. Measurements and comparisons are performed within the same space.

[0079] Equation (3)

[0080] In the formula, Indicates computing device The device embedding vector, This represents a nonlinear mapping function that embeds features into an input vector, with parameters... Determined through pre-training.

[0081] here, Various deep learning models can be used, such as the Multilayer Perceptron (MLP) architecture, trained using historical or simulation data. For each device... ,use Process its attribute vector Generate the corresponding embedding vector This enables the device to have operator requirement awareness in the operation vector space.

[0082] In step S220, for each operator operation vector, a similarity score is calculated with all device embedding vectors.

[0083] Here, the adaptation relationship between the operator and the device is measured by the similarity in the vector space, which intuitively expresses the matching degree of the operator adapted to run on different devices.

[0084] Specifically, the cosine similarity formula can be used to calculate the similarity score between the operator and the device :

[0085] , formula (4)

[0086] In the formula, represents the similarity score of the operator corresponding to the node and the device , and ||·|| is the Euclidean norm of the vector.

[0087] Thus, the parallel scoring of the adaptation degree of the operator to all devices is realized, and a quantitative operator-device affinity matrix is provided for the operation segment division. In addition, the similarity value is stabilized in the interval [-1, 1] by vector normalization, which is beneficial to subsequent softmax normalization.

[0088] In step S230, the similarity score distribution of the operator operation vector for all computing devices is obtained by normalization through the softmax function.

[0089] In order to convert the similarity score into an additive probability distribution, the softmax function is introduced to normalize the device score corresponding to each operator, and the probability form of the device preference distribution is obtained.

[0090] Specifically, for each operator , the softmax is applied to the scoring of all devices:

[0091] , formula (5)

[0092] In the formula, represents the normalized similarity score of the operator corresponding to the node adapted to the computing device , reflecting the tendency of the operator corresponding to the node to be preferentially allocated to the computing device ; is the normalization term, representing the sum of the exponential scores of all computing devices.

[0093] Thus, the discrete scoring is mapped to a continuous probability distribution, allowing the system to make fuzzy segment division rather than mandatory assignment.

[0094] In step S240, according to the directed edges in the task directed acyclic graph, the operator operation vector pairs with existing data dependency are directly merged into the same operation segment; for each operator operation vector without directed edges, at least one group of operator operation vector pairs is constructed through vector combination, and the distribution distance corresponding to the similarity score distribution of the operator operation vector pairs is calculated; when the distribution distance corresponding to the first operator operation vector pair is not greater than a preset distance threshold, the first operator operation vector pair is merged into the same operation segment.

[0095] On the one hand, in the task DAG graph, the data dependency between the operator nodes often means that there is continuity in the calculation context, and if dispatched to different devices, not only significant data migration overhead will be introduced, but also the overall execution efficiency will be reduced due to inconsistent synchronization between devices. Therefore, the operator operation vector pairs with direct data dependency are directly merged into the same operation segment to reduce data migration and synchronization waiting, and improve the execution efficiency of the data flow within the segment.

[0096] On the other hand, the similarity score probability distribution of the operator and the device is taken as the core index to measure the adaptability, and the distribution distance is used to measure the approximate mode of the adaptability between the operators. Specifically, if two operators have no dependency but the distribution distance is close enough, they can be merged into the same operation segment, which can ensure that the whole segment has higher resource utilization efficiency and execution consistency when dispatched to the target device, avoid resource waste, execution bottleneck or complex heterogeneous compilation within the segment caused by the difference in operator adaptability, and thus improve the overall load balancing and scheduling effect of each device in the computing power pool.

[0097] Specifically, for two operator operation nodes with directed edges in the DAG (for example ), the two are preferentially directly classified into the same operation segment, which ensures that the dependency order of operator execution will not be disturbed, maximizes data locality, and reduces cross-segment communication cost.

[0098] In addition, for operator operation nodes without directed edges, when merging, it can be judged whether the adaptability distribution distance of the two operators is within the allowed range, so as to avoid forcibly allocating operators with poor adaptability to the same device, causing resource waste or execution bottleneck.

[0099] Specifically, for any two operators without dependency edges, the Euclidean distance is used to quantify the difference between the device adaptability distributions of the two operators:

[0100] , formula (6)

[0101] In the formula, and respectively represent the nodes and the nodes The corresponding operator, Indicates a node The corresponding operators are adapted to computing devices. Normalized similarity score; express and Distance in the similarity score distribution.

[0102] In some implementations, a threshold is set. ,when This indicates that the resource compatibility of the two operators is highly similar, and they can be assigned to the same operation segment. Therefore, by using segment partitioning driven by distribution distance, it is ensured that the compatibility of operators within each segment is close, and when the same segment is sent to the same device, the execution performance is optimal and the fluctuation is minimal.

[0103] In step S250, the operation segments are recursively expanded for the operation vector space until all operator operation vectors are assigned to the corresponding operation segments.

[0104] Here, the operation vector space is iteratively expanded by continuously traversing it. For example, for all operator operation vectors that have not yet been assigned to a segment, distribution calculation and merging judgment are repeatedly performed. Operator operations with consistent suitability are continuously merged into the corresponding segments until all operators are assigned. After each round of expansion, the segment set is dynamically maintained, and the suitability distribution statistics and assignment status of operator operations within each segment are updated. Finally, when all operator operations are covered or cannot be further merged, the final assignment result of all operation segments is output.

[0105] Through the embodiments of this application, it is ensured that each operation segment obtained is optimal in terms of both resource adaptability and task dependency, so that the segmentation automatically adapts to the heterogeneity of task structure and device capabilities, thereby improving the overall system elasticity and scalability.

[0106] Figure 3 A flowchart illustrating an example of the intermediate representation of a segment corresponding to a generation operation segment according to an embodiment of this application is shown.

[0107] like Figure 3 As shown, in step S310, the codes of each operator node within the operation segment are extracted and arranged according to the original dependency order to obtain the operator code sequence of the operation segment.

[0108] Within an operation fragment, there are execution dependencies between operator nodes. Disrupting this order can lead to incorrect execution of the scheduled execution graph or even computational errors. Therefore, in the first step of generating the intermediate representation of the fragment, the traceability of the operator order in the original task graph can be preserved so that the compiler can correctly construct the execution sequence.

[0109] For example, for each segment Extract the type codes of the internal operator nodes according to the topological sort of the original task's DAG, forming an operator code sequence:

[0110] Equation (7)

[0111] In the formula, Indicates the first One operation segment, Indicates operation fragment Operator encoding sequence within, Indicates operation fragment Inner Type encoding of each operator.

[0112] Here, each element in the sequence is the operator type number and operator index corresponding to that operator. Through sequential encoding, it is ensured that the execution dependencies between operators will not be lost due to the scheduling strategy.

[0113] In step S320, based on the computational and memory requirements in the operator operation vector, the total computational and memory requirements corresponding to the operation segment are calculated.

[0114] Fragment scheduling must be performed under global resource constraints. Failure to pre-calculate computational and memory requirements may result in allocations exceeding the physical capabilities of the target device, leading to scheduling failures. Therefore, the total computational resource requirements of the entire fragment must be calculated during the fragment formation phase. Total demand for video memory resources .

[0115] In some implementations, the computational requirements for all operator nodes in the fragment are... Summing, we get:

[0116] Equation (8)

[0117] Equation (9)

[0118] In the formula, Indicates operation fragment Total computing resource requirements Indicates operation fragment The total demand for video memory resources. Memory requirements for all nodes Take the maximum value (reflecting the peak video memory usage during runtime).

[0119] As for the description of formulas (8) and (9), the calculation amount is accumulative, and the memory occupation bottleneck comes from the operator with the largest resource requirement in the segment (which may be determined by the intermediate activation tensor, cache, etc.). Therefore, the and , can significantly reduce the risk of resource conflict and task rollback in the scheduling process, and provide clear hard constraints for device matching.

[0120] In step S330, the similarity score distribution of each operator in the operation segment is weighted and summed according to the operator calculation amount to obtain the adaptation probability distribution of the operation segment on all devices.

[0121] It should be noted that although the similarity distribution for different devices has been calculated , the segment is a combination of multiple operators, so it is necessary to fuse the operator-level similarity distribution into the segment-level adaptation probability distribution to guide the device scheduling of the segment.

[0122] , formula (10)

[0123] , formula (11)

[0124] In the formula, represents the operation segment adaptation probability of the device , and are the calculation resource weighting coefficient and the memory resource weighting parameter, ; is the calculation-memory joint normalization weighting coefficient of the operator corresponding to the node .

[0125] In the fusion process, operators with high calculation amount and operators with high memory demand will obtain higher weights, thereby having a greater impact on the final result when selecting devices. Therefore, through the weighted fusion method, the overall execution bottleneck of the segment can be more truly reflected, avoiding the excessive influence of operators with small calculation amount or low memory demand on the selection of the target device of the segment, and improving the accuracy of the target device matching.

[0126] In step S340, the device architecture type corresponding to the maximum value of the overall adaptation probability distribution of the segment is taken as the target device architecture type of the operation segment.

[0127] The scheduling system must specify the optimal target device architecture type for each segment, otherwise even if the adaptation probability distribution is available, it will not be able to enter the compilation and deployment stage due to the lack of clear device type selection.

[0128] Specifically, the maximum value corresponding to the device architecture type is selected from the fused adaptive probability distribution:

[0129] , formula (12)

[0130] In the formula, indicates the target device architecture type of the operation segment .

[0131] Here, not only as a reference for scheduling allocation, but also directly used in subsequent cross-architecture compiler calls. The optimal execution architecture type is bound for each operation segment, ensuring that subsequent compilation, memory allocation and operator kernel binding have clear targets, reducing the number of compilation attempts and cross-device migration overhead.

[0132] In step S350, the operator encoding sequence, the total amount of computing resource requirement, the total amount of video memory resource requirement, the adaptive probability distribution and the target device architecture type are aggregated into structured metadata to obtain a segment intermediate representation corresponding to the operation segment.

[0133] In order to support subsequent compilation, optimization and runtime scheduling, the multi-dimensional information of the segment needs to be packaged into a unified data structure, i.e., a segment intermediate representation, so as to be passed between the scheduling engine and the compiler.

[0134] , formula (13)

[0135] In the formula, indicates the segment intermediate representation of the finally generated operation segment .

[0136] It should be understood that the data structure of the intermediate representation can be diversified, such as JSON, Protocol Buffers or other formats, etc.

[0137] In the embodiments of the present application, the target device architecture is selected on the basis of the comprehensive trade-off of operator dependency, resource constraint and adaptive probability. By outputting a unified intermediate representation containing operator sequence, resource index, adaptive distribution and target architecture type, the scheduling and compilation process can reduce invalid attempts while ensuring the executability and stability of the segment on the target device, achieving more efficient task allocation and resource utilization.

[0138] Figure 4 An operation flowchart of an example of allocating an operation segment to a target computing device according to an embodiment of the present application is shown.

[0139] As Figure 4As shown, in step S410, at the scheduling time, the device architecture type of each computing device is matched with the target device architecture type in the intermediate representation of the fragment, and the device resource indicators of each computing device that matches the target device architecture type are collected in real time to construct a real-time remaining resource vector.

[0140] Here, during the scheduling phase, based on the target device architecture type already determined in the intermediate representation of the fragment, all computing devices matching the architecture type are first selected from the computing power pool to avoid the matching consumption and compilation time caused by scheduling calculations on devices with incompatible architectures.

[0141] Then, for each device with a matching architecture, its remaining computing power, remaining video memory capacity, and operator operation queue length (i.e., the current queue length of operator operations) are collected in real time to form a resource indicator vector that reflects the current device load and available resource status, thereby dynamically reflecting the carrying capacity of each computing device at the scheduling moment.

[0142] In step S420, based on resource demand metadata and real-time remaining resource vector, the device matching score of the segment intermediate representation on each device is calculated.

[0143] Segment scheduling needs to take into account both resource requirements and the current load status of the equipment. Here, a weighted calculation of multiple indicators is used to obtain the equipment matching score, so as to quantify the suitability of each equipment to undertake the segment at the current moment.

[0144] Equation (14)

[0145] In the formula, Indicates at time fragments Dispatched to computing device The device matching score, Indicates the first The computing device at time The current remaining computing power, Indicates the first The computing device at time The remaining video memory capacity, express The computing device at time The queue length of operator operations, This indicates the maximum queue length among all devices currently in use. Representing fragments The number of operator operations contained within. This indicates the maximum number of operators among all segments in the current scheduling batch. These are the weighting coefficients of each indicator in the matching scoring function.

[0146] Regarding formula (14), the first part The ratio of the total computing requirement of the segment to the remaining computing capacity of the device is evaluated, reflecting the degree of computing resource adaptation. The second part The ratio of the video memory requirement of the segment to the remaining video memory capacity of the device is evaluated to prevent insufficient video memory from causing execution failure. The third part The device queue length and the number of operators in the segment are considered at the same time to balance the risk of task congestion and the complexity of execution. The last part The device adaptation probability distribution of the segment calculated in the previous stage is introduced as a positive incentive or negative inhibition factor. The above items are combined according to the weight coefficients to form a matching score that comprehensively reflects the device adaptation situation, achieving global optimality and stability of task allocation.

[0147] In step S430, the device with the smallest device matching score is selected as the target computing device of the segment intermediate representation.

[0148] Specifically, each computing device in the computing power pool is traversed, and the value is compared to select the device number corresponding to the minimum value as the final scheduling target of the segment.

[0149] , formula (15)

[0150] In the formula, indicates the device number of the target computing device.

[0151] Here, by minimizing the matching score, it is ensured that the segment is assigned to the currently most suitable device for execution, which helps to improve the overall task scheduling efficiency and reduce resource conflicts.

[0152] In step S440, the segment intermediate representation is compiled into executable code according to the device architecture type of the target computing device, and submitted to the target computing device for execution.

[0153] It should be noted that due to differences in instruction sets, memory access modes and operator implementation methods between different computing device architectures (such as CUDA-based GPUs, ROCm-based GPUs, or AI accelerators based on special instruction sets), the intermediate representation of the operation segment needs to be converted into executable code that can be recognized and efficiently run by the target device after scheduling is completed and the target device is determined.

[0154] In some embodiments, the operator sequence in the segment intermediate representation is read each operator is mapped to a low-level instruction or library function supported by the target device, such as a cuBLAS interface call for a matrix multiplication operator and a cuDNN or MIOpe interface call for a convolution operator. Then, resource allocation information (such as the starting address of the video memory, the thread block and grid division strategy) of the operation fragment is embedded in the compilation stage to reduce the initialization overhead at runtime. Further, the binary file generated by compilation is packaged with the meta-information required for execution (such as the dependency library path, input and output tensor description, etc.) into a task package to ensure that it can be directly called on the target device. Finally, through the device execution queue interface of the scheduling system, the task package is added to the execution queue of the target device and waits for execution according to the priority or queuing strategy.

[0155] Thus, through the above compilation and optimization for the target device architecture, the special compilation path based on the architecture type avoids waste of compilation resources and supports heterogeneous compilation automation. Not only does it ensure the direct executability of the operation fragment on different architecture devices, but it also reduces the additional configuration and compilation overhead at runtime, improving the overall execution efficiency and resource utilization of the system.

[0156] To verify the effectiveness of the scheme, we built a computing power pool simulation environment and compared the traditional static polling scheduling (baseline scheme) with the adaptive scheduling scheme of the present application. Two high-performance GPUs (denoted as GPU A) and two medium-performance GPUs (denoted as GPU B) were considered, with GPU A having a computing capacity of 10 GFLOPs and 16 GB of video memory, and GPU B having a computing capacity of 4 GFLOPs and 8 GB of video memory, with two of each. 100 tasks to be executed were generated, each randomly containing a computation amount of 0.5-5 GFLOPs and a video memory requirement of 0.1-4 GB. The simulation used the baseline polling scheduling and the adaptive scheduling algorithm of the present application to execute all tasks, respectively, and compared the throughput (number of tasks completed per unit time) and average waiting time.

[0157] Table 1. Performance comparison of the adaptive scheduling algorithm of the present application and the baseline polling scheduling algorithm

[0158]

[0159] ​It should be understood that the higher the throughput per unit time, the lower the average waiting time. The adaptive algorithm has a clear advantage in throughput, increasing by about 15% (compared to the baseline algorithm), proving that the adaptive task allocation strategy improves the number of tasks completed by the system per unit time. In addition, by dynamically scheduling and preferentially allocating high-computing tasks to more powerful GPU devices (GPU A), the adaptive algorithm reduces the average waiting time by about 14%, further improving user experience and system response speed. Through data comparison, the dual advantages of the adaptive scheduling algorithm in throughput and waiting time optimization are highlighted, solving the efficiency bottleneck of traditional methods in high-load environments.

[0160] Figure 5 A distribution histogram of all task waiting times under the baseline polling scheduling algorithm and the adaptive scheduling algorithm of the present disclosure is shown.

[0161] As shown in Figure 5 , the horizontal axis represents the waiting time of the task (unit: time unit), and the vertical axis represents the frequency of the task in each waiting time interval. It can be seen that the task waiting time of the adaptive scheduling algorithm is mostly concentrated in the lower waiting time interval (0-4 time units), and the task frequency in the higher waiting time interval (6-10 time units) is significantly reduced, showing an effective reduction in long-tail waiting time. In contrast, the baseline algorithm still has more tasks in the higher waiting time interval (especially 7-10 time units), and the distribution of task waiting time is more dispersed. This result confirms the advantages of the adaptive scheduling algorithm in reducing extreme waiting time, improving task scheduling efficiency, and optimizing service quality.

[0162] Figure 6 A simulation effect diagram showing an example of the relationship between the computing demand of the task operation segment and the allocation of the device in the adaptive scheduling algorithm of the present disclosure is shown. The horizontal axis in the figure represents the computing demand of the operation segment after the task is split (unit: GFLOPs), and the vertical axis represents the device index to which the operation segment is allocated (device 0 represents the device with the strongest computing power, and device 2 represents the device with the weakest computing power).

[0163] As shown in Figure 6 , operation segments with high computing demand (close to 5 GFLOPs) are mainly allocated to device 0, while operation segments with low computing demand (close to 1-2 GFLOPs) are allocated to device 2. Thus, by splitting the task into multiple segments and dynamically allocating them to different devices according to the computing demand, it ensures that high-computing-demand segments are supported by devices with strong computing power, thereby optimizing resource utilization efficiency and overall task execution performance, and embodying the optimization effect of the adaptive scheduling algorithm in task splitting and resource allocation.

[0164] Based on the analysis of the limitations of existing GPU virtualization and computing power pooling scheduling technologies, a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method is proposed. By vectorizing task operators, matching heterogeneous resource characteristics, and completing resource mapping based on an adaptive weighting function, cross-architecture intermediate representation generation and dynamic scheduling are achieved. Compared with traditional round-robin scheduling, the method optimizes both throughput rate and average waiting time, and the task distribution is more reasonable. In addition, since the IR contains resource requirement information, this method also facilitates migration between multiple compiler backends, making the algorithm scalable and platform compatible.

[0165] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of actions, but those skilled in the art should know that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0166] Figure 7 A structural block diagram of an example of a heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation system according to an embodiment of the present application is shown.

[0167] As Figure 7 shown, the heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation system 700 includes a device basic attribute modeling unit 710, a task graph modeling unit 720, an intermediate representation generation unit 730, and a scheduling and compilation execution unit 740.

[0168] The device basic attribute modeling unit 710 is configured to construct a device basic attribute vector corresponding to each computing device in the computing power pool according to the device resource basic attributes of each computing device.

[0169] The task graph modeling unit 720 is configured to collect the operator operation list of each to-be-scheduled task and the operation resource requirement of each operator operation, and generate a corresponding directed acyclic graph for each to-be-scheduled task. The node of the directed acyclic graph represents an operator operation vector, which is defined by the operation resource requirement of the corresponding operator operation. The directed edge is defined by the operator dependency relationship of the connected nodes.

[0170] The intermediate representation generation unit 730 is configured to project each device base attribute vector to the operation vector space to calculate a similarity score distribution of the operator operation vector for each device base attribute vector, divide each operator operation vector into an operation segment according to the similarity score distribution and the operator dependency relationship, and generate a corresponding segment intermediate representation carrying resource requirement metadata; the operation vector space is a feature space constructed according to the operator operation vectors corresponding to each node in all directed acyclic graphs; the resource requirement metadata includes an operator code sequence in the operation segment, an operator resource requirement total amount, and a target device architecture type.

[0171] The scheduling compilation execution unit 740 is configured to, for each segment intermediate representation, collect a device resource real-time index of each computing device to construct a real-time residual resource vector of each computing device, calculate a device matching score between the real-time residual resource vector and the resource requirement metadata corresponding to the segment intermediate representation, determine a target computing device corresponding to the segment intermediate representation according to the device matching score, compile the segment intermediate representation into executable code according to the device architecture type of the target computing device, and submit the executable code to the target computing device for execution.

[0172] In some embodiments, the embodiments of the present application provide a non-volatile computer readable storage medium, the storage medium storing one or more programs including execution instructions, the execution instructions being readable and executable by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to perform the steps of any of the above-mentioned heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation methods.

[0173] In some embodiments, the embodiments of the present application also provide a computer program product, the computer program product including a computer program stored on a non-volatile computer readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, causing the computer to perform the steps of any of the above-mentioned heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation methods.

[0174] In some embodiments, the embodiments of the present application also provide an electronic device, including at least one processor, and a memory connected to the at least one processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method.

[0175] The product can execute the method provided by the embodiments of the application, has the corresponding function modules and beneficial effects of executing the method. Technical details not described in detail in the embodiments can be referred to the method provided by the embodiments of the application.

[0176] The electronic device of the embodiments of the application exists in various forms, including but not limited to: a mobile communication device, an ultra-mobile personal computer device, a portable entertainment device, or other onboard electronic devices with data interaction functions.

[0177] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e. can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the application.

[0178] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus a general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0179] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation method for a computing power pool composed of computing devices of multiple architectures, characterized in that, The method comprises: According to the device resource basic attribute of each computing device in the computing power pool, a device basic attribute vector corresponding to each computing device is constructed; An operator operation list and operation resource demand of each operator operation of each to-be-scheduled task are collected, and a corresponding directed acyclic graph is generated for each to-be-scheduled task; the node of the directed acyclic graph represents an operator operation vector, and the operator operation vector is defined by the operation resource demand of the corresponding operator operation; the directed edge is defined by the operator dependency relationship of the connected nodes; Each device basic attribute vector is projected into an operation vector space to calculate a similarity score distribution of the operator operation vector for each device basic attribute vector; each operator operation vector is divided into an operation segment according to the similarity score distribution and the operator dependency relationship, and a corresponding segment intermediate representation carrying resource demand metadata is generated; the operation vector space is a feature space constructed according to the operator operation vector corresponding to each node in all directed acyclic graphs; the resource demand metadata includes an operator code sequence, an operator resource demand total amount and a target device architecture type in the operation segment; For each segment intermediate representation, a real-time residual resource vector of each computing device is constructed by collecting device resource real-time indicators of each computing device; a device matching score between each real-time residual resource vector and the resource demand metadata corresponding to the segment intermediate representation is calculated; a target computing device corresponding to the segment intermediate representation is determined according to the device matching score; the segment intermediate representation is compiled into executable code according to the device architecture type of the target computing device, and is submitted to the target computing device for execution.

2. The method of claim 1, wherein, The method comprises: For each computing device of a set of devices in a computing power pool , the device architecture type , the device computing power , the device memory capacity , and the device GPU capacity of the computing device are normalized to form a device base attribute vector : ， wherein, and are normalization functions for the device computational power and device memory capacity respectively.

3. The method of claim 1, wherein, The method comprises: The nodes in the respective directed acyclic graph are defined based on each operator in the task to be scheduled, and the nodes are respectively defined The corresponding operator operation vector is constructed : ， In the formula, is a node corresponding to the operator type code, is a node The original computing requirement of the node, reflecting the basic operation amount required to complete the operator operation; is a normalized value of the computing requirement of the node , is the original memory requirement of the node , reflecting the memory resource consumption required by the operator at runtime; is a normalized value of the memory requirement of the node , is the parallelism degree of the node .

4. The method of claim 1, wherein, The method comprises: The method comprises: ， wherein denotes a device embedding vector of a computing device , denotes a non-linear mapping function that performs a feature embedding on the input vector, the parameters are determined by pre-training; For each device basic attribute vector, a nonlinear embedding function is used to map the device basic attribute vector to an operation vector space to obtain a device embedding vector: ， wherein representing a node the operator corresponding to the device a similarity score, is the Euclidean norm of the vector; For each operator operation vector, the similarity score of the operator operation vector and all device embedding vectors is calculated respectively: ， wherein, denotes the normalized similarity score of the node corresponding operator to the computing device reflecting the tendency of the node corresponding operator to be assigned preferentially to the computing device ; is a normalization term, denoting the sum of the exponential scores of all computing devices; The similarity score distribution of the operator operation vector for all computing devices is obtained by normalization through a softmax function: According to the directed edges in the task directed acyclic graph, the operator operation vector pairs with data dependency relationship are directly combined and classified into the same operation segment; For each operator operation vector without directed edges, at least one group of operator operation vector pairs is constructed by vector combination, and the distribution distance corresponding to the similarity score distribution of the operator operation vector pairs is calculated; When the distribution distance corresponding to the first operator operation vector pair is not greater than a preset distance threshold, the first operator operation vector pair is classified into the same operation segment; ， wherein, and respectively represent the operators corresponding to the nodes and the nodes respectively, represent the normalized similarity score of the nodes corresponding to the operator adapted to compute the normalized similarity score of the computing device ; represent the distance of the and the in the similarity score distribution; The operation vector space is recursively expanded with operation segments until all operator operation vectors are attributed to corresponding operation segments.

5. The method of claim 4, wherein, The corresponding segment intermediate representation carrying resource requirement metadata is generated, including: The encoding of each operator node in the operation segment is extracted and arranged in the original dependency order to obtain an operator encoding sequence of the operation segment: ， wherein represents a first operational segment, represents an operator encoding sequence within the operational segment represents a type encoding of a first operator within the operational segment ​​ According to the calculation requirement and the memory requirement in the operator operation vector, the total calculation resource requirement and the total memory resource requirement corresponding to the operation segment are counted: ， ， In the formula, represents the total amount of computing resource requirements of the operation fragment represents the total amount of video memory resource requirements of the operation fragment represents the total amount of video memory resource requirements of the operation fragment​ The similarity score distribution of each operator in the operation segment is weighted and summed according to the operator calculation amount to obtain the adaptation probability distribution of the operation segment on all devices: ， ， In the formula, Operation fragment For the device The adaptation probability, And The computing resource weighting coefficient and the video memory resource weighting parameter are respectively, ; The computing-video memory joint normalization weighting coefficient of the operator corresponding to the node ; The device architecture type corresponding to the maximum value of the overall adaptation probability distribution of the segment is taken as the target device architecture type of the operation segment: ， In the formulae, represents the target device architecture type of the operating fragment ​ The operator encoding sequence, the total calculation resource requirement, the total memory resource requirement, the adaptation probability distribution and the target device architecture type are aggregated as structured metadata to obtain the segment intermediate representation corresponding to the operation segment: ， In the formula, represents the final generated operation fragment fragment intermediate representation.

6. The method of claim 5, wherein, The device resource real-time indicators of each computing device are collected to construct real-time residual resource vectors of each computing device, and the device matching scores between each real-time residual resource vector and the resource requirement metadata corresponding to the segment intermediate representation are calculated based on the resource requirement metadata corresponding to the segment intermediate representation, and the target computing device corresponding to the segment intermediate representation is determined according to the device matching scores, including: At the scheduling moment, the device architecture type of each computing device is matched with the target device architecture type in the segment intermediate representation, and the device resource indicators of each computing device matched with the target device architecture type are collected in real time to construct real-time residual resource vectors, including the current residual calculation capability, the residual memory capacity and the operator operation queue length; Based on the resource requirement metadata and the real-time residual resource vectors, the device matching scores of the segment intermediate representation on each device are calculated: ， In the formula, represents the device matching score of the segment scheduled to the computing device at time , represents the current remaining computing capability of the nth computing device at time , , represents the remaining video memory capacity of the nth computing device at time , , represents the operator operation queue length of the nth computing device at time , , represents the maximum queue length among all devices at present, represents the number of operator operations contained in the segment , represents the maximum number of operators among all segments in the current scheduling batch, are weight coefficients of each index in the matching score function, respectively. The device with the smallest device matching score is selected as the target computing device of the segment intermediate representation: ， In the formula, device number representing the target computing device.

7. A heterogeneous computing power fusion driven multi-architecture compilation intermediate representation generation system, facing a computing power pool composed of computing devices of multiple architectures, characterized in that, The system includes: A device basic attribute modeling unit is configured to construct a device basic attribute vector corresponding to each computing device in the algorithm power pool according to the device resource basic attributes of each computing device in the algorithm power pool. A task graph modeling unit is configured to collect an operator operation list of each to-be-scheduled task and operation resource requirements of each operator operation, and generate a corresponding directed acyclic graph for each to-be-scheduled task; the nodes of the directed acyclic graph represent operator operation vectors, and the operator operation vectors are defined by the operation resource requirements of the corresponding operator operations; the directed edges are defined by the operator dependency relationships of the connected nodes. The intermediate representation generation unit is configured to project each device base attribute vector to an operation vector space to calculate a similarity score distribution of an operator operation vector for each device base attribute vector, divide each operator operation vector into an operation segment according to the similarity score distribution and an operator dependency relationship, and generate a corresponding segment intermediate representation carrying resource requirement metadata; the operation vector space is a feature space constructed according to operator operation vectors corresponding to each node in all directed acyclic graphs; the resource requirement metadata includes an operator code sequence in the operation segment, an operator resource requirement total amount, and a target device architecture type; The scheduling compilation execution unit is configured to, for each segment intermediate representation, collect device resource real-time indexes of each computing device to construct real-time residual resource vectors of each computing device, calculate device matching scores between the real-time residual resource vectors and resource requirement metadata corresponding to the segment intermediate representation based on the resource requirement metadata, determine a target computing device corresponding to the segment intermediate representation according to the device matching scores, compile the segment intermediate representation into executable code according to a device architecture type of the target computing device, and submit the executable code to the target computing device for execution.

Images