Cluster computing resource allocation method

Abstract

The invention discloses a cluster computing resource allocation method. The method comprises the following steps: constructing a heterogeneous computing resource pool comprising an MT CPU, an FT CPU and an MTDSP cluster; an intelligent algorithm basic calculation library is deployed, hardware mapping of a deep learning algorithm is realized through an irregular matrix multiplication optimization module, and the irregular matrix multiplication optimization module comprises a microkernel automatic generation unit which dynamically generates an assembly instruction assembly line supporting three FMAC units according to matrix dimensions; the multi-core parallel strategy unit is used for providing M-dimensional and K-dimensional parallel calculation strategies and a dynamic scheduling mechanism; dynamically selecting a row block size parameter # imgabs0 # and a column block size parameter # imgabs1 # based on the matrix shape to carry out self-adaptive adjustment on the block size; physical resources are abstracted into containers or virtual machine instances through resource virtualization, and computing resources are dynamically allocated according to matrix dimension characteristics of the intelligent training module. According to the method, automatic resource deployment strategy generation and flexible computing resource scheduling are realized, and high-performance cluster computing resource allocation and deployment services are provided for users.

Description

Technical Field

[0001] The present invention relates to the fields of computer network technology, artificial intelligence, and cloud computing, and particularly designs a cluster computing resource allocation method. Background Art

[0002] For the large sample and high concurrency requirements based on multi-tenancy, it is necessary to support the rapid deployment of typical virtual machine instances, and support the orchestration, scheduling, deployment and operation of containerized applications, elastic scaling, load balancing, and high availability. Traditional computing cluster resource allocation methods are difficult to meet the needs of multi-user / multi-task / multi-sample online collaborative parallel deployment, and are difficult to meet new requirements such as the generation of automatic deployment strategies for test resources, the efficient distribution and one-click deployment of large-scale test resource images, and the release of test resources.

[0003] Therefore, there is an urgent need to propose a cluster computing resource allocation method that meets the needs of large samples and high concurrency artificial intelligence simulation tests for multiple users. Summary of the Invention

[0004] Aiming at the problems existing in the prior art, the purpose of the present invention is to provide a cluster computing resource allocation method, which provides high-performance virtual computing environments such as virtual machines and containers for intelligent applications, and uniformly plans the cluster resources.

[0005] To achieve the above purpose, the present invention provides a cluster computing resource allocation method, which constructs a heterogeneous computing resource pool including MT CPU, FT CPU, and MTDSP clusters; Deploy an intelligent algorithm basic computing library, and implement the hardware mapping of deep learning algorithms through an irregular matrix multiplication optimization module. The irregular matrix multiplication optimization module includes: A microkernel automatic generation unit that dynamically generates an assembly instruction pipeline supporting three FMAC units according to the matrix dimensions; A multi-core parallel strategy unit that provides two parallel computing strategies of M dimension and K dimension and a dynamic scheduling mechanism; Dynamically select the row block size parameter and the column block size parameter for adaptive adjustment of the block size; Abstract physical resources into containers or virtual machine instances through resource virtualization, and dynamically allocate computing resources according to the matrix dimension characteristics of the intelligent training module.

[0006] Further, the dynamic scheduling mechanism includes: Construct a matrix dimension feature recognition to real-time monitor the matrix length-width ratio and precision type; When it is detected that the M / N dimension ratio > 5, activate the M-dimensional parallel strategy and allocate MTDSP cluster resources based on the DMA three-level ping-pong strategy; When it is detected that the K dimension < M dimension / 3, activate the K - dimensional parallel strategy, and implement cross - core data reduction through the GSM shared memory.

[0007] Furthermore, the micro - kernel automatic generation unit includes: The first generation mode: When 64 < ≤96, configure the column block size = 1, and map the three FMAC units of 16 VPEs to the vector register VFMULAS32 instruction pipeline respectively; The second generation mode: When 0 < ≤64, configure the column block size > 1 and = 6, and implement scalar data expansion and broadcast synchronization of the SVBCAST2 instruction through the SBALE2H instruction; where is the number of columns of the B matrix, is the micro - kernel row block size.

[0008] Furthermore, the resource virtualization includes: Deploying in a containerized form, and building a multi - layer image repository for the intelligent training module: The base image layer contains the Python3.25GB basic library and a 200MB neural network dataset; The runtime image layer integrates the ftIMM optimization library and the dynamic link library.so file; The application image layer encapsulates the distributed training framework, supporting parallel data collection by 1000 agents; Dynamically allocate Pod instances containing 64MT CPU cores + 16MTDSP clusters through the Kubernetes orchestrator.

[0009] Furthermore, deploy in a containerized form and build a micro - kernel compilation environment container, including: Cross - platform assembly, supporting the conversion of VLDW / VFMULAS32 instructions of the processor; Instruction latency compensation, automatically inserting NOP instructions according to the tVLDW and tSBR latencies; Register conflict detection, optimizing the vector register allocation of the SVBCAST instruction through static analysis.

[0010] Furthermore, the heterogeneous computing resource pool includes: An intelligent training resource partition, configured with a dedicated MTDSCP cluster to achieve: During M - dimensional parallelism, each DSP core is assigned an independent L2 cache area to store the A sub - matrix; During K - dimensional parallelism, cross - core broadcast of the B sub - matrix is achieved through the GSM shared cache.

[0011] Further, by collecting the FMAC utilization rate index of the VPE in real time, dynamic rescheduling is triggered when it is lower than 70%.

[0012] Further, when it is detected that multiple cores modify the same cache line simultaneously, sequential writing is performed according to the core priority order.

[0013] Further, when a long vector calculation task is detected, the MTDSP cluster is preferentially allocated and the main frequency is increased to 2.1 GHz; When performing a small batch inference task, the energy-saving mode of the FT CPU is enabled and the number of cores is limited.

[0014] Further, when the utilization rate > 90%, the VPE voltage is increased to 1.2 V and overclocked by 15%; when the utilization rate < 60%, the near-threshold computing mode is enabled to reduce the VPE voltage to 0.9 V.

[0015] The beneficial effects of the present invention are as follows: Based on the intelligent algorithm basic computing library, the present invention realizes the hardware mapping of the deep learning algorithm through the irregular matrix multiplication optimization module, realizes the generation of the resource automatic deployment strategy, the elastic scheduling and control of the computing resources, and provides users with a high-performance, highly reliable, service-oriented, and easy-to-use cluster computing resource allocation and deployment service. Description of the Drawings

[0016] Figure 1 It is a schematic diagram of the framework of the present invention; Figure 2 It is a schematic diagram of the process for generating the test resource container image of the present invention. Detailed Embodiments

[0017] Next, the technical solutions of the present invention will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0018] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0019] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", and "linkage" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0020] The following will Figure 1 - Figure 2 describe the specific embodiments of the present invention in detail. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0021] A method for allocating cluster computing resources according to the present invention is specifically as follows: Construct a heterogeneous computing resource pool including MT CPUs, FT CPUs, and MTDSP clusters; Deploy an intelligent algorithm basic computing library, and implement the hardware mapping of deep learning algorithms through an irregular matrix multiplication optimization module. The irregular matrix multiplication optimization module includes: A microkernel automatic generation unit that dynamically generates an assembly instruction pipeline supporting three types of FMAC units according to the matrix dimensions; A multi-core parallel strategy unit that provides two parallel computing strategies of M dimension and K dimension and a dynamic scheduling mechanism; A block size adaptive adjustment unit that dynamically selects the block size parameter based on the matrix shape and ; Abstract physical resources into container or virtual machine instances through resource virtualization, and dynamically allocate computing resources according to the matrix dimension characteristics of the intelligent training module.

[0022] The heterogeneous computing resource pool includes an intelligent training resource partition, and configures a dedicated MTDSCP cluster to implement resource allocation and adjustment during M-dimensional parallelism and K-dimensional parallelism. Through the dynamic scheduling mechanism, when a long vector calculation task is detected, the MTDSP cluster is preferentially allocated and the main frequency is increased to 2.1 GHz; when a small batch inference task is executed, the energy-saving mode of the FT CPU is enabled and the number of cores is restricted.

[0023] The present invention focuses on the large sample and high concurrency requirements based on multi-tenancy, supports the rapid deployment of typical virtual machine instances, supports the orchestration, scheduling, deployment and operation of containerized applications, capabilities such as elastic scaling, load balancing and high availability, provides high-performance virtual computing environments such as high-performance virtual machines and containers for intelligent applications, and makes a unified plan for cluster resources.

[0024] I. Deployment plan: 1. Object access module Deployed in the form of virtual machines, the computing resource requirement is 8 FT CPU cores, and the memory resource and storage resource requirements are 8GB and 50GB respectively.

[0025] 2. Intelligent Management and Control Module (1) Sub-module 1 Deployed in the form of containers, the size of the functional system image is 410M, the computing resource requirement is 4 FT CPU cores, and the memory resource and storage resource requirements are 2GB and 1GB respectively.

[0026] (2) Sub-module 2 Deployed in the form of containers, the size of the functional system image is 410M, the computing resource requirement is 4 FT CPU cores, and the memory resource and storage resource requirements are 2GB and 1GB respectively.

[0027] (3) Sub-module 3 Deployed in the form of containers, the size of the functional system image is 420M, the computing resource requirement is 4 FT CPU cores, and the memory resource and storage resource requirements are 2GB and 1GB respectively.

[0028] (4) Sub-module 4 Deployed in the form of containers, the size of the functional system image is 400M, the computing resource requirement is 4 FT CPU cores, and the memory resource and storage resource requirements are 2GB and 1GB respectively.

[0029] 3. Intelligent Training Module (1) Trainer 1 (Base Image Layer) Deployed in the form of containers, the size of the functional system image is about 6.6GB, the computing resource requirement is 64 MT CPU cores and 16 MTDSP clusters, and the memory resource and storage resource requirements are 256GB and 512GB respectively.

[0030] (2) Trainer 2 (Runtime Image Layer) Deployed in the form of containers, running in a distributed parallel manner, it can support the distributed data collection of 1000 agents. Among them, the CPU is used to execute the simulation program, and the DSP is used for training and inference. The size of the functional system image of a single container is about 6GB, the computing resource requirement is 100 FT CPU cores, 64 MT CPU cores and 16 MTDSP clusters, and the memory resource and storage resource requirements are 512GB and 512GB respectively.

[0031] 4. Intelligent Evaluation Module Deployed in a containerized form, the functional system image size is 4.8GB, and the computing resource requirements are 32 FT CPU cores, 8 MT CPU cores, and 8 MTDSP clusters. The memory resource and storage resource requirements are 32GB and 512GB respectively.

[0032] 5. Resource Management Module Mainly deploy cloud desktops, model libraries, resource cataloging, resource recommendation, and cloud portals.

[0033] Among them, the cloud desktop and model library are implemented by deploying a unified workbench portal management node and a unified workbench application node. Resource cataloging and resource recommendation are implemented by deploying the cataloging system Tapdata and stateful applications, and the cataloging system functional system image.

[0034] (1) Unified Workbench Portal Management Node The unified workbench portal management node is deployed on bare metal. The computing resource requirement is 16 FT CPU cores, and the memory resource and storage resource requirements are 64GB and 1024GB respectively. It is necessary to configure environment dependencies such as postGresql9.6, redis6.2.5, and tomcat8.5.63 in advance.

[0035] (2) Unified Workbench Application Node The unified workbench application node is deployed on bare metal. The computing resource requirement is 16 FT CPU cores, and the memory resource and storage resource requirements are 64GB and 1024GB respectively. It is necessary to configure docker20.10.11 in advance.

[0036] (3) Cataloging System Tapdata and Stateful Applications The cataloging system Tapdata and stateful applications are deployed on bare metal. The computing resource requirement is 16 FT CPU cores, and the memory resource and storage resource requirements are 64GB and 1024GB respectively. It is necessary to configure docker20.10.11 in advance.

[0037] (4) Cataloging System Functional System Image The cataloging system functional system image is deployed in a containerized form. The computing resource requirement is 16 FT CPU cores, and the memory resource and storage resource requirements are 16GB and 20GB respectively.

[0038] (6) Cloud Portal The cloud portal is deployed in a containerized form. The computing resource requirement is 2 FT CPU cores, and the memory resource and storage resource requirements are 2GB and 10GB respectively.

[0039] 6. Application 1 Deployed in a containerized form, the functional system image size for a single sample is 1.8 GB, and the computing resource requirements are 64 FT CPU cores, 8 MT CPU cores, and 2 MTDSP clusters. The memory resource and storage resource requirements are 8 GB and 30 GB respectively.

[0040] 7. Application 2 Deployed in a containerized form, the functional system image size for a single sample is 1 GB, and the computing resource requirements are 64 FT CPU cores. The memory resource and storage resource requirements are 8 GB and 30 GB respectively.

[0041] 8. Application 3 Deployed through virtual machines, the computing resource requirements for a single sample are 4 FT CPU cores. The memory resource and storage resource requirements are 16 GB and 256 GB respectively, and the vs2010sp1 patch needs to be pre-installed.

[0042] 9. Application 4 Deployed in a containerized form, the Python basic library in the base image is 3 GB, and the Python neural network data totals 200 M. The application program image size is 50 M, and the total image size is 3.25 GB. The computing resource requirements for a single sample are 32 FT CPU cores, 16 MT CPU cores, and 2 MTDSP clusters. The memory resource and storage resource requirements are 12 GB and 50 GB respectively.

[0043] 10. Other software tools (1) Data recording software The data recording software is deployed in a containerized form. The functional system image size is 20 M, and the computing resource requirements are 16 FT CPU cores. The memory resource and storage resource requirements are 8 GB and 20 GB respectively.

[0044] (2) Kafka middleware The deployment of Kafka middleware requires 3 FT bare machines to build a Kafka + Zookeeper cluster. When the system fails, in order to ensure that the service does not stop, it is necessary to provide an additional part of computing nodes to ensure disaster-tolerant operation.

[0045] II. Intelligent algorithm basic computing library: Based on high-performance parallel computing technology and software-hardware collaborative intelligent optimization technology, the present invention has carried out the mapping and optimization of core algorithms on the long vector architecture, carried out research on methods such as operator fusion and batch processing fusion, and proposed an optimization method for irregular matrix multiplication for the autonomous long vector architecture, providing a basic support for the construction of the entire algorithm library.

[0046] In the field of deep learning, matrix multiplication is the fundamental core of various algorithms. In the calculation of convolutional neural networks, convolutional calculations are usually transformed into matrix multiplications for large-scale parallel processing. In large language models, matrix multiplication and matrix-vector multiplication are the core parts of the Transformer architecture. However, there are many differences between matrix multiplication in the field of deep learning and that in the field of scientific computing. In the matrix multiplication calculation in the field of deep learning, one dimension of the matrix is usually much larger than the other dimension, and special optimization is required. To address this issue, research on optimization techniques for irregular matrix multiplication oriented to processors has been carried out, and efficient matrix multiplication methods with various precisions including single precision and half precision have been achieved.

[0047] By analyzing in detail the problems and challenges of optimizing matrices on processors. The research found that in order to achieve high performance, various micro-kernels and parallelization methods are indispensable for irregular-shaped GEMM.

[0048] For a specific processor, an implementation (ftIMM) for three irregular-shaped GEMMs is proposed. ftIMM integrates multiple micro-kernels through automatic code generation, adopts different parallelization strategies, and can automatically select the best block size and parallelization strategy for irregular-shaped input matrices through dynamic adjustment. The MT3000 processor is used in this embodiment.

[0049] Experiments on the MT3000 processor show that ftIMM can provide multiple automatically generated micro-kernels with performance close to the theoretical value. For the three irregular-shaped GEMMs, compared with traditional implementations, ftIMM can achieve a performance acceleration of up to 7.2 times. In addition, compared with the OpenBLAS library running on the multi-core CPU of the MT3000, ftIMM can obtain a 3.1-fold performance improvement.

[0050] To effectively process three types of irregular-shaped matrix-matrix multiplications on multi-core DSPs, the present invention proposes an effective implementation: ftIMM. First, a scheme for automatically generating assembly micro-kernels is designed to support matrices of various shapes in irregular-shaped GEMM. Then, a multi-core algorithm for irregular-shaped GEMM with two parallelization strategies is designed. Due to the special shape of the matrices in irregular-shaped GEMM, the parallelization strategy and block size are dynamically adjusted to make full use of the multi-core and reduce the additional overhead.

[0051] 1. Micro-kernel design and generation, which is executed by the micro-kernel automatic generation unit and deployed in a containerized form to build a micro-kernel compilation environment container: a) Design principle: The microkernel implementation aims to effectively utilize the three FMAC units in the VPE and minimize the cost of on-chip memory access (SM and AM). By collecting the FMAC utilization metrics of the VPE in real time, dynamic rescheduling is triggered when it is below 70%. This goal is achieved by making full use of instruction-level parallelism, scalar and vector registers, and the broadcast bandwidth between the SPU and VPU. In FT-m7032, due to instruction conflicts, the SPU can at most convert two 32-bit scalar data into two vector variables per instruction cycle. Through register conflict detection, static analysis optimizes the vector register allocation of the SVBCAST instruction. When it is detected that multiple cores modify the same cache line simultaneously, sequential writes are executed according to the core priority order. The VPU can load at most 128 32-bit data into 4 vector registers per cycle and execute at most three vector fused multiply-add operations per cycle. Therefore, the bandwidth and computing power between the VPE and AM can meet the requirements of irregular-shaped GEMM with N ≤ 96, while the broadcast bandwidth between the SPU and VPU is likely to become a performance bottleneck. In the design and automatic generation of the microkernel, it is very important to improve the parallelism of broadcast operations. In practical applications, when the utilization rate > 90%, the VPE voltage is increased to 1.2V and overclocked by 15%; when the utilization rate < 60%, the near-threshold computing mode is enabled to reduce the VPE voltage to 0.9V to effectively and dynamically allocate computing resources.

[0052] Based on the above principles, a general implementation of the microkernel in ftIMM is proposed, and the row block size parameter and column block size parameter are dynamically selected based on the matrix shape for adaptive adjustment of the block size. As shown in Algorithm 1, compared with the particles of TGEMM, there are mainly three differences. The first is that ftIMM adds two additional loops (indexed by and ) through loop blocking, and these two layers of loops will be unfolded in the assembly program. The block size (row block size parameter and column block size parameter ) depends on how to make full use of the three FMAC units of the DSP and hide its latency. The second is that when the column block size parameter is greater than 1, there is a reduction operation in ftIMM (lines 12 - 13 of Algorithm 1). The third is that the VPU performs the loading operation according to the size of , so there is no need for implicit padding in TGEMM, improving the space utilization efficiency of AM.

[0053]

[0054] Algorithm 1: Microkernel assembly pseudocode in the ftIMM algorithm, and the specific interpretation of this algorithm is as follows: Input: (Blocked A matrix), (Blocked B matrix) Output: (Blocked C matrix) S1. Set ,

[0055] Initialize vector parameters: : Data length of a single operation of the vector register (32 elements); : Number of columns of matrix B According to the vector length The number of blocks after blocking, calculated by ceiling division.

[0056] S2. for : : do Outer row loop: According to the row block size Traverse The rows of is the starting row index of the current block; Loop range: , step size (For example =6, the blocks are 0-5, 6-11,...).

[0057] S3. VPU inits to zero Initialize the accumulation register: The VPU initializes the three-dimensional accumulation register (Dimension ) to 0 for storing intermediate calculation results.

[0058] S4. for do Middle column loop (K dimension): According to the column block size Traverse The columns of (i.e., the rows of is the starting column index of the current block; Loop range: , step size .

[0059] S5. for do Inner row expansion loop: Unroll granularity by row Traverse the rows of the current block as the current row offset (single-step loop, actually unrolled in assembly).

[0060] S6. for do Inner column unroll loop: Unroll granularity by column Traverse the columns of the current block as the current column offset (single-step loop, unrolled in assembly).

[0061] S7. SPU loads to scalar register

[0062] Load scalar data: The SPU loads scalar elements from into the scalar register ; Address calculation: .

[0063] S8. SPU broadcasts to vector register

[0064] Scalar broadcast to vector: Broadcast the value of the scalar register to the vector register , generating a repeated vector of length (e.g., ).

[0065] S9. for do Column block loop (in the column direction of the B matrix): Traverse by the number of blocks the columns of for as the current column block index.

[0066] S10. VPU loads to vector register

[0067] Load B matrix vector data: Load column block data from into the vector register ; Address range: (If the last column is insufficient , pad with 0).

[0068] S11.

[0069] Vector multiply-accumulate calculation: Perform fused multiply-accumulate (FMAC) operation: Each vector element is calculated independently, and the result is accumulated into .

[0070] S12. for do Intermediate result reduction (K-dimensional accumulation): Traverse = 1 to , and reduce the accumulated results of multiple to the register where = 0.

[0071] S13.

[0072] Reduction operation: Add the value of by row ( ) and column block ( ) and accumulate it into ; Formula: .

[0073] S14. VPU store back to

[0074] Write the result back to memory: Store the reduced result to the corresponding position of ; Address calculation: .

[0075] b) Assembly microkernel generation: Based on the given microkernel ( , and ), the key to generating effective assembly code is how to map the calculations in the microkernel to the FMAC units of the DSP core in MT3000 and try to hide the execution cycles of these FMAC units. According to 's size, there are two methods for generating microkernels in ftIMM as follows: The first generation mode: When 64 < ≤ 96, it is the same as in TGEMM, Three FMAC units mapped in parallel to 16 VPEs. Through instruction latency compensation, NOP instructions are automatically inserted according to the tVLDW and tSBR latencies. According to different schemes are used to hide the latency of the FMAC units according to the size of For micro-kernels larger than the FMAC instruction latency (tfma), the column block size is set to 1, and is used as large as possible to hide the latency. For <tfma micro-kernels, is set to toms, and is set to greater than 1 to hide the latency tfma. Based on the current instruction set, the assembly pipelines generated for micro-kernels with different (i.e., ≥tfma) are shown in Table 1, where tVLDW and tSBR represent the latencies of vector data load instructions (i.e., VLDW and VLDDW) and jump instructions (i.e., SBR), respectively. All three FMAC units are filled, and all operations are in pipeline mode, such as scalar data load (i.e., SLDH and SLDW), scalar data extension (i.e., SFEXTS32L and SBALE2H), broadcast (i.e., SVBCAST and SVBCAST2), vector data load, and fused multiply-add instruction (i.e., VFMULAS32).

[0076] Table 1: Micro-kernel assembly code pipeline when ≥tfma and 64 < ≤96

[0077] Second generation mode: 0 < ≤64. For this case, for the three FMAC units in 16 VPEs, the parallelism in is not sufficient. To improve the utilization of each loop FMAC unit, the column block size should be set to greater than 1. At the same time is also set as large as possible to hide the latency of the FMAC units. The assembly pipelines generated for micro-kernels in the cases of 32 < ≤64 and 0 < ≤32 are shown in Tables 2 and 3, where

[0078] Table 2: Micro-kernel assembly code pipeline when =6 and 32 < ≤64

[0079] Table 3: When = 6 and 0 < ≤ 32, the micro-kernel assembly code pipeline

[0080] 2. Multi-core parallel strategy, executed by the multi-core parallel strategy unit M-dimensional parallel strategy: Algorithm 2 shows the M-dimensional multi-core parallel strategy. A three-level ping-pong strategy based on DMA is designed to accelerate matrix-matrix multiplication. First, block in the N dimension and the K dimension, and calculate the M × sub-matrix of A and the of B × sub-matrix multiplication in the ping-pong loop represented by the index. Further block in the M, and dimensions, execute the ping-pong strategy in the dimension based on DMA, and execute the product of the × sub-matrix of A and the × sub-matrix of B. The multiplication of the × sub-array of A and the × sub-matrix of B can be implemented based on the ping-pong on . In this strategy, the M-dimensional loop is parallelized to utilize multiple cores to implement irregular-shaped GEMM. When parallelizing in the M dimension, each DSP core is assigned an independent L2 cache area to store the A sub-matrix; in addition, GSM is used to cache the sub-matrix of B shared among multiple cores. During the calculation process, each core loads its own private data A and C from the DDR, and loads the first shared sub-matrix from the GSM, which can fully achieve data reuse and reduce the memory access overhead.

[0081]

[0082] Algorithm 2: Implementation algorithm for irregular matrix multiplication with M-dimensional parallelism. The specific interpretation of this algorithm is as follows: Input: ,

[0083] Output:

[0084] S1. for do Global column block loop: According to the global column block size Traverse and output the matrix The columns of are the starting index of the first current block

[0085] S2. for do / / Ping-pong Global K-dimension block loop Traverse the rows of the input matrix according to the global block size The columns of are the starting index of the second current block are the starting index of the second current block

[0086] S3. DMA( ) DMA load the block of matrix B Transfer the sub-matrix of to the first shared sub-matrix through DMA through DMA 。

[0087] S4. for do in parallel Multi-core parallel row block loop Traverse the row blocks of according to the core-level row block size The row blocks of are the starting index of the third current block, and multiple cores process different 。

[0088] S5. for do Core-level column block loop Traverse the columns in the shared according to the core-level column block size The columns of are the starting index of the fourth current block

[0089] S6. DMA( ) DMA load the block of matrix C Load the sub-matrix of the output matrix The sub-matrix of into the first core private memory 。

[0090] S7. for do / / Ping-pong Core-level K-dimension block loop Traverse according to the core-level block size Traverse The rows, is the starting index of the fifth current block.

[0091] S8. DMA( ) DMA loads the B sub - matrix: Load into the second core private memory .

[0092] S9. for do / / Ping - pong Micro - kernel row - block loop: Traverse by the micro - kernel row - block size through the rows, which is the starting index of the sixth current block.

[0093] S10. DMA( ) DMA loads the A sub - matrix: Load into the third core private memory .

[0094] S11.

[0095] Micro - kernel matrix multiply - accumulate: Perform matrix multiplication ; Specifically, call the micro - kernel implementation of Algorithm 1 for calculation.

[0096] S12. DMA( ) DMA writes back the C - matrix block: Write the result in the first core private memory back to the global memory .

[0097] b) K - dimensional parallel strategy: Algorithm 3 presents a multi - core parallelization strategy based on K - dimensions. Similar to the parallelization strategy based on M - dimensions, this strategy uses a ping - pong strategy based on two - level DMA to implement matrix - matrix multiplication. First, in the K - dimension, perform the product of the × sub - matrix of A and the × sub - matrix of B. In the ping - pong loop in the M - dimension, the × sub - array in A and the × Multiplication of sub-matrices. Parallelize the K-dimension loop, use GSM to cache sub-matrices of C, perform reduction among multiple DSP cores based on GSM, effectively achieve data reuse, and reduce the memory access overhead caused by reduction. When parallelizing in the K-dimension, broadcast the sub-matrix of B across cores through GSM shared cache; in addition, due to the additional reduction overhead brought by the parallel strategy based on the K-dimension, this strategy is applicable to the multiplication operations of two small-sized irregular-shaped GEMMs in the M-dimension and N-dimension, so the ping-pong mode is not used in the outermost loop.

[0098]

[0099] Algorithm 3: Implementation algorithm for irregular matrix multiplication with K-dimension parallelism. The specific interpretation of this algorithm is as follows: Input: Matrix ,

[0100] Output: Matrix

[0101] S1. for do Global row block loop: Traverse the rows of the output matrix according to the global row block size , is the starting index of the first current block.

[0102] S2. for do Global column block loop: Traverse the columns of the output matrix according to the global column block size , is the starting index of the second current block.

[0103] S3. DMA( ) DMA load Matrix block: Load into the second shared sub-matrix .

[0104] S4. for do Core-level row block loop: Traverse the rows in the shared according to the core-level row block size , is the starting index of the fourth current block.

[0105] S5. for do​​ Core-level column block loop: In the shared Traverse columns according to the core-level column block size , Is the starting index of the fifth current block.

[0106] S6. Init sub-matrix to zero Initialize the core private Sub-matrix: Initialize the first core private memory To all 0.

[0107] S7. for Do in parallel / / Ping-pong Multi-core parallel Dimension block loop: Traverse according to the core-level K block size Of Columns and Rows of, Is the starting index of the third current block, and multiple cores process different .

[0108] S8. DMA( ) DMA load Sub-matrix: Will Load into the second core private memory .

[0109] S9. for Do / / Ping-pong Micro-kernel row block loop: Traverse according to the micro-kernel row block size Of Rows, Is the starting index of the seventh current block.

[0110] S10. DMA( ) DMA load A sub-matrix: Will Load into the third core private memory .

[0111] S11.

[0112] Micro-kernel matrix multiply and accumulate: Execute matrix multiplication ; The specific calculation calls the microkernel implementation of Algorithm 1.

[0113] S12.

[0114] Multi-core reduction operation: Through the GSM shared cache, the of multiple cores are accumulated into the shared , Formula: .

[0115] III. Deployment and implementation: The present invention uses resource virtualization technology to uniformly construct various physical computing resources, storage resources, and network resources into a joint shared resource pool, and uses a large-scale container / virtual machine cluster to quickly deploy and allocate resources according to requirements, providing an operating foundation support for large-scale intelligent deduction and scheduling optimization services.

[0116] 1. Deployment in the form of containerization, constructing a microkernel compilation environment container (1) Generation of container images The generation of container images for functional systems / applications is planned to adopt the method based on the Dockerfile file to quickly create container images. The specific implementation process is as follows: Step 1: Encapsulation of functional system / application resources. Encapsulate the functional system / application and its running dependency packages into an image, load it into the container for compilation, running, and testing. For functional systems / applications developed in compiled languages, including C, C++, and Go languages, dynamically call the corresponding code compilers to compile the code and generate an executable code package; for functional systems / applications developed in interpreted languages, including Java and Python languages, call the pre-set code packaging environment to package the code and generate an executable program or code package.

[0117] Step 2: Generation of the Dockerfile file. Load the container base image on which the functional system / application runs, set the environment variables on which the container image runs, declare the ports listened to by the services in the image, and specify the default entry instruction of the container image, the username and ID when running the container.

[0118] Step 3: Creation of container images. Load the Dockerfile file, generate container images by calling the Build command, and upload the generated container images to the image library. Once the container images are uploaded successfully, they can be queried and reused through the image service.

[0119] The process of generating container images for functional systems / applications based on the Dockerfile file is specifically asFigure 2 as shown

[0120] (2)Container-based Image Distribution and Deployment To improve the deployment speed of container images, container (Docker) and container cluster (Kubernetes) technologies are used to complete the dynamic generation and deployment of containers. The scheduling container image is loaded from the image repository, and the proxy in the virtual server deploys and starts the container image to run.

[0121] Step 1: Generate a container deployment file, including version number, Pod name, container image name, container startup command parameters, container working mode, port number on the virtual host where the container is located, port protocol, CPU and memory resources required by the container image.

[0122] Step 2: Load and deploy the container image. The deployment proxy sends a request to the Master node's APIServer and creates a Controller object Deployment. The controller Deployment loads the corresponding image file from the image repository and requests the APIserver to create a certain number of Pod objects. At the same time, it is assigned by the scheduler on the Master to the selected worker node and the container is started to run.

[0123] Step 3: To facilitate providing a fixed access entry for the Pod object, request the APIServer in the Master node to create a Service object, dynamically allocate a dedicated ClusterIP address and port, so that the functional system / application can be accessed through its service name and ClusterIP.

[0124] 2. Deployment in the form of virtual machines (1)Generation of virtual machine images Based on cloud environment virtualization technology, generate virtual machine images for functional systems / applications. It is necessary to prepare the dependent running support tools in advance and use the cloud environment management tool to create virtual machine images. The specific implementation process is described as follows: Step 1: Start and run a blank image instance; Step 2: Install and run the dependent running support tools; Step 3: Save the virtual machine instance as a virtual machine image; Step 4: Package, upload and register the image into the repository.

[0125] Once the image is successfully made and uploaded, it can be queried and reused through the image service.

[0126] (2)Image Distribution and Deployment Based on Virtual Machines The automatic deployment process of the virtual machine-based functional system / application mainly consists of two steps, and the specific implementation process is as follows: Step 1: Dynamically create a virtual machine and associate the mapping relationship between the functional system / application to be deployed and the virtual machine; Step 2: The virtual machine automatically downloads the functional system / application to be deployed from the database, loads the configuration file, and runs the functional system / application.

[0127] 3. Deployment in the form of bare metal To meet the high stability requirements of relevant software tools and databases for hard disk storage resources, some software tools of the cloud portal, Kafka middleware, databases, etc. are deployed and installed in the form of bare metal.

[0128] The beneficial technical effects of the present invention: Through a proposed cluster computing resource allocation method, the present invention solves problems such as multi-user / multi-task / multi-sample online collaborative parallel deployment in intelligent experiments, generation of automatic deployment strategies for experimental resources, efficient distribution and one-key deployment of large-scale experimental resource images, and release of experimental resources, and efficiently supports the development of artificial intelligence joint simulation experiments and test evaluations. Based on the intelligent algorithm basic computing library, the present invention realizes the hardware mapping of deep learning algorithms through the non-regular matrix multiplication optimization module, abstracts physical resources into containers or virtual machine instances through resource virtualization, dynamically allocates computing resources according to the matrix dimension characteristics of the intelligent training module, and realizes the adaptive adjustment of block size through the method of dynamic block size and selection of parallel strategies, so as to make the most of computing resources and storage resources and provide users with high-performance, highly reliable, service-oriented, and easy-to-use cluster computing resource allocation and deployment services.

[0129] Any process or method description in the flowchart of the present invention or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process, which can be implemented in any computer scale medium for an instruction execution system, apparatus, or device. The computer-readable medium can be any medium including storage, communication, propagation, or transmission of a program for use by an instruction execution system, apparatus, or device. It includes read-only memory, magnetic disks, or optical discs, etc.

[0130] In the description of this specification, the description with reference to terms such as "embodiment", "example", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. In addition, those skilled in the art can combine or combine different embodiments or examples described in this specification and the features therein without conflict.

[0131] Although the above content has shown and described embodiments of the present invention, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can perform update operations such as changes, modifications, substitutions, and variations on the above embodiments within the scope of the present invention.

Claims

1. A method for allocating cluster computing resources, characterized in that, Build a heterogeneous computing resource pool including an MT CPU, an FT CPU, and an MTDSP cluster; Deploy a basic computing library for intelligent algorithms, and implement hardware mapping of deep learning algorithms through an irregular matrix multiplication optimization module, where the irregular matrix multiplication optimization module includes: A microkernel automatic generation unit that dynamically generates an assembly instruction pipeline supporting three types of FMAC units according to matrix dimensions; A multi-core parallel strategy unit that provides two parallel computing strategies of M dimension and K dimension and a dynamic scheduling mechanism; Dynamically select the row block size parameter and the column block size parameter to perform adaptive adjustment of the block size; Abstract physical resources into containers or virtual machine instances through resource virtualization, and dynamically allocate computing resources according to the matrix dimension characteristics of the intelligent training module.

2. The cluster computing resource allocation method according to claim 1, characterized in that, The dynamic scheduling mechanism includes: Build matrix dimension feature recognition to monitor the matrix length-width ratio and precision type in real time; When it is detected that the M / N dimension ratio > 5, activate the M-dimensional parallel strategy and allocate MTDSP cluster resources based on the DMA three-level ping-pong strategy; When it is detected that the K dimension < M dimension / 3, activate the K-dimensional parallel strategy and achieve cross-core data reduction through GSM shared memory.

3. The cluster computing resource allocation method according to claim 2, characterized in that The microkernel automatic generation unit includes: The first generation mode: when 64 < ≤ 96, configure the column block size = 1, and map the three FMAC units of 16 VPEs to the vector register VFMULAS32 instruction pipeline respectively; Second generation mode: when 0 < ≤ 64, configure the column block size > 1 and = 6, implement scalar data expansion and broadcast synchronization of the SVBCAST2 instruction through the SBALE2H instruction; where is the number of columns of the B matrix, is the microkernel row block size.

4. The cluster computing resource allocation method according to claim 1, wherein The resource virtualization includes: deploying in a containerized form, and building a multi-layer image repository for the intelligent training module: The base image layer contains a 25GB Python3 base library and a 200MB neural network dataset; The runtime image layer integrates the ftIMM optimization library and the dynamic link library.so file; The application image layer encapsulates a distributed training framework, supporting parallel collection of 1000 agents; dynamically allocate Pod instances containing 64 MT CPU cores + 16 MTDSP clusters through the Kubernetes orchestrator.

5. The cluster computing resource allocation method according to claim 4, wherein, Deploy in a containerized form and build a microkernel compilation environment container, including: Cross-platform assembly, supporting the conversion of VLDW / VFMULAS32 instructions of the processor; Instruction latency compensation, automatically insert NOP instructions according to the tVLDW and tSBR latencies; Register conflict detection, optimize the vector register allocation of the SVBCAST instruction through static analysis.

6. The cluster computing resource allocation method according to claim 1, wherein, The heterogeneous computing resource pool includes: an intelligent training resource partition, configured with a dedicated MTDSCP cluster to achieve: During M-dimensional parallelism, each DSP core is allocated an independent L2 cache area to store the A sub-matrix; During K-dimensional parallelism, cross-core broadcast of the B sub-matrix is achieved through GSM shared cache.

7. The cluster computing resource allocation method according to claim 6, wherein Trigger dynamic rescheduling when the FMAC utilization rate index of the VPE is lower than 70% through real-time acquisition.

8. The cluster computing resource allocation method according to claim 5, wherein, When it is detected that multiple cores modify the same cache line simultaneously, perform sequential writing according to the core priority order.

9. The cluster computing resource allocation method according to claim 7, wherein When a long vector calculation task is detected, preferentially allocate the MTDSP cluster and increase the main frequency to 2.1 GHz; When performing a small batch inference task, enable the energy-saving mode of the FT CPU and limit the number of cores.

10. The cluster computing resource allocation method according to claim 9, wherein, When the utilization rate > 90%, increase the VPE voltage to 1.2V and overclock by 15%; when the utilization rate < 60%, enable the near-threshold computing mode to reduce the VPE voltage to 0.9V.

Images