A reduction class kernel function optimization method for mu xi xi cloud C500
By optimizing the thread configuration and parallel processing of kernel functions on the Muxi Cloud C500 chip, the problem of time-consuming kernel function writing and optimization on the MXMACA architecture was solved, achieving efficient development of reduction-type kernel functions and improved computational performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- KYLIN CORP
- Filing Date
- 2024-07-23
- Publication Date
- 2026-07-03
AI Technical Summary
Writing and optimizing kernel functions on the MXMACA architecture is a lengthy and labor-intensive process, and the parallelization of reduction-type kernel functions is complex, making it difficult to perform efficiently on the Muxi Cloud C500 chip.
By optimizing the thread configuration parameters of the kernel function, including setting the number of threads, vector register capacity, and parallel processing mode, the data volume is reasonably divided and parallel reduction operation is achieved, and efficient parallel computing is performed by utilizing the hardware characteristics of the Muxi Cloud C500 chip.
This significantly improves the efficiency of kernel function development, simplifies the optimization and writing process of reduction-type kernel functions on the Muxi Cloud C500 chip, and enhances computing performance.
Smart Images
Figure CN118966322B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, specifically providing a reduction-class kernel function optimization method for Muxi Cloud C500. Background Technology
[0002] MXMACA is a computing platform developed by MXM that uses a general-purpose parallel computing architecture to solve complex computational problems. It includes a self-developed instruction set architecture (ISA) and a parallel computing engine inside the GPU, integrating general-purpose computing and machine learning frameworks, providing experts in various application fields with a highly flexible and high-performance open computing platform.
[0003] This computing platform provides a simple and easy-to-use C-like programming language for users to write programs for the MXMACA architecture, enabling it to run at ultra-high efficiency on METAX GPU processors. The language features a simple and flexible syntax, is easy to read and write, and boasts excellent expressive power; it is also compatible with mainstream heterogeneous computing languages such as C / C++, allowing users to easily adapt software and develop efficient client operators.
[0004] Kernel functions, as a key component in deep learning algorithms, need to satisfy specific mathematical conditions to ensure the effectiveness of the algorithm. Writing kernel functions based on the MXMACA architecture is a challenging task, requiring consideration of multiple aspects such as symmetry, positive semidefiniteness, parameter selection, computational complexity, generalization ability, and interpretability to fully leverage the hardware parallel acceleration performance of the MXMACA computing platform.
[0005] Current kernel function writing and optimization methods are all applicable to the NVIDIA CUDA ecosystem. However, for kernel function writing and optimization based on the MXMACA architecture, researchers must conduct in-depth research and continuous practice to gradually improve the quality and performance of kernel functions; otherwise, the computational performance of the MXMACA computing platform cannot be maximized. This requires researchers to have a strong background in mathematics and statistics to understand and construct suitable kernel functions, and continuous iterative testing is necessary to finally determine the kernel function optimization parameters. The process is lengthy and labor-intensive.
[0006] Reduction kernel functions involve performing aggregation operations on a set of data, such as summation, maximum / minimum calculation, and product. These kernel functions are characterized by the need to progressively merge numerous independently calculated intermediate results into a final result. Typical reduction operations include: global / local summation: calculating the sum of all elements in an array or the sum of a contiguous sub-interval in an array; maximum / minimum value search: finding the maximum or minimum value in an array or within a specific sub-interval; average calculation: calculating the average of the array elements, typically requiring summation followed by division by the number of elements.
[0007] Parallelization of reduction kernel functions is relatively complex. It requires consideration of how to effectively distribute computational tasks among multiple threads and design appropriate reduction strategies (such as phased reduction, tree reduction, etc.) to merge intermediate results, while avoiding data races and ensuring correctness. Summary of the Invention
[0008] To overcome the above-mentioned shortcomings, this invention is proposed to solve the problems that when processing reduction-type kernel functions on the Muxi Cloud C500 chip, not only is the process of determining kernel function optimization parameters in the MXMACA architecture lengthy and labor-intensive, but also the technical problems of parallelization reduction are complex.
[0009] This invention provides a reduction-type kernel function optimization method for Muxi Cloud C500, comprising the following steps:
[0010] S1: Obtain the data length N of the subject to be executed, and select the subject whose data length N is greater than the preset value to execute the following steps S2-S5;
[0011] S2: Set the number of threads in the kernel function, block_size, to 1024, and the vector register capacity of each thread in block_size is 64*4B;
[0012] S3: Calculate the number of data items that each thread needs to process, NUM = N / 1024;
[0013] S4: Determine the number of processing times W, vector length X, number of parallel processing vectors Z, and number of parallel processing vectors Z' for each thread based on the NUM;
[0014] S5: Based on the vector length X, the number of vectors processed in parallel Z, the number of vectors processed in parallel Z' in the last time, and the number of times each thread processes data W, set the kernel function startup parameters, and perform reduction operations on the intra-vector, inter-vector, number of times each thread processes data, and data of all threads in sequence.
[0015] Furthermore, step S4 includes the following steps:
[0016] S41: Calculate the vector length X and the total number of vectors Y for each thread, where X*Y=NUM, X is an integer power of 2 and X≤64, and Y takes the smallest integer value that satisfies the conditions;
[0017] S42: Determine whether X*Y≤64 is true. If yes, then the number of parallel processing vectors Z=Y, the number of processing times for each thread W=1, and determine the number of parallel processing vectors Z'=Z for the last time. If no, then the number of parallel processing vectors Z=64 / X, obtain the number of processing times for each thread W=Y / Z, round up to the nearest integer value for W, and determine the number of parallel processing vectors Z'=Y-(W-1)*Z for the last time.
[0018] Furthermore, step S5 includes the following steps:
[0019] S51: Based on the vector length X and the number of vectors processed in parallel Z, perform parallel reduction operations on the data within and between vectors processed in the first W-1 times in each thread; based on the vector length X and the number of vectors processed in parallel Z' in the last time, perform parallel reduction operations on the data within and between vectors processed in the Wth time in each thread.
[0020] S52: Based on the number of processing times W of each thread, perform the reduction operation in parallel on the reduction results of all processing times of each thread;
[0021] S53: Based on the fact that a single Warp contains 64 threads, sort the 1024 threads and divide them into multiple Warp, and perform a reduction operation on the data of all threads in each Warp in parallel;
[0022] S54: Reduce all Warp data.
[0023] Furthermore, the parallel reduction operations on the intra-vector and inter-vector data processed in each thread include:
[0024] Index the address of each thread in the current block;
[0025] Index the address of each vector in each thread;
[0026] Based on the index address, the data within and between vectors in each thread are reduced in parallel.
[0027] The working principle and beneficial effects of this invention are as follows:
[0028] In implementing the technical solution of this invention, based on the characteristics of the Muxi Cloud C500 chip and the MXMACA architecture, for a reduction subject with a large amount of input data, the configuration parameters are optimized within 1024 threads in a block according to the data length of the reduction subject. By calculating the amount of data processed in each thread, and then dividing the data amount into reasonable vector lengths and number of vectors according to the capacity of each thread, parallel reduction operations with better performance are achieved. This helps R&D personnel to quickly perform reduction optimization of existing operators and write new operators on the Muxi Cloud C500 chip, greatly improving the efficiency of kernel function development. Attached Figure Description
[0029] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:
[0030] Figure 1 This is a schematic diagram of the main steps of a reduction-type kernel function optimization method for Muxi Cloud C500 according to the present invention. Detailed Implementation
[0031] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.
[0032] This invention optimizes the kernel function of the Muxi Cloud C500 chip. First, the architecture of the Muxi Cloud C500 is compared with that of the NVIDIA A100 Ampere architecture to identify key performance bottlenecks and verify the worst-performing hardware unit.
[0033] The NVIDIA A100 Ampere architecture consists of a GPU processing cluster and 108 SM (Streaming Multiprocessors). SMs are the core computing units of the NVIDIA GPU, each containing a large number of CUDA cores, which are the smallest units for performing general-purpose parallel computing tasks. The A100's GPU processing cluster contains 6912 32-bit floating-point units, 6912 32-bit integer units, 3456 64-bit floating-point units, 432 tensor units, and a pair of memory controllers controlling one HBM2 DRAM stack.
[0034] The core computing architecture of the Muxi Cloud C500 consists of a Data Processing Cluster (DPC), Accelerated Processing Units (APs), Processing Element Units (PEUs), and a matrix computation unit. Each PEU supports multiple parallel waves, each AP can accommodate up to 2048 threads, and each AP's vector register capacity is 2048*64*4B. The MXMACA software stack architecture is divided into blocks and warps, distributed across the APs. The grid_size determines the number of blocks across all APs. A block is further divided into multiple warps, each block can accommodate up to 1024 threads, and a single warp contains 64 threads.
[0035] Example 1
[0036] Figure 1 This is a schematic diagram illustrating the main steps of a reduction-type kernel function optimization method for MuXiXiYun C500 according to the present invention. Figure 1 As shown, the reduction kernel function optimization method for Muxi Cloud C500 in this embodiment mainly includes the following steps S1-S5.
[0037] S1: Obtain the data length N of the subject to be executed, and select the subject whose data length N is greater than the preset value to execute the following steps S2-S5.
[0038] The MXMACA software stack architecture typically uses maca_kernel<<<grid_size,block_size,0,stream> A kernel function is started using the >>(N,...) method. When the maca_kernel function is defined for reduction computation, the length of the input data, i.e., the amount of data N, is obtained.
[0039] In practical applications, if the value of N is small, there is no optimization significance, and the XiYun C500 can continue to use the startup program to perform reduction calculations. However, if the amount of data N being processed is large, since the startup program is a general program of the MXMACA software stack architecture, the execution effect of large-scale reduction calculations on the XiYun C500 is not good based on the configuration parameters grid_size (grid dimension and size) and block_size (block dimension and size) in the startup program. Therefore, this invention optimizes the configuration parameters of the startup kernel function according to the input data length N and the architecture of the XiYun C500.
[0040] S2: Set the number of threads in the kernel function, block_size, to 1024. The vector register capacity of each thread in block_size is 64*4B.
[0041] In this embodiment, the preset value in step S1 is at least greater than 1024, because a block includes 1024 threads. For the reduction body with a data length N less than 1024, each thread executes one data item, allowing the entire reduction operation to be completed in parallel. Furthermore, in the XiYun C500 hardware architecture, each AP includes 2048 threads, and the vector register capacity of each AP is 2048*64*4B. Since the capacity of each thread is 64*4B, when creating a virtual block on the AP, the capacity of each thread is 64*4B. Each data item in the reduction body is 4B in size. S3: Calculate the number of data items that each thread needs to process, NUM = N / 1024.
[0042] In this embodiment, the input length of large language models is generally 4096 or 8192, and the data length N of the general reduction body is a multiple of 1024.
[0043] S4: Determine the number of processing times W, vector length X, number of parallel processing vectors Z, and number of parallel processing vectors Z' for each thread based on the NUM.
[0044] Furthermore, the specific calculation steps of step S4 include the following steps S41-S42.
[0045] S41: Calculate the vector length X and the total number of vectors Y for each thread, where X*Y = NUM, X is an integer power of 2 and X≤64, and Y takes the smallest integer value that satisfies the conditions.
[0046] S42: Determine whether X*Y≤64 is true. If yes, then the number of parallel processing vectors Z=Y, the number of processing times for each thread W=1, and determine the number of parallel processing vectors Z'=Z for the last time. If no, then the number of parallel processing vectors Z=64 / X, calculate the number of processing times for each thread W=Y / Z. When W is not an integer, round up W to the nearest integer, and determine the number of parallel processing vectors Z'=Y-(W-1)*Z for the last time.
[0047] In this embodiment, when the input length N is very large, the value of Y may be large, resulting in the space of X*Y elements being much larger than 64*4B. That is, when the amount of data NUM processed by each thread exceeds the vector register capacity of each thread 64*4B, i.e., X*Y>64, it is necessary to further group Y so that the number of vectors in each group does not exceed Z, among which the space of X*Z elements is closest to 64*4B.
[0048] S5: Based on the vector length X, the number of vectors processed in parallel Z, the number of vectors processed in the last parallel operation Z', and the number of times each thread processes data W, set the kernel function to start the program maca_kernel.<Float,Y,X> <<<Grid,1024> >> performs reduction operations sequentially on vectors within each thread, between vectors, the number of processing iterations for each thread, and the data for all threads. Here, Y = Z*(W-1) + Z' is used to start the program through a kernel function that includes Y, where the kernel function controls the loop to use these parameters in parallel.
[0049] In one implementation, step S5 specifically includes the following steps, wherein the kernel function starts the program maca_kernel.<Float,Y,X> <<<Grid,1024> The execution logic of >> is as follows.
[0050] S51: Based on the vector length X and the number of vectors processed in parallel Z, perform parallel reduction operations on the data within and between vectors processed in the first W-1 times in each thread. Based on the vector length X and the number of vectors processed in parallel Z' in the last time, perform parallel reduction operations on the data within and between vectors processed in the Wth time in each thread.
[0051] The processing logic for parallel reduction operations on intra-vector and inter-vector data processed in each thread is as follows:
[0052] The pseudocode `index = i * (1024 * Y) + j` is used to index the address of a single thread within the current block. Here, `i` represents the index address of the current block within the grid, and `j` represents the index address of the current thread within the current block.
[0053] The pseudocode `x_vec[ii] = IN_vecX[index + ii * step]` indexes the address of a single vector in each thread. Here, `ii` represents the index of the current vector among all vectors in the current thread, `IN_vecX` represents the current thread's input data `IN` split and loaded according to the vector length `X`, and `step` represents the distance between adjacent vectors in all vectors of the current process.
[0054] The pseudocode `val = op.reduce(val, x_vec[ii].val[jj])` performs parallel reduction operations on the intra-vector and inter-vector data processed in each thread. This function first reduces all elements within a vector, and then reduces the reduced results of all vectors within a thread. Here, `op.reduce` represents the reduction operator, `ii` represents the index of the current vector in the current thread's vectors, and `jj` represents the index of an element within a vector.
[0055] In this scheme, the vector length X represents the number of data points that need to be reduced in each vector. Specifically, "within a vector" means reducing all data within each vector in each thread. "Between vectors" means reducing all vectors that have been reduced in each thread.
[0056] S52: Based on the number of processing times W for each thread, reduce the reduction result of all processing times for each thread again.
[0057] W controls the number of iterations. First, W-1 iterations are performed, with one thread processing Z vectors in parallel each time. Then, the Wth iteration is performed, with one thread processing Z' vectors in parallel each time. Finally, the reduction results of all processing iterations for each thread are reduced again. S53: Based on the fact that a single Warp contains 64 threads, the 1024 threads are sorted and divided into multiple Warp, and the data of all threads within each Warp is reduced.
[0058] The pseudocode `val = WarpReduce(val, op)` sorts 1024 threads and divides them into Warps, then performs a reduction operation within each Warp. Here, `WarpReduce` represents the reduction operator, and `op` represents the reduction operator.
[0059] S54: Perform reduction operations on all Warp data in parallel.
[0060] The pseudocode `shared[k] = op.reduce(val[k], val[k+offset])` performs a parallel reduction operation on the 16 warp data, and the final `shared[0]` is the final reduction result. Here, `k` represents the thread number. `op.reduce` represents the reduction operator, `val` represents the reduction result of the 16 warp data, and `offset` controls the distance between the two warps being reduced.
[0061] The progressive calculation process of Warp is as follows: the 16 Warp values are divided into two groups, with the offset first set to 8; then the offset is set to 4... and finally to 1, reducing the value to 0. This completes the optimization and writing of the entire reduction kernel function on the MuXiXiCloud C500.
[0062] Based on steps S1-S5 above, and considering the characteristics of the Muxi Cloud C500 chip and the MXMACA architecture, for reduction subjects with large input data volumes, the configuration parameters are optimized within 1024 threads in a block according to the data length of the reduction subject. By calculating the amount of data processed in each thread and then dividing the data volume into reasonable vector lengths and number of vectors based on the capacity of each thread, parallel reduction operations with better performance are achieved. This helps researchers quickly optimize existing operators and write new operators on the Muxi Cloud C500 chip, significantly improving kernel function development efficiency. Existing operators refer to currently developed GPU operators; new operators refer to those newly written by researchers in the future.
[0063] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.
[0064] Example 2
[0065] Based on the technical solution of Embodiment 1 above, this embodiment proposes a specific implementation method.
[0066] S1: Obtain the data length N of the subject of the specification to be executed, which is 40960;
[0067] S2: Set the number of threads in the kernel function, block_size, to 1024, and the vector register capacity of each thread in block_size is 64*4B;
[0068] S3: Calculate the number of data items that each thread needs to process: NUM = 40960 / 1024 = 40;
[0069] S4: Determine the number of processing times W, vector length X, and number of parallel processing vectors for each thread based on the NUM.
[0070] First, ensure that X*Y = 40, X is an integer power of 2 and X ≤ 64, and Y takes the smallest integer value that satisfies the conditions. Then the vector length X = 8 and the total number of processed vectors Y = 5.
[0071] Since X*Y≤64 at this time, the number of parallel processing vectors Z=5, and the number of processing times per thread W=1.
[0072] S5: Based on the reduction operation where each thread processes a vector of length 8 once and the number of vectors processed in parallel is 5, set the kernel function to start the program maca_kernel.<Float,Y,X> <<<Grid,1024> >>, performs reduction operations sequentially on the vectors within each thread, between vectors, the number of processing times for each thread, and the data of all threads.
[0073] Example 3
[0074] Based on the technical solution of Embodiment 1 above, this embodiment proposes a specific implementation method.
[0075] S1: Obtain the data length N of the subject of the specification to be executed, which is 102400;
[0076] S2: Set the number of threads in the kernel function, block_size, to 1024, and the vector register capacity of each thread in block_size is 64*4B;
[0077] S3: Calculate the number of data items that each thread needs to process: NUM = 102400 / 1024 = 100;
[0078] S4: Determine the number of processing times W, vector length X, and number of parallel processing vectors for each thread based on the NUM.
[0079] First, ensure that X*Y = 100, X is an integer power of 2 and X ≤ 64, and Y takes the smallest integer value that satisfies the conditions. Then the vector length X = 4 and the total number of processed vectors Y = 25.
[0080] Since X*Y > 64 at this time, the number of parallel processing vectors Z = 64 / 4 = 16, and the number of processing times per thread W = 2. In the first processing, the number of parallel processing vectors per thread is 16, and in the second processing, the number of parallel processing vectors per thread is 9.
[0081] S5: Based on the principle that each thread first processes a reduction operation with a vector length of 4 and 16 vectors processed in parallel, and then processes a reduction operation with a vector length of 4 and 9 vectors processed in parallel, set the kernel function to start the program maca_kernel.<Float,Y,X> <<<Grid,1024> >>, performs reduction operations sequentially on the vectors within each thread, between vectors, the number of processing times for each thread, and the data of all threads.
[0082] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. A reduction-based kernel function optimization method for the Sunyaev-Zel'dovich effect (SZ effect) cloud C500, characterized in that, Includes the following steps: S1: Obtain the data length N of the subject to be executed, and select the subject whose data length N is greater than the preset value to execute the following steps S2-S5; S2: Set the number of threads in the kernel function, block_size, to 1024, and the vector register capacity of each thread in block_size is 64*4B; S3: Calculate the number of data items that each thread needs to process, NUM = N / 1024; S4: Determine the number of processing times W, vector length X, number of parallel processing vectors Z, and number of parallel processing vectors Z' for each thread based on the NUM; S5: Based on the vector length X, the number of vectors processed in parallel Z, the number of vectors processed in parallel Z' in the last time, and the number of times each thread processes data W, set the kernel function startup parameters, and perform reduction operations on the intra-vector, inter-vector, number of times each thread processes data, and data of all threads in sequence.
2. The reduction-class kernel function optimization method for Muxi Cloud C500 according to claim 1, characterized in that, Step S4 includes the following steps: S41: Calculate the vector length X and the total number of vectors Y for each thread, where X*Y=NUM, X is an integer power of 2 and X≤64, and Y takes the smallest integer value that satisfies the conditions; S42: Determine whether X*Y≤64 is true. If yes, then the number of parallel processing vectors Z=Y, the number of processing times for each thread W=1, and determine the number of parallel processing vectors Z'=Z for the last time. If no, then the number of parallel processing vectors Z=64 / X, obtain the number of processing times for each thread W=Y / Z, round up to the nearest integer value for W, and determine the number of parallel processing vectors Z'=Y-(W-1)*Z for the last time.
3. The reduction-class kernel function optimization method for Muxi Cloud C500 according to claim 1, characterized in that, Step S5 includes the following steps: S51: Based on the vector length X and the number of vectors processed in parallel Z, perform parallel reduction operations on the data within and between vectors processed in the first W-1 times in each thread; based on the vector length X and the number of vectors processed in parallel Z' in the last time, perform parallel reduction operations on the data within and between vectors processed in the Wth time in each thread. S52: Based on the number of processing times W of each thread, perform the reduction operation in parallel on the reduction results of all processing times of each thread; S53: Based on the fact that a single Warp contains 64 threads, sort the 1024 threads and divide them into multiple Warp, and perform a reduction operation on the data of all threads in each Warp in parallel; S54: Reduce all Warp data.
4. The reduction-class kernel function optimization method for Muxi Cloud C500 according to claim 3, characterized in that, Parallel reduction operations on intra-vector and inter-vector data processed in each thread include: Index the address of each thread in the current block; Index the address of each vector in each thread; Based on the index address, the data within and between vectors in each thread are reduced in parallel.