Method for optimizing reduction type kernel function of Xichun cloud C500

By optimizing the thread processing of kernel functions and vector length configuration, the problem of time-consuming kernel function optimization parameters and complex parallelization of reduced kernel functions on the MXMACA architecture is solved, and efficient development of kernel functions on Mu Xixiyun C500 chip is realized and Performance improvement.

CN118966322AActive Publication Date: 2024-11-15KYLIN CORP

Patent Information

Application Number
CN202410986611.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15
Estimated Expiration
2044-07-23

AI Technical Summary

Technical Problem

In the MXMACA architecture, the process of optimizing the parameters of the kernel function is long and labor-intensive, especially the parallelization of the reduced kernel function is complex, making it difficult to effectively utilize the computing performance of the Mu Xixiyun C500 chip.

Method used

By obtaining the data length of the subject to execution of the regulations, setting the number of threads and vector register capacity, calculating the number of processing times and vector length of each thread, optimizing the kernel function startup parameters, and using phased parallel processing and vector reduction strategies to ensure data merging and the effectiveness of regulations.

Benefits of technology

It greatly improves the efficiency of kernel function development, simplifies the optimization of existing operators and the writing of new operators, and maximizes the computing performance of Mu Xixiyun C500 chip.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966322A_ABST
    Figure CN118966322A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computers, and particularly provides a reduction-type kernel function optimization method for Xichun cloud C500, which comprises the following steps: acquiring the data length N of a protocol main body to be executed, and selecting the protocol main body with the data length N greater than a preset value to execute the following steps; setting the thread count in the kernel function as 1024; calculating the number NUM of data needing to be processed by each thread; determining the processing times W, the vector length X, the parallel processing vector number Z and the last parallel processing vector number Z'of each thread according to the NUM; and then setting kernel function starting parameters, and sequentially carrying out protocol operation on the internal vector, the inter-vector, the processing times of each thread and the data of all the threads in each thread. By means of the technical scheme, for the input protocol main body with the large data size, optimization of the configuration parameters is achieved in 1024 threads in one block according to the data length of the protocol main body, and the kernel function development efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and specifically provides a reduction-type kernel function optimization method for Muxixi Cloud C500. Background Art

[0002] MXMACA is a computing platform launched by Muxi that uses a general-purpose parallel computing architecture to solve complex computing problems. It includes a self-developed instruction set architecture (ISA) and a parallel computing engine inside the GPU, integrating general computing and machine learning frameworks, and provides a highly flexible and high-performance open computing platform for experts in various application fields.

[0003] The computing platform provides a simple and easy-to-use C-like programming language for users to write programs for the MXMACA architecture, so that it can run at super high efficiency on the METAX GPU processor. The programming language has simple and flexible syntax, is easy to read and write, and has outstanding expressive power; at the same time, it is compatible with mainstream C / C++ heterogeneous computing languages, allowing users to obtain convenient software adaptation and efficient customer operator development.

[0004] As a key component in deep learning algorithms, kernel functions need to meet specific mathematical conditions to ensure the effectiveness of the algorithm. Writing kernel functions based on the MXMACA architecture is a challenging task that requires consideration of symmetry, semi-positive definiteness, parameter selection, computational complexity, generalization ability, and interpretability to fully utilize the hardware parallel acceleration performance of the MXMACA computing platform.

[0005] The current kernel function writing and optimization methods are all applicable to the NVIDIA CUDA ecosystem. To write and optimize kernel functions based on the MXMACA architecture, researchers must conduct in-depth research and continuous practice to gradually improve the writing quality and performance of kernel functions, otherwise the computing performance of the MXMACA computing platform cannot be maximized. This requires researchers to have a deep background in mathematics and statistics to understand and build appropriate kernel functions, and continuous iterative testing is required to finally determine the kernel function optimization parameters. The process is long and labor-intensive.

[0006] Among them, reduction kernel functions involve performing some kind of aggregation operation on a set of data, such as summation, maximum value, minimum value, product, etc. The characteristic of this type of kernel function is that it needs to gradually merge a large number of intermediate results of independent calculations into a final result. Typical reduction operations include: global / local summation: calculating the sum of all elements in an array or calculating the sum of a continuous sub-interval in an array; maximum / minimum search: finding the maximum or minimum value in an array or finding the maximum or minimum value in a sub-interval; average calculation: calculating the average value of array elements, usually it is necessary to first sum and then divide by the number of elements.

[0007] The parallelization of reduction kernel functions is relatively complex. It is necessary to consider how to effectively distribute the computing tasks among multiple threads and design appropriate reduction strategies (such as staged reduction, tree reduction, etc.) to merge intermediate results while avoiding data contention and ensuring correctness. Summary of the invention

[0008] In order to overcome the above-mentioned defects, the present invention is proposed to solve the problem that when processing reduction-type kernel functions on the Muxixiyun C500 chip, not only the process of determining the kernel function optimization parameters in the MXMACA architecture is long and labor-intensive, but also the technical problem of complex parallelization of the reduction is faced.

[0009] The present invention provides a reduction-like kernel function optimization method for Muxixi Cloud C500, comprising the following steps:

[0010] S1: Obtain the data length N of the protocol body to be executed, and select the protocol body whose data length N is greater than a preset value to execute the following steps S2-S5;

[0011] S2: Set the number of threads in the kernel function block_size to 1024, and the vector register capacity of each thread in block_size is 64*4B;

[0012] S3: Calculate the number of data that each thread needs to process, NUM = N / 1024;

[0013] S4: Determine the number of processing times W, the vector length X, the number of parallel processing vectors Z and the last parallel processing vector number Z' of each thread according to the NUM;

[0014] S5: according to the vector length X, the number of parallel processing vectors Z, the number of vectors processed in parallel for the last time Z' and the number of processing times of each thread W, set the kernel function startup parameters, and perform reduction operations on the vectors within each thread, between vectors, the number of processing times of each thread, and the data of all threads in turn.

[0015] Further, the step S4 comprises the steps of:

[0016] S41: Calculate the vector length X of each thread and the total number of vectors Y, where X*Y=NUM, X is an integer power of 2 and X≤64, and Y takes the minimum integer value that satisfies the condition;

[0017] S42: Determine whether X*Y≤64 holds. If so, the number of parallel processing vectors Z=Y, the number of processing times of each thread W=1, and the last number of parallel processing vectors Z'=Z is determined; if not, the number of parallel processing vectors Z=64 / X, and the number of processing times of each thread W=Y / Z is obtained, W takes an integer value according to the rounding method, and the last number of parallel processing vectors Z'=Y-(W-1)*Z is determined.

[0018] Further, the step S5 comprises the steps of:

[0019] S51: performing reduction operations on the data in the vectors and between the vectors processed for the first W-1 times in each thread according to the vector length X and the number of vectors processed in parallel Z, and performing reduction operations on the data in the vectors and between the vectors processed for the Wth time in each thread according to the vector length X and the number of vectors processed in parallel Z' for the last time;

[0020] S52: according to the processing times W of each thread, the reduction results of all the processing times of each thread are reduced in parallel;

[0021] S53: Since a single Warp contains 64 threads, the 1024 threads are sorted and divided into multiple Warps, and the data of all threads in each Warp are reduced in parallel;

[0022] S54: Perform reduction operation on all Warp data.

[0023] Furthermore, the reduction operation is performed in parallel on the data in the vector and between the vectors processed each time in each thread, including:

[0024] Index the address of each thread in the current block;

[0025] Index the address of each vector in each thread;

[0026] The intra-vector and inter-vector data in each thread are reduced in parallel according to the index address.

[0027] The working principle and beneficial effects of the present invention are:

[0028] In the technical solution for implementing the present invention, according to the characteristics of the Muxixiyun C500 chip and the MXMACA architecture, for the specification body with a large amount of input data, the configuration parameters are optimized according to the data length of the specification body in 1024 threads in a block. By calculating the amount of data processed in each thread, and then dividing the data amount into reasonable vector lengths and vector numbers according to the capacity of each thread, better parallel reduction operations are achieved, which helps R&D personnel to quickly perform reduction optimization of existing operators and write new operators on the Muxixiyun C500 chip, greatly improving the efficiency of kernel function development. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which:

[0030] Figure 1 It is a flow chart of the main steps of a reduction-type kernel function optimization method for Muxixi Cloud C500 of the present invention. DETAILED DESCRIPTION

[0031] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0032] The present invention optimizes the kernel function of the Muxixiyun C500 chip. Here, the Muxixiyun C500 architecture is first compared with the NVIDIA A100 Ampere architecture to find the key performance bottleneck and verify the worst time-consuming hardware unit.

[0033] NVIDIA A100 Ampere architecture consists of a GPU processing cluster and 108 SM stream multiprocessors. SM stream multiprocessors are the core computing units of NVIDIA GPUs. Each SM stream multiprocessor contains a large number of CUDA cores, which are the smallest units for performing general parallel computing tasks. A100's GPU processing cluster contains 6912 32-bit floating-point units, 6912 32-bit integer units, 3456 64-bit floating-point units, 432 tensor units, and a pair of memory controllers controlling an HBM2 DRAM stack.

[0034] The computing core of the MXMACA C500 architecture (hardware) consists of a data processing cluster DPC, an acceleration processor AP, a processing element unit PEU, and a matrix computing unit. Each PEU supports multiple parallel waves, and each acceleration processor AP can accommodate up to 2048 threads. The vector register capacity of each AP is 2048*64*4B. The MXMACA software stack architecture is divided into blocks and warps, which are distributed on the acceleration processor AP. The grid_size determines how many blocks there are on all APs. The block is divided into multiple warps. Each block can accommodate up to 1024 threads, and a single warp contains 64 threads.

[0035] Example 1

[0036] Figure 1 This is a schematic flow chart of the main steps of a reduction kernel function optimization method for Muxixi Cloud C500 of the present invention. Figure 1 As shown, a reduction-like kernel function optimization method for Muxixi Cloud C500 in this embodiment mainly includes the following steps S1 to S5.

[0037] S1: Obtain the data length N of the protocol body to be executed, and select the protocol body whose data length N is greater than a preset value to execute the following steps S2-S5.

[0038] MXMACA software stack architecture generally uses maca_kernel<<<grid_size,block_size,0,stream> >>(N,...) method to start a kernel function. When the maca_kernel function is defined as a reduced calculation, the input data length, that is, the data volume N, is obtained.

[0039] In practical applications, if the N value is small, it does not have optimization significance, and Xiyun C500 can continue to use the startup program to perform the reduction calculation; if the amount of data processed N is large, since the startup program is a general program of the MXMACA software stack architecture, according to the configuration parameters grid_size (grid dimension and size) and block_size (block dimension and size) in the startup program, the execution effect of the reduction operation of big data on Xiyun C500 is not good. Therefore, the present invention optimizes the configuration parameters of the startup kernel function according to the input data length N and the Muxixiyun C500 architecture.

[0040] S2: Set the number of threads in the kernel function block_size to 1024, and the vector register capacity of each thread in block_size is 64*4B.

[0041] In this embodiment, the preset value in step S1 is at least a value greater than 1024, because a block includes 1024 threads. For the protocol body with a data length N less than 1024, each thread executes one data, and the entire protocol operation can be completed in parallel. In addition, because each AP in the Xiyun C500 hardware structure includes 2048 threads, and the vector register capacity of each AP is 2048*64*4B, because the capacity of each thread is 64*4B, when a virtual block is created on the AP, the capacity of each thread is 64*4B. For each data in the protocol body, the size is 4B. S3: Calculate the number of data that each thread needs to process NUM=N / 1024.

[0042] In this embodiment, the input length of the current large language model is generally 4096 or 8192, and the data length N of the protocol body for general calculation is a multiple of 1024.

[0043] S4: Determine the number of processing times W, the vector length X, the number of parallel processing vectors Z and the number of the last parallel processing vectors Z' of each thread according to the NUM.

[0044] Furthermore, the specific operation steps of step S4 include the following steps S41-S42.

[0045] S41: Calculate the vector length X of each thread and the total number of vectors Y, where X*Y=NUM, X is an integer power of 2 and X≤64, and Y takes the minimum integer value that meets the conditions.

[0046] S42: Determine whether X*Y≤64 holds. If so, the number of parallel processing vectors Z=Y, the number of processing times of each thread W=1, and the last number of parallel processing vectors Z'=Z is determined; if not, the number of parallel processing vectors Z=64 / X, the number of processing times of each thread W=Y / Z is calculated, when W is not an integer, W takes an integer value according to the rounding method, and the last number of parallel processing vectors Z'=Y-(W-1)*Z is determined.

[0047] In this embodiment, when the input length N is very large, the value of Y may be large, resulting in the X*Y element space being much larger than 64*4B. That is, when the amount of data processed by each thread NUM exceeds the vector register capacity of each thread 64*4B, that is, X*Y>64, it is necessary to group Y again so that the number of vectors in each group does not exceed Z, and the X*Z element space is closest to 64*4B.

[0048] S5: Set the kernel function startup program maca_kernel according to the vector length X, the number of parallel processing vectors Z, the number of vectors processed in the last parallel processing Z', and the number of processing times of each thread W.<Float,Y,X> <<<Grid,1024> >>, perform reduction operations on the vectors in each thread, between vectors, the number of times each thread is processed, and the data of all threads in turn. Where Y = Z*(W-1)+Z', start the program through the kernel function including Y, and control the loop in the kernel function to use these parameters in parallel.

[0049] In one embodiment, step S5 specifically includes the following steps, wherein the kernel function starts the program maca_kernel<Float,Y,X> <<<Grid,1024> >>The execution logic is as follows.

[0050] S51: Perform reduction operations on the data in the vectors processed for the first W-1 times and between vectors in each thread in parallel according to the vector length X and the number of vectors processed in parallel Z, and perform reduction operations on the data in the vectors processed for the Wth time and between vectors in each thread in parallel according to the vector length X and the number of vectors processed in parallel Z' for the last time.

[0051] The processing logic for parallel reduction operations on data within and between vectors processed in each thread is as follows:

[0052] The address of a single thread in the current block is indexed by the pseudo code index=i*(1024*Y)+j, where i represents the index address of the current block in the grid, and j represents the index address of the current thread in the current block.

[0053] The address of a single vector in each thread is indexed by the pseudo code x_vec[ii]=IN_vecX[index+ii*step], where ii represents the sequence number of the current vector in all vectors of the current thread, IN_vecX represents the current thread input data IN divided and loaded according to the vector length X, and step represents the distance between adjacent vectors in all vectors of the current process.

[0054] Through the pseudo code val = op.reduce(val,x_vec[ii].val[jj]), the vectors and inter-vectors processed in each thread are reduced in parallel. The meaning of this function is to first reduce all elements in a vector, and then reduce the results of all vectors reduced in a thread. Among them, op.reduce represents the reduction operator, ii represents the sequence number of the current vector in all vectors of the current thread, and jj represents the sequence number of elements in a vector.

[0055] The vector length X in this solution means that each vector includes X data that need to be reduced, that is, within the vector means that all data in each vector in each thread are reduced. Between vectors means that all reduced vectors in each thread are reduced.

[0056] S52: According to the processing times W of each thread, the reduction results of all processing times of each thread are reduced again.

[0057] W controls the number of loops. First, W-1 loops are performed, and one thread processes Z vectors in parallel each time. Then, the Wth processing is performed, and one thread processes Z' vectors in parallel each time. Finally, the reduction results of all processing times of each thread are reduced again. S53: According to the fact that a single Warp contains 64 threads, 1024 threads are sorted and divided into multiple Warps, and the data of all threads in each Warp are reduced;

[0058] The pseudo code val=WarpReduce(val,op) is used to sort the 1024 threads and divide them into warps, and a reduction operation is performed in each warp, where WarpReduce represents the reduction operator and op represents the reduction operator.

[0059] S54: Perform reduction operations on all Warp data in parallel.

[0060] The above 16 warp data are reduced in parallel through the pseudo code shared[k]=op.reduce(val[k],val[k+offset]), and the final shared[0] is the final reduction result. Among them, k represents the thread number. Among them, op.reduce represents the reduction operator, val represents the reduction result of the upper 16 warps, and offset controls the distance between the two warps being reduced.

[0061] The progressive process of obtaining the value of Warp is that 16 Warps are divided into two groups, and the offset is first taken as 8; then the offset is taken as 4... and finally 1, reducing the value to 0. So far, the optimization and writing of the entire reduction kernel function in Muxixiyun C500 is completed.

[0062] Based on the above steps S1-S5, according to the characteristics of the Muxixiyun C500 chip and the MXMACA architecture, for the specification body with a large amount of input data, the configuration parameters are optimized according to the data length of the specification body in the 1024 threads in a block. By calculating the amount of data processed in each thread, and then dividing the data amount according to the capacity of each thread into a reasonable vector length and vector number, a better parallel specification operation is achieved, which helps R&D personnel to quickly perform reduction optimization of existing operators and write new operators in the Muxixiyun C500 chip, greatly improving the efficiency of kernel function development. Existing operators refer to GPU operators that have been developed and exist; new operators refer to those newly written by researchers in the future.

[0063] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0064] Example 2

[0065] According to the technical solution of the above-mentioned embodiment 1, this embodiment proposes a specific implementation method.

[0066] S1: Get the data length N of the protocol body to be executed, which is 40960;

[0067] S2: Set the number of threads in the kernel function block_size to 1024, and the vector register capacity of each thread in block_size is 64*4B;

[0068] S3: Calculate the number of data that each thread needs to process NUM = 40960 / 1024 = 40;

[0069] S4: Determine the number of processing times W, the vector length X, and the number of parallel processing vectors Z for each thread according to the NUM.

[0070] First, ensure that X*Y=40, X is an integer power of 2 and X≤64, and Y takes the minimum integer value that satisfies the conditions, then the vector length X=8 and the total number of processed vectors Y=5.

[0071] Since X*Y≤64 at this time, the number of parallel processing vectors Z=5 is calculated, and the number of processing times of each thread W=1.

[0072] S5: According to the reduction operation that each thread processes a vector length of 8 and the number of parallel processing vectors is 5, set the kernel function startup program maca_kernel<Float,Y,X> <<<Grid,1024> >>, perform reduction operations on the vectors within each thread, between vectors, the number of times each thread is processed, and the data of all threads in turn.

[0073] Example 3

[0074] According to the technical solution of the above-mentioned embodiment 1, this embodiment proposes a specific implementation method.

[0075] S1: Get the data length N of the protocol body to be executed, which is 102400;

[0076] S2: Set the number of threads in the kernel function block_size to 1024, and the vector register capacity of each thread in block_size is 64*4B;

[0077] S3: Calculate the number of data that each thread needs to process NUM = 102400 / 1024 = 100;

[0078] S4: Determine the number of processing times W, the vector length X, and the number of parallel processing vectors Z for each thread according to the NUM.

[0079] First, ensure that X*Y=100, X is an integer power of 2 and X≤64, and Y takes the minimum integer value that satisfies the conditions, then the vector length X=4 and the total number of processed vectors Y=25.

[0080] Since X*Y>64 at this time, the number of parallel processing vectors Z=64 / 4=16 is calculated, and the number of processing times of each thread W=2. In the first processing, the number of parallel processing vectors of each thread is 16, and in the second processing, the number of parallel processing vectors of each thread is 9.

[0081] S5: Each thread first processes a reduction operation with a vector length of 4 and a number of parallel processing vectors of 16, and then processes a reduction operation with a vector length of 4 and a number of parallel processing vectors of 9, and sets the kernel function startup program maca_kernel<Float,Y,X> <<<Grid,1024> >>, perform reduction operations on the vectors within each thread, between vectors, the number of times each thread is processed, and the data of all threads in turn.

[0082] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A reduction kernel function optimization method for Muxixi Cloud C500, It is characterized in that The following steps are involved: S1: Obtain the data length N of the protocol body to be executed, and select the protocol body whose data length N is greater than a preset value to execute the following steps S2-S5; S2: Set the number of threads in the kernel function block_size to 1024, and the vector register capacity of each thread in block_size is 64*4B; S3: Calculate the number of data that each thread needs to process, NUM = N / 1024; S4: Determine the number of processing times W, the vector length X, the number of parallel processing vectors Z and the last parallel processing vector number Z' of each thread according to the NUM; S5: according to the vector length X, the number of parallel processing vectors Z, the number of vectors processed in parallel for the last time Z' and the number of processing times of each thread W, set the kernel function startup parameters, and perform reduction operations on the vectors within each thread, between vectors, the number of processing times of each thread, and the data of all threads in turn.

2. According to claim 1, a reduction kernel function optimization method for Muxixi Cloud C500, It is characterized in that The step S4 comprises the steps of: S41: Calculate the vector length X of each thread and the total number of vectors Y, where X*Y=NUM, X is an integer power of 2 and X≤64, and Y takes the minimum integer value that satisfies the condition; S42: Determine whether X*Y≤64 holds. If so, the number of parallel processing vectors Z=Y, the number of processing times of each thread W=1, and the last number of parallel processing vectors Z'=Z is determined; if not, the number of parallel processing vectors Z=64 / X, and the number of processing times of each thread W=Y / Z is obtained, W takes an integer value according to the rounding method, and the last number of parallel processing vectors Z'=Y-(W-1)*Z is determined.

3. According to claim 1, a reduction-like kernel function optimization method for Muxixi Cloud C500, It is characterized in that The step S5 comprises the steps of: S51: performing reduction operations on the data in the vectors and between the vectors processed for the first W-1 times in each thread according to the vector length X and the number of vectors processed in parallel Z, and performing reduction operations on the data in the vectors and between the vectors processed for the Wth time in each thread according to the vector length X and the number of vectors processed in parallel Z' for the last time; S52: according to the processing times W of each thread, the reduction results of all the processing times of each thread are reduced in parallel; S53: Since a single Warp contains 64 threads, the 1024 threads are sorted and divided into multiple Warps, and the data of all threads in each Warp are reduced in parallel; S54: Perform reduction operation on all Warp data.

4. According to claim 3, a reduction kernel function optimization method for Muxixi Cloud C500, It is characterized in that The reduction operations for data within and between vectors processed in each thread in parallel include: Index the address of each thread in the current block; Index the address of each vector in each thread; The intra-vector and inter-vector data in each thread are reduced in parallel according to the index address.

Citation Information

Patent Citations

  • GPU architecture-based self-adaption recursive least squares clutter cancellation method

    CN103226193A

  • GPU parallel acceleration method for solving phononic Boltzmann equation based on finite volume method

    CN110275733A

  • Parallelization signal quality evaluation method

    CN111624631A

  • Uniform interface, server and calling method for runtime of artificial intelligence framework

    CN117648211A

  • Image processing method and apparatus, and electronic device and storage medium

    WO2023279739A1

Cited By

  • Method and system for identifying two-dimensional structure of tropical external cyclone

    CN119851096A