Assembly line GEMV calculation and Bayesian optimization tuning method based on CamconMLU platform

Abstract

The invention relates to a pipeline GEMV calculation and Bayesian optimization tuning method based on a CamconMLU platform, and the method comprises the steps: taking a GEMV kernel execution performance index as a target function in a GEMV calculation process, and carrying out the iterative calculation of an optimal parameter combination enabling the function value of the target function to be maximized through a Bayesian optimization algorithm; the optimal parameter combination comprises a target row block number, a target floating point block size and a target buffer area number; the size of the target floating point block represents column dimension division granularity; segmenting the matrix subjected to GEMV calculation according to a target row block number and a target floating point block size to obtain a plurality of block data, and setting double buffer areas on an on-chip memory of the MLU chip based on a target buffer area number; and sequentially carrying the multiple pieces of block data to the double buffer regions and executing calculation operation, and simultaneously carrying the block data needing to be calculated next time to the other buffer region in parallel while executing the calculation operation on the carried block data in one buffer region in the double buffer regions. According to the method, the GEMV calculation efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of high-performance computing and artificial intelligence chip optimization technology, and more specifically, to a pipelined GEMV computation and Bayesian optimization tuning method based on the CambriconMLU platform. Background Technology

[0002] General Matrix Vector Multiplication (GEMV) is one of the core operators in scientific computing, machine learning, and deep learning. In applications such as solving sparse linear equations, graph computation, recommender systems, and forward and backward propagation in neural networks, GEMV operations often account for the majority of overall computation time and energy consumption. With the development of artificial intelligence and large-scale scientific computing, higher demands are being placed on the performance and energy efficiency of GEMV operations.

[0003] On traditional general-purpose CPUs, GEMV primarily relies on cache hierarchy and SIMD instructions for optimization. On GPU platforms, GEMV often improves efficiency through large-scale thread parallelism, shared memory caches, and warp-level instruction scheduling. However, dedicated AI accelerator chips like Cambricon MLU (designed and manufactured by Cambricon Technologies specifically for accelerating AI computing) employ an architecture different from GPUs: their on-chip memory (NRAM, SRAM) has limited capacity, and data is frequently moved between HBM (High Bandwidth Memory) and on-chip memory; explicit management is required between the memory hierarchy and operator execution, and improper handling can lead to memory access latency and idle computing units.

[0004] Therefore, optimizing the GEMV computation efficiency of the Cambricon MLU platform is crucial. Summary of the Invention

[0005] Therefore, it is necessary to provide a pipelined GEMV computation and Bayesian optimization tuning method, system, computer equipment, computer-readable storage medium, and computer program product based on the CambriconMLU platform that can improve computational efficiency in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a pipelined GEMV computation and Bayesian optimization tuning method based on the CambriconMLU platform, which includes:

[0007] In the matrix-vector multiplication GEMV calculation process, the GEMV kernel execution performance index is used as the objective function, and the optimal parameter combination that maximizes the function value of the objective function is calculated iteratively using a Bayesian optimization algorithm. The parameters in the optimal parameter combination include the target number of row blocks, the target floating-point block size, and the target number of buffers. The target floating-point block size is used to represent the granularity of column dimension partitioning.

[0008] The matrix for GEMV calculation is divided into multiple data blocks according to the target number of row blocks and the target floating-point block size, and a double buffer is set on the on-chip memory of the MLU chip based on the target number of buffers.

[0009] The multiple data blocks are sequentially moved to the dual buffer and computation is performed. While performing computation on the moved data blocks in one of the dual buffers, the next data blocks to be computed are moved to the other buffer in parallel.

[0010] In one embodiment, the dual buffer includes a computation buffer and a data buffer; the step of performing computation operations on the transferred data chunks in one buffer of the dual buffer while simultaneously transferring the next batch of data chunks to be computed to the other buffer includes:

[0011] While performing calculation operations on the block data moved to the calculation buffer, the next block data is moved to the data buffer in parallel. After the moving is completed, a ready flag is added to the data buffer.

[0012] In the next calculation, the data buffer with the ready flag is switched to the calculation buffer to perform the calculation operation, and the calculation buffer from the previous calculation is switched to the data buffer to perform the data transfer operation, so as to continue transferring the block data.

[0013] In one embodiment, the parameters in the optimal parameter combination also include a target cycle expansion factor;

[0014] The method further includes:

[0015] During the computation operation on the block data in one of the double buffers, multiple elements in each row of the block data are expanded based on the target loop expansion factor to form a multi-parallel multiply-accumulate chain for computation.

[0016] In one embodiment, expanding multiple elements in each row of the segmented data based on a target loop expansion factor to form a multi-parallel multiply-accumulate chain for computation includes:

[0017] If the number of columns in the block data is not divisible by the alignment width of the target loop expansion factor, the block data is padded with 0s so that the number of columns in the padded block data is an integer multiple of the target loop expansion factor; multiple elements in each row of the padded block data are expanded based on the target loop expansion factor and multiplied and added together.

[0018] or,

[0019] The block data is split into a first sub-data block and a second sub-data block by columns, wherein the number of columns in the first sub-data block is divisible by the alignment width of the target loop expansion factor, and the number of columns in the second sub-data block is not divisible by the alignment width of the target loop expansion factor; the second sub-data block is padded with 0s so that the number of columns in the padded data block is an integer multiple of the target loop expansion factor; both the first sub-data block and the padded data block are expanded based on the target loop expansion factor and subjected to multiplication-addition accumulation calculation.

[0020] In one embodiment, corresponding GEMV kernel binary files were pre-compiled for each of the multiple preset parameter combinations;

[0021] The step of dividing the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size to obtain multiple data blocks, and setting up a double buffer on the on-chip memory of the MLU chip based on the target number of buffers includes:

[0022] The pre-compiled target GEMV kernel binary file corresponding to the optimal parameter combination is loaded using the dynamic link library loading interface, and the target GEMV kernel entry function is parsed from the target GEMV kernel binary file;

[0023] The target GEMV kernel entry function is called to divide the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size to obtain multiple data blocks, and a double buffer is set on the on-chip memory of the MLU chip based on the target number of buffers.

[0024] In one embodiment, the target row block size is controlled by a first parameter SRAM_BLOCK_ROWS, which limits the number of rows of the matrix block loaded into the SRAM of the MLU chip in each computation batch; the target floating-point block size is controlled by a second parameter NRAM_CHUNK_FLOATS, which limits the number of floating-point blocks moved into the NRAM of the MLU chip each time.

[0025] In one embodiment, the step of iteratively calculating the optimal parameter combination that maximizes the function value of the objective function using a Bayesian optimization algorithm includes:

[0026] Establish a set of values ​​corresponding to multiple preset parameter items;

[0027] Multiple sets of candidate parameter combinations are generated based on the aforementioned value sets;

[0028] The Bayesian optimization algorithm is used to iteratively calculate the function value of the objective function based on each set of candidate parameter combinations until the iteration stopping condition is met. The candidate parameter combination that maximizes the function value of the objective function is determined as the optimal parameter combination.

[0029] Secondly, this application provides a pipelined GEMV computation and Bayesian optimization tuning system based on the CambriconMLU platform, the system comprising:

[0030] The optimal combination determination module is used in the matrix-vector multiplication GEMV calculation process to iteratively calculate the optimal parameter combination that maximizes the function value of the objective function by using the GEMV kernel execution performance index as the objective function and employing a Bayesian optimization algorithm. The parameters in the optimal parameter combination include the target number of row blocks, the target floating-point block size, and the target number of buffers. The target floating-point block size is used to represent the granularity of column dimension partitioning.

[0031] The partitioning module is used to divide the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size to obtain multiple data blocks, and to set up a double buffer on the on-chip memory of the MLU chip based on the target number of buffers.

[0032] The calculation module is used to sequentially move the multiple data blocks to the dual buffer and perform calculation operations, wherein while performing calculation operations on the moved data blocks in one of the dual buffers, the next data blocks to be calculated are moved to the other buffer in parallel.

[0033] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the embodiments of this application.

[0034] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in the embodiments of this application.

[0035] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the embodiments of this application.

[0036] The aforementioned pipelined GEMV computation and Bayesian optimization tuning methods, systems, computer equipment, computer-readable storage media, and computer program products based on the Cambricon MLU platform introduce intelligent parameter optimization processing at the upper layer of the Cambricon MLU platform. Specifically, for GEMV computation, the optimal kernel configuration (i.e., the optimal parameter combination for optimal performance) is automatically, quickly, and accurately explored through Bayesian optimization algorithms, achieving adaptive performance improvement for GEMV computation. This allows the kernel to maintain efficient, stable, and adaptive operation under different matrix sizes and hardware resource constraints. Furthermore, at the lower level of the Cambricon MLU platform, a pipelined design for on-chip storage (NRAM / SRAM) is constructed for GEMV computation. This involves splitting and transporting the GEMV computation matrix into row blocks and floating-point blocks based on the optimal parameter combination, utilizing a Ping-Pong (double buffering) mechanism to achieve overlap between data transport and computation. This effectively solves the problems of memory access latency and idle computation units, achieving deep overlap between memory access and computation, significantly improving GEMV kernel throughput and computational efficiency. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the pipelined GEMV computation and Bayesian optimization tuning method based on the CambriconMLU platform in one embodiment.

[0038] Figure 2 This is a block diagram of a pipelined GEMV computation and Bayesian optimization tuning system based on the CambriconMLU platform in one embodiment.

[0039] Figure 3 This is a structural block diagram of the partitioning module in one embodiment;

[0040] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] It should be understood that, unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by those skilled in the art. The technical terms used herein are for the purpose of describing specific examples only and are not intended to limit the scope of this application.

[0043] In some solutions, the NRAM block size, SRAM row block number, and cycle expansion factor are manually specified, and adjusted empirically to suit matrices of different sizes. However, due to the large parameter space and complex hardware characteristics, manual parameter tuning is time-consuming and difficult to achieve optimal results. Moreover, it is difficult to apply in actual engineering deployments.

[0044] Some solutions rely on compiler-based automatic vectorization and scheduling optimization, but on dedicated AI acceleration chips like CambriconMLU, compiler optimization is not ideal.

[0045] Based on this, a novel scheme is proposed in this application embodiment, which can effectively optimize and improve the computational efficiency of GEMV. See the description below for details.

[0046] like Figure 1 As shown, in one embodiment, a pipelined GEMV calculation and Bayesian optimization tuning method based on the CambriconMLU platform is provided. This method is executed by a computer device and specifically includes the following steps:

[0047] S11, in the matrix-vector multiplication GEMV calculation process, the GEMV kernel execution performance index is used as the objective function, and the optimal parameter combination that maximizes the function value of the objective function is calculated iteratively using the Bayesian optimization algorithm; the parameters in the optimal parameter combination include the target number of row blocks, the target floating-point block size, and the target number of buffers; the target floating-point block size is used to represent the granularity of column dimension partitioning.

[0048] The GEMV kernel execution performance metrics refer to quantitative indicators that measure the operational efficiency or performance of the GEMV kernel. The GEMV kernel refers to the core computational code segment that executes GEMV operations; it can also be called the computational kernel that implements GEMV operations. It should be understood that "number of line blocks, floating-point block size, number of buffers, and loop unrolling factor" are all configurable parameters of the GEMV kernel, and these parameters directly determine the operational efficiency or performance of the GEMV kernel (i.e., GEMV operation performance). Therefore, a Bayesian optimization algorithm is used to iteratively calculate the optimal parameter combination to optimize GEMV operation performance.

[0049] In some embodiments, the optimal combination of parameters that maximizes the objective function is calculated iteratively using a Bayesian optimization algorithm, including:

[0050] (1) Establish a set of values ​​corresponding to multiple preset parameter items.

[0051] The preset parameter items may include at least one of the following: number of line blocks, floating-point block size, and number of buffers. For example, the preset parameter items may also include a loop expansion factor.

[0052] Each preset parameter has a corresponding set of values. For example, the number of line blocks ∈ {16,32,64,128,256}, the size of the floating-point block ∈ {64,128,256,512}, the number of buffers ∈ {1,2}, and the loop expansion factor ∈ {1,2,4,8}.

[0053] It should be noted that candidate values ​​for parameters that do not meet on-chip capacity, alignment, or ABI constraints can be directly determined as infeasible and will not be added to the corresponding value set for subsequent evaluation.

[0054] (2) Generate multiple sets of candidate parameter combinations based on each set of values.

[0055] Each candidate parameter combination has values ​​corresponding to each preset parameter item. The values ​​of each preset parameter item are not exactly the same in different candidate parameter combinations. Multiple candidate parameter combinations are equivalent to multiple preset parameter combinations, and a Bayesian optimization algorithm is needed to select the optimal parameter combination from them.

[0056] (3) Using the Bayesian optimization algorithm, the function value of the objective function is calculated iteratively based on each set of candidate parameter combinations until the iteration stopping condition is met. The candidate parameter combination that maximizes the function value of the objective function is determined as the optimal parameter combination.

[0057] It should be understood that multiple sets of candidate parameter combinations constitute a parameter search space, from which a Bayesian optimization algorithm can be used to automatically search for approximately optimal parameter combinations. Specifically, a Bayesian optimization method can be used as an intelligent optimization algorithm. With the support of a surrogate model and a data acquisition function, it can automatically explore the optimal parameter combination from multiple sets of candidate parameter combinations in the parameter search space, and approximate the globally optimal configuration—that is, the optimal parameter combination—within a finite number of iterations. This achieves adaptive improvement and efficient tuning of GEMV computational performance, replacing manual exhaustive search and experience-based parameter tuning, significantly reducing trial-and-error costs and improving cross-scenario performance stability.

[0058] In this model, a surrogate model is used to predict the performance of different candidate parameter combinations (i.e., predict the function value of the objective function), while a data acquisition function is used to strike a balance between exploring new parameters and utilizing existing optimal solutions, thereby finding the optimal parameter configuration (i.e., finding the optimal parameter combination that maximizes the function value of the objective function) within a limited number of evaluations. In some examples, a Gaussian process (GP) can be used as a surrogate model to characterize the posterior distribution of the "parameter-performance" relationship, and Expected Improvement can be chosen as the data acquisition function to achieve a balance between exploring unknown regions and utilizing the current best solution.

[0059] Specifically, the proxy model can be initialized first. For example, a small number of candidate parameter combinations can be randomly selected. For each candidate parameter combination, it can be submitted to the runtime compilation and loading module via an external evaluation callback. The function value of the corresponding objective function (i.e., the measured GEMV kernel execution performance index value) can be obtained through parsing the standard output. The proxy model is then trained based on each set of candidate parameter combinations and their corresponding function values.

[0060] Then, the predicted function value (i.e. the function value of the objective function when using each candidate parameter combination) and the corresponding uncertainty metric value are determined by the trained surrogate model.

[0061] Next, the acquisition function dynamically selects the next set of candidate parameter combinations based on the prediction function value and uncertainty metric value corresponding to each candidate parameter combination.

[0062] Furthermore, by submitting the candidate parameter combination for the selected collection function to the runtime compilation and loading module via an external evaluation callback, the actual function value of the target function obtained using this candidate parameter combination (i.e., the measured GEMV kernel execution performance index value) is obtained through standard output parsing. Then, the newly selected candidate parameter combination and its corresponding function value are used as new training data to update the surrogate model's hyperparameters and posterior distribution. Based on the updated surrogate model, the predicted function values ​​corresponding to each candidate parameter combination are re-predicted, and subsequent steps are executed. This iterative process continues until the iteration stopping condition is met. When the iteration stopping condition is met, the candidate parameter combination that maximizes the target function value (i.e., maximizes the GEMV kernel execution performance index value) is determined to be the optimal parameter combination. For example, the iteration stopping condition may include the current iteration count meeting a pre-set maximum iteration count, or the surrogate model converging.

[0063] It should be understood that for hard boundaries such as capacity constraints and alignment constraints, feasibility filtering can be set up at the outer layer of the optimizer or constraint processing can be introduced in the acquisition function, thereby limiting the optimization process to an executable parameter subspace.

[0064] It's important to note that Bayesian optimization algorithms incorporate data from real-world operating environments to search for and predict the optimal parameter combination, making them more accurate than static methods that predict the optimal parameter combination. This is because real-world operating environments contain interference such as memory access jitter and inter-thread interference; simply deriving the optimal parameter combination statically based on an existing model would result in a significant discrepancy between the actual performance and the actual results, leading to inaccuracies.

[0065] In some examples, the GFLOPS (GigaFloatingPointOperationsPerSecond, or one billion floating-point operations per second) of the GEMV kernel is used as the objective function. The GFLOPS of a GEMV kernel refers to the number of one billion floating-point operations that the GEMV kernel (i.e., the highly optimized GEMV computation code) can perform per second. A higher GFLOPS value means that the GEMV kernel executes faster and has better performance. Therefore, for a selected candidate parameter combination (e.g., a candidate parameter combination selected by the acquisition function or a candidate parameter combination randomly selected during the initialization of the surrogate model), this candidate parameter combination can be submitted to the runtime compilation and loading module via an external evaluation callback. The actual GFLOPS value obtained using this candidate parameter combination is obtained through standard output parsing (i.e., the GFLOPS value corresponding to the GEMV kernel performing GEMV computation based on this candidate parameter combination). This newly selected candidate parameter combination and its corresponding GFLOPS value are then added to the training data set of the surrogate model.

[0066] When calculating GFLOPS, the timing method can be specified as the average of the start and end times of the device-side queue notification. Specifically, the kernel floating-point operation load can be determined; that is, first calculate the total number of floating-point operations (in FLOPs) required for the GEMV kernel to perform one GEMV calculation based on this candidate parameter combination. Then, through the device-side queue notification, run the GEMV kernel multiple times and record the execution time each time to calculate the average execution time. For example, the execution time of the first cold start (first run) can be excluded. Finally, the average execution time is calculated as "total floating-point operation load ÷ average execution time ÷ 10". 9 This yields the GFLOPS value calculated by the GEMV kernel based on this candidate parameter combination.

[0067] In some examples, the evaluation can be repeated multiple times, and the mean or median can be taken. That is, a candidate parameter combination can be repeatedly used to perform multiple calculations, and the mean or median can be taken as the function value of the objective function corresponding to this candidate parameter combination (such as the final GFLOPS value). This repeated evaluation and denoising strategy can reduce random jitter and improve the stability and effectiveness of optimization.

[0068] In some examples, the objective function returns 0.0 as a penalty signal when compilation fails, symbol resolution fails, or a GEMV kernel exception occurs. When a timeout or insufficient resources occur, a retry is preferred, or automatic degradation to more conservative parameters (e.g., downgrading to a candidate parameter combination with smaller values) is implemented. In this example, the infeasibility penalty and timeout rollback strategy improves the stability and effectiveness of the optimization. System-level exception handling and feasibility constraints make the optimization process robust and controllable, ensuring availability and maintainability in industrial production environments.

[0069] In some examples, the optimal configuration (i.e. the optimal combination of parameters), timing configuration, device information, evaluation logs, and historical results can be archived and cached to facilitate reproduction (reduce the additional overhead of repeated evaluations) and auditing, ensuring the controllability, reproducibility, and engineering availability of the optimization process.

[0070] In some embodiments, to ensure the stable operation of the above process in an engineering environment, the following supporting details are also proposed in this application embodiment. First, in terms of timing and evaluation, a warm-up can be performed to eliminate the cold start effect, and the average or robust statistics are taken for N repeated executions; GFLOPS calculation adopts 2·m·n / time (seconds) / 1e9, and if necessary, the fusion calculation of β·y is included in the memory access overhead without repeatedly including it in the arithmetic operation. Second, in terms of resource and exception handling, strict return value checks are set for device initialization, memory allocation, queue creation, and notification creation and destruction, and orderly recycling and readable log output are performed in case of exceptions to prevent resource leakage and deadlock. Third, in terms of numerical stability, compensated summation or block reduction strategies can be enabled under high expansion to reduce the impact of rounding errors on the final result; in scenarios with higher accuracy requirements, a mixed precision or double precision path can be switched. Finally, regarding the scope of application, the pipeline and optimization ideas of this invention are not only applicable to standard GEMV, but can also be extended to transposed GEMV, boundary paths of block GEMM, and other linear algebra operators that require explicit on-chip buffer management; in multi-device or multi-node scenarios, multiple sets of candidate parameters can be evaluated in parallel in one iteration to further shorten the optimization convergence time.

[0071] In some embodiments, the final output optimal parameter combination (or near-global optimal parameter combination) can be used to recompile and generate the corresponding target GEMV kernel binary file, parse the target GEMV kernel binary file, call the target GEMV kernel entry function to implement GEMV operation based on the optimal parameter combination (i.e., optimal or near-optimal parameter configuration), and output the final performance index and GEMV operation result vector.

[0072] In other embodiments, corresponding GEMV kernel binary files are pre-compiled for each parameter combination. After determining the optimal parameter combination, the corresponding target GEMV kernel binary file can be loaded and parsed to call the target GEMV kernel entry function to perform GEMV operations based on the optimal parameter combination, outputting the final performance metrics and GEMV operation result vector.

[0073] For details on performing GEMV operations based on the optimal parameter combination, please refer to the relevant descriptions of steps S12 to S13 below.

[0074] S12 divides the matrix for GEMV calculation according to the target row block number and target floating-point block size to obtain multiple data blocks, and sets up a double buffer on the on-chip memory of the MLU chip based on the target buffer number.

[0075] It should be understood that GEMV computation is the process of multiplying a matrix and a vector to obtain the resulting vector. For clarity, the following explanation uses matrix A (size m×n, single-precision floating-point matrix), vector x (length n), and the resulting vector y (length m) in GEMV computation as an example, but this does not constitute a limitation on data types or sizes.

[0076] The target row block count limits the number of matrix rows involved in a single computation on the MLU chip's on-chip memory. The target floating-point block size limits the number of floating-point units moved to the MLU chip's on-chip memory in a single operation. The target floating-point block size represents the granularity of data movement / partitioning along the column dimensions of matrix A. The target row block count and target floating-point block size together determine the space occupancy and temporal locality of the on-chip data. Thus, matrix A can be divided into submatrices (i.e., block data) using the target row block count and target floating-point block size. Simultaneously with partitioning matrix A, an adaptive partitioning of vector x is also necessary; that is, vector x is segmented based on the target floating-point block size to match the column dimensions of the block data in matrix A, thereby ensuring subsequent computations.

[0077] In some embodiments, the target number of rows is controlled by a first parameter, SRAM_BLOCK_ROWS, which limits the number of rows of matrix blocks loaded into the MLU chip (Static Random Access Memory) for each computation batch. That is, SRAM_BLOCK_ROWS refers to the number of data block rows processed at one time in the SRAM (Synchronous Dynamic Random Access Memory) of the MLU chip.

[0078] The target floating-point block size is controlled by the second parameter, NRAM_CHUNK_FLOATS, which limits the number of floating-point numbers transferred to the NRAM in the MLU chip each time. That is, NRAM_CHUNK_FLOATS refers to the number of floating-point numbers processed in NRAM (Neural Network Random Access Memory) at one time.

[0079] Specifically, for a matrix A of dimension m×n and a vector x of length n, the target row block parameter R = SRAM_BLOCK_ROWS is selected, and A is divided according to the row dimension. Each row block is divided into several segments. The column dimension transport granularity is set to C = NRAM_CHUNK_FLOATS, dividing the column dimension into several segments, where C is the target floating-point block size. For each row block, the C consecutive elements of vector x and the corresponding C column sub-blocks of matrix A are loaded into SRAM / NRAM for multiplication-accumulation operations. For example, the segmented data (including matrix blocks and vector sub-segments) can be loaded into SRAM first, and then further loaded from SRAM into NRAM for computation, as NRAM has a faster computation speed, thus improving the efficiency of multiplication-accumulation operations.

[0080] After completing the multiplication, addition, and accumulation of this segment, the next segment of C consecutive elements is moved to the corresponding C column sub-blocks of matrix A for further calculation. In this way, the multiplication, addition, and accumulation results of each row block are completed, and then the multiplication, addition, and accumulation results of all row blocks are summarized to obtain the GEMV calculation result (i.e., the result vector y).

[0081] For example, to ensure on-chip capacity feasibility, it is preferable to constrain the sum of R×C×sizeof(float) and buffer overhead (the memory size occupied by the dual buffers) to not exceed the available capacity of on-chip memory (NRAM / SRAM). Here, sizeof(float) refers to the number of bytes of memory space required to store a floating-point number.

[0082] For example, the transport granularity is matched with the throughput of the operator array to avoid "over-coarse transport leading to overflow" and "over-fine transport leading to high instruction overhead".

[0083] For alignment, the target floating-point block size C is aligned with the underlying memory access channel granularity to enable API calls to the CambriconMLU chip package, reducing on-chip memory access replay and bus rounds. For tail blocks where m or n is not an integer multiple of the block length, boundary checks are used to avoid out-of-bounds reads and writes. Specifically, for matrix A, boundary checks are used. If the tail block size is less than R×C, reads and writes are no longer performed according to the R×C size, but according to the original tail block size. The tail block is directly transferred to on-chip memory for calculation, for example, directly loaded into SRAM, and then further loaded from SRAM into NRAM for operation.

[0084] For example, regarding data layout, regardless of whether matrix elements are stored in column-major order (matrix elements are stored row-major order in memory) or row-major order (matrix elements are stored column-major order in memory), the step size (lda, incx, incy) can be set to adapt to the loading order within the block to ensure the continuity of memory access. In scenarios where transpose or conjugate transpose needs to be supported, the access continuity can be maintained by setting the trans flag at the interface layer and changing the traversal order within the block.

[0085] It should be understood that after determining the optimal combination of parameters, the data is divided into blocks based on the optimal target number of row blocks and the target floating-point block size. That is, the on-chip data is organized into blocks in a structured manner, which can significantly reduce the pressure of single data transfer, reduce unnecessary on-chip swapping in / out, improve the overall bandwidth utilization efficiency, and at the same time improve the temporal and spatial locality of the data, providing a data foundation for subsequent pipeline parallelism.

[0086] The above-mentioned block-based strategy can improve the on-chip cache reuse rate and significantly reduce the pressure of single-pass handling and memory access thrashing, laying the foundation for subsequent pipelined parallel processing.

[0087] For example, a set of buffers is established in the on-chip memory (SRAM) of the MLU chip based on the target number of buffers, and organized into an alternating structure. The target number of buffers B = PIPELINE_BUFFERS, preferably B = 2, to form a double buffer; a three-stage pipeline structure of "prefetch-compute-writeback" is formed. Subsequently, a Ping-Pong buffering mechanism can be used to allow data transfer and computation to overlap in parallel in different buffers, thereby further improving concurrency and computational efficiency under long latency channels. For detailed processing, please refer to the relevant description of step S13.

[0088] S13: Move multiple data blocks to the double buffer sequentially and perform calculation operations. While performing calculation operations on the moved data blocks in one of the double buffers, move the next data blocks to be calculated to the other buffer in parallel.

[0089] It should be understood that by using double buffering alternately, while one buffer is performing calculations, the other buffer can complete the next batch of data transfer in parallel, thereby significantly reducing the waiting time of the computing unit and achieving deep overlap between memory access and computation.

[0090] For example, a circular or alternating buffer management system with lightweight synchronization points, such as setting synchronization points or lightweight barriers within the computation queue or kernel, ensures the ordered nature of "computing only when data is ready and reusing only after computation is complete." Specifically, after the transport thread (or transport phase) writes the block data and its corresponding vector sub-segment into buffer i, a ready flag is set; the computation phase only reads from the "ready" buffer. Non-blocking queue submission and event notification are preferred to reduce host-device control overhead; simultaneously, for on-chip multi-memory structures, the buffer start address and step size are preferably planned to reduce memory conflicts.

[0091] Specifically, role switching occurs between consecutive batches of segmented data. That is, in a dual-buffer configuration, one buffer acts as a computation buffer (for data computation), while the other acts as a data buffer for data movement. While performing computation on the segmented data moved from the computation buffer, the next segmented data is simultaneously moved to the data buffer. After movement is complete, a ready flag is added to the data buffer. In the next computation, the computation buffer from the previous computation is switched to the data buffer to perform data movement (i.e., the computation buffer from the previous batch becomes the data buffer for the next batch) to continue moving segmented data. The data buffer with the ready flag is then switched to the computation buffer to perform computation (i.e., the data buffer from the previous batch becomes the computation buffer for the next batch) to continue moving segmented data.

[0092] For example, to avoid write-after-read or read-after-write conflicts, a memory fence or insert queue synchronization event can be set at the switching point (where "switching point" refers to the moment when the current batch processing is complete and ready to switch buffer roles) to ensure data consistency.

[0093] In some examples, when the data size is large or the bandwidth-to-computation ratio is unbalanced, the number of buffers can be expanded to three or more to build a deeper three-stage "prefetch-computation-writeback" pipeline, further hiding long-latency data transfer. This mechanism can significantly reduce the idle waiting time of computing units, improve the effective utilization of the operator array and the overall throughput.

[0094] In some examples, in higher-concurrency implementations, the write-back phase can be incorporated into the pipeline. Immediately after computation, the local accumulated result is written back to the result vector y in global memory, or written to an intermediate buffer before being uniformly reduced (e.g., first written from NRAM to SRAM, then uniformly reduced from SRAM to global memory, i.e., a pipelined approach with SRAM as an intermediate buffer). This further shortens the tail bubble write-back time. For exceptions or timeouts, timeout retry and failure rollback mechanisms can be set at the queue layer to ensure system stability and controllability.

[0095] The aforementioned method introduces intelligent parameter optimization at the upper layer of the Cambricon MLU platform. Specifically, for GEMV computation, a Bayesian optimization algorithm is used to automatically, quickly, and accurately explore the optimal kernel configuration (i.e., the optimal parameter combination for best performance), achieving adaptive performance improvement for GEMV computation. This allows the kernel to maintain efficient, stable, and adaptive performance under different matrix sizes and hardware resource constraints. Furthermore, at the lower layer of the Cambricon MLU platform, a pipelined design for on-chip storage (NRAM / SRAM) is constructed for GEMV computation. Based on the optimal parameter combination, the matrix for GEMV computation is split and transported in row blocks and floating-point blocks. A Ping-Pong (double buffering) mechanism is used to overlap data transport and computation, effectively solving the problems of memory access latency and idle computation units. This achieves deep overlap between memory access and computation, significantly improving GEMV kernel throughput and computational efficiency.

[0096] Furthermore, through a refined block strategy and Ping-Pong buffering mechanism, on-chip memory reuse is maximized and latency of high-bandwidth memory accesses is hidden without changing the mathematical equivalence of GEMV. In other words, the advantages of on-chip memory are fully utilized through a pipelined architecture.

[0097] Furthermore, this method innovatively unifies the "low-level pipeline kernel design" and the "upper-level intelligent tuning algorithm" within the same closed-loop system, constructing a GEMV acceleration solution capable of adapting to different constraints and application requirements. This solution significantly improves matrix computation performance and resource utilization on the Cambricon MLU platform, and significantly enhances GEMV throughput and portability on the MLU platform. In other words, the method and the corresponding system possess good versatility and portability, adapting to matrices of different sizes, data types, and Cambricon MLU chips of different models, and can be smoothly extended to scenarios such as transposed paths, mixed precision, and multi-device parallel evaluation.

[0098] In some embodiments, the parameters in the optimal parameter combination further include a target cycle expansion factor. The method also includes: during the computation operation on the block data in one of the buffers of the double buffer, expanding multiple elements in each row of the block data based on the target cycle expansion factor to form a multi-path parallel multiply-accumulate chain for computation.

[0099] In other words, a loop expansion strategy is introduced into the core loop computation of matrix-vector multiplication to reduce the instruction overhead caused by loop control and branch prediction, and to improve the instruction-level parallelism of the operator array. Specifically, by controlling the target expansion factor, multiple scalar multiplication and addition operations are aggregated and executed at the register level, reducing the instruction density of loop control and the uncertainty of branch prediction, thereby further improving the utilization of the internal computing units of the Cambricon MLU.

[0100] Specifically, the target expansion factor U = UNROLL_FACTOR is set based on register pressure, on-chip bandwidth, the parallelism of the operator array, and the compiler scheduling capability. During the computation of each data block, the C elements (i.e., C columns) in each row of the data block are expanded at a granularity of U, forming a U-way parallel multiply-accumulate chain. For example, partial sum registers are used to reduce the number of memory accesses; when U increases to the point of causing register overflow or memory congestion, the system can automatically fall back to a smaller U value to ensure throughput and stability. It should be understood that when the C elements (i.e., C columns) in each row of the data block of matrix A are expanded at a granularity of U, the corresponding C elements in vector X to be multiplied are also expanded at a granularity of U, thus maintaining accurate multiply-accumulate operations.

[0101] Furthermore, the scheduling and memory access alignment of loop unrolling need to be considered together: when U and C / alignment width are not divisible, loading alignment can be maintained by padding with 0 or splitting the path. Specifically, when the number of columns in the block data is not divisible by the alignment width of the target loop unrolling factor, the block data can be padded with 0 so that the number of columns in the padded block data is an integer multiple of the target loop unrolling factor; multiple elements in each row of the padded block data are then unrolled based on the target loop unrolling factor and multiplied-added to accumulate the results. Alternatively, the block data can be split into a first sub-block and a second sub-block by columns, where the number of columns in the first sub-block is divisible by the alignment width of the target loop unrolling factor, while the number of columns in the second sub-block is not divisible by the alignment width of the target loop unrolling factor; the second sub-block is padded with 0 so that the number of columns in the padded data block is an integer multiple of the target loop unrolling factor; both the first sub-block and the padded data block are then unrolled based on the target loop unrolling factor and multiplied-added to accumulate the results.

[0102] For example, numerical techniques such as compensated summation can be combined to reduce floating-point rounding errors and ensure numerical stability under high expansion.

[0103] In some embodiments, corresponding GEMV kernel binary files are pre-compiled for multiple preset parameter combinations. That is, key parameters in the multiple parameter combinations are pre-parameterized and compiled to generate the GEMV kernel binary files. These GEMV kernel binary files essentially "hard-coded" the configuration parameters into the functions. Subsequently, during the runtime computation phase, the corresponding configured GEMV kernel binary files can be directly loaded and called through a dynamic loading mechanism to perform GEMV operations.

[0104] Specifically, during the compilation phase, macro definitions are injected into the compiler command line to generate GEMV kernel binary files (also known as shared kernel binary files) corresponding to various preset parameter combinations (or candidate parameter combinations). That is, by injecting parameter combinations through compiler macros, GEMV kernel binary files corresponding to each parameter combination are quickly generated. Each parameter combination includes parameter values ​​for various items such as the number of line blocks, floating-point block size, number of buffers, and loop unrolling factor; the parameter values ​​for each item differ between different parameter combinations.

[0105] For example, to avoid the time overhead of repeated compilation, each parameter combination (R,C,B,U) can be hashed and used as the target file name or cache key; when the same parameter combination appears repeatedly, the existing GEMV kernel binary file can be reused directly.

[0106] In this embodiment, the matrix for GEMV calculation is divided into multiple data blocks according to the target number of row blocks and the target floating-point block size, and a double buffer is set on the on-chip memory of the MLU chip based on the target buffer quantity. This includes: loading a pre-compiled target GEMV kernel binary file corresponding to the optimal parameter combination using a dynamic link library loading interface; parsing the target GEMV kernel entry function from the target GEMV kernel binary file; calling the target GEMV kernel entry function to divide the matrix for GEMV calculation into multiple data blocks according to the target number of row blocks and the target floating-point block size, and setting a double buffer on the on-chip memory of the MLU chip based on the target buffer quantity.

[0107] That is, during the computation phase, dynamic link library loading and symbol resolution techniques can be used to load the target GEMV kernel entry function corresponding to the optimal parameter combination, enabling the system to flexibly switch kernel configurations without exiting the main process. This enhances the versatility and adaptability of the method, and makes the GEMV kernel configurable for different problem sizes and device conditions.

[0108] Specifically, during GEMV calculation, the main program can use a dynamic link library loading interface to load a pre-compiled target GEMV kernel binary file corresponding to the optimal parameter combination. Symbolic resolution technology is then used to parse the target GEMV kernel entry function within this binary file. After successful parsing, the target GEMV kernel entry function can be called to divide the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size, obtaining multiple data blocks. A double buffer is then set up on the MLU chip's on-chip memory based on the target buffer quantity. It should be understood that the target GEMV kernel entry function already hard-coded the values ​​of each parameter in the optimal parameter combination; therefore, calling the target GEMV kernel entry function allows direct use of these values ​​for block processing and double buffer setup.

[0109] For example, to enhance robustness, the target GEMV kernel entry point can be bound immediately. An explicit error code is returned on resolution failure. It should be understood that the implementation of the target GEMV kernel entry point can only be called after it has been bound.

[0110] In some examples, if loading or parsing fails, or if device resources are insufficient, an invalid performance value is returned and an error is logged (i.e., a traceable error message is generated), allowing the upper-level scheduler and optimizer to avoid this configuration in subsequent iterations. Furthermore, even if parsing fails, if resources have already been allocated, these resources should be released as needed to ensure system leak-free and reentrant operation.

[0111] In some examples, when the data layout (whether to transpose, step size settings, etc.) changes, it can be adapted and routed at runtime through a unified interface to ensure consistency and portability under different calling scenarios.

[0112] In summary, this invention constructs a GEMV acceleration system on the Cambricon MLU platform that is feasible, scalable, and reproducible through an integrated implementation method of "block transport—Ping-Pong pipeline—loop unrolling—parameterized compilation and dynamic loading—Bayesian optimization tuning." While ensuring numerical accuracy, the system can stably achieve high GFLOPS performance under various matrix sizes and device conditions, significantly reducing manual parameter tuning costs, and possessing good engineering maintainability and cross-scenario portability. Any equivalent substitutions or modifications made by those skilled in the art to the parameter set, pipeline depth, surrogate model and acquisition function, timing and evaluation strategies, data layout and alignment methods, and anomaly rollback strategies without departing from the spirit and substance of this invention should be included within the scope of protection of this invention.

[0113] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0114] Based on the same inventive concept, this application also provides a system for implementing the pipelined GEMV computation and Bayesian optimization tuning system based on the CambriconMLU platform described above. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the pipelined GEMV computation and Bayesian optimization tuning system based on the CambriconMLU platform provided below can be found in the limitations of the pipelined GEMV computation and Bayesian optimization tuning method based on the CambriconMLU platform described above, and will not be repeated here.

[0115] In one embodiment, such as Figure 2 As shown, a pipelined GEMV computation and Bayesian optimization tuning system based on the CambriconMLU platform is provided. This system includes:

[0116] The optimal combination determination module 202 is used to calculate the optimal parameter combination that maximizes the function value of the objective function by using the GEMV kernel execution performance index as the objective function during the matrix-vector multiplication GEMV calculation process; the parameters in the optimal parameter combination include the target number of row blocks, the target floating-point block size, and the target number of buffers; the target floating-point block size is used to represent the column dimension partitioning granularity;

[0117] The partitioning module 204 is used to divide the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size to obtain multiple data blocks, and to set up a double buffer on the on-chip memory of the MLU chip based on the target number of buffers.

[0118] The calculation module 206 is used to sequentially move the multiple data blocks to the dual buffer and perform calculation operations, wherein while performing calculation operations on the moved data blocks in one of the dual buffers, the next data blocks to be calculated are moved to the other buffer in parallel.

[0119] In some embodiments, the dual buffer includes a computation buffer and a data buffer;

[0120] The calculation module 206 is also used to perform calculation operations on the block data moved to the calculation buffer while simultaneously moving the next block data to the data buffer. After the moving is completed, a ready flag is added to the data buffer. In the next calculation, the data buffer with the ready flag is switched to the calculation buffer to perform calculation operations, and the calculation buffer from the previous calculation is switched to the data buffer to perform data moving operations, so as to continue moving the block data.

[0121] In some embodiments, the parameters in the optimal parameter combination further include a target cycle expansion factor;

[0122] The calculation module 206 is also used to expand multiple elements in each row of the block data based on a target loop expansion factor during the calculation operation on the block data in one of the buffers of the double buffer, forming a multi-path parallel multiply-accumulate chain for calculation.

[0123] In some embodiments, the calculation module 206 is further configured to: fill the block data with 0s when the number of columns in the block data is not divisible by the alignment width of the target loop expansion factor, such that the number of columns in the filled block data is an integer multiple of the target loop expansion factor; expand multiple elements in each row of the filled block data based on the target loop expansion factor and perform multiplication-addition-cumulative calculations; or, split the block data into a first sub-data block and a second sub-data block by columns, wherein the number of columns in the first sub-data block is divisible by the alignment width of the target loop expansion factor, and the number of columns in the second sub-data block is not divisible by the alignment width of the target loop expansion factor; fill the second sub-data block with 0s, such that the number of columns in the filled data block is an integer multiple of the target loop expansion factor; expand both the first sub-data block and the filled data block based on the target loop expansion factor and perform multiplication-addition-cumulative calculations.

[0124] In some embodiments, corresponding GEMV kernel binary files are pre-compiled for multiple preset parameter combinations; the partitioning module 204 is further configured to load the pre-compiled target GEMV kernel binary file corresponding to the optimal parameter combination using a dynamic link library loading interface, parse the target GEMV kernel entry function from the target GEMV kernel binary file, call the target GEMV kernel entry function, divide the matrix for GEMV calculation according to the target row block number and the target floating-point block size to obtain multiple block data, and set up a double buffer on the on-chip memory of the MLU chip based on the target buffer number.

[0125] In some embodiments, the target row block size is controlled by a first parameter SRAM_BLOCK_ROWS, which limits the number of rows of the matrix block loaded into the SRAM of the MLU chip in each computation batch; the target floating-point block size is controlled by a second parameter NRAM_CHUNK_FLOATS, which limits the number of floating-point blocks moved to the NRAM of the MLU chip each time.

[0126] like Figure 3 As shown, in some embodiments, the optimal combination determination module 202 includes:

[0127] The candidate combination generation module 2021 is used to establish a set of values ​​corresponding to multiple preset parameter items; and to generate multiple sets of candidate parameter combinations based on each set of values.

[0128] The Bayesian optimization algorithm module 2022 is used to iteratively calculate the function value of the objective function based on each set of candidate parameter combinations using the Bayesian optimization algorithm until the iteration stopping condition is met, and the candidate parameter combination that maximizes the function value of the objective function is determined as the optimal parameter combination.

[0129] The modules in the aforementioned CambriconMLU-based pipelined GEMV computation and Bayesian optimization tuning system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in a computer device, or stored in software within the computer device's memory, allowing the processor to invoke and execute the corresponding operations of each module.

[0130] In one embodiment, a computer device is provided, which may be a terminal or a server. The internal structure diagram of the computer device may be as follows: Figure 4As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a pipelined GEMV computation and Bayesian optimization tuning method based on the CambriconMLU platform.

[0131] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the embodiments of this application.

[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the embodiments of this application.

[0134] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the embodiments of this application.

[0135] It should be noted that the user information (including but not limited to user device information, user attribute content, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0136] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The above embodiments are merely illustrative of several implementation methods of this application and should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A pipelined GEMV calculation and Bayesian optimization tuning method based on the CambriconMLU platform, characterized in that, The method includes: In the matrix-vector multiplication GEMV calculation process, the GEMV kernel execution performance index is used as the objective function, and the optimal parameter combination that maximizes the function value of the objective function is calculated iteratively using a Bayesian optimization algorithm. The parameters in the optimal parameter combination include the target number of row blocks, the target floating-point block size, and the target number of buffers. The target floating-point block size is used to represent the granularity of column dimension partitioning. The matrix for GEMV calculation is divided into multiple data blocks according to the target number of row blocks and the target floating-point block size, and a double buffer is set on the on-chip memory of the MLU chip based on the target number of buffers. The multiple data blocks are sequentially moved to the dual buffer and computation is performed. While performing computation on the moved data blocks in one of the dual buffers, the next data blocks to be computed are moved to the other buffer in parallel.

2. The method according to claim 1, characterized in that, The dual buffer includes a computation buffer and a data buffer; The process of performing computation on the transferred data chunks in one buffer of the dual buffers while simultaneously transferring the next batch of data chunks to be computed to the other buffer includes: While performing calculation operations on the block data moved to the calculation buffer, the next block data is moved to the data buffer in parallel. After the moving is completed, a ready flag is added to the data buffer. In the next calculation, the data buffer with the ready flag is switched to the calculation buffer to perform the calculation operation, and the calculation buffer from the previous calculation is switched to the data buffer to perform the data transfer operation, so as to continue transferring the block data.

3. The method according to claim 1, characterized in that, The parameters in the optimal parameter combination also include the target cycle expansion factor; The method further includes: During the computation operation on the block data in one of the double buffers, multiple elements in each row of the block data are expanded based on the target loop expansion factor to form a multi-parallel multiply-accumulate chain for computation.

4. The method according to claim 3, characterized in that, The step of expanding multiple elements in each row of the segmented data based on a target loop expansion factor to form a multi-parallel multiply-accumulate chain for computation includes: If the number of columns in the block data is not divisible by the alignment width of the target loop expansion factor, the block data is padded with 0s so that the number of columns in the padded block data is an integer multiple of the target loop expansion factor; multiple elements in each row of the padded block data are expanded based on the target loop expansion factor and multiplied and added together. or, The block data is split into a first sub-data block and a second sub-data block by columns, wherein the number of columns in the first sub-data block is divisible by the alignment width of the target loop expansion factor, and the number of columns in the second sub-data block is not divisible by the alignment width of the target loop expansion factor; the second sub-data block is padded with 0s so that the number of columns in the padded data block is an integer multiple of the target loop expansion factor; both the first sub-data block and the padded data block are expanded based on the target loop expansion factor and subjected to multiplication-addition accumulation calculation.

5. The method according to claim 1, characterized in that, The corresponding GEMV kernel binary files were pre-compiled for each of the multiple preset parameter combinations; The step of dividing the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size to obtain multiple data blocks, and setting up a double buffer on the on-chip memory of the MLU chip based on the target number of buffers includes: The pre-compiled target GEMV kernel binary file corresponding to the optimal parameter combination is loaded using the dynamic link library loading interface, and the target GEMV kernel entry function is parsed from the target GEMV kernel binary file; The target GEMV kernel entry function is called to divide the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size to obtain multiple data blocks, and a double buffer is set on the on-chip memory of the MLU chip based on the target number of buffers.

6. The method according to claim 1, characterized in that, The target row block size is controlled by the first parameter SRAM_BLOCK_ROWS, which limits the number of rows of the matrix block loaded into the SRAM of the MLU chip in each computation batch; the target floating-point block size is controlled by the second parameter NRAM_CHUNK_FLOATS, which limits the number of floating-point blocks moved into the NRAM of the MLU chip each time.

7. The method according to any one of claims 1 to 6, characterized in that, The optimal parameter combination that maximizes the objective function value by iteratively calculating using the Bayesian optimization algorithm includes: Establish a set of values ​​corresponding to multiple preset parameter items; Multiple sets of candidate parameter combinations are generated based on the aforementioned value sets; The Bayesian optimization algorithm is used to iteratively calculate the function value of the objective function based on each set of candidate parameter combinations until the iteration stopping condition is met. The candidate parameter combination that maximizes the function value of the objective function is determined as the optimal parameter combination.

8. A pipelined GEMV computation and Bayesian optimization tuning system based on the CambriconMLU platform, characterized in that, The system includes: The optimal combination determination module is used in the matrix-vector multiplication GEMV calculation process to iteratively calculate the optimal parameter combination that maximizes the function value of the objective function by using the GEMV kernel execution performance index as the objective function and employing a Bayesian optimization algorithm. The parameters in the optimal parameter combination include the target number of row blocks, the target floating-point block size, and the target number of buffers. The target floating-point block size is used to represent the granularity of column dimension partitioning. The partitioning module is used to divide the matrix for GEMV calculation according to the target number of row blocks and the target floating-point block size to obtain multiple data blocks, and to set up a double buffer on the on-chip memory of the MLU chip based on the target number of buffers. The calculation module is used to sequentially move the multiple data blocks to the dual buffer and perform calculation operations, wherein while performing calculation operations on the moved data blocks in one of the dual buffers, the next data blocks to be calculated are moved to the other buffer in parallel.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When a computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images