Method, device and equipment for optimizing reasoning operator of large model based on mercuric chloride chip

Abstract

The technical scheme can be applied to the field of financial science and technology / medical health. The invention discloses a mercuric chloride chip-based large model reasoning operator optimization method, device and equipment, and the method comprises the steps: carrying out the blocking processing of original input data according to the parallel calculation capability of a mercuric chloride chip, and generating the blocking data of an adaptive chip calculation unit; optimizing a memory access path of a matrix multiplication operator by combining a memory hierarchical structure and a calculation core type of a mercuric chloride chip based on the block data, and adjusting a sliding step length and a filling mode of convolution operation; a plurality of operators continuously executed in the large model are fused into a composite operator, the data storage process of the composite operator is optimized, and collaborative execution is achieved by dynamically allocating computing resources; and integrating and decoding block calculation results after collaborative optimization, and adjusting an input data block strategy and operator execution parameters through a verification feedback mechanism to form closed-loop optimization. According to the technical scheme, the reasoning efficiency of a large model can be improved.

Claims

1. A large model inference operator optimization method based on Ascend chips, characterized by: The following steps are involved: Based on the parallel computing capabilities of the Ascend chip, the original input data is processed in blocks to generate block data adapted to the chip's computing units. Based on the block data, combined with the memory hierarchy and computing core type of the Ascend chip, the memory access path of the matrix multiplication operator is optimized, and the sliding step size and padding method of the convolution operation are adjusted; Fusion of multiple operators executed continuously in large models into composite operators, optimization of the data storage process of composite operators, and coordinated execution through dynamic allocation of computing resources; The block calculation results after collaborative optimization are integrated and decoded, and the input data block strategy and operator execution parameters are adjusted through the verification feedback mechanism to form a closed-loop optimization.

2. The large model inference operator optimization method based on the Ascend chip according to claim 1 is characterized in that: The memory access path of the optimized matrix multiplication operator specifically includes: After dividing the input matrix into blocks by rows or columns, the high-frequency access matrix blocks are dynamically allocated to the high-speed memory layer based on the high-speed to low-speed access characteristics of the Ascend chip memory layer and the access frequency statistics of historical computing data. Plan the order in which the compute cores access matrix blocks to reduce memory access latency.

3. The large model inference operator optimization method based on the Ascend chip according to claim 1 is characterized in that: The sliding step size and filling method of adjusting the convolution operation specifically include: Convert the data format of the input feature map and convolution kernel to the format supported by the chip hardware acceleration unit; Based on the parallel processing capabilities of the Ascend chip's computing core, the sliding step size and padding method of the convolution kernel on the input feature map are adjusted to ensure that the convolution kernel sliding matches the layout of the chip's parallel task processing units.

4. The large model inference operator optimization method based on the Ascend chip according to claim 1 is characterized in that: The process of fusing multiple operators executed continuously in a large model into a composite operator and optimizing the data storage of the composite operator specifically includes: Fusion of convolutional layers, batch normalization layers, and activation function layers into a composite operator; This is achieved by storing the convolutional output data directly in memory locations where batch normalization and activation layers can read them efficiently.

5. The large model inference operator optimization method based on the Ascend chip according to claim 1, characterized in that: The collaborative execution achieved by dynamically allocating computing resources specifically includes: Dynamically allocate computing resources for convolutional layers, fully connected layers, and pooling layers based on the large model inference task process and the total amount of chip computing resources.

6. The large model inference operator optimization method based on the Ascend chip according to claim 1, characterized in that: The block processing of the original input data specifically includes: Divide the text sequence into predefined block lengths to generate continuous text blocks with consistent lengths; The image data is divided into spatial regions to generate sub-image blocks whose sizes are adapted to the parallel computing units of the chip.

7. The large model inference operator optimization method based on the Ascend chip according to claim 6, characterized in that: The adjustment of input data partitioning strategy and operator execution parameters through the verification feedback mechanism includes: Based on the accuracy index of the output results, identify the source of the error as data partitioning strategy or operator execution parameters; If the error is caused by the data block strategy, adjust the block length or image sub-block size; If the error is caused by operator execution parameters, re-optimize memory allocation or collaborative resource allocation ratio.

8. A large model inference operator optimization device based on Ascend chip, characterized in that: include: The input data preprocessing module is used to process the original input data in blocks based on the parallel computing capabilities of the Ascend chip, generating block data adapted to the chip's computing units. An operator execution optimization module, which is used to optimize the memory access path of the matrix multiplication operator based on the block data and in combination with the memory hierarchy and computing core type of the Ascend chip, and adjust the sliding step size and padding method of the convolution operation; The operator fusion and collaborative optimization module is used to fuse multiple operators executed continuously in a large model into composite operators, optimize the data storage process of composite operators, and achieve collaborative execution by dynamically allocating computing resources; The output result post-processing module is used to integrate and decode the block calculation results after collaborative optimization, and adjust the input data block strategy and operator execution parameters through the verification feedback mechanism to form a closed-loop optimization.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the large model inference operator optimization method based on the Ascend chip are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the large model inference operator optimization method based on the Ascend chip are implemented as described in any one of claims 1 to 7.

Images