Modeling analysis and calculation optimization method for operator performance on intelligent chip

Abstract

The invention discloses a modeling analysis and calculation optimization method for operator performance on an intelligent chip. The modeling analysis and calculation optimization method comprises the following steps: carrying out graph representation modeling on an intelligent chip hardware architecture and an interconnection mode; designing a link mechanism of operator loop tree nodes, so that operator loops are described and mapped in a hardware architecture model represented by a graph; and performing delay performance analysis on the single operators and the fusion operators in various combination forms by using a pipeline analysis method to obtain the performance of the operators under a hardware architecture, and further performing calculation optimization acceleration. By the adoption of the technical scheme, sufficient guidance can be provided for the initial stage of optimization of the hardware architecture, guidance is provided for optimization space exploration of operators, and therefore the calculation performance of intelligent chip hardware is improved.

Description

Technical Field

[0001] This invention belongs to the field of electronic information technology, and relates to the performance analysis and computational optimization technology of operators in deep learning models (also known as deep neural networks, DNN) on smart chips. Specifically, it relates to a hardware modeling analysis and computational optimization method for smart chips based on graph representation. Background Technology

[0002] Deep learning models, also known as deep neural networks (DNNs), are artificial neural network structures composed of multiple neural network layers. These structures possess the ability to automatically extract features from data and model complex nonlinear relationships. In recent years, they have become a core component of artificial intelligence (AI) technology and are widely used in various technical fields such as speech recognition, image processing, and natural language processing. In DNNs, tensors serve as the basic data representation form, used to describe the inputs, outputs, and intermediate variables in the model. Tensors can also be called multidimensional arrays (N-dimensional arrays, or ND-arrays). The number of dimensions of a tensor defines the number of its tensor axes. Let tensor X be an N-dimensional tensor; then it has N tensor axes, denoted as Xa, where the i-th tensor axis is denoted as X.ai. The shape of a tensor is determined by the lengths along each tensor axis. For example, when the shape of tensor X is 2×3, it means that X has a length of 2 on the first axis and a length of 3 on the second axis. Each element in a tensor is a scalar value, accessible by index. For example, A[0,2] represents the element in the 0th row and 2nd column of the two-dimensional tensor A. The basic computational unit in deep neural networks is the operator. Each operator typically accepts one or more input tensors and produces an output tensor. Operators can perform basic numerical operations (such as addition and multiplication) or complex mathematical operations (such as matrix multiplication, convolution, and pooling). Most operators can be described using tensor expressions (TEs). Tensor expressions typically include loop axes, input and output tensors, a summation symbol, and several scalar operations. For example, matrix multiplication can be represented as: C[m,n]=∑kA[m,k]×B[k,n], where m, n, and k are loop axes, A, B, and C are tensors, A and B are input tensors, and C is the output tensor. In the above expression, the cyclic axes (m, n) involved in indexing the output tensor C are called spatial axes, which correspond one-to-one with the tensor axes of the output tensor; the cyclic axis (k) involved in the summation operation is called the temporal axis. Let operator X have all its spatial axes denoted as Xs and all its temporal axes denoted as Xt, where the i-th spatial axis and temporal axis are denoted as Xs and Xt, respectively. iA complete deep neural network can be abstractly represented as a Directed Acyclic Graph (DAG). In this graph structure, each node represents an operator, and the directed edges between nodes represent tensor data dependencies between operators. If the output tensor of an operator u is used as the input tensor of another operator v, then there exists a directed edge from u to v in the DAG. The scheduling of a DNN is the execution order of operators in the DAG, and this order must satisfy the data dependency constraint; that is, if there is a directed edge from operator u to operator v, then the execution of u must be earlier than that of v in the schedule.

[0003] Intelligent chips are dedicated spatial accelerators optimized for artificial intelligence scenarios, taking Huawei's Ascend AI Processors as an example. This series of chips adopts a heterogeneous multi-core architecture, integrating multiple computing and communication modules, including general-purpose computing cores, dedicated acceleration units, and high-speed interconnects, to achieve high throughput and low power consumption for deep learning inference and training performance. The Ascend series mainly includes models such as Ascend 310 and Ascend 910. The Ascend 310 is primarily designed for edge computing scenarios, supporting real-time inference tasks for various mainstream deep neural network models, and features low power consumption and flexible deployment. The Ascend 910 is positioned for data center training scenarios, possessing higher computing power and stronger parallel processing capabilities. This series of chips integrates Huawei's self-developed AI acceleration core, including modules such as the Multiply Accumulate Unit (MAC), vector acceleration unit, and scalar processor. Among them, the MAC can efficiently execute common matrix multiplication operators in deep learning. The chip employs a high-bandwidth, low-latency Network-on-Chip (NoC) for data communication between computing units, and supports multi-level on-chip storage structures to reduce data transfer latency and improve computing efficiency. To adapt to the hardware characteristics of the Ascend series chips, Huawei also provides a matching software stack, CANN (Compute Architecture for Neural Networks). This software stack supports operator-level optimization, graph-level fusion, model quantization, and scheduling, enabling efficient mapping of models from mainstream frameworks (such as TensorFlow and PyTorch) to the chip's underlying instruction execution units, achieving performance optimization through hardware-software collaboration.

[0004] Operator performance analysis models are widely used analytical tools in the design phase and implementation optimization of artificial intelligence chips. These models can predict the performance of operators under specific hardware architectures, providing hardware designers with a basis for identifying performance bottlenecks and making structural adjustments. Simultaneously, they can provide quantitative performance evaluation standards for optimizing operator scheduling and implementation at the software level, guiding candidate solution search and cost estimation during the Design Space Exploration (DSE) process.

[0005] Existing research has yielded various modeling methods and frameworks for analyzing operator performance on smart chips. For example, the Timeloop framework proposes a tree-structure-based hardware abstraction modeling approach, dividing the mapping of operator loop axes in hardware into spatial and temporal dimensions, thus enabling joint analysis of mapping strategies and performance. The Maestro framework proposes a data-centric dataflow representation method and analyzes data reuse relationships under different mapping methods. Building upon this, frameworks such as Interstellar and TileFlow have extended the Timeloop method: Interstellar integrates Timeloop's modeling capabilities into the Halide compilation framework, improving its integration with the actual compilation process; TileFlow further expands the analytical scope of Timeloop, extending from single operators to operator fusion scenarios.

[0006] However, the above methods are mostly based on the assumptions of typical smart chip architectures with hierarchical storage structures and a single type of computing unit. For Huawei's Ascend series of space acceleration chips, which adopt a more complex on-chip storage organization and heterogeneous computing units, the existing methods are difficult to apply directly to the performance analysis of this type of chip.

[0007] Therefore, there is an urgent need to research a modeling method for intelligent chips with complex on-chip memory structures, and based on this, to construct a novel operator performance analysis model suitable for space accelerators. This model should not only have the ability to accurately model tensor data in multi-level storage and transmission paths, but also consider the impact of different mapping strategies on data reusability, computational concurrency, and communication bandwidth, thereby providing more targeted analysis and optimization guidance for chip design and operator implementation optimization. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention proposes a method for operator performance modeling, analysis, and computational optimization on intelligent chips (space accelerators), and applies it to the optimization of large language models in the field of natural language processing. This invention can be used for common computational models in the field of deep learning, such as accelerating the computation of large language models in natural language processing.

[0009] This invention first models the hardware architecture and interconnection methods using a graph representation. Second, it designs a linking mechanism for operator loop tree nodes, enabling operator loops to be described and mapped within the hardware architecture represented by the graph. Finally, it uses pipeline analysis to perform latency performance analysis on single and fused operators under various combinations, obtaining the performance of the operators under the current architecture, which can be further optimized and accelerated for computation.

[0010] The technical solution of the present invention is as follows:

[0011] A method for modeling, analyzing, and optimizing the performance of operators on intelligent chips is proposed. This method analyzes and optimizes computational models such as large language models in natural language processing (extracting computational operators and their hardware mapping), target device memory architecture, interconnection methods, communication bandwidth capabilities, operator input / output tensor dimensions and shapes, and operator loop expressions as inputs. Specific steps include:

[0012] 1) Extract the core operators from the application object.

[0013] This invention extracts the core computational processes involved in application task algorithms requiring accelerated analysis (such as natural language processing and image processing), identifies the operators with the highest computational time, and extracts their functions and hardware mapping forms as inputs to subsequent analysis models. The extracted objects include matrix multiplication operators and attention operators.

[0014] 2) Describe the hardware architecture of the smart chip and construct a graph structure of the smart chip based on graph representation.

[0015] This invention transforms the hardware architecture of a smart chip into a hardware graph structure based on graph representation. This graph structure consists of multiple nodes, each representing a type of memory hierarchy or computational unit. Specifically, memory hierarchies mainly include High Bandwidth Memory (HBM), buffers, and registers, while computational units mainly include matrix multiplication acceleration units and vector acceleration units. The node description will be stored in Arch.yaml data serialization format.

[0016] 3) Describe the interconnection structure of the smart chip and construct a directed graph of the smart chip, including interconnection nodes, interconnection bandwidth, and interconnection type.

[0017] This invention describes the interconnections between different nodes in a hardware architecture using a directed graph. Start and end nodes are labeled between different interconnected nodes, and the actual interconnect bandwidth and interconnect type are described. Specifically, interconnect types include half-duplex (HD), full-duplex (FD), unicast (UC), and broadcast (BC). This interconnect description will be stored in the Interconnection.yaml data serialization format.

[0018] 4) Describe the type and form of the core operators to obtain the operator description of the application object.

[0019] After obtaining the core computational operators in step 1), this invention will describe the input operators from several aspects, including input and output tensors, computational dimensions, computation type, and actual computational values. Overall, this invention first describes the input and output tensors required throughout the computation process, names the tensor dimensions used in the process, and specifies the actual size of the corresponding dimensions according to computational needs. Subsequently, for each of the several operators included in the computation process, this invention will describe the input and output tensors and dimension information corresponding to each operator, and specify the computation type of the operator (including matrix multiplication and vector computation). During this process, each tensor and its corresponding dimension will be bound together. This operator description will be stored in Prob.yaml data serialization format.

[0020] 5) Describe the operator mapping method for the application object. This invention will map operators to the graph representation of the actual intelligent chip hardware architecture in the form of a cyclic tree. During the mapping process, this invention will first perform multiple tiling operations on the cyclic axis of the operator and map it to the corresponding hardware. Specifically, for a single vector addition, the input vectors are A[M,N] and B[M,N], and the output vector is C[M,N], i.e., the vector dimension is [M,N]. Its initial computation cyclic axis is [M,N]. To improve the transfer efficiency, this cyclic axis can be tiled. Taking two tiling operations as an example, this cyclic axis will be transformed into [M2,N2,M1,N1,M0,N0], where the dimension M is transformed into M2, M1, and M0, and M = M2 * M1 * M0, and the dimension N is transformed into N2, N1, and N0, and N = N2 * N1 * N0. After segmentation, the corresponding segments are mapped to the intelligent chip hardware, such as [{M2,N2}-HBM,{M1,N1}-Buffer,{M0,N0}-Reg,{1}-PE]. The segments in HBM are considered the root nodes of the cyclic tree, the segments in the Processing Engine (PE) are considered leaf nodes of the cyclic tree, called ComputeNodes (Compute), where computation is performed. The remaining segments at the memory level are considered intermediate nodes of the cyclic tree, called MemoryNodes (Mem). Furthermore, since the structure of the cyclic tree is difficult to directly correspond to the hardware structure in the graph representation, this invention provides TransactionNodes (Trans) as additional leaf nodes inserted at locations requiring cross-level data transfer. Finally, for the analysis of fused operators, this invention provides ScopeNodes (Scope), where the operators represented by the first-level child nodes under this node are analyzed in a fused form. The specific fusion methods (i.e. node relationships) include four types: sequential execution (Sequential, Seq), shared input data (Sharing, Shar), pipeline parallelism (Pipeline, Pipe), and parallel execution (Parallel, Parallel).

[0021] 6) After completing the description of the input in steps 2)-4), the present invention performs a legality check on the description, specifically including loop parameter matching, loop dimension matching, mapping relationship matching, etc., to ensure that the operator mapping complies with the hardware architecture limitations and the parameter limitations of the operator type.

[0022] 7) After completing step 6), the present invention analyzes the delay of the operators on the mapping matching on the smart chip hardware.

[0023] First, this invention will traverse the operator's cyclic tree using Depth-First Search (DFS). During the traversal, this invention will record the range of variable changes in each memory level during each traversal, taking into account changes in the cyclic and hardware levels, thereby obtaining the data transfer / computational load. Second, after obtaining the data transfer load, this invention will obtain the data transfer time (including data inbound and data outbound times) for each memory level under the current traversal, based on the hardware connection relationships and bandwidth obtained in steps 1) to 2). In addition, this invention will also determine the time required by the computing unit based on the actual computing power of the computing unit and the obtained computational load. Finally, this invention will construct a top-down latency pipeline based on the obtained data inbound, data outbound, and computation times of each node. The latency calculation of nodes under the parallel layer will be calculated according to the four node relationships described in 4), thereby obtaining the latency of the entire operator.

[0024] 8) After completing the above steps, the present invention adjusts and optimizes the block strategy of the input operator based on the delay analysis results obtained in step 7). The adjustment methods are all based on the operator mapping method provided in step 5), including (1) adjusting the order of the loop axis, such as changing the order of the last two dimensions [M2,N2,M1,N1,M0,N0]→[M2,N2,M1,N1,N0,M0]; (2) adjusting the size of the loop parameter, including increasing and decreasing the upper limit of the loop parameter in different blocks, such as [M2,N2,M1,N1,M0,N0]→[M2,N2,M1,N1*2,M0,N0 / 2]; (3) adjusting the block strategy, such as changing the form of the range node (changing sequential execution to parallel execution). Through these optimizations and model analysis results, the optimized block strategy is returned to the source application of the input operator to optimize the computational performance of the application.

[0025] Furthermore, the various legality checks performed in step 6) are obtained by traversing the operator cycle tree using DFS to obtain information about the tensor parameters, dimensions, and mapping relationships corresponding to the specific mapping.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] This invention proposes a method for modeling and analyzing operator performance on intelligent chips. It abstractly describes the architecture of the intelligent chip and analyzes the performance of its operators. Implementation verification shows that the correlation coefficient between the analytical capability of the proposed method and actual performance reaches 0.99 (1 being the highest), accurately reflecting hardware performance. This invention provides sufficient guidance for the initial optimization of the hardware architecture and for exploring the optimization space of operators, thereby improving the computational performance of intelligent chip hardware. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the main components constituting the present invention;

[0029] Figure 2 This is an architecture modeling diagram of the Huawei Ascend 910B series chip in an embodiment of the present invention;

[0030] Figure 3 This is a complete operator model diagram for the Huawei Ascend 910B series chip in a specific embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram comparing the analysis results of the present invention with the running results of Huawei's official operator ACLNN on an actual chip in a specific embodiment of the present invention. Subgraph a) shows matrix calculation operators of various shapes, subgraph b) shows vector calculation operators of various shapes, and subgraph c) shows vector calculation fusion operators of various shapes.

[0032] Figure 5 This is a specific embodiment of the optimization results of the matrix multiplication operator on different large language model networks in the field of natural language processing. Detailed Implementation

[0033] The invention will now be further explained clearly and completely with reference to the accompanying drawings and specific examples.

[0034] This invention provides a method for modeling, analyzing, and optimizing the performance of operators on intelligent chips, taking the optimization of common operators (matrix multiplication, binocular vector computation, and vector fusion computation) in large language models in the field of natural language processing as an example. Figure 1 This is a schematic diagram of the main components constituting the present invention, specifically:

[0035] A. Operator Description Component: Provides functional and mapping descriptions for the input operators, specifically including:

[0036] a) Functional Description: Describe the function, computation type, tensor type, operator shape, and input / output format of the input operator and store them in the Prob.yaml file. This file will describe the following: 1) Define the overall input and output tensors required by the operator, represented by one or more tensor names; 2) Define the names of the dimensions involved in the operator; 3) Define the actual numerical information of the dimensions involved in the operator; 4) Define the computations involved in the operator.

[0037] For step 4) in the functional description, this method will describe each computation separately through multiple parallel description nodes.

[0038] The specific functions described include the computation name, the dimensions involved in the computation, the computation type (e.g., scalar computation, vector computation, matrix computation, etc.), the names of the input and output tensors designed for the computation, and the dimensions of the tensors involved in the computation.

[0039] b) Mapping Description: The mapping relationship between the input operators and the actual hardware is described and stored in the Map.yaml file. Different operator blocks and functional blocks will be divided into four types of nodes: storage nodes bound to memory mapping; computation nodes bound to computation mapping; range nodes describing the relationships between nodes; and transmission nodes for data transfer between nodes in a data stream manner. This file will describe the following: 1) the node types and names of different blocks; 2) the data stream transmission mode under the current block (including write-back transmission and data stream transmission); 3) the mapping mode within the current block (including time mapping and space mapping); and 4) the block parameter information of the current block.

[0040] (Limited to compute and storage nodes); 5) Define the target hardware hierarchy for the current block. Different nodes are connected through a tree structure.

[0041] B. Hardware Description and Interconnection Description: For a given hardware platform, we first abstract and describe it: different memory levels and computing units are abstracted as nodes in a graph, and the interconnection relationships between different nodes are abstracted as edges, thereby constructing a complete directed hardware graph structure through nodes and edges.

[0042] a) Taking Huawei's Ascend 910B smart chip as an example, its hardware architecture is as follows: Figure 2 As shown. The architecture includes...

[0043] The system consists of main memory (HBM), L2 cache, and 20 matrix computation cores (AICs).

[0044] It also includes 40 vector computation cores (AIVs). The matrix computation unit and the vector computation unit are located on the matrix computation core and the vector computation core, respectively. The AIC and AIV contain computation units for performing matrix multiplication and vector operations, respectively.

[0045] b) Hardware Description: The characteristics of each level of the hardware architecture are abstracted and described in the Arch.yaml file. This file will describe the following: 1) the name of the current hardware level; 2) the type of the current hardware level, such as DRAM, SRAM, VEC, etc.; 3) the number of independent units accommodated by the current hardware level; 4) the specific characteristics involved in this level, including read / write bit width, capacity, computing power, etc.

[0046] c) Interconnection Description: Abstractly describes and stores the interconnection relationships between different levels of units in the hardware architecture.

[0047] The file Interconnect.yaml will describe the following: 1) Construct interconnect nodes between different hardware layers by combining hardware layers; 2) In each interconnect node, define the start node and end node of the hardware layer involved in the node to describe the directed data transmission channel; 3) Define the transmission types supported by the hardware corresponding to the node, such as unicast, broadcast, etc.; 4) Define the read and write bandwidth information of the node.

[0048] C. Modeling: Based on the above hardware and interconnection description, a complete hardware and software integration model is further constructed. Figure 3 This paper presents a complete modeling example of a matrix multiplication and vector operation fusion operator on a Huawei smart chip. The operator first performs matrix multiplication, dividing the memory into blocks according to dimensions i1 and j1 at both the HBM and L1 storage levels. Subsequently, the matrix multiplication is further divided into k1 dimensions at the L0AB level, and matrix multiplication is iteratively implemented within Cube cells, performing 16*16 multiplications of 16*16 sub-blocks each time. After completing all k1-dimensional multiplications, the results are sequentially transmitted to the L0C and UB storage via transmission nodes to complete subsequent vector calculations. Vector calculations in the UB storage also utilize a block division in the iv dimension, and vector operations are performed sequentially through vector cells in the jv dimension. The final result is then transmitted back to the HBM via transmission nodes.

[0049] D. Legality Check: To ensure the correctness of the constructed model, this invention performs multiple legality checks on the generated model, specifically including the following aspects:

[0050] a) Cyclic dimension matching: For any input operator, the cyclic dimension involved in the functional description must exist completely in the operator mapping process to ensure that the functional description and the mapping are consistent;

[0051] b) Loop parameter matching: For any input operator, the mapping process must ensure strict consistency with the description value for each loop dimension. For example, for an operator [m0,m1,m2] that performs three-part division on dimension M=1024, m0×m1×m2=1024 must be satisfied. In addition, to enhance the representation of the tail block, the loop representation proposed in this invention introduces a tail block description mechanism, that is, each loop consists of a main block and a tail block within parentheses. The tail block is only executed when all loops before this layer have been executed to the last loop. For example, the loop representation of dimension M=576 in this invention is [5,8(4),16], and its expansion calculation is (5-1)×8×16+1×4×16=576, thereby ensuring the accurate expression of the tail block;

[0052] c) Mapping Relationship Matching: For any operator's mapping description, input / output data is transferred between nodes at different levels in its corresponding cyclic tree through hierarchical structures or transmission nodes. This check ensures that the transmission relationships in the operator mapping description are consistent with the data transmission paths in the actual hardware architecture, guaranteeing the mapping's legality and executability.

[0053] E. Delay Analysis: This invention analyzes the delay of operators on the mapping on the hardware. This analysis consists of two steps, as follows:

[0054] a) Data Transfer Volume Analysis: This analysis determines how the data transfer volume changes with the number of iterations when the loop level changes. This invention employs a depth-first search (DFS) approach, traversing each node in the operator loop tree and its internal loop dimensions layer by layer. It statistically analyzes the tensor changes of each node during loop changes, thereby calculating the data transfer volume of each node during execution. In the calculation, tensor changes are categorized into input tensor changes and output tensor changes: for input tensors, the change is the new tensor size minus the intersection of the old and new tensors; for output tensors, the change is directly equal to the new tensor size.

[0055] b) Latency Analysis: After obtaining the data transfer volume of each node, the overall computation latency is further calculated, specifically including:

[0056] i. Calculate the data ingress and egress latency corresponding to each node based on the data transfer volume of each node and the bandwidth information given in Bc; if the node is a computing node, calculate the computing latency of the node in addition to the computing volume and the computing power of the node.

[0057] ii. Based on the node delays and operator loop tree structure, a Depth-First Search (DFS) traversal is used to construct a delay pipeline from top to bottom. The analysis assumes that the operator continuously utilizes a double-buffering mechanism during execution, allowing computation and communication processes within the pipeline, as well as communication between different levels, to overlap. Furthermore, the structure of nodes with different ranges will lead to different delay coverage effects: the delays of child nodes in sequential nodes do not overlap; the delays of child nodes in pipeline nodes and shared nodes can overlap during the input and output stages, i.e., the output delay of the previous child node and the input delay of the next child node are superimposed; the delays of child nodes in parallel nodes completely overlap, and the total delay depends on the longest delay among all child nodes.

[0058] iii. Calculate the total latency of the computation pipeline starting from the root node, and obtain the total latency of the current operator under the current hardware architecture.

[0059] F. Real-world performance measurement and comparison: This invention compares the performance estimated by this invention with that of ACLNN, the official computing library provided by Huawei's intelligent chips, under the same operator mapping method and hardware architecture. The comparison chart is shown in... Figure 4 As shown in the figure, the present invention compared the two in three scenarios: matrix multiplication, vector computation, and vector fusion computation. The vertical axis represents latency, and the horizontal axis represents the operator number tested. By calculating the Pearson correlation coefficient, it can be seen that the model proposed in this invention achieves correlation coefficients of 0.996, 0.998, and 0.997 with the actual test results, respectively, reflecting the accurate trend prediction of the model for actual hardware performance.

[0060] G. Performance Optimization Guidance: To optimize operator performance using the analysis results provided in this invention, this invention demonstrates the computational optimization process for matrix multiplication operators in various large language model networks (including BERT-S / B / L, VIT14-L / B / H, and T5 networks) in the field of natural language processing, guided by the analysis results (e.g., Figure 5 (As shown in the figure). Original block partitioning refers to not performing any block optimization operations on the operator; block partitioning on L1 refers to partitioning only at the L1 cache level; block partitioning on L0AB refers to partitioning only at the L0AB cache level; and block partitioning on both L1 and L0AB indicates block optimization at both levels. The figure illustrates the impact of adjusting different block partitioning strategies on operator computation performance under the guidance of this invention. After adjusting and optimizing the block partitioning strategy, compared to the original block partitioning, the block partitioning strategy guided by this invention can provide a 22-120 times speedup for matrix multiplication operators, improving computation speed.

[0061] Finally, it should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the scope of the claims.

Claims

1. A method for modeling analysis and computational optimization of operators on smart chips, characterized in that, The hardware architecture and interconnection mode of the intelligent chip are modeled by graph representation; a linkage mechanism of the node of the operator circulation tree is designed, so that the operator circulation is described and mapped in the hardware architecture model of the graph representation; a pipeline analysis method is used to analyze the delay performance of single operators and fusion operators in various combinations, to obtain the performance of the operators under the hardware architecture, and further to perform calculation optimization and acceleration; including: 1) extracting core operators from the core calculation process; 2) constructing an intelligent chip hardware graph structure based on the graph representation for the hardware architecture of the intelligent chip; The intelligent chip hardware graph structure includes multiple nodes, each of which represents a memory level or a calculation unit; 3) describing the interconnection structure of the intelligent chip, constructing an intelligent chip directed graph, including interconnected nodes, interconnection bandwidth, and interconnection type; 4) describing the type and form of the core operator to obtain operator description; 5) describing the operator mapping mode; 6) checking the legality of the description, including matching the circulation parameters, matching the circulation dimensions, and matching the mapping relationship, so that the operator mapping conforms to the restrictions of the hardware architecture and the parameter restrictions of the operator type; 7) analyzing the delay of the mapped and matched operators on the intelligent chip hardware; 8) adjusting and optimizing the block strategy of the input operator according to the operator mapping mode and the obtained delay analysis result, and the adjustment and optimization methods include adjusting the circulation axis order, adjusting the circulation parameter size, and adjusting the block strategy; The optimized block strategy is returned to the application of the input operator to optimize the application calculation performance.

2. The method of claim 1, wherein the modeling, analysis and computation optimization of the operator performance on the smart chip is characterized by, The core operators extracted from the core calculation process include matrix multiplication operators and attention operators.

3. The method of claim 1, wherein the modeling, analysis and computation optimization of the operator performance on the smart chip is characterized by, In step 2), the memory level includes high-bandwidth storage, cache, and register; the calculation unit includes a matrix multiplication acceleration unit and a vector acceleration unit; the node description is stored in the Arch.yaml data serialization format.

4. The method of claim 3, wherein the modeling, analysis and optimization of the performance of the operator on the smart chip is performed by a computer system. In step 3), in the constructed intelligent chip directed graph, the starting node and the ending node are marked between different interconnection nodes, and the interconnection bandwidth and the interconnection type are described; the interconnection type includes half duplex, full duplex, unicast, and broadcast; the interconnection description is stored in the Interconnection.yaml data serialization format.

5. The method of claim 4, wherein the modeling, analysis and optimization of the performance of the operator on the smart chip is performed by a computer system. In step 4), the core calculation operator is described, including: input and output tensors, calculation dimensions, calculation types, and calculation actual values, each tensor is bound to the corresponding dimension; the operator description is stored in the Prob.yaml data serialization format.

6. The method for modeling, analysis and computation optimization of operator performance on smart chip as claimed in claim 5 wherein, In step 5), the operator is mapped to the graph representation of the intelligent chip hardware architecture by the form of the circulation tree; during the mapping process, the circulation axis of the operator is first subjected to multiple block operations and is mapped to the corresponding hardware; for a one-vector addition, the input vectors A[M,N] and B[M,N] are outputted as the vector C[M,N], i.e., the vector dimension is [M,N], the initial calculation circulation axis is [M,N], the circulation axis is subjected to a block operation, and the corresponding block after the block operation is mapped to the intelligent chip hardware.

7. The method of claim 6, wherein the modeling, analysis and optimization of the performance of the operator on the smart chip is performed by a computer system. Two partition operations are performed, specifically converting the calculation loop axis into [M2, N2, M1, N1, M0, N0], where the dimension M=M2*M1*M0, and the dimension N=N2*N1*N0; the corresponding partition obtained after the two partition operations is [{M2, N2}-HBM, {M1, N1}-Buffer, {M0, N0}-Reg, {1}-PE], where HBM represents high bandwidth memory, the partition of HBM is the root node of the loop tree, the partition on the processing unit PE is the leaf node of the loop tree, that is, the calculation node, and the calculation is completed in the leaf node; the rest of the partitions on the memory level are intermediate nodes of the loop tree, that is, storage nodes; define the data transfer node as an additional leaf node inserted at the position where cross-level data transfer is required; design the range node for fusion operator analysis, and the operators represented by the first-level child nodes under the range node are analyzed in a fusion method.

8. The method of claim 7, wherein the modeling, analysis and computation optimization of the operator performance on the smart chip is performed by a computer system. The fusion method includes sequential execution, shared input data, pipelined parallelism, and parallel execution.

9. The method of claim 8, wherein the modeling, analysis and computation optimization of the operator performance on the smart chip is performed by a computer system. In step 7), the method for analyzing and calculating the delay specifically includes: First, the operator loop tree is traversed in a depth-first traversal method, and in the traversal process, the range of variables contained in each memory level is recorded when the loop and the hardware level change each time, and the data transfer amount / computational amount is obtained; Second, after obtaining the data transfer amount, the data transfer time of each memory level under the current traversal is obtained according to the obtained hardware connection relationship and bandwidth; Third, according to the computing power of the calculation unit and the obtained computational amount, the time required by the calculation unit to consume is obtained; Finally, a top-down delay pipeline is constructed according to the data transfer in, data transfer out, and calculation time of each node, the delay of the node under the parallel level is calculated according to the node relationship of the fusion method, and thus the delay of the entire operator is obtained.

Images