Transformer model mapping method under distributed computing architecture

Abstract

The present invention discloses a Transformer model mapping method under a distributed computing architecture, comprising the following steps: S1: dividing a single window attention calculation task branch and distributing it to nodes of a P×H rectangular distribution; S2: setting a mapping strategy and a weight parameter sharing strategy for the window attention calculation task after division; S3: aiming to minimize the end-to-end execution time of model reasoning on a distributed computing chip architecture, a division and mapping scheme for the window attention calculation task in the network layer of all stages of the Transformer model is formulated based on a division and mapping algorithm of integer linear programming; the division and mapping method and system provide an optimal window attention calculation task division and mapping scheme to maximize the utilization of computing resources and minimize the running time of the Transformer model reasoning task.

Description

Technical Field

[0001] The present invention relates to the field of distributed computing technology, and in particular to a Transformer model mapping method under a distributed computing architecture. Background Art

[0002] Transformer is a deep neural network based on the self-attention mechanism, which has achieved great success in many fields such as natural language processing, computer vision and video analysis. The self-attention mechanism is considered to be a storage-intensive operation.

[0003] Computing architectures based on on-chip distributed storage (such as in-memory computing and near-memory computing accelerators) have the advantages of scalable high storage bandwidth, low memory access overhead, and large-scale parallel processing. They can effectively alleviate the performance bottleneck of the Transformer model and significantly accelerate model reasoning.

[0004] The Transformer model based on the self-attention mechanism has powerful global modeling capabilities, and the time complexity of attention calculation is O(n 2 ), dense global attention will lead to excessive memory and computational costs, limiting the scalability of the model. Summary of the invention

[0005] Based on the technical problems existing in the background technology, the present invention proposes a Transformer model mapping method under a distributed computing architecture, and gives the optimal window attention computing task division and mapping scheme to maximize the utilization of computing resources and minimize the execution time of the Transformer model reasoning task on the distributed computing architecture.

[0006] The Transformer model mapping method under the distributed computing architecture proposed in the present invention, the Transformer model is composed of multiple network layers connected in series, and each network layer contains several attention-based computing task branches. The mapping method includes the following steps:

[0007] S1: Divide the single window attention calculation task branch and assign it to nodes with a P×H rectangular distribution. Set the current node to receive the data transmitted by the previous adjacent node and transmit data to the next adjacent node. The last node transmits data to the first node. P means that the input image patch is divided into P parts, and H means that the attention head is divided into H parts.

[0008] S2: Set the mapping strategy and weight parameter allocation strategy for the window attention calculation task after division;

[0009] S3: With the goal of minimizing the end-to-end execution time of model inference on a distributed computing chip architecture, a partitioning and mapping algorithm based on integer linear programming is developed to partition and map the window attention calculation tasks in the network layers of all stages of the Transformer model.

[0010] The partitioning and mapping scheme is specifically formulated as follows:

[0011] Objective function:

[0012]

[0013] Constraints:

[0014]

[0015]

[0016]

[0017]

[0018] Para_volume+Dynamic_volume≤Given_capacity

[0019] in, They are the dot multiplication time of the P×H matrix after the window attention calculation task is divided, the data transmission time and the weight sharing time. is the number of network layers in the sth stage of the network model, X s,i,j,u,v Indicates that each network layer in the sth stage has i window attention tasks to do U at the jth time layer s [u]×V s [v] is divided into 1≤s≤N stage , U s and V s represents an array, u and v represent array U respectively s and V s The index of Indicates the maximum number of batches when all window attention tasks on the same network layer in the sth stage of the network model are processed in batches, which is equal to the number of windows represents the number of all window attention tasks on the same network layer in the sth stage of the network model, N stage Indicates that the network model has N stagestages, Para_volume represents the storage space required for the network parameter weights on each node, Dynamic_volume represents the storage space required for the intermediate data generated during network inference calculations, Given_capacity represents the memory space of the node, · represents the product, N u,v Indicates the division of window attention tasks as U s [u]×V s [v] is the maximum number of window attention tasks that can be accommodated on the chip's computing node array, and u indicates that the multiple heads in the window attention task are divided into U s [u] equal parts, v means the patch is divided into V in the window attention task s [v] Equal parts, Num s,j Represents the number of window attention tasks at the jth temporal layer for each network layer in the sth stage of the network model.

[0020] Furthermore, in the step S2: setting the mapping strategy and weight parameter allocation strategy of the window attention calculation task after division, the mapping strategy of the window attention calculation task after division is specifically:

[0021] Parallel processing: The window attention calculation tasks on the Transformer Block of the same network layer are assigned to the computing nodes of the chip for parallel processing;

[0022] Batch processing: Process window attention calculation tasks in batches in parallel. The chip's computing array has the ability to handle multiple window attention calculation tasks;

[0023] Equal division: The window attention calculation tasks processed in the same batch are divided in the same way, and different batches form different time layers;

[0024] Storage capacity constraint: After the window attention calculation task is divided, the storage unit of each node saves the weight matrix parameters of the current window attention calculation task, and stores the intermediate calculation results of the current window attention calculation task. The storage space of the weight parameters and the intermediate calculation results is not greater than the capacity of the storage unit corresponding to the node.

[0025] Furthermore, in equal division, the attention calculation task of a single window after division is processed in the same time layer due to the data flow between nodes.

[0026] Furthermore, in the step S2: setting the mapping strategy and weight parameter allocation strategy of the window attention calculation task after division, the weight parameter allocation strategy of the window attention calculation task after division is specifically:

[0027] The weight matrix is ​​divided into rows and evenly distributed to the node array of the window attention calculation task;

[0028] The weight parameters of window attention calculation tasks of different batches are reused.

[0029] Furthermore, in the reuse of weight parameters of window attention calculation tasks in different batches, it specifically includes:

[0030] Condition 1) All weight parameters already exist: If a single storage unit contains all weight parameters of the window attention calculation task, all batches reuse the above weight parameters;

[0031] Condition 2) The division method is the same: If the division method of the window attention calculation tasks of different batches is the same, the weight parameters are reused.

[0032] Furthermore, when the weight parameters of window attention calculation tasks of different batches are reused, the weight parameters Para_volume and vol on each node s,j , as follows:

[0033] Para_volume=∑ j vol s,j

[0034]

[0035] Parameter reuse is converted into linear expression as follows:

[0036]

[0037]

[0038]

[0039] Among them, vol s,j represents the storage space required for the weight parameters of the jth network layer in the sth stage of the network model, C s represents the feature size at the sth stage of the Transformer model, Y s,j,j′,u,v represents the division of the jth time layer and the j′th time layer in the sth stage of the network model. s,j,j′,u,v =0 means the same division method, when Y s,j,j′,u,v =1 indicates different division methods, auxiliary binary variable Z s,j,u,v Indicates whether the j-th time layer of the s-th stage of the network model reuses the weight parameters of other batches. s,j,u,v = 0 means that the weight parameters of other batches cannot be reused. s,j,u,v =1 means that the weight parameters of other batches will be reused, X s,i,j′,0,0Indicates that there are i windows of attention tasks on each network layer in the sth stage, and no division is done on the j′th time layer, that is, each window is processed on one computing node, X s,i,j′,u,v Indicates that each network layer in the sth stage has i window attention tasks to do U on the j′th time layer s [u]×V s [v] Division method.

[0040] Further, in step S1: the window attention calculation task is divided into P×H parts, and the divided window attention calculation task is assigned to the nodes of the P×H rectangular distribution, the current node is set to receive the transmission data of the previous adjacent node and transmit data to the next adjacent node, and the last node transmits data to the first node. P represents the patch of multi-head attention, and H represents the head of multi-head attention. The window attention calculation task is divided into P×H parts, which specifically includes:

[0041] Divide the input feature map of the window attention calculation task into P equal parts, assign them to different computing nodes, and synchronously process the computing tasks in the same network layer;

[0042] Each network layer contains multi-head self-attention. According to the computational tasks of different heads in the multi-head self-attention mechanism, the computational tasks are divided into H equal parts;

[0043] The window attention calculation task is divided into P×H equal parts. When the single window attention calculation task branch of the upper network layer of the Transformer model at stage s contains Ω s attention heads and Φ s When there are patches, they are assigned to the subarray P×H, where P is Any value in , H is Any value in Among them, Λ min represents the minimum granularity of a patch applicable to a single computing node on the computing architecture, and u and v are arrays U s and V s The index of .

[0044] Furthermore, after dividing the input feature map of the window attention calculation task into P equal parts, the key matrix and query matrix of self-attention are transferred between nodes.

[0045] The advantages of the Transformer model mapping method under the distributed computing architecture provided by the present invention are: the Transformer model mapping method under the distributed computing architecture provided in the structure of the present invention divides the window attention calculation tasks of the model, processes them in batches in parallel on the chip computing nodes, and sets the data flow form between nodes; a partitioning and mapping algorithm based on integer linear programming is given, and under the storage space constraints of the computing nodes, all window attention calculation tasks are divided, and the chip computing nodes are mapped to maximize the chip computing resource utilization. According to the node calculation time cost after the division and the data flow transmission time cost between nodes, the optimal window attention calculation task partitioning and mapping scheme is given to maximize the computing resource utilization and minimize the running time of the Transformer model reasoning task. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a structural schematic diagram of the present invention;

[0047] Figure 2 Schematic diagram of data flow between nodes after the window attention calculation task is divided and assigned to the nodes in the P×H rectangular distribution. DETAILED DESCRIPTION

[0048] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0049] Transformer neural networks often use a local attention mechanism to balance performance, computational complexity, and memory usage. The model as a whole is a serial structure, usually with four stages. Each stage has several network layers in series, and the input feature map of each network is divided into several fixed areas as the input of the local attention mechanism. For example, in Swin Transformer, the input feature map is divided into local (window) attention calculation tasks of size M. The pixels in the local (window) are used as the input of the network layer (Transformer Block). The calculation tasks of the network layer mainly include four parts: row normalization (LayerNorm), multi-head self-attention (MSA), row normalization (LayerNorm), and multi-layer perceptron (MLP).

[0050] The on-chip distributed neural network accelerator targeted by this embodiment is usually composed of N nodes, each of which is composed of local storage (which can reach more than 8MB), a neural network computing core (matrix point multiplication computing core, such as a systolic array), a storage controller, an on-chip network router, etc. The processing core can directly access the local storage unit through the storage controller, and can also access the memory units in other remote nodes through the on-chip network; the nodes are connected by an on-chip network structure based on a regular topology such as a mesh.

[0051] This embodiment provides a partitioning and mapping method for the Transformer model on a distributed computing architecture on a chip. The network layer computing tasks in the four stages of the Transformer model will be mapped to the chip one by one for execution, and each network layer will occupy all computing nodes on the chip. First, the partitioning method and data flow form of the model are given; then, a partitioning and mapping method based on integer linear programming is given to minimize the running time of the model. In this article, the local (window) attention computing task corresponds to the window attention computing task, but the expression method is different, as follows:

[0052] like Figure 1 and 2 As shown, the Transformer model mapping method under the distributed computing architecture proposed in the present invention, the Transformer model is composed of multiple network layers connected in series, and each network layer contains several attention-based computing task branches, and the mapping method includes the following steps S1 to S3:

[0053] S1: Divide the single window attention calculation task branch and assign it to nodes with a P×H rectangular distribution. Set the current node to receive the data transmitted by the previous adjacent node and transmit data to the next adjacent node. The last node transmits data to the first node. P means that the input image patch is divided into P parts, and H means that the attention head is divided into H parts.

[0054] Both P and H are variables to achieve flexible division. P means that the input image patch is divided into P parts, and H means that the attention head is divided into H parts. When the single computational task branch of the upper network layer of the Transformer model in the sth stage contains Ω s attention heads and Φ s When there are patches, they are allocated to a subarray of variable size P×H, then P can be Any value in , and H can be Any value in And min is the minimum granularity of a patch applicable to a single computing node on the computing architecture, u and v are the arrays U s and Vs The index of .

[0055] According to the characteristics of the Transformer model based on local attention, the division method and data flow form of the local (window) attention task are set to achieve high computing resource utilization on the distributed computing chip frame; the Transformer model based on local attention (Local Attention Enhanced Transformer) divides the input feature map into several windows at each stage of the network layer and uses it as the input of the network layer (Transformer Block) to realize local attention calculation. In order to realize the parallel processing of local (window) attention calculation tasks, it is divided equally according to the patches in the window and the heads of multi-head attention. If the local (window) attention calculation task is set to U s [u]×V s [v] is divided into U s [u] Equal parts, the patch in the window is divided into V s [v] Equal parts. Local (window) attention calculation task Local (window) attention calculation task, specifically:

[0056] Division method: If the model is allocated and mapped to the computing nodes of the chip in units of window attention computing tasks, it will lead to extremely low computing node utilization. Therefore, in order to maximize the utilization of computing nodes, this embodiment provides two dimensions for dividing the window attention computing tasks: ① row division, the input feature map of the window attention computing task is divided into P equal parts, allocated to different computing nodes, and the computing tasks in the same network layer are processed synchronously; ② column division, the network layer contains multi-head self-attention, and the computing tasks of each head can be processed in parallel. According to the different head computing tasks in the multi-head self-attention mechanism, the computing tasks are divided into H equal parts. In this way, the attention computing task of a single local (window) will be divided into P equal parts, and then mapped to different computing nodes for parallel processing.

[0057] Layout and data flow form after division: The window attention calculation task consists of four parts. After division, there is a lot of data dependency between nodes and a strong coupling relationship exists. This embodiment gives the task layout after division and the data flow form between nodes.

[0058] In order to simplify the data flow form and data routing complexity, the divided local (window) attention tasks are assigned to P computing nodes distributed in a rectangular pattern; after being divided into P equal parts by row, multi-head attention related data flows will be generated between nodes, that is, the key matrix and query matrix of self-attention are transmitted between nodes; after being divided into H equal parts by column, matrix operations such as multi-layer perceptron and multi-head attention linear transformation will be generated between these nodes, such as Figure 2 , here a circular data flow method is adopted. Except for the last node transmitting data to the first node, other nodes only transmit data to one adjacent node and receive data from another adjacent node. After each node passes data P-1 times or H-1 times, all nodes will receive data from other nodes.

[0059] S2: Set the mapping strategy and weight parameter allocation strategy for the window attention calculation task after division;

[0060] In view of the limited storage space on the nodes of the distributed architecture, in order to maximize the communication efficiency of the system and minimize the communication cost, the mapping strategy and weight parameter allocation strategy of the local (window) attention calculation task after division are set as follows:

[0061] Mapping strategy of local (window) attention calculation task after division: On N nodes, perform local (window) attention calculation task in the network layer of Transformer model, assuming that there are There are local (window) attention calculation tasks. The mapping criteria for the local (window) attention calculation tasks are as follows:

[0062] 1) Parallel processing: local (window) attention calculation tasks on the same network layer can be assigned to the computing nodes of the chip for parallel processing;

[0063] 2) Batch (time layer) processing: Process local (window) attention calculation tasks in batches and in parallel;

[0064] 3) Equal division: The local (window) attention calculation tasks processed in the same batch must be divided in the same way. Different batches form different time layers. In particular, the single local (window) attention calculation task after division must be processed in the same time layer due to the data flow between nodes.

[0065] 4) Satisfy storage capacity constraints: After the window attention calculation task is divided, the storage unit of each node saves the weight matrix parameters of the current window attention calculation task, and stores the intermediate calculation results of the current window attention calculation task. It is necessary to ensure that the storage space of the weight parameters and the intermediate calculation results is not greater than the capacity of the storage unit corresponding to the node.

[0066] The weight parameters after division are distributed and stored: the weight matrix is ​​divided by rows and evenly distributed to the node array of the local (window) attention calculation task; under the storage resource constraints of the distributed computing chip architecture, the weight parameters of the local (window) attention calculation tasks of different batches (time layers) are reused to reduce the storage space requirements during model calculation:

[0067] Condition 1) All weight parameters are available: If a single storage unit (Bank) contains all weight parameters for a single attention calculation task, all temporal layers can reuse these parameters.

[0068] Condition 2) The division method is the same: If the local (window) division methods of different time layers are the same,

[0069] Then the weight parameters can be reused.

[0070] S3: With the goal of minimizing the end-to-end execution time of model inference on a distributed computing chip architecture, a partitioning and mapping algorithm based on integer linear programming is developed to partition and map the window attention calculation tasks in the network layers of all stages of the Transformer model.

[0071] With the goal of maximizing computing resource utilization and minimizing the end-to-end execution time of the Transformer model inference task, a task partitioning and mapping algorithm based on integer linear programming is given below: According to the mapping strategy of the window attention calculation task after partitioning and the weight parameter sharing strategy given above, a partitioning and mapping algorithm based on integer linear programming is given below to achieve the minimum network model running time.

[0072] Since the whole model has s stages, each stage has B s The computing node needs to store the weight parameters of all network layers. The partitioning and mapping method provided in this embodiment can simultaneously provide a partitioning and mapping scheme for local (window) tasks in all network layers under the constraint of the storage unit capacity in the node. The specific algorithm steps are as follows:

[0073] Objective functionMinimize Obj:

[0074]

[0075] in, They are the dot multiplication time of the P×H matrix after the window attention calculation task is divided, the data transmission time and the weight sharing time. is the number of network layers in the sth stage of the network model.

[0076] Constraints:

[0077]

[0078]

[0079]

[0080]

[0081] Para_volume+Dynamic_volume≤Given_capacity (5)

[0082] Among them, X s,i,j,u,v Indicates that each network layer in the sth stage has i window attention tasks to do U at the jth time layer s [u]×V s [v] is divided into 1≤s≤N stage , U s and V s represents an array, u and v represent array U respectively s and V s The index of Indicates the maximum number of batches when all window attention tasks on the same network layer in the sth stage of the network model are processed in batches, which is equal to the number of windows represents the number of all window attention tasks on the same network layer in the sth stage of the network model, N stage Indicates that the network model has N stage stages, Para_volume represents the storage space required for the network parameter weights on each node, Dynamic_volume represents the storage space required for the intermediate data generated during network inference calculations, Given_capacity represents the memory space of the node, · represents the product, N u,v Indicates the division of window attention tasks as U s [u]×V s [v] is the maximum number of window attention tasks that can be accommodated on the chip's computing node array, and u indicates that the multiple heads in the window attention task are divided into U s [u] equal parts, v means the patch is divided into V in the window attention task s [v] Equal parts, Num s,j Represents the number of window attention tasks at the jth temporal layer for each network layer in the sth stage of the network model.

[0083] Constraint (1) represents the constraint on the number of local (window) attention computation tasks processed in parallel at each temporal layer;

[0084] Constraint (2) means that all windows are assigned to the temporal layer;

[0085] Constraint (3) means that the local (window) attention calculation task of each temporal layer adopts the same partitioning scheme;

[0086] Constraint (4) indicates that the local (window) attention computation task on the temporal layer is incremental;

[0087] Constraint (5) represents the storage constraint, that is, the sum of the memory space of the parameter weight and the memory space of the intermediate data on each node does not exceed the memory space corresponding to the node.

[0088] What needs to be explained is the reuse of weight parameters:

[0089]

[0090]

[0091] It is converted into a linear expression:

[0092] Auxiliary parameter 1:

[0093] The linear expression is:

[0094] 0≤j′ <j<layer num

[0095] Auxiliary parameter 2:

[0096] The linear expression is:

[0097] at last:

[0098]

[0099] Para_volume=∑ j vol j

[0100] Among them, vol s,j represents the storage space required for the weight parameters of the jth network layer in the sth stage of the network model, C s represents the feature size at the sth stage of the Transformer model, Y s,j,j′,u,v represents the division of the jth time layer and the j′th time layer in the sth stage of the network model. s,j,j′,u,v =0 means the same division method, when Y s,j,j′,u,v =1 indicates different division methods, auxiliary binary variable Z s,j,u,v Indicates whether the j-th time layer of the s-th stage of the network model reuses the weight parameters of other batches. s,j,u,v = 0 means that the weight parameters of other batches cannot be reused.s,j,u,v =1 means that the weight parameters of other batches will be reused, X s,i,j′,0,0 Indicates that there are i windows of attention tasks on each network layer in the sth stage, and no division is done on the j′th time layer, that is, each window is processed on one computing node, X s,i,j′,u,v Indicates that each network layer in the sth stage has i window attention tasks to do U on the j′th time layer s [u]×V s [v] Division method.

[0101] It should be noted that condition ① corresponds to: Condition 1) All weight parameters already exist: If a single storage unit contains all weight parameters of the window attention calculation task, all batches reuse the above weight parameters; Condition ② corresponds to: Condition 2) The division method is the same: If the division method of the window attention calculation tasks of different batches is the same, the weight parameters are reused.

[0102] Existing research work only focuses on the parallel processing of the attention mechanism of the Transformer model on the hardware accelerator, and does not involve the partitioning and mapping of the model's end-to-end reasoning execution. In addition, the currently popular Transformer model introduces a local self-attention strategy, and each layer of the network consists of a large number of self-attention computing tasks. In order to achieve large-scale and efficient parallel processing of Transformer model reasoning tasks based on local attention, a partitioning, mapping and mapping method for the end-to-end execution of the Transformer model on an on-chip distributed storage accelerator is proposed in the compilation stage; according to steps S1 to S3, the local (window) attention computing tasks of the model are divided and processed in batches on the chip computing nodes in parallel, and the data flow form between nodes is set; a partitioning and mapping algorithm based on integer linear programming is given, and under the storage space constraints of the computing nodes, all local (window) attention computing tasks are divided, and chip computing nodes are mapped to maximize the chip computing resource utilization. According to the node computing time cost after the division and the data flow transmission time cost between nodes, the optimal local (window) attention computing task partitioning and mapping scheme is given to maximize the computing resource utilization and minimize the running time of the reasoning task.

[0103] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. Transformer model mapping method under distributed computing architecture, characterized in that: The Transformer model is composed of multiple network layers connected in series, and each network layer contains several attention-based computing task branches. The mapping method includes the following steps: S1: Divide the single window attention calculation task branch and assign it to nodes with a P×H rectangular distribution. Set the current node to receive the data transmitted by the previous adjacent node and transmit data to the next adjacent node. The last node transmits data to the first node. P means that the input image patch is divided into P parts, and H means that the attention head is divided into H parts. S2: Set the mapping strategy and weight parameter allocation strategy for the window attention calculation task after division; The mapping strategy is as follows: Parallel processing: The window attention calculation tasks on the Transformer Block of the same network layer are assigned to the computing nodes of the chip for parallel processing; Batch processing: Process window attention calculation tasks in batches in parallel. The chip's computing array has the ability to handle multiple window attention calculation tasks; Equal division: The window attention calculation tasks processed in the same batch are divided in the same way, and different batches form different time layers; Storage capacity constraint: After the window attention calculation task is divided, the storage unit of each node saves the weight matrix parameters of the current window attention calculation task, and stores the intermediate calculation results of the current window attention calculation task. The storage space of the weight parameters and the intermediate calculation results is not greater than the capacity of the storage unit corresponding to the node; Among them, the weight parameter allocation strategy is specifically as follows: The weight matrix is ​​divided into rows and evenly distributed to the node array of the window attention calculation task; The weight parameters of window attention calculation tasks in different batches are reused; S3: With the goal of minimizing the end-to-end execution time of model inference on a distributed computing chip architecture, a partitioning and mapping algorithm based on integer linear programming is developed to partition and map the window attention calculation tasks in the network layers of all stages of the Transformer model. The partitioning and mapping scheme is specifically formulated as follows: Objective function: Constraints: Para_volume+Dynamic_volume≤Given_capacity in, They are the point multiplication calculation time of the P×H matrix after the window attention calculation task is divided, the data transmission time and the weight sharing time. is the number of network layers in the sth stage of the network model, X s,i,j,u,v Indicates that there are i window attention tasks on each network layer in the sth stage, and U is performed on the jth time layer s [u]×V s [v] is divided into 1≤s≤N stage , U s and V s represents an array, u and v represent array U respectively s and V s The index of Indicates the maximum number of batches when all window attention tasks on the same network layer in the sth stage of the network model are processed in batches. Equal to the number of windows Represents the number of all window attention tasks on the same network layer in the sth stage of the network model, N stage Indicates that the network model has N stage stages, Para_volume represents the storage space required for the network parameter weights on each node, Dynamic_volume represents the storage space required for the intermediate data generated during network inference calculations, Given_capacity represents the memory space of the node, · represents the product, N u，v Indicates the division of window attention tasks as U s [u]×V s [v] is the maximum number of window attention tasks that can be accommodated on the chip's computing node array, and u indicates that the multiple heads in the window attention task are divided into U s [u] equal parts, v means the patch is divided into V in the window attention task s [v] Equal parts, Num s，j Represents the number of window attention tasks at the jth time layer for each network layer in the sth stage of the network model.

2. The Transformer model mapping method under the distributed computing architecture according to claim 1, characterized in that: In equal division, the attention calculation task of a single window after division is processed in the same time layer because of the data flow between nodes.

3. The Transformer model mapping method under the distributed computing architecture according to claim 1, characterized in that: In the reuse of weight parameters of window attention calculation tasks in different batches, specifically including: Condition 1) All weight parameters already exist: If a single storage unit contains all weight parameters of the window attention calculation task, all batches reuse the above weight parameters; Condition 2) The division method is the same: If the division method of the window attention calculation tasks of different batches is the same, the weight parameters are reused.

4. The Transformer model mapping method under the distributed computing architecture according to claim 3 is characterized in that: When the weight parameters of window attention calculation tasks of different batches are reused, the weight parameters Para_volume and vol on each node s,j , as follows: For_volume=∑ j vol s，j Parameter reuse is converted into linear expression as follows: Among them, vol s，j represents the storage space required for the weight parameters of the jth network layer in the sth stage of the network model, C s represents the feature size at the sth stage of the Transformer model, Y s，j，j′，u，v represents the division of the jth time layer and the j′th time layer in the sth stage of the network model. s,j，j′,u,v =0 means the same division method, when Y s，j，j′，u，v =1 indicates different division methods, auxiliary binary variable z s,j,u，v Indicates whether the j-th time layer of the s-th stage of the network model reuses the weight parameters of other batches. s，j，u，v = 0 means that the weight parameters of other batches cannot be reused. s，j，u，v =1 means that the weight parameters of other batches will be reused, X s,i，j′，0，0 Indicates that there are i window attention tasks on each network layer in the sth stage without any division at the j′th time layer, X s，i，j′,u,v Indicates that each network layer in the sth stage has i window attention tasks to do U on the j′th time layer s [u]×V s [v] Division method.

5. The Transformer model mapping method under the distributed computing architecture according to claim 1, characterized in that: In step S1: the window attention calculation task is divided into P×H parts, and the divided window attention calculation task is assigned to the nodes of the P×H rectangular distribution. The current node is set to receive the transmission data of the previous adjacent node and transmit data to the next adjacent node. The last node transmits data to the first node. P represents the patch of multi-head attention, and H represents the head of multi-head attention. The window attention calculation task is divided into P×H parts, which specifically includes: Divide the input feature map of the window attention calculation task into P equal parts, assign them to different computing nodes, and synchronously process the computing tasks in the same network layer; Each network layer contains multi-head self-attention. According to the computational tasks of different heads in the multi-head self-attention mechanism, the computational tasks are divided into H equal parts; The window attention calculation task is divided into P×H equal parts. When the single window attention calculation task branch of the upper network layer of the Transformer model at stage s contains Ω s attention heads and Φ s When there are patches, they are assigned to the subarray P×H, where P is Any value in , H is Any value in Among them, A min represents the minimum granularity of a patch applicable to a single computing node on the computing architecture, and u and v are arrays U s and V s The index of .

6. The Transformer model mapping method under the distributed computing architecture according to claim 4, characterized in that: After dividing the input feature map of the window attention calculation task into P equal parts, the key matrix and query matrix of self-attention are transferred between nodes.

Images