A neural network load mapping method and device of a multi-core storage and computing integrated chip

Abstract

The application discloses a neural network load mapping method and device of a multi-core storage and computing integrated chip, and belongs to the field of artificial intelligence hardware acceleration and in-memory computing. The method extracts a matrix multiplication operator of a neural network and a data dependency relationship between the operators, obtains a neural network load directed computation graph, and then generates a storage and computing integrated chip load mapping result according to the neural network load directed computation graph and architecture parameters of the storage and computing integrated chip, and further generates a chip configuration program in combination with configuration register information in the chip architecture parameters, so as to realize automatic and efficient mapping of the neural network.

Description

Technical Field

[0001] This invention relates to a neural network load mapping method and apparatus for a multi-core in-memory computing chip, belonging to the field of artificial intelligence hardware acceleration and in-memory computing. Background Technology

[0002] Neural networks have generated immense practical value in today's society. With the iteration of algorithm technology, the number of parameters and computational load of neural network algorithms are experiencing explosive growth. However, the development of computing power in computing chips is hampered by constraints such as the memory wall and power consumption wall, making it difficult to meet the deployment and application tasks of neural network algorithms. Compute-in-Memory (CIM) technology, as a new architecture in the post-Moore's Law era, is expected to break through the computing power and power consumption bottlenecks of classic architectures, serving the deployment and application of neural network algorithms.

[0003] In-memory computing architecture is essentially based on an array of in-memory computing devices to achieve high-efficiency matrix multiplication (MM). By programming the weights of the in-memory computing devices using neural network algorithms, multiplication and accumulation calculations can be performed in-situ on the memory devices. Since the original neural network computation graph differs from the structural parameters of the in-memory computing architecture, the neural network workload needs to be decomposed and mapped before it can be deployed on the in-memory computing chip.However, current mapping tools are mostly based on manual arrangement or coarse-grained simple arrangement schemes. For example, the paper Zhu Z, Sun H, Xie T, et al. Mnsim 2.0: A behavior-level modeling tool for processing-in-memory architectures[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2023, 42(11): 4112-4125. proposes to model based on fully associative and equal data flow dependencies between source and destination nodes. This method will lead to a large number of non-existent dependencies in the data flow graph, which will block the data flow and cause a large number of memory computing units to be idle and waiting, greatly reducing hardware utilization. The paper Shafiee A, Nag A, Muralimanohar N, et al. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars[J]. ACM SIGARCH Computer Architecture News, 2016, 44(3): The methods proposed in 14-26 rely on manual mapping of computing nodes to avoid pipeline conflicts, but this is no longer feasible when mapping large-scale algorithm networks. The paper Qu S, Zhao S, Li B, et al. Cim-mlc: A multi-level compilation stack for computing-in-memoryaccelerators[C]. Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages ​​and Operating Systems, Volume 2. 2024: 185-200. proposes a chip-array-bitline level mapping strategy, but focuses on the quantitative analysis of virtual workloads and does not consider actual physical computation dependencies. Because these methods ignore the actual algorithm data flow or hardware architecture parameters, they struggle to automate the mapping of large-scale complex workloads and also fail to leverage the high computational efficiency of in-memory computing chips.

[0004] Hardware-software co-operation of computing chips is a key technology widely studied in the field of computer architecture, and efficient load mapping of neural networks is a crucial part of it. In applications with high requirements for power consumption, computing resources, or performance, automated neural network load mapping tools will play a vital role, but the existing mapping strategies mentioned above cannot achieve highly efficient automated mapping. Summary of the Invention

[0005] To accomplish the neural network load mapping task on a multi-core in-memory computing chip, this invention proposes an automated method and apparatus that integrates data flow analysis, optimization, and load mapping functions for matrix multiplication-intensive neural networks such as convolutional and fully connected networks.

[0006] The first objective of this invention is to provide a neural network load mapping method for a multi-core in-memory computing chip, comprising:

[0007] Step 1: Extract the original list of operators and the data dependencies between operators to construct the directed computation graph of the neural network load. ;

[0008] Step 2, based on the directed computation graph of the neural network load Size of cross array with in-memory computing chip The cross-array level directed computation graph is obtained. ;

[0009] Step 3, based on the directed computation graph of the cross array level The computational core-level computational graph was derived. ;

[0010] Step 4, based on the computational core-level computational graph Derivation of multi-core chip-level computing graph ;

[0011] Step 5, based on the multi-core chip-level computing graph The computational pipeline-level computational graph was derived. .

[0012] Optionally, step 1 includes:

[0013] Step 1.1, determine the original list of operators Each operator in The type corresponds to the node in the constructed computation graph;

[0014] Step 1.2: Construct a directed computation graph for the neural network load based on the data dependencies between nodes and operators in the computation graph. ;in For a neural network load, there is a set of nodes in a directed computation graph. Let the neural network load be a set of directed computation graph edges, with weights equal to the weights of its source nodes. Size.

[0015] Optionally, step 2 includes:

[0016] Step 2.1: For each directed computation graph node of the neural network load Flatten its dimensions into a two-dimensional tensor consisting of an accumulation dimension and a parallel dimension. ;

[0017] Step 2.2, flatten each node Divided into A set of child nodes ={ }; , and These represent the number of word lines and bit lines of the cross-array of the in-memory computing chip, respectively; the set of all child nodes constitutes... Nodes of a cross-array-level directed computation graph ;

[0018] Step 2.3: Based on the relationship between each node in the cross-array-level directed computation graph and the nodes in the original computation graph, extract the nodes of the original computation graph. A subset of attributes, which are attributes of child nodes, including subsets of input / output channels. Input / output feature map subset ;

[0019] Step 2.4: Reconstruct the edges between nodes in the cross-array level directed computation graph;

[0020] If there is a dependency between nodes, it means that there is a directed edge between the two nodes. The weight of the directed edge is calculated according to the following formula.

[0021]

[0022] in, For nodes The amount of output feature map data, for Number of output channels, for rely The number of channels for a node is calculated based on whether two nodes belong to the same computation graph. :

[0023]

[0024] Representative child node The node to which it belongs in the computation graph.

[0025] Optionally, step 3 includes:

[0026] Step 3.1, calculate the number of cross arrays in the core. The number of nodes in the cross-array level computation graph ,structure A set of computing core nodes Each computing core node consists of no more than It is formed by the union of nodes in the cross array level;

[0027] Step 3.2, for For each node, its data dependencies are calculated, including the amount of input / output feature map data. The amount of data reused for input / output feature maps within the computational core. And construct the clustering objective function: ;in These are non-negative hyperparameters;

[0028] Step 3.3: Based on the objective function in Step 3.2, minimize the objective function using the optimization algorithm. The sum of the objective functions corresponding to each core-level computing node;

[0029] Step 3.4: Construct the core-level computation graph by using the stable solution of the optimization algorithm as the final computational core-level computation graph node scheme. , and These are the set of nodes and the set of edges of the core-level computation graph, respectively.

[0030] Optionally, step 4 includes:

[0031] Step 4.1: For a 2D-mesh multi-core in-memory computing chip, perform computation graph node analysis for each computing core. Assign one-to-one corresponding multi-core chip physical nodes Its physical mapping location coordinates are ;

[0032] Step 4.2: Based on the physical mapping location, map the data dependency edges of the computation core-level computation graph to the NoC path, and count the data volume of each data dependency edge. and communication hop count ; t Indicates time;

[0033] Step 4.3: Minimize the objective function by swapping the physical node positions or by using an optimization algorithm that selects the optimal candidate node mapping one by one. , t Indicates time;

[0034] Step 4.4: Take the mapping result at the end of the optimization algorithm execution as the result of the load physical mapping, based on the mapping result of each physical computing core. Generates parameter information that the RISC-V register configuration program depends on. .

[0035] Optionally, step 5 includes:

[0036] Step 5.1: Set up the pipeline stage set based on the computational pipeline. For each stage of the process Extracting subgraphs from multi-core chip-level computation graphs ; and These represent subgraphs of a multi-core chip-level computation graph. The set of nodes and the set of edges;

[0037] Step 5.2, for each NoC physical link Construct its betweenness centrality measure as follows:

[0038]

[0039] in, and Subgraph The two nodes, Represents a multi-core chip-level computational subgraph Middle node and The number of shortest paths between them. Indicates that it passed through The number of shortest paths, represent to Data communication volume;

[0040] Step 5.3: Calculate subgraph edges based on multi-core chip level. Initialization status, statistics of physical links Congestion metric :

[0041]

[0042] in, Through physical link And the data volume is arranged from smallest to largest as follows: Bit-level connection path data volume To pass Total number of paths; when ,definition This means there is no congestion.

[0043] Step 5.4, Set of opposite edges by path Sort by shortest distance and data size in descending order, As for each NoC physical link The weights are used to calculate the path with the least congestion, which is then used as the scheduling result.

[0044] Step 5.5, based on the edge set The scheduling results are used to generate a set of packets for NoC route scheduling. .

[0045] The second objective of this invention is to provide a neural network load mapping device for a multi-core in-memory computing chip. The device is used to implement the above-mentioned method and includes: a load parsing unit, a load mapping unit, and a configuration information generation unit, which are connected in sequence. At the same time, the load parsing unit receives the externally input neural network, and the load mapping unit and the configuration information generation unit receive the externally input architecture parameters of the in-memory computing chip.

[0046] The load parsing unit is used to parse the input neural network ONNX representation. And extract the matrix multiplication operator from it. Data dependencies between operators The output requires a directed computation graph of a neural network workload that needs to be accelerated by an in-memory computing chip. The load mapping unit is used to perform directed computation of the neural network load graph. Architecture parameters of in-memory computing chips Generate in-memory computing chip load mapping results The configuration information generation unit is used to generate configuration information based on the load mapping results. and chip architecture parameters The configuration register information is used to generate the chip configuration program. .

[0047] Optionally, the load parsing unit includes an operator extraction subunit and a data stream reconstruction subunit, wherein the operator extraction subunit is used to extract data based on the input neural network ONNX representation. The metadata blocks of the operators are extracted, and the data flow reconstruction subunit uses these metadata blocks to extract computational load information. The window size of the convolution kernel is multiplied by the input channel size to obtain the cumulative dimension, and the parallel dimension is obtained based on the output channel size. Based on these two dimensions, computational nodes and data dependencies are constructed, resulting in a directed computational graph of the neural network load. .

[0048] Optionally, the load mapping unit includes an architecture data flow generation subunit, a physical mapping subunit, and a communication scheduling subunit, wherein the architecture data flow generation subunit constructs a directed computation graph of the neural network load based on the output of the load parsing unit. Based on the cross-array size in the input chip architecture parameters, the original computation graph nodes are split into cross-array level computation graphs. According to the number of cross-arrays for computation cores in the chip architecture parameters, the cross-array level nodes are merged to generate a computation core level computation graph. Based on the number of computation cores, on-chip network topology, physical mapping subunit, and communication scheduling subunit in the chip architecture parameters, the computation core level computation graph is sequentially mapped in the spatial and temporal domains to generate a multi-core chip-level computation graph and a computation pipeline level computation graph. Based on the multi-core chip-level computation graph and the computation pipeline level computation graph, the final load mapping result is generated. .

[0049] Optionally, the configuration information generation unit includes a mapping result parsing subunit and a configuration program generation subunit. The mapping result parsing subunit receives the mapping result Res output by the load mapping unit and outputs physical mapping information MInfo and communication scheduling information CInfo, which are respectively used to compile the configuration register writing program of the RISC-V CPU and the header information of the NoC data packet distribution logic. The configuration program generation subunit automatically generates the corresponding configuration program Conf based on the physical mapping information MInfo and communication scheduling information CInfo, combined with the register address information TAddr in the input chip architecture parameters.

[0050] The beneficial effects of this invention are:

[0051] First, a directed computation graph of the neural network load is constructed by extracting the original list of operators and the data dependencies between them. Then, based on the flattening and partitioning of each node in the directed computation graph, nodes in a cross-array-level directed computation graph are constructed. Next, the union of these nodes is performed to obtain the core-level nodes, thus constructing a core-level computation graph. Then, combined with the relevant parameters of the target in-memory computing chip architecture, the core-level computation graph is mapped to the actual physical chip, constructing a multi-core chip-level computation graph. Finally, based on the congestion analysis and optimization of the on-chip network in the actual computation pipeline, a pipeline-level computation graph is obtained. This invention achieves highly efficient automated mapping of large-scale complex neural network loads, enabling in-memory computing chips to effectively leverage their high computational efficiency and find wide application in fields with high power consumption, computational resource requirements, or high performance demands. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is an overall architecture diagram of the neural network load mapping device for a multi-core in-memory computing chip provided in an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the load parsing unit in the neural network load mapping device of the multi-core in-memory computing chip provided in this embodiment of the invention.

[0055] Figure 3 This is a schematic diagram of the load mapping unit in the neural network load mapping device of the multi-core in-memory computing chip provided in the embodiment of the present invention.

[0056] Figure 4 This is a schematic diagram of the configuration information generation unit in the neural network load mapping device of the multi-core in-memory computing chip provided in this embodiment of the invention.

[0057] Figure 5 This is a cross-array level computation graph obtained by the neural network load mapping method for multi-core in-memory computing chips provided in this embodiment of the invention.

[0058] Figure 6 This is a computational core-level computational graph obtained by the neural network load mapping method for multi-core in-memory computing chips provided in this embodiment of the invention.

[0059] Figure 7 This is a multi-core chip-level computation graph obtained by the neural network load mapping method for multi-core in-memory computing chips provided in this embodiment of the invention.

[0060] Figure 8 This is a computational pipeline-level computational graph obtained by the neural network load mapping method for multi-core in-memory computing chips provided in this embodiment of the invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0062] Example 1

[0063] This embodiment provides a neural network load mapping device and corresponding mapping method for a multi-core in-memory computing chip; see [link to documentation]. Figure 1The neural network load mapping device includes a load parsing unit, a load mapping unit, and a configuration information generation unit; these three are connected in sequence. Simultaneously, the load parsing unit receives external input from the neural network, and the load mapping unit and configuration information generation unit receive external input from the architecture parameters of the in-memory computing chip. Specifically:

[0064] Load parsing unit, used to parse the input neural network ONNX representation. And extract the matrix multiplication operator from it. Data dependencies between operators The output requires a directed computation graph of a neural network workload that needs to be accelerated by an in-memory computing chip. , where the set of nodes edge set . The first term of the directed computation graph is represented by the second term. i Each graph node , n The number of nodes in the graph; The first term of the directed computation graph is represented by the second term. j Edge, , k Let be the number of edges.

[0065] Load mapping unit, used to perform directed computation based on the load graph of the neural network. Architecture parameters of in-memory computing chips Generate in-memory computing chip load mapping results Architecture parameters Typically includes device data accuracy Cross array size Calculate the number of core cross arrays Calculate the number of cores On-chip network topology Register address information wait.

[0066] Configuration information generation unit, used to generate configuration information based on load mapping results. and chip architecture parameters The configuration register information is used to generate the chip configuration program. .

[0067] like Figure 2 As shown, the load parsing unit includes an operator extraction subunit and a data stream reconstruction subunit, wherein the operator extraction subunit is used to extract data based on the input neural network ONNX representation. The metadata blocks of the operators are extracted, and the data flow reconstruction subunit uses these metadata blocks to extract computational load information. The window size of the convolution kernel is multiplied by the input channel size to obtain the cumulative dimension, and the parallel dimension is obtained based on the output channel size. Based on these two dimensions, computational nodes and data dependencies are constructed, resulting in a directed computational graph of the neural network load. .

[0068] like Figure 3 As shown, the load mapping unit includes an architecture data flow generation subunit, a physical mapping subunit, and a communication scheduling subunit. The architecture data flow generation subunit constructs a directed computation graph of the neural network load based on the output of the load parsing unit. Based on the cross-array size in the input chip architecture parameters, the original computation graph nodes are split into cross-array level computation graphs. According to the number of cross-arrays for computation cores in the chip architecture parameters, the cross-array level nodes are merged to generate a computation core level computation graph. Based on the number of computation cores, on-chip network topology, physical mapping subunit, and communication scheduling subunit in the chip architecture parameters, the computation core level computation graph is sequentially mapped in the spatial and temporal domains to generate a multi-core chip-level computation graph and a computation pipeline level computation graph. Based on the multi-core chip-level computation graph and the computation pipeline level computation graph, the final load mapping result is generated. .

[0069] like Figure 4 As shown, the configuration information generation unit includes a mapping result parsing subunit and a configuration program generation subunit. The mapping result parsing subunit receives the mapping result Res output by the load mapping unit and outputs physical mapping information MInfo and communication scheduling information CInfo, which are used to compile the configuration register writing program of the RISC-V CPU and the header information of the NoC data packet distribution logic, respectively. The configuration program generation subunit automatically generates the corresponding configuration program Conf based on the physical mapping information MInfo and communication scheduling information CInfo, combined with the register address information TAddr in the input chip architecture parameters.

[0070] The neural network load mapping method of the present invention includes:

[0071] Step 1: Receive the ONNX representation of the neural network to obtain the original list of operators for the neural network algorithm. By considering the data dependencies between operators, a directed computation graph of the neural network load is constructed. .

[0072] Step 1.1, determine the original list of operators Each operator in The type corresponds to the node in the constructed computation graph:

[0073] 1) If it is a matrix multiplication operator, then directly construct the load computation graph nodes. Its attributes include the size of the input feature map. Output feature map size and operator size ;

[0074] 2) If it is a non-matrix multiplication operator, then merge it into the associated load computation graph node and update the load computation graph node attributes caused by the operator, including the input feature map size. Output feature map size and operator size Based on the node fusion situation, the computational dependencies of the original operators are also folded accordingly.

[0075] Step 1.2: Construct a directed computation graph for the neural network load based on the data dependencies between nodes and operators in the computation graph. .in For a neural network load, there is a set of nodes in a directed computation graph. Let the neural network load be a set of directed computation graph edges, with weights equal to the weights of its source nodes. Size.

[0076] Step 2, according to Size of cross array with in-memory computing chip The directed computation graph at the cross array level is derived. . and These represent the number of word lines and the number of bit lines in the crossbar array of the in-memory computing chip, respectively. and These represent the set of nodes and the set of edges in a cross-array-level directed computation graph, respectively.

[0077] The specific steps are as follows:

[0078] Step 2.1: For each directed computation graph node of the neural network load Flatten its dimensions into a two-dimensional tensor consisting of an accumulation dimension and a parallel dimension. , and Representing graph nodes The cumulative dimension and parallel dimension.

[0079] For load nodes in matrix form, such as fully connected layers, their dimensions remain unchanged; for multidimensional tensors such as convolutions, the dimensions that need to be accumulated are flattened into one dimension. The other dimensions remain unchanged.

[0080] Step 2.2, flatten each node Divided into A set of child nodes ={ }; The set of all child nodes constitutes Nodes of a cross-array-level directed computation graph ;

[0081] The specific partitioning method is to partition each two-dimensional tensor... According to the size of the cross array of the in-memory computing chip Divide it into blocks. If the block size at the boundary is smaller than... If the original size of these blocks is maintained, this partitioning method ensures that each block corresponds to one child node, and the size does not exceed [the specified value]. Therefore, it can be mapped onto a compute-in-memory cross array to perform computations.

[0082] Step 2.3: Based on the relationship between each child node in the cross-array-level directed computation graph and the nodes in the original computation graph, extract the nodes of the original computation graph. A subset of attributes, which are attributes of child nodes, including subsets of input / output channels. Input / output feature map subset .

[0083] Step 2.4: Reconstruct the edges between nodes in the cross-array level directed computation graph.

[0084] If there is a dependency between nodes, it means there is a directed edge between the two nodes, and the weight of the directed edge needs to be calculated; for example:

[0085] When node Output dependency nodes If the output is such that a new directed edge exists between the two, then a new directed edge exists between them. Its weight is:

[0086]

[0087] in, For nodes The amount of output feature map data; for The number of output channels; For nodes Dependency Nodes The number of channels for a node is calculated based on whether two nodes belong to the same computation graph. The specific calculation method is as follows:

[0088]

[0089] Representative child node The node to which it belongs in the computation graph.

[0090] Construct a cross-array level directed computation graph based on the nodes of the cross-array level directed computation graph determined in step 2.2 and the edges of the cross-array level directed computation graph determined in step 2.4, as follows: Figure 5 As shown.

[0091] Step 3: Derive the computational core-level computational graph based on the cross-array-level directed computational graph. .

[0092] and These represent the set of nodes and the set of edges in the computational core-level computational graph, respectively.

[0093] The specific steps are as follows:

[0094] Step 3.1, calculate the number of cross arrays in the core. The number of nodes in the cross-array level computation graph ,structure A set of computing core nodes Each computing core node consists of no more than It is formed by the union of nodes in the cross array level.

[0095] Step 3.2, for For each node, its data dependencies are statistically analyzed.

[0096] Including the amount of input / output feature map data The amount of data reused for input / output feature maps within the computational core. Construct the clustering objective function:

[0097]

[0098] in This is a non-negative hyperparameter used to adjust the importance of the optimization objective. A larger value indicates greater importance of the data stream in the analysis process; a value of 0 indicates that the data stream analysis does not consider the impact of this type of data.

[0099] Step 3.3: Based on the objective function in Step 3.2, minimize the objective function using the optimization algorithm. The sum of the objective functions corresponding to each computational core node.

[0100] The optimization algorithm can be any algorithm that can minimize the objective function, such as iterative clustering algorithm, genetic algorithm, simulated annealing algorithm, etc.

[0101] Step 3.4: The stable solution of the optimization algorithm is used as the final computational core-level computational graph node scheme.

[0102] The weighted edges of the core-level computation graph are obtained by taking the weighted edges of the cross-array-level computation graph. The weights are obtained by calculating the union of the core-level edges, and are summed from the weights of the non-overlapping parts of these edges.

[0103]

[0104] in This represents the node pairs in the computational core graph. Represents node pairs Weighted edges in the core-level computation graph between computational nodes. For nodes in a cross-array-level directed computation graph, Need to meet Belongs to the composition Cross-array level directed computation graph nodes, Belongs to the composition Cross-array level directed computation graph nodes.

[0105] For example, node 1 in the core-level computation graph is composed of the union U1 of nodes in several cross-array-level computation graphs, and another node 2 in the core-level computation graph is composed of the union U2 of nodes in several other cross-array-level computation graphs. Whether there is a weighted edge between node 1 and node 2 in the core-level computation graph is determined by whether there are edges between each node in union U1 and each node in union U2 (i.e., the weighted edges in the core-level computation graph are obtained by taking the weighted edges from the cross-array-level computation graphs). (Obtained by calculating the union at the core level), the weight of the edge is obtained by summing the weights of the non-overlapping edges between each node in union U1 and each node in union U2.

[0106] The constructed core-level computation graph like Figure 6 As shown.

[0107] Step 4: Derive the multi-core chip-level computing graph based on the core-level computing graph. .

[0108] and These represent the set of nodes and the set of edges in a multi-core chip-level computation graph, respectively.

[0109] The specific steps are as follows:

[0110] Step 4.1: For a 2D-mesh multi-core in-memory computing chip, perform computation graph node analysis for each computing core. Assign one-to-one corresponding multi-core chip physical nodes Its physical mapping location coordinates are .

[0111] Step 4.2: Based on the physical mapping location, map the data dependency edges of the computation core-level computation graph to the NoC path, and count the data volume of each data dependency edge. and communication hop count .

[0112] Step 4.3: Minimize the objective function by swapping the physical node positions or by using an optimization algorithm that selects the optimal candidate node mapping one by one. .

[0113] Step 4.4: Take the mapping result at the end of the optimization algorithm execution as the result of the load physical mapping, based on the mapping result of each physical computing core. Generates parameter information that the RISC-V register configuration program depends on. .

[0114] The derived multi-core chip-level computation graph like Figure 7 As shown.

[0115] Step 5: Derive the computation pipeline-level computation graph based on the multi-core chip-level computation graph. .

[0116] and These represent the set of nodes and the set of edges in the computation pipeline-level computation graph, respectively.

[0117] The specific steps are as follows:

[0118] Step 5.1, Set up the pipeline stage set based on the computational pipeline For each stage of the process Extracting subgraphs from multi-core chip-level computation graphs The nodes and edges of each subgraph are constructed using parameters from the cross array that is actually used in the current computation pipeline, and the construction method is the same as before.

[0119] Step 5.2, for each NoC physical link Construct its betweenness centrality measure as follows:

[0120]

[0121] in, Represents a multi-core chip-level computation subgraph Middle node The number of shortest paths between them. Indicates that it went through The number of shortest paths, represent to Data communication volume.

[0122] Step 5.3: Calculate subgraph edges based on multi-core chip level. Initialization status, statistics of physical links Congestion metric :

[0123]

[0124] in, Through physical link And the data volume is arranged from smallest to largest as follows: Bit-level connection path data volume To pass The total number of paths. When ,definition This means there is no congestion.

[0125] Step 5.4, for all sets of connection paths, i.e., the set of edges by path Sort by shortest distance and data size in descending order, For each NoC physical link The weights are used to calculate the path using Dijkstra's algorithm, which yields the scheduling result with the minimum congestion.

[0126] Step 5.5, according to The scheduling results are used to generate a set of packets for NoC route scheduling. .

[0127] The derived computational pipeline-level computational graph like Figure 8 As shown.

[0128] At this point, the entire neural network load mapping process is complete.

[0129] Example 2

[0130] This embodiment provides a neural network load mapping method for a multi-core in-memory computing chip. Taking the mapping of AlexNetConv3-Conv5 to a multi-core in-memory computing chip as an example, the method distributes the computing tasks of Conv3-Conv5 to different cores of the multi-core in-memory computing chip to achieve efficient parallel computing and data storage.

[0131] AlexNet is a classic deep convolutional neural network architecture that includes multiple convolutional layers, pooling layers, and fully connected layers. Conv3-Conv5 are three key consecutive convolutional layers that gradually extract high-level features of the image through successive convolutional operations and the ReLU activation function.

[0132] (1) The AlexNet operator parameters and the multi-core in-memory computing chip parameters used in this embodiment are shown in Table 1 and Table 2 below, respectively.

[0133] Table 1 Neural Network Operator Parameters

[0134]

[0135] Table 2 Parameters of Multi-core In-memory Computing Chip

[0136]

[0137] The operator information is input via AlexNet's ONNX representation, and the chip parameters are input in parsable formats such as .json and .csv. The load mapping will be completed sequentially through the following processing units:

[0138] Load parsing unit such as Figure 2 As shown, it includes an operator extraction unit and a data stream reconstruction unit. First, the operator extraction unit parses the original ONNX input, and when it matches Conv3-Conv5 related operators, it extracts the metadata blocks of the operators. The data stream reconstruction unit extracts computational load information using metadata blocks, multiplies the window size of the convolution kernel by the input channel size to obtain the accumulated dimension value, and extracts the number of convolution kernels as the output dimension value. For the Conv3-Conv5 operators mentioned in this embodiment, the accumulated dimension parameters are 2304, 3456, and 3456 respectively, and the output dimensions are 384, 256, and 256 respectively. Dependencies It is a linear cascade. The nodes are two-dimensional tensors of (cumulative dimension, output dimension), and the edges represent dependencies. In this embodiment, It consists of three nodes arranged in a sequential cascade, with the node dimensions described above.

[0139] Load mapping unit such as Figure 3 As shown, it includes an architecture data flow generation unit, a physical mapping unit, and a communication scheduling unit. The architecture data flow generation unit first determines the data flow based on the computational graph. Based on the cross-array size in the chip architecture information, the original computation graph nodes are split into cross-array level nodes. In this example, Conv3 is split into two 1152×256 nodes and one 1152×128 node; Conv4 is split into three 1152×256 nodes and three 1152×128 nodes; and Conv5 is split into three 1152×128 nodes. The reconstructed computation graph is shown below. like Figure 4As shown. Then, based on the number of single-core cross arrays in the chip architecture information, and adopting an analysis strategy that prioritizes intra-layer accumulation output multiplexing and balances parallelism, the cross arrays are merged into the computing cores, resulting in the computation graph. like Figure 5 As shown. Further, based on the NoC topology of the chip architecture information, and the data dependencies between Tile1, Tile3, and Tile4, the three nodes are mapped to their nearest neighbors. Tile2 is then mapped to its nearest neighbors, Tile3 and Tile4. Since Tile3 and Tile5 have a stronger data dependency, Tile5 is mapped to its nearest neighbor, Tile3. (Computation graph) like Figure 6 As shown, the physical mapping unit generates the physical mapping results of the computing nodes based on the computation graph. Finally, based on the computation pipeline partitioning of the chip architecture information, the communication scheduling unit processes the generated computation pipeline-level computation graph. Perform communication congestion scheduling. For example... Figure 7 As shown, in this embodiment, the computational pipeline depth is 1, meaning only one layer of convolution operators is allowed to perform computation simultaneously. Furthermore, the scheme based on XY routing initialization has no communication congestion, so scheduling terminates directly, yielding the final mapping result and outputting NoC data routing information. The outputs of the physical mapping unit and the communication scheduling unit are combined into the mapping result. Output.

[0140] Configuration information generation unit such as Figure 8 As shown, it includes a mapping result parsing unit and a configuration program generation unit. The mapping result parsing unit accepts the mapping result. Output physical mapping information Communication scheduling information These are used to compile the configuration register writer for the RISC-V CPU and the header information for configuring the NoC packet distribution logic, respectively. Taking typical CPU MMIO access configuration information as an example, the process of writing the value to ADDR_BASE+REG_OFFT is as follows:

[0141] li t0, ADDR_BASE+REG_OFFT

[0142] li t1, value

[0143] sw t1, 0(t0)

[0144] Since the instruction sequence is fixed, the variables are the value generated by the configuration information unit and the address of the register to be configured. Therefore, the mapping result parsing unit can maintain the key-value pairs between different types of values ​​and the target configuration register in the mapping result, and then pass it to the configuration program generation unit, combining it with the address information of the chip configuration register. Automatically generate the corresponding configuration program .

[0145] This embodiment demonstrates an automated analysis and optimization process for workload mapping tasks from neural networks to multi-core in-memory computing chips. While fully utilizing computing resources, it also performs architecture parameter analysis and optimization based on a hierarchical mapping strategy. This approach improves algorithm execution efficiency for specific chip architectures while maintaining versatility.

[0146] (2) The mapping tool of this invention is used to map various complete neural networks to 2D-mesh multi-core in-memory computing chips. The computation and on-chip communication latency are evaluated by MNSIM and BookSim2 respectively, and the performance is compared with the baseline scheme of sequential mapping-XY routing. MNSIM is a simulation platform focused on simulating and evaluating neuromorphic computing systems (Processing-In-Memory, PIM) based on memristors. It is mainly used to study and optimize the performance of PIM architecture in terms of hardware performance and neural network computation accuracy. BookSim2 is a cycle-accurate interconnect network simulator used to simulate and evaluate the performance of different network topologies and routing algorithms (for a detailed introduction to BookSim, please refer to the introduction in Jiang N, Becker DU, Michelogiannakis G, et al. A detailed and flexible cycle-accurate network-on-chip simulator[C]. 2013 IEEE international symposium on performance analysis of systems and software (ISPASS). IEEE, 2013: 86-96.).

[0147] The architecture parameters and evaluation results are shown in Tables 3 and 4 below. Experimental results show that the mapping method provided by this invention can effectively improve the execution efficiency of neural network algorithms on in-memory computing chips. As can be seen from Table 4, compared with the baseline scheme of sequential mapping-XY routing, the mapping method provided by this invention is applicable to neural network algorithms with various network structures and can effectively reduce the inference latency and throughput of neural network algorithms.

[0148] Table 3. Parameters of the memory computing chip architecture in this embodiment

[0149]

[0150] Table 4 Neural Network Algorithm and Performance Optimization Data

[0151]

[0152] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A neural network load mapping method for a multi-core computing-on-chip, characterized in that, The method comprises: Step 1, extract the original operator list of the neural network and the data dependency between the operators, and construct a neural network load directed computation graph ; Step 2, the neural network load directed computation graph and the memory-compute integrated chip cross array size , get the cross array level directed computation graph ; Step 3, deriving the computation core level computation graph from the cross-array level directed computation graph , deriving the computation core level computation graph from the cross-array level directed computation graph ; Step 4, compute the core-level computation graph according to the computation core , derive the multi-core chip-level computation graph ; Step 5, deriving a compute pipeline level computation graph from the multi-core chip level computation graph , deriving a compute pipeline level computation graph from the multi-core chip level computation graph ; The step 3 comprises: Step 3.1, number of cross-array stages of a compute core and number of nodes of a cross-array stage compute graph , constructing a set of compute core stage nodes each compute core stage node consisting of a union of no more than cross-array stage nodes; Step 3.2, for each node in the graph, count its data dependency, including input / output feature map data volume , calculate the input / output feature map reuse data volume in the core ; and construct a clustering objective function: ; wherein is a non-negative hyperparameter;​ Step 3.

3. Minimizing the objective function based on the optimization algorithm according to the objective function in step 3.2 the sum of the objective functions corresponding to each computing core level node Step 3.4, taking the stable solution of the optimization algorithm as the final core-level computation graph node scheme to construct a core-level computation graph , and are respectively a node set and an edge set of the core-level computation graph; The step 4 comprises: Step 4.1, for the multi-core storage-computing integrated chip of the 2D-mesh structure, calculating the graph node for each computing core level allocating a one-to-one corresponding multi-core chip physical node , and the physical mapping position coordinates are ; Step 4.2, according to the physical mapping position, mapping the data dependent edge of the calculation core level calculation graph to the NoC channel, counting the data volume of each data dependent edge and communication hop number ; t representing time; Step 4.3, Minimizing the objective function by swapping the physical node locations, or optimization algorithm that selects the best candidate node mapping one by one ; Step 4.4, taking the mapping result at the end of the optimization algorithm as the result of the load physical mapping, according to the mapping result of each physical computing core , generating parameter information relied on by the RISC-V register configuration program ; The step 5 comprises: Step 5.1, Set up the pipeline stage set based on the computational pipeline For each stage of the process Extracting subgraphs from multi-core chip-level computation graphs ; and These represent subgraphs of a multi-core chip-level computation graph. The set of nodes and the set of edges; Step 5.2, for each NoC physical link , construct its betweenness centrality metric as follows: wherein, and denote two nodes of a subgraph , denote the number of shortest paths between nodes and in a multi-core chip-level computing subgraph , denote the number of shortest paths passing through , represent the data traffic from to ; Step 5.3, according to the initialization case of the multi-core chip-level computing subgraph edge , the congestion metric of the physical link is counted : wherein, is the total path number through , and the data amount from small to large is arranged in the connection path data amount of the bit, is the total path number through , define , that is, there is no congestion;​ Step 5.4, edge set , shortest distance by path and data volume descending order, to each NoC physical link weight, dijkstra algorithm solution path as the minimum congestion scheduling results; Step 5.5, generating a data packet set of NoC routing schedule according to the scheduling result of the edge set .​ 2. The method of claim 1, wherein, The step 1 comprises: Step 1.1, determine the original list of operators Each operator in The type corresponds to the node in the constructed computation graph; Step 1.2, constructing a neural network load directed computation graph according to the computation graph nodes and data dependency relationship between operators ; wherein is a set of neural network load directed computation graph nodes, is a set of neural network load directed computation graph edges, and the weight is the size of the source node of the edge.

3. The method of claim 2, wherein, The step 2 comprises: Step 2.1, for each neural network load a directed computation graph node flattening its dimensions into a two-dimensional tensor consisting of an accumulation dimension and a parallel dimension ; Step 2.2, divide each node of the flattened graph into a set of sub-nodes { } , and respectively represent the number of word lines and bit lines of the integrated chip cross array; all sub-node sets constitute nodes of the cross array level directed computation graph , ;​ Step 2.

3. Extract a subset of attributes of each node in the original computation graph as attributes of the sub-nodes, including a subset of input / output channels , a subset of input / output feature maps , according to the relationship between each node in the cross-array level directed computation graph and the node of the original computation graph ​ Step 2.4, reconstructing edges between nodes in the cross array level directed computation graph; If there is a dependency relationship between nodes, it means that there is a directed edge between the two nodes, and the weight of the directed edge is calculated according to the following formula: in, For nodes The amount of output feature map data, for Number of output channels, for rely The number of channels for a node is calculated based on whether two nodes belong to the same computation graph. : representing child nodes The parent computational graph node to which this node belongs.

4. A neural network load mapping device of a multi-core computing integrated chip, characterized in that, The device is used to implement the method of any one of claims 1-3, comprising: a load analysis unit, a load mapping unit and a configuration information generation unit, which are connected in sequence; at the same time, the load analysis unit receives the externally input neural network, and the load mapping unit and the configuration information generation unit receive the externally input architecture parameters of the computing chip. The load parsing unit is used to parse the input neural network ONNX representation. And extract the matrix multiplication operator from it. Data dependencies between operators The output requires a directed computation graph of a neural network workload that needs to be accelerated by an in-memory computing chip. The load mapping unit is used to perform directed computation of the neural network load graph. Architecture parameters of in-memory computing chips Generate in-memory computing chip load mapping results The configuration information generation unit is used to generate configuration information based on the load mapping results. and chip architecture parameters The configuration register information is used to generate the chip configuration program. .

5. The apparatus of claim 4, wherein, The load resolution unit comprises an operator extraction subunit and a data flow reconstruction subunit, wherein the operator extraction subunit is configured to extract an operator according to an input neural network ONNX representation The meta-information block of the extracted operator is obtained, the data flow reconstruction subunit extracts the calculation load information by using the meta-information block, multiplies the window size of the convolution kernel and the input channel size to obtain the value of the accumulated dimension, obtains the value of the parallel dimension based on the output channel size, constructs the calculation node and the data dependency according to the values of the two dimensions, and obtains the neural network load directed computation graph .

6. The apparatus of claim 5, wherein, The load mapping unit comprises an architecture data flow generation subunit, a physical mapping subunit and a communication scheduling subunit, wherein the architecture data flow generation subunit splits the original computation graph node into a cross array level computation graph according to the cross array size in the input chip architecture parameter According to the cross array number of calculation cores in the chip architecture parameter, the cross array level node is merged to generate a calculation core level computation graph; according to the number of calculation cores and the on-chip network topology in the chip architecture parameter, the physical mapping subunit and the communication scheduling subunit perform spatial and temporal mapping on the calculation core level computation graph in sequence to generate a multi-core chip level computation graph and a calculation flow level computation graph; Based on the multi-core chip-level computing graph and the computing pipeline-level computing graph, a final load mapping result is generated .

7. The apparatus of claim 6, wherein, The configuration information generation unit comprises a mapping result analysis subunit and a configuration program generation subunit, wherein the mapping result analysis subunit receives the mapping result Res output by the load mapping unit, and outputs physical mapping information MInfo and communication scheduling information CInfo, which are respectively used for compiling the configuration register writing program of the RISC-V CPU and the packet header information of the NoC data packet distribution logic; the configuration program generation subunit automatically generates the corresponding configuration program Conf according to the physical mapping information MInfo and the communication scheduling information CInfo, and in combination with the address information TAddr of the register in the input chip architecture parameters.

Images