On-chip-inter-chip interconnected neural network chip hardware architecture design method and system

Abstract

The present invention provides a method and system for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnection, and conducts hardware architecture exploration for computing power division and cache hierarchy design. During the exploration process, the most important convolutional layer in the neural network is calculated and data access modeling is performed to obtain chip delay performance and energy consumption indicators. Based on the scale configuration of each level obtained by exploration, common topologies are evaluated, and interconnection topologies suitable for neural network communication traffic are explored. Through the study of chip scale, cache hierarchy and on-chip interconnection topology, a complete neural network chip hardware architecture based on on-chip and inter-chip interconnection is obtained. The final designed neural network chip hardware architecture optimizes system delay and energy, achieving the goals of low latency and high energy efficiency.

Description

Technical Field

[0001] The present invention relates to the field of neural network chip hardware architecture exploration, and in particular, to a neural network chip hardware architecture design method and system based on chiplet on-chip-to-chip interconnection. A corresponding terminal and computer-readable storage medium are also provided. Background Art

[0002] In recent years, deep neural networks (DNN), especially convolutional neural networks (CNN), have achieved outstanding performance in many fields including face recognition, language recognition and autonomous driving, and have been successfully applied to various artificial intelligence devices. Compared with traditional machine learning algorithms, neural networks do not need to rely on manual feature extraction, and can perform recognition or classification tasks through end-to-end training. However, these operations require a lot of computing and storage, and the traditional central processing unit (CPU)-based architecture can no longer meet its performance and energy efficiency requirements, and various artificial intelligence (AI) chips have emerged.

[0003] With the rapid development of deep learning, the growing computing and storage resources have also led to an increase in chip area and a significant increase in manufacturing costs, which has promoted the development of chiplet technology. Chiplet technology divides the chip into multiple "chiplets", and the area of ​​a single chiplet is smaller than that of traditional SoCs. Therefore, on AI chips, the design of connecting multiple neural network chiplets using network on package (NoP) has become a promising research direction. However, chiplet technology also brings new challenges to the hardware architecture design of neural network chips.

[0004] On the one hand, due to the unique characteristics of neural network data flow, the complex neuron connections and communication modes in neural network chip design have brought new challenges to the interconnection architecture design. It is not efficient to directly introduce the interconnection architecture of traditional multi-core processors into neural network chips. How to design an on-chip-to-interchip interconnection architecture with high energy efficiency, low latency, and adaptability to neural network data flow has become a new challenge. Interconnection topology is the core of the interconnection network. The existing research on non-mesh interconnection topology is not sufficient. In view of the energy and area overhead brought by the complex hardware structure of routers, the existing loop-based non-routing design is a promising solution, but it has not yet been evaluated and optimized for neural network data flow.

[0005] On the other hand, after the introduction of chiplet technology in neural network chips, the configuration of storage and computing resources of each chiplet becomes an important issue that needs to be explored. Under given performance requirements, how to choose the appropriate chiplet scale: that is, the number of MAC units and the storage size of each cache level. A system with fewer chiplets will result in a larger single chiplet area, but with low communication overhead. On the contrary, a large-scale system with a large number of small-sized chiplets will lead to complex inter-chip interconnection, while saving chip development costs. Existing research pays less attention to the problem of computing power partitioning, and often only focuses on the number of data accesses, without considering the impact of communication in the interconnection architecture on the system. In addition, the number of chiplets involved is often small, and the scalability is not strong.

[0006] At present, no description or report of similar technology to the present invention has been found, and similar information at home and abroad has not been collected yet. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and system for designing a hardware architecture of a neural network chip with on-chip-inter-chip interconnection. A corresponding terminal and a computer-readable storage medium are also provided. Among them, on-chip-inter-chip refers to a complete on-chip system plus an inter-chip system.

[0008] According to one aspect of the present invention, a method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnection is provided, comprising:

[0009] Based on the computing power and storage size of the neural network chip, determine the chip hardware scale and explore the design space;

[0010] exploring a design space based on the hardware scale and sequentially selecting a hardware architecture scale parameter configuration therefrom;

[0011] For the hardware architecture scale parameter configuration, a performance evaluation is performed, and the optimal performance is selected as the performance of the hardware architecture scale parameter;

[0012] The performance based on the hardware architecture scale parameter is compared with the existing hardware architecture performance, and if it is better than the existing hardware architecture performance, the performance is retained as the optimal architecture performance;

[0013] Traverse the entire hardware scale exploration design space to obtain the final optimal architecture performance. The hardware architecture scale parameter corresponding to the final optimal architecture performance is the result of the chip hardware scale exploration, and then the optimal computing power division and cache hierarchy scale under given computing power and storage size are obtained;

[0014] Based on the results of the chip hardware scale exploration, the characteristics of the neural network data flow are analyzed, the neural network communication data flow is extracted, and the neural network communication data flow configuration space is obtained;

[0015] Based on the neural network communication data flow configuration space, the on-chip interconnect topology performance is evaluated, the performance of each topology is compared, and the optimal on-chip interconnect topology is obtained.

[0016] Based on the results of the chip hardware scale exploration and the optimal on-chip interconnection topology, the final output based on the neural network chip hardware architecture exploration is obtained, which is the optimal neural network chip hardware architecture.

[0017] Optionally, determining the chip hardware scale to explore the design space includes:

[0018] Based on the three-level hardware architecture of Package-Chiplet-PE, given the chip computing power, the number of MACs inside PE, the number of PEs in each chiplet, and the number of chiplets are determined, and the design space of computing power division is determined to meet the following formula:

[0019] N MAC =N chip ×N PE ×N size ×N lane

[0020] Among them, N MAC Indicates the number of multipliers and accumulators corresponding to the computing power, N c h ip Indicates the number of chiplets, N PE Indicates the number of PEs in each chiplet, N size Indicates the vector-MAC size, N lane Indicates the number of vector-MACs in each PE; Package indicates that the complete system contains multiple chiplets, Chiplet indicates chiplet, PE indicates processing unit, MAC indicates multiply-accumulator, vector-MAC: a vector multiply-accumulator composed of multiple multiply-accumulators.

[0021] For the L1 cache and L2 cache in PE, cache partitioning is performed respectively, and the size of input feature map cache, weight cache and output feature map cache under L1 cache and L2 cache after partitioning is explored;

[0022] Based on the above steps, the chip hardware scale exploration design space is obtained.

[0023] Optionally, the performing performance evaluation on the hardware architecture scale parameter configuration and selecting the optimal performance as the performance of the hardware architecture scale parameter includes:

[0024] Based on the selected hardware architecture scale parameters, performing neural network model mapping;

[0025] Based on the hardware architecture scale parameter and the neural network model obtained by mapping, randomly generate the neural network model mapping code, and based on the neural network model mapping code, extract the computational amount and communication amount of the corresponding mapping scheme through the constructed computational model and data access model;

[0026] Based on the extracted computational amount and communication amount, the delay and energy are calculated to obtain the energy-delay product performance of the mapping scheme, and compared with the historical optimal performance under the hardware architecture scale parameter, and the better performance is retained as the performance corresponding to the mapping scheme;

[0027] After multiple iterations of the mapped neural network model, the optimal performance is selected as the performance of the hardware architecture scale parameter.

[0028] Optionally, the calculation model is constructed to obtain a delay performance indicator non_stall_lat caused by the calculation task; wherein the calculation method includes:

[0029] runtimeP=P3×P2×P1×PP3×PP2

[0030] runtimeQ=Q3×Q2×Q1×PQ3×PQ2

[0031] runtimeK=K3×K2×K1×PK3×PK2×PK0

[0032] runtimeC=C3×C2×C1×PC0

[0033]

[0034]

[0035] In the formula, runtimeP represents the amount of computation required for the P dimension in the loop expansion and loop blocking of the actual mapping scheme, P represents the theoretical amount of computation for the P dimension in the convolution layer, util. represents the utilization of computing resources, and the non-blocking delay non_stall_lat is calculated as the delay performance by calculating the resource utilization; C is the input feature map channel, P, Q, K are the output feature map, height, and channel respectively, R and S are the convolution kernel width and height respectively, runtimeQ represents the amount of computation required for the Q dimension under the mapping scheme, runtimeK represents the amount of computation required for the K dimension under the mapping scheme, runtimeC represents the amount of computation required for the C dimension under the mapping scheme, P1~PP3 are the loop blocking parameters of the P dimension under the three hardware levels (PE, Chiplet, Package), and PP3 is the loop of the P dimension in the Package level (between different Chiplet) Loop expansion parameters, PP2 is the loop expansion parameter of P dimension in chiplet level (between different PEs), Q1~Q3 are loop block parameters of Q dimension in three hardware levels (PE, chiplet, package), K1~K3 are loop block parameters of K dimension in three hardware levels (PE, chiplet, package), C1~C3 are loop block parameters of C dimension in three hardware levels (PE, chiplet, package), PK0 is the loop expansion parameter of K dimension in PE level (between different multipliers and accumulators), PK2 and PK3 are loop expansion parameters of K dimension in chiplet level and package level, PC0 is the loop expansion parameter of C dimension in PE level (between different multipliers and accumulators), MAC_num is the total number of multipliers and accumulators in the complete system, i.e., package.

[0036] Optionally, the working process of the data access model includes:

[0037] Determine whether each dimension parameter in the data stream is a relevant parameter for the data of the input feature map, weight, and output feature map respectively;

[0038] Starting from the innermost for loop of the data flow, the storage resource requirements of the input feature map, output feature map, and weight at the for loop of this layer are calculated based on whether the parameters are related;

[0039] For a total of 6 storage units of input feature maps, weights and output feature maps under L1 and L2 caches, determine the bottleneck points of the 6 storage units according to the capacity of each storage unit and the storage resource requirements at each layer of for loop;

[0040] According to the bottleneck points of each storage unit and the amount of data transmission required at the bottleneck points, the total amount of data transmission between the PE level and the L2 cache and between the chiplet level and the DRAM of the convolution layer is calculated, which is the communication volume of the mapping scheme.

[0041] Based on the calculated total amount of data transmission information, the energy cost caused by data transmission is obtained according to the set indicators, which is the energy cost of the system.

[0042] Optionally, the step of analyzing the characteristics of neural network data streams based on the result of the chip hardware scale exploration and extracting neural network communication data streams includes:

[0043] Based on the results of chip hardware scale exploration, the neural network communication traffic is analyzed, where the communication traffic is the input feature map, weights in the L2 cache of the PE, and the data interaction between the output feature map and the PE cluster;

[0044] Based on the number of PEs in the hardware architecture scale, determine the source of PE parallelism, group PEs, obtain data sharing between PEs, and extract communication flows;

[0045] Based on different neural network models and different layers of neural network models, data flow analysis is performed, and it is found that different layers of the neural network model can be represented by different proportions of three types of data flow: input feature map data volume, weight data volume, and output feature map data volume. Among them, the convolution layer may have more input data volume and output data volume and less weight data volume, and the fully connected layer may have more weight data volume and less input data volume and output data volume. The characteristics of different neural network layers are characterized by scanning the proportions of the three types of data flow (i.e., input feature map data volume, weight data volume, and output feature map data volume) with L1 cache, L2 cache, and DRAM;

[0046] Combining the communication flow with the data flow, a complete neural network communication data flow configuration space is obtained.

[0047] Optionally, the performing on-chip interconnect topology performance evaluation includes:

[0048] Based on the neural network communication data flow configuration space, for different PE grouping situations, the ratio of L1 cache, L2 cache and DRAM data flow is scanned as input for performance evaluation;

[0049] Evaluate the performance of the interconnect architecture under the communication data flow by evaluating the congestion degrate_ratio of each link and finding the maximum congestion max(degrate_ratio) as the performance of the interconnect architecture under the communication data flow;

[0050] Traverse the on-chip interconnect topology and iterate the above steps to obtain the maximum congestion degree of each topology under different PE grouping conditions as the data traffic ratio changes, and complete the performance evaluation of the on-chip interconnect topology.

[0051] Optionally, the data flow characteristics include: communication mode and statistical information of the data flow; wherein:

[0052] The communication mode includes: unicast, multicast and broadcast;

[0053] The statistical information of the data flow includes: a sending node, a destination node and a communication volume.

[0054] Optionally, the on-chip interconnection topology structure includes: a common topology structure and a loop-based non-routing on-chip topology structure; wherein:

[0055] The common topologies include: Mesh topology, CMesh topology and Torus topology;

[0056] The ring-based routing-free on-chip topology includes: 3 corresponding to the input features Figure 2 Level cache AL2, weight level cache WL2 and output features Figure 2 The secondary cache of the level cache OL2 and 16 processor units PE have a total of 19 nodes, and the 19 nodes interact with the four loops through their corresponding injection-receiving interfaces. The four loops realize data communication between different nodes; wherein the injection-receiving interface has an arbitrary data flow direction, the four loops are unidirectional loops, and the data packet advances one hop in each cycle on the loop;

[0057] Through the performance evaluation of the on-chip interconnection topology, the optimal on-chip interconnection topology is the loop-based non-routing on-chip topology.

[0058] According to another aspect of the present invention, a neural network chip hardware architecture design system with on-chip-inter-chip interconnection is provided, where on-chip-inter-chip means that the system considered is a complete on-chip system plus an inter-chip system, including:

[0059] A parameter configuration module, which determines the chip hardware scale exploration design space based on the computing power and storage size of the neural network chip; based on the hardware scale exploration design space, sequentially selects a hardware architecture scale parameter configuration;

[0060] A performance evaluation module, which performs performance evaluation on the hardware architecture scale parameter configuration and selects the optimal performance as the performance of the hardware architecture scale parameter; compares the performance of the hardware architecture scale parameter with the existing hardware architecture performance, and if it is better than the existing hardware architecture performance, retains the performance as the optimal architecture performance; traverses the entire hardware scale exploration design space to obtain the final optimal architecture performance, and the hardware architecture scale parameter corresponding to the final optimal architecture performance is the result of the chip hardware scale exploration, thereby obtaining the optimal computing power division and cache hierarchy scale under given computing power and storage size;

[0061] A communication data stream extraction module, which analyzes the characteristics of the neural network data stream based on the results of the chip hardware scale exploration, extracts the neural network communication data stream, and obtains the neural network communication data stream configuration space;

[0062] A hardware architecture building module, which evaluates the performance of the on-chip interconnect topology structure based on the neural network communication data flow configuration space to obtain the optimal on-chip interconnect topology structure; based on the results of the chip hardware scale exploration and the optimal on-chip interconnect topology structure, the final output based on the neural network chip hardware architecture exploration is obtained, which is the optimal neural network chip hardware architecture.

[0063] According to a third aspect of the present invention, there is provided a neural network chip hardware architecture with on-chip and inter-chip interconnections, which is constructed using any of the design methods described above.

[0064] According to a fourth aspect of the present invention, there is provided a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute any of the above-described methods when executing the program.

[0065] According to a fifth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any of the methods described above.

[0066] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0067] The on-chip and inter-chip interconnected neural network chip hardware architecture design method and system provided by the present invention establish detailed calculation and storage access models at all levels based on a comprehensive analysis of the data flow of the convolutional layer, and obtain accurate and reliable delay and energy consumption indicators.

[0068] The on-chip-inter-chip interconnected neural network chip hardware architecture design method and system provided by the present invention explore the hardware design space for chip granularity division, L1 and L2 cache size configuration, and allocation of various data cache sizes. Through the hardware design space exploration, the obtained solution achieves an EDP reduction of 2.47% to 87.99% compared with other solutions.

[0069] The present invention provides a method and system for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections, and proposes a loop-based on-chip network-without-routing topology targeting the characteristics of on-chip communication traffic of the neural network, thereby improving communication performance while reducing the on-chip network area. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0071] Figure 1 The present invention is a flowchart of a method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections according to an embodiment of the present invention.

[0072] Figure 2 The present invention is a flowchart of a method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections in a preferred embodiment of the present invention.

[0073] Figure 3 Schematic diagram of the overall hardware architecture of a neural network chip with on-chip and inter-chip interconnections in a preferred embodiment of the present invention.

[0074] Figure 4 This is an on-chip communication data flow diagram in a preferred embodiment of the present invention.

[0075] Figure 5 Schematic diagram of a common interconnection topology structure in a preferred embodiment of the present invention.

[0076] Figure 6 The figure is a schematic diagram of a loop-based non-routing topology structure in a preferred embodiment of the present invention.

[0077] Figure 7 A schematic diagram of the components of a neural network chip hardware architecture design system for on-chip and inter-chip interconnection in one embodiment of the present invention. DETAILED DESCRIPTION

[0078] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0079] Figure 1 This is a workflow diagram of a method for designing a hardware architecture of a neural network chip with on-chip-inter-chip interconnection provided by an embodiment of the present invention, wherein on-chip-inter-chip refers to a complete on-chip system plus an inter-chip system.

[0080] like Figure 1 As shown, the method for designing the hardware architecture of a neural network chip with on-chip and inter-chip interconnections provided in this embodiment may include the following steps:

[0081] S100, based on the computing power and storage size of the neural network chip, determines the chip hardware scale and explores the design space;

[0082] S200, exploring the design space based on the hardware scale, and sequentially selecting a hardware architecture scale parameter configuration;

[0083] S300, performing performance evaluation on the hardware architecture scale parameter configuration, and selecting the best performance as the performance of the hardware architecture scale parameter;

[0084] S400, comparing the performance based on the hardware architecture scale parameter with the existing hardware architecture performance, and if it is better than the existing hardware architecture performance, retaining the performance as the optimal architecture performance;

[0085] S500, traverse the entire hardware scale exploration design space to obtain the final optimal architecture performance, and the hardware architecture scale parameter corresponding to the final optimal architecture performance is the result of the chip hardware scale exploration, thereby obtaining the optimal computing power division and cache hierarchy scale under the condition of given computing power and storage size;

[0086] S600, based on the results of chip hardware scale exploration, analyzes the characteristics of neural network data flow, extracts neural network communication data flow, and obtains the neural network communication data flow configuration space;

[0087] S700, based on the neural network communication data flow configuration space, evaluate the performance of the on-chip interconnect topology, compare the performance of each topology, and obtain the optimal on-chip interconnect topology;

[0088] S800, based on the results of chip hardware scale exploration and the optimal on-chip interconnection topology, obtains the final output based on neural network chip hardware architecture exploration, which is the optimal neural network chip hardware architecture.

[0089] In a preferred embodiment of S600, the data flow characteristics focus on the communication mode (unicast, multicast, broadcast) and the statistical information of the data flow (including the sending node and the destination node and the communication volume).

[0090] In a preferred embodiment of S700, the on-chip interconnection topology includes a common topology and a loop-based non-routing topology. Furthermore, the loop-based non-routing topology is proposed based on the characteristics of the data flow (multicast data flow) and the most congested link of on-chip communication (the link corresponding to the secondary cache). The loop-based non-routing topology is the optimal on-chip interconnection topology obtained. The structure verifies that in combination with the data sharing between PEs, it better adapts to the multicast data flow under data sharing and alleviates the most congested link of on-chip communication (the link corresponding to the secondary cache).

[0091] Furthermore, the loop-based non-routing topology includes: 3 secondary caches (corresponding to the input feature Figure 2 Level 2 cache AL2, weight level 2 cache WL2, output features Figure 2 The 19 nodes (3 L2 cache nodes + 16 PE nodes) interact with four loops (loop 1, loop 2, loop 3, loop 4) through their corresponding injection-receive interfaces. The four loops realize data communication between different nodes. The injection-receive interface is different from the traditional router. It only determines which loop the local data packet goes to (injection process) and whether the data packet of the loop is received by the local node (receiving process). It does not select the direction. Therefore, compared with the traditional router, its design complexity is lower and the area overhead is smaller. The four loops are unidirectional loops, and the data packet advances one hop (hops) in each cycle on the loop. Under such a design, there are more links and larger bandwidth at the L2 cache node, which can alleviate the congestion of the on-chip network on the L2 cache. At the same time, the design of the loop is naturally quartz and multicast data flow, because the data transmitted to the loop will pass through all nodes on the loop.

[0092] In this embodiment, the inter-chip interconnection may adopt a commonly used mesh. This embodiment conducts hardware exploration for the system on chip and the inter-chip system, wherein the inter-chip part focuses on the hardware scale exploration.

[0093] Although the interconnection network has gradually matured in the design of multi-core chips and can solve the problem of complex multi-core interconnection, the complex neuron connections and communication modes in the design of neural network chips have brought new challenges to the design of interconnection architecture. The on-chip-to-inter-chip interconnected neural network chip hardware architecture design method provided in the above embodiment of the present invention explores the hardware architecture for computing power division and cache hierarchy design. During the exploration process, the most important layer in the neural network, the convolution layer, is calculated and data access modeling is performed to obtain chip throughput and energy consumption indicators. Based on the scale configuration of each level obtained by the exploration, common topologies are evaluated, and interconnection topologies suitable for neural network communication traffic are explored. Through the study of chip scale, cache hierarchy and on-chip interconnection topology, a complete neural network chip hardware architecture based on on-chip-to-inter-chip interconnection is obtained. The neural network chip hardware architecture finally designed optimizes system delay and energy, achieving the goals of low latency and high energy efficiency.

[0094] The technical solution of the on-chip-inter-chip interconnected neural network chip hardware architecture design method provided by the above embodiment of the present invention is further explained below in conjunction with the accompanying drawings and corresponding preferred embodiments and specific application examples.

[0095] Figure 2 A flowchart of a method for designing a neural network chip hardware architecture with on-chip and inter-chip interconnections provided in accordance with a preferred embodiment of the present invention.

[0096] like Figure 2 As shown, the on-chip-to-inter-chip interconnected neural network chip hardware architecture design method provided in this embodiment targets the energy efficiency and latency of the neural network chip based on the on-chip-to-inter-chip interconnection, and better adapts the advantages of the on-chip-to-inter-chip interconnection architecture to the neural network chip. Through convolutional layer calculation and data access models, the hardware scale exploration of the neural network chip based on the on-chip-to-inter-chip interconnection is realized, and the interconnection topology structure is explored, thereby constructing an interconnection topology architecture suitable for neural network communication traffic.

[0097] like Figure 2 As shown, the preferred embodiment provides a method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections, which may include the following steps:

[0098] Step 1: Based on the computing power and storage size of the neural network chip, clarify the chip hardware scale exploration design space (including computing power division and cache level scale);

[0099] Step 2: Explore the design space based on the hardware scale in step 1, and select a hardware architecture scale parameter configuration in sequence;

[0100] Step 3: For the hardware architecture scale parameter configuration selected in step 2, a performance evaluation is performed, and the performance of the corresponding optimal mapping effect is selected as the performance of the hardware scale parameter;

[0101] Step 4: Based on the architecture performance of step 3, compare whether it is better than the existing architecture performance. If it is better, retain it as the optimal architecture performance;

[0102] Step 5: Determine whether the entire hardware scale parameter design space has been traversed. If not, jump to step 2. If the traversal is complete, the hardware scale parameter corresponding to the optimal architecture performance is the result of the chip hardware scale exploration, which clarifies the optimal computing power division and cache hierarchy scale under given computing power and storage size;

[0103] Step 6: Based on the chip hardware scale parameters in step 5, analyze the characteristics of the neural network data flow and extract the neural network communication data flow;

[0104] Step 7: Based on the communication data stream extracted in step 6, the on-chip interconnection topology performance is evaluated, and the performance of each topology is compared to obtain the optimal interconnection topology; wherein the on-chip interconnection topology includes a common topology and a loop-based non-routing on-chip topology established in this preferred embodiment; the loop-based non-routing on-chip topology includes: 3 corresponding to the input features Figure 2 Level cache AL2, weight level cache WL2 and output features Figure 2 The secondary cache of the level cache OL2 and 16 processor units PE have a total of 19 nodes, and the 19 nodes interact with the four loops through their corresponding injection-receiving interfaces. The four loops realize data communication between different nodes; wherein the injection-receiving interface has an arbitrary data flow direction, the four loops are unidirectional loops, and the data packet advances one hop in each cycle on the loop;

[0105] Step 8: Based on the optimal hardware scale parameters of step 5 and the optimal on-chip interconnect topology of step 7, the final output of the neural network chip hardware architecture exploration algorithm is obtained, which is the optimal hardware architecture.

[0106] Specifically, the steps to define the chip hardware scale design space in step 1 are:

[0107] Step 1-1: Based on the three-level hardware architecture of Package-Chiplet-PE, given the chip computing power, the number of MACs inside PE, the number of PEs in each chiplet, and the number of chiplets are determined, and the design space of computing power division is determined to satisfy the following formula:

[0108] N MAC =N chip ×N PE ×Nsize ×N lane

[0109] Among them, N MAC Indicates the number of multipliers and accumulators corresponding to the computing power, N c h ip Indicates the number of chiplets, N PE Indicates the number of PEs in each chiplet, N size Indicates the vector-MAC size based on NVDLA, N lane Indicates the number of vector-MACs in each PE; Package indicates that the complete system contains multiple chiplets, Chiplet indicates chiplet, PE indicates processing unit, MAC indicates multiplication accumulator, vector-MAC: a vector multiplication accumulator composed of multiple multiplication accumulators;

[0110] Step 1-2: For L1 cache and L2 cache, based on the basic cache size, perform cache partitioning and explore the size of input feature map cache (AL1, AL2), weight cache (WL1, WL2), and output feature map cache (OL1, OL2);

[0111] Step 1-3: Based on steps 1-1 and 1-2, clarify the complete hardware scale to explore the design space;

[0112] The specific steps for performance evaluation of the chip hardware scale parameter solution in step 3 are:

[0113] Step 3-1: Perform neural network model mapping based on the selected chip hardware scale parameters;

[0114] Step 3-2: Based on the chip hardware scale parameters, the encoding of the random neural network model mapping, and the computing model and data access model of the present invention, the computing amount and communication amount of the corresponding mapping scheme are extracted;

[0115] Step 3-3: Based on the computational workload and communication traffic in step 3-2, calculate the delay and energy, and obtain the energy-delay product performance of the mapping scheme, compare it with the historical optimal performance under the hardware scale parameters of the chip, and retain the better performance as the optimal performance;

[0116] Step 3-4: After the random neural network model mapping of step 3-2 and step 3-3 is iterated M times, the optimal performance is selected as the performance of the chip hardware scale parameter;

[0117] The specific steps of neural network communication traffic analysis and extraction in step 6 are:

[0118] Step 6-1: Neural network communication traffic analysis, based on the on-chip hardware architecture, the communication traffic is the data interaction between AL2, WL2, OL2 and PE clusters;

[0119] Step 6-2: Based on the number of PEs in the hardware scale, the source of PE parallelism is clarified, and the data sharing between PEs is obtained by grouping the PEs, and the communication flow is extracted;

[0120] Step 6-3: Based on different neural network models and different layers of the neural network models, data flow analysis is performed to characterize the characteristics of different neural network layers by scanning the proportions of three types of data flows corresponding to the three types of caches;

[0121] Step 6-4: Combining the communication flow extracted in step 6-2 with the data flow extracted in step 6-3, a complete neural network communication data flow configuration space is obtained;

[0122] The specific steps of the on-chip network topology performance evaluation in step 7 are:

[0123] Step 7-1: Based on the neural network communication data flow configuration space obtained in step 6, for different PE grouping situations, scan the proportions of the three types of data flows as input for performance evaluation;

[0124] Step 7-2: Evaluate the performance of the interconnect architecture under the communication data flow by evaluating the congestion degrate_ratio of each link and finding the maximum congestion max(degrate_ratio) as the performance of the interconnect architecture under the data flow;

[0125] Step 7-3: Traverse the common Mesh, CMesh, Torus and the loop-based non-routing topology proposed in the present invention, iterate steps 7-1 and 7-2, and obtain the maximum congestion of each topology structure under different PE grouping conditions that changes with the data traffic ratio.

[0126] The technical solution provided by the above embodiment of the present invention is further described below in combination with specific application examples.

[0127] In this specific application example, the computing power is set to 128TOPS, the PE operating frequency is 1Ghz, the size of the vector-MAC based on NVDLA is 16, and the number of vector-MACs in each PE is 16. The overall hardware architecture of the neural network chip with on-chip and inter-chip interconnection is as follows: Figure 3 shown.

[0128] The algorithm flow of the present invention is as follows Figure 2 As shown, the following steps are included:

[0129] Step 1: Based on the computing power and storage size of the neural network chip, clarify the chip hardware scale exploration design space (including computing power division and cache level scale);

[0130] Step 1-1: Based on the three-level hardware architecture of Package-Chiplet-PE, given a chip computing power of 128TOPS, calculate the corresponding number of multiplier-accumulator MACs:

[0131]

[0132] According to the given vector-MAC parameters (N size =16, N lane =16), according to the formula:

[0133] N MAC =N chip ×N PE ×N size ×N lane

[0134] The design space for computing power partitioning is to determine the number of chiplets N. chip and PE number N PE , satisfying the following formula:

[0135] N chip ×N PE =256

[0136] Then (N chip ,N PE ) can take values ​​of (1,256), (2,128), (4,64), (8,32), (16,16), (32,8), and (64,4), forming the design space for computing power partitioning:

[0137] Step 1-2: For L1 cache and L2 cache, based on the basic cache size, perform cache partitioning and explore the size of input feature map cache (AL1, AL2), weight cache (WL1, WL2), and output feature map cache (OL1, OL2);

[0138] Step 1-2-1: Based on the basic size of 11KB of L1 cache in PE, divide L1 cache. First, compress or expand the basic size of L1 cache to a certain extent, traversing from 0.5× to 16×, which is equivalent to traversing several L1 cache sizes. Secondly, for various L1 cache sizes, traverse 5 WAO ratios WAO_L1_ratio. Assuming that the WAO ratio is [8,2,1] and the L1 basic unit size is 1KB, the corresponding WL1 size is 8KB, the AL1 size is 2KB, and the OL1 size is 1KB;

[0139] Step 1-2-2: Based on the L1 cache partition, perform L2 cache partition. By changing the L2_ratio to expand or compress the size of the L2 cache, the following formula is satisfied:

[0140] WL2_size = N PE ×WL1_size×L2_ratio

[0141] AL2_size = N PE ×AL1_size×L2_ratio

[0142] OL2_size = N PE ×OL1_size×L2_ratio

[0143] L2_ratio = 2, N PE =16 as an example, based on the example result of step 1-2-1, the size of WL2 is 256KB, the size of AL2 is 64KB, and the size of OL2 is 32KB;

[0144] Step 1-3: Based on steps 1-1 and 1-2, clarify the complete hardware scale exploration design space. The complete design space parameters are shown in Table 1.

[0145] Table 1

[0146]

[0147] Step 2: Explore the design space based on the hardware scale in step 1, and select a hardware architecture scale parameter configuration in sequence;

[0148] Step 3: For the hardware architecture scale parameter configuration selected in step 2, perform performance evaluation and select the performance of the corresponding optimal mapping effect as the performance of the hardware scale parameter; below, assume that the hardware configuration selected in step 2 is N Chip =16, N PE =16, L1 b ase=1KB、 L2 ratio =2;

[0149] Step 3-1: Based on the selected chip hardware scale parameters, perform neural network model mapping; the neural network mapping considered in the present invention is mainly for the For loop expression of the convolution layer, and explores loop expansion, loop blocking, and loop order. All three are based on the three levels of Package-Chiplet-PE. The program example of the specific For loop mapping result is as follows. Each level contains for and parallel_for. For represents loop blocking, and parallel_for represents loop expansion. The order of for statements in each level represents the loop order. The dimensions in the for statement and parrallel_for statement are mainly 6 dimensions, namely, input feature map channel C, output feature map width P, height Q, channel K, convolution kernel width R, height S. Specifically, taking the K dimension and the number of PEs as an example, the relationship that needs to be satisfied is as follows:

[0150] PK0×PK2×PK3×K1×K2×K3≥K

[0151] PP2×PQ2×PK2≥N PE

[0152] Program example:

[0153]

[0154] Step 3-2: Based on the chip hardware scale parameters, the encoding of the random neural network model mapping, and the computing model and data access model of the present invention, the computing amount and communication amount of the corresponding mapping scheme are extracted;

[0155] Step 3-2-1: Neural network model mapping code, a code is composed of 34 data, of which 8 data represent loop expansion, 14 data represent loop block, and 14 data represent loop order. The neural network mapping code is randomly generated by randomly setting the values ​​of 34 data under the specified constraints;

[0156] Step 3-2-2: Based on the neural network mapping code, extract the corresponding calculation amount. The formula of the calculation model is as follows:

[0157] runtimeP=P3×P2×P1×PP3×PP2

[0158] runtimeQ=W3×W2×W1×PQ3×PQ2

[0159] runtimeK=K3×K2×K1×PK3×PK2×PK0

[0160] runtimeC=C3×C2×C1×PC0

[0161]

[0162] Taking the P dimension as an example, runtimeP represents the amount of computation required for the P dimension in the loop expansion and loop blocking of the actual mapping scheme, P represents the theoretical amount of computation of the P dimension in the convolution layer, and util. represents the utilization of computing resources. By calculating the resource utilization, we can calculate the non-blocking delay non_stall_lat as the delay performance; C is the input feature map channel, P, Q, K are the output feature map, height, and channel respectively, R and S are the convolution kernel width and height respectively, runtimeQ represents the amount of computation required for the Q dimension under the mapping scheme, runtimeK represents the amount of computation required for the K dimension under the mapping scheme, and runtimeC represents the amount of computation required for the C dimension under the mapping scheme. P1~P3 are the loop blocking parameters of the P dimension under three hardware levels (PE, Chiplet, Package), and PP3 is the P dimension in the Package level (different Chiplet PP2 is the loop expansion parameter of P dimension in chiplet level (between different PEs), Q1~Q3 are loop block parameters of Q dimension in three hardware levels (PE, chiplet, Package), K1~K3 are loop block parameters of K dimension in three hardware levels (PE, chiplet, Package), C1~C3 are loop block parameters of C dimension in three hardware levels (PE, chiplet, Package), PK0 is the loop expansion parameter of K dimension in PE level (between different multipliers and accumulators), PK2 and PK3 are loop expansion parameters of K dimension in chiplet level and Package level, PC0 is the loop expansion parameter of C dimension in PE level (between different multipliers and accumulators), MAC_num is the total number of multipliers and accumulators in the complete system, i.e., Package;

[0163] Step 3-2-3: Based on the neural network mapping code, the corresponding communication volume is extracted, wherein the specific working process of the data access model proposed by the present invention is as follows:

[0164] Step 3-2-3-1: For input feature maps, weights and output features respectively Figure 3For each type of data, determine whether each dimension parameter in the data stream is a related parameter. A related parameter means that changes in the value of the dimension parameter will directly cause changes in the corresponding data. For example, the timing for loop parameter C1 is a related parameter for the input feature map and weight, because after the value of c1 changes, the input feature map and weight will switch channels; while this parameter is a non-related parameter for the output feature map, because if only the value of c1 changes, the corresponding output feature map pixels remain the same, which will not affect the storage requirements of the output cache;

[0165] Step 3-2-3-2: Starting from the innermost for loop of the data flow, calculate the storage resource requirements of the input feature map, output feature map and weight at the for loop of this layer based on whether the parameters are related. If this storage resource requirement can be met, it means that during the internal calculation process of the for loop of this layer, there is no need to initiate read and write requests to the outer storage;

[0166] Step 3-2-3-3: Determine the bottleneck points (Stuck Position, SP) of the six storage units, namely WL1, AL1, OL1, WL2, AL2 and OL2, according to their capacities and the storage resource requirements at each for loop.

[0167] Step 3-2-3-4: Based on the SP points of each storage unit and the amount of data transfer required at the SP points, the total amount of data transfer between the PE and L2 cache, and between the chiplet and DRAM of the convolution layer can be calculated;

[0168] According to the above steps 3-2-3-1 to 3-2-3-4, Table 2 shows whether each parameter is a related parameter of the weight W, input feature map A and output feature map O under some data flows. T represents related and F represents unrelated. From bottom to top, it represents the change from the inside to the outside of the for loop. According to the correlation between the parameters, starting from the innermost loop and pushing outward, the storage requirements of WL1, AL1 and OL1 can be calculated according to the formula in the figure. Based on the extracted total data transmission information, the energy overhead caused by data transmission can be obtained according to certain indicators;

[0169] Table 2

[0170]

[0171] Step 3-3: Based on the computational workload and communication traffic in step 2-2, calculate the delay and energy, and obtain the energy-delay product performance of the mapping scheme, compare it with the historical optimal performance under the hardware scale parameters of the chip, and retain the better performance as the optimal performance;

[0172] Step 3-4: After the random neural network model mapping of step 2-2 and step 2-3 is iterated M times, the optimal performance is selected as the performance of the chip hardware scale parameter;

[0173] Step 4: Based on the architecture performance of step 3, compare whether it is better than the existing architecture performance. If it is better, retain it as the optimal architecture performance;

[0174] Step 5: Determine whether the entire hardware scale parameter design space has been traversed. If not, jump to step 2. If the traversal is complete, the hardware scale parameter corresponding to the optimal architecture performance is the result of the chip hardware scale exploration, which clarifies the optimal computing power division and cache hierarchy scale under given computing power and storage size;

[0175] Step 6: Based on the chip hardware scale parameters in step 5, analyze the characteristics of the neural network data flow and extract the neural network communication data flow;

[0176] Step 6-1: Neural network communication traffic analysis, based on the on-chip hardware architecture, the communication traffic is the data interaction between AL2, WL2, OL2 and PE clusters;

[0177] Step 6-2: Based on the number of PEs in the hardware scale, clarify the source of PE parallelism, group the PEs, obtain the data sharing between PEs, extract the communication flow, and use N PE = 16, the communication traffic from L2 cache to PE can be mainly divided into the following categories: Figure 4 The three types shown in the figure, PEs with the same color in the figure require the same data, and the data sharing of input feature maps and weights under different PE parallelism configurations are listed respectively. Taking PP2 or PQ2 as 16 and PK2 as 1 as an example, the 16 PEs on the chip share weights, which can be obtained through WL2 broadcast, but the required input feature maps are different, so data needs to be supplied from AL2 unicast;

[0178] Step 6-3: Based on different neural network models and different layers of neural network models, data flow analysis is performed. The characteristics of different neural network layers are characterized by scanning the ratios of the three data flows corresponding to the three caches. The ratios of the three data flows are changed by scanning act_ratio and wgt_ratio. act_ratio and wgt_ratio respectively represent how many input feature map data and weight data are to be sent from AL2 and WL2 to PE under one output result (sent from PE to OL2). For example, act_ratio = 4 means that on average, one output feature map result can be calculated and written for every four input feature map data read. The difference in this ratio reflects the different computational characteristics of different network layers, and widely covers different conditions of the number of input feature maps, weights, and output numbers;

[0179] Step 6-4: Combine the communication flow extracted in step 6-2 with the data flow extracted in step 6-3 to obtain the complete neural network communication data flow configuration space. The communication flow clarifies which PEs the three types of caches (WL2, AL2, OL2) need to communicate with, which part of the data, and how much data is communicated. Figure 4 Taking PP2 or PQ2 as 16 and PK2 as 1 as an example, the communication flow makes it clear that WL2 needs to broadcast the same weighted data to all PEs, and AL2 needs to unicast different data to 16 PEs. The data flow makes it clear the number of data that WL2, AL2, and OL2 need to give to PE. Taking wgt_ratio as 4 and agt_ratio as 2 as an example, combined with the communication flow, it shows that for each output data generated by PE, WL2 needs to broadcast the same 4 weighted data to all PEs, and AL2 needs to unicast 2 different input data to 16 PEs, that is, unicast 32 input data.

[0180] Step 7: Based on the communication data stream extracted in step 6, the performance of the on-chip interconnection topology is evaluated, and the common topology is compared with the loop-based non-routing topology proposed in the present invention to obtain the optimal interconnection topology;

[0181] Step 7-1: Based on the neural network communication data flow configuration space obtained in step 6, for different PE grouping situations, scan the proportions of the three types of data flows as input for performance evaluation;

[0182] Step 7-2: Evaluate the performance of the interconnection architecture under the communication data flow. The present invention is based on the XY routing algorithm, by evaluating the congestion degrate_ratio of each link, where the congestion is the total amount of communication on the link / link bandwidth, and finding the maximum congestion max(degrate_ratio) among all links as the performance of the interconnection architecture under the data flow;

[0183] Step 7-3: Traverse the common Mesh, CMesh, Torus and the loop-based non-routing topology proposed by the present invention, such as Figure 5 and Figure 6 The common topology and the loop-based non-routing topology proposed in the present invention are shown. Steps 7-1 and 7-2 are iterated to obtain the maximum congestion degree of each topology under different PE grouping conditions as the data traffic ratio changes.

[0184] Step 8: Based on the optimal hardware scale parameters of step 5 and the optimal on-chip interconnect topology of step 7, the final output of the neural network chip hardware architecture exploration algorithm is obtained, which is the optimal hardware architecture.

[0185] Figure 7A schematic diagram of the component modules of a neural network chip hardware architecture design system based on on-chip-inter-chip interconnection provided by one embodiment of the present invention.

[0186] like Figure 7 As shown, the on-chip-inter-chip interconnected neural network chip hardware architecture design system provided by this embodiment may include the following modules:

[0187] A parameter configuration module, which determines the chip hardware scale exploration design space based on the computing power and storage size of the neural network chip; based on the hardware scale exploration design space, a hardware architecture scale parameter configuration is selected in sequence;

[0188] Performance evaluation module, which performs performance evaluation on the hardware architecture scale parameter configuration and selects the optimal performance as the performance of the hardware architecture scale parameter; compares the performance of the hardware architecture scale parameter with the existing hardware architecture performance, and if it is better than the existing hardware architecture performance, retains the performance as the optimal architecture performance; traverses the entire hardware scale exploration design space to obtain the final optimal architecture performance, and the hardware architecture scale parameter corresponding to the final optimal architecture performance is the result of the chip hardware scale exploration, thereby obtaining the optimal computing power division and cache hierarchy scale under given computing power and storage size;

[0189] Communication data stream extraction module: This module analyzes the characteristics of neural network data stream based on the results of chip hardware scale exploration, extracts neural network communication data stream, and obtains the configuration space of neural network communication data stream;

[0190] The hardware architecture building module evaluates the performance of the on-chip interconnect topology based on the neural network communication data flow configuration space to obtain the optimal on-chip interconnect topology. The module combines the results of the chip hardware scale exploration with the optimal on-chip interconnect topology to obtain the final output based on the neural network chip hardware architecture exploration, which is the optimal neural network chip hardware architecture.

[0191] It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system, that is, the embodiments in the method can be understood as preferred examples for constructing the system, which will not be elaborated here.

[0192] An embodiment of the present invention provides a neural network chip hardware architecture based on on-chip-to-inter-chip interconnection, and the neural network chip hardware architecture is constructed using any one of the design methods in the above embodiments of the present invention.

[0193] An embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any one of the methods in the above embodiments of the present invention.

[0194] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it can be used to execute any one of the methods in the above embodiments of the present invention.

[0195] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile lememory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0196] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0197] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0198] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0199] The on-chip-inter-chip interconnected neural network chip hardware architecture design method and system provided in the above-mentioned embodiment of the present invention establish detailed data access models at all levels of the computing model based on a comprehensive analysis of the data flow of the convolutional layer, and obtain accurate and reliable delay and energy consumption indicators; a hardware design space exploration method for chip granularity division, L1 and L2 cache size configuration, and allocation of various data cache sizes is proposed. Through hardware design space exploration, the obtained solution achieves an EDP reduction of 2.47% to 87.99% compared with other solutions; a Ring-based on-chip network-without-routing topology is proposed for the characteristics of on-chip communication traffic of the neural network, which improves communication performance while reducing the on-chip network area.

[0200] All matters not covered in the above embodiments of the present invention are well known in the art.

[0201] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnection, comprising: Based on the computing power and storage size of the neural network chip, determine the chip hardware scale and explore the design space; exploring a design space based on the hardware scale and sequentially selecting a hardware architecture scale parameter configuration therefrom; For the hardware architecture scale parameter configuration, a performance evaluation is performed, and the optimal performance is selected as the performance of the hardware architecture scale parameter; The performance based on the hardware architecture scale parameter is compared with the existing hardware architecture performance, and if it is better than the existing hardware architecture performance, the performance is retained as the optimal architecture performance; Traverse the entire hardware scale exploration design space to obtain the final optimal architecture performance. The hardware architecture scale parameter corresponding to the final optimal architecture performance is the result of the chip hardware scale exploration, and then the optimal computing power division and cache hierarchy scale under given computing power and storage size are obtained; Based on the results of the chip hardware scale exploration, the characteristics of the neural network data flow are analyzed, the neural network communication data flow is extracted, and the neural network communication data flow configuration space is obtained; Based on the neural network communication data flow configuration space, an on-chip interconnect topology performance evaluation is performed, and the performance of each topology is compared to obtain an optimal on-chip interconnect topology; Based on the result of the chip hardware scale exploration and the optimal on-chip interconnection topology, a final output based on the neural network chip hardware architecture exploration is obtained, which is the optimal neural network chip hardware architecture; The on-chip interconnection topology structure includes: a common topology structure and an established loop-based non-routing on-chip topology structure; wherein: The common topologies include: Mesh topology, CMesh topology and Torus topology; The loop-based routing-free on-chip topology structure includes: 3 L2 caches corresponding to the input feature map L2 cache AL2, the weight L2 cache WL2 and the output feature map L2 cache OL2 respectively and 16 processor units PE, a total of 19 nodes, the 19 nodes interact with four loops through their respective corresponding injection-receiving interfaces, and the four loops realize data communication between different nodes; wherein the injection-receiving interface has an arbitrary data flow direction, the four loops are unidirectional loops, and the data packet advances one hop in each cycle on the loops; Through the performance evaluation of the on-chip interconnection topology, the optimal on-chip interconnection topology is obtained, which is the loop-based non-routing on-chip topology.

2. The method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections according to claim 1, characterized in that: Determining the chip hardware scale to explore the design space includes: Based on the three-level hardware architecture of Package-Chiplet-PE, given the chip computing power, the number of MACs inside PE, the number of PEs in each chiplet, and the number of chiplets are determined, and the design space of computing power division is determined to meet the following formula: N MAC =N chip ×N PE ×N size ×N lane Among them, N MAC Indicates the number of multipliers and accumulators corresponding to the computing power, N chip Indicates the number of chiplets, N PE Indicates the number of PEs in each chiplet, N size Indicates the vector-MAC size, N lane Indicates the number of vector-MACs in each PE; Package indicates that the complete system contains multiple chiplets, Chiplet indicates chiplet, PE indicates processing unit, MAC indicates multiplication accumulator, vector-MAC: a vector multiplication accumulator composed of multiple multiplication accumulators; For the L1 cache and L2 cache in PE, cache partitioning is performed respectively, and the size of input feature map cache, weight cache and output feature map cache under L1 cache and L2 cache after partitioning is explored; Based on the above steps, the chip hardware scale exploration design space is obtained.

3. The method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections according to claim 1, characterized in that: The step of performing performance evaluation on the hardware architecture scale parameter configuration and selecting the optimal performance as the performance of the hardware architecture scale parameter includes: Based on the selected hardware architecture scale parameters, performing neural network model mapping; Based on the hardware architecture scale parameter and the neural network model obtained by mapping, randomly generate the neural network model mapping code, and based on the neural network model mapping code, extract the computational amount and communication amount of the corresponding mapping scheme through the constructed computational model and data access model; Based on the extracted computational amount and communication amount, the delay and energy are calculated to obtain the energy-delay product performance of the mapping scheme, and compared with the historical optimal performance under the hardware architecture scale parameter, and the better performance is retained as the performance corresponding to the mapping scheme; After multiple iterations of the mapped neural network model, the optimal performance is selected as the performance of the hardware architecture scale parameter.

4. The method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections according to claim 3, characterized in that: The calculation model is constructed to obtain the delay performance indicator non_stall_lat caused by the calculation task; wherein the calculation method includes: runtimeP=P3×P2×P1×PP3×PP2 runtimeQ=Q3×Q2×Q1×PQ3×PQ2 runtimeK=K3×K2×K1×PK3×PK2×PK0 runtimeC=C3×C2×C1×PC0 In the formula, runtimeP represents the amount of computation required for the P dimension in the loop expansion and loop blocking of the actual mapping scheme, P represents the theoretical amount of computation for the P dimension in the convolution layer, util. represents the utilization of computing resources, and the non-blocking delay non_stall_lat is calculated as the delay performance by calculating the resource utilization; C is the input feature map channel, P, Q, and K are the output feature map, height, and channel, R and S are the convolution kernel width and height, runtimeQ represents the amount of computation required for the Q dimension under the mapping scheme, runtimeK represents the amount of computation required for the K dimension under the mapping scheme, and runtimeC represents the amount of computation required for the C dimension under the mapping scheme. P1~P3 are the loop blocking parameters of the P dimension under three hardware levels, namely PE, Chiplet, and Package, and PP3 is the P dimension between different Chiplet in the Package level. PP2 is the loop expansion parameter of P dimension between different PEs in chiplet level, Q1~Q3 are loop block parameters of Q dimension at three hardware levels, namely PE, chiplet and Package, K1vK3 are loop block parameters of K dimension at three hardware levels, namely PE, chiplet and Package, C1~C3 are loop block parameters of C dimension at three hardware levels, namely PE, chiplet and Package, PK0 is the loop expansion parameter of K dimension between different multipliers and accumulators at PE level, PK2 and PK3 are loop expansion parameters of K dimension at chiplet level and Package level, PC0 is the loop expansion parameter of C dimension between different multipliers and accumulators at PE level, MAC_num is the total number of multipliers and accumulators in the complete system, namely Package; The working process of the data access model includes: Determine whether each dimension parameter in the data stream is a relevant parameter for the data of the input feature map, weight, and output feature map respectively; Starting from the innermost for loop of the data flow, the storage resource requirements of the input feature map, output feature map, and weight at the for loop of this layer are calculated based on whether the parameters are related; For a total of 6 storage units of input feature maps, weights and output feature maps under L1 and L2 caches, determine the bottleneck points of the 6 storage units according to the capacity of each storage unit and the storage resource requirements at each layer of for loop; According to the bottleneck points of each storage unit and the amount of data transmission required at the bottleneck points, the total amount of data transmission between the PE level and the L2 cache and between the chiplet level and the DRAM of the convolution layer is calculated, which is the communication volume of the mapping scheme. Based on the calculated total amount of data transmission information, the energy overhead caused by data transmission is obtained according to the set indicators.

5. The method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections according to claim 1, characterized in that: The method of analyzing the characteristics of neural network data flow based on the result of the chip hardware scale exploration and extracting the neural network communication data flow includes: Based on the results of chip hardware scale exploration, the neural network communication traffic is analyzed, where the communication traffic is the input feature map, weights in the L2 cache of the PE, and the data interaction between the output feature map and the PE cluster; Based on the number of PEs in the hardware architecture scale, determine the source of PE parallelism, group PEs, obtain data sharing between PEs, and extract communication flows; Based on different neural network models and different layers of neural network models, data flow analysis is performed to obtain different proportions of three types of data flow: input feature map data volume, weight data volume, and output feature map data volume. The characteristics of different neural network layers are characterized by scanning the proportions of the three types of data flow with L1 cache, L2 cache, and DRAM. Combining the communication flow with the data flow, a complete neural network communication data flow configuration space is obtained.

6. The method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections according to claim 1, characterized in that: The on-chip interconnect topology performance evaluation comprises: Based on the neural network communication data flow configuration space, for different PE grouping situations, the ratio of L1 cache, L2 cache and DRAM data flow is scanned as input for performance evaluation; Evaluate the performance of the interconnect architecture under the communication data flow by evaluating the congestion degrate_ratio of each link and finding the maximum congestion max(degrate_ratio) as the performance of the interconnect architecture under the communication data flow; Traverse the on-chip interconnect topology and iterate the above steps to obtain the maximum congestion degree of each topology under different PE grouping conditions as the data traffic ratio changes, and complete the performance evaluation of the on-chip interconnect topology.

7. The method for designing a hardware architecture of a neural network chip with on-chip and inter-chip interconnections according to any one of claims 1 to 6, characterized in that: Also includes any one or more of the following: - The data flow characteristics include: communication mode and statistical information of the data flow; wherein: The communication mode includes: unicast, multicast and broadcast; The statistical information of the data flow includes: a sending node, a destination node and a communication volume.

8. A neural network chip hardware architecture design system with on-chip-inter-chip interconnection, where on-chip-inter-chip means that the system under consideration is a complete on-chip system plus an inter-chip system, and its characteristics include: Parameter configuration module, which determines the chip hardware scale and explores the design space based on the computing power and storage size of the neural network chip; exploring a design space based on the hardware scale and sequentially selecting a hardware architecture scale parameter configuration therefrom; A performance evaluation module, which performs performance evaluation on the hardware architecture scale parameter configuration and selects the optimal performance as the performance of the hardware architecture scale parameter; compares the performance of the hardware architecture scale parameter with the existing hardware architecture performance, and if it is better than the existing hardware architecture performance, retains the performance as the optimal architecture performance; traverses the entire hardware scale exploration design space to obtain the final optimal architecture performance, and the hardware architecture scale parameter corresponding to the final optimal architecture performance is the result of the chip hardware scale exploration, thereby obtaining the optimal computing power division and cache hierarchy scale under given computing power and storage size; A communication data stream extraction module, which analyzes the characteristics of the neural network data stream based on the results of the chip hardware scale exploration, extracts the neural network communication data stream, and obtains the neural network communication data stream configuration space; A hardware architecture building module, which performs an on-chip interconnect topology performance evaluation based on the neural network communication data flow configuration space to obtain an optimal on-chip interconnect topology; based on the results of the chip hardware scale exploration and the optimal on-chip interconnect topology, a final output based on the neural network chip hardware architecture exploration is obtained, which is the optimal neural network chip hardware architecture; The on-chip interconnection topology structure includes: a common topology structure and an established loop-based non-routing on-chip topology structure; wherein: The common topologies include: Mesh topology, CMesh topology and Torus topology; The loop-based routing-free on-chip topology structure includes: 3 L2 caches corresponding to the input feature map L2 cache AL2, the weight L2 cache WL2 and the output feature map L2 cache OL2 respectively and 16 processor units PE, a total of 19 nodes, the 19 nodes interact with four loops through their respective corresponding injection-receiving interfaces, and the four loops realize data communication between different nodes; wherein the injection-receiving interface has an arbitrary data flow direction, the four loops are unidirectional loops, and the data packet advances one hop in each cycle on the loops; Through the performance evaluation of the on-chip interconnection topology, the optimal on-chip interconnection topology is obtained, which is the loop-based non-routing on-chip topology.

9. A neural network chip hardware architecture with on-chip and inter-chip interconnection, characterized in that: The method is constructed by using the design method described in any one of claims 1 to 6.

10. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it can be used to perform the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to execute the method according to any one of claims 1 to 7.

Images