Decentralization method and system for neuromorphic chip

Abstract

The invention provides a decentration method and system for a neuromorphic chip. The decentration method for the neuromorphic chip comprises the following steps: designing the number of MAC (Media Access Control) in a single computing core; designing the number of cores based on the number of the MAC in the single calculation core; and designing the MEM capacity and the MAC structure of the single core based on the exertion of the calculation efficiency, and completing the decentration design of the neuromorphic chip. According to the technical scheme, the design of a decentralized architecture chip is optimized, and the number of internal cores is designed; and the MAC number adaptive to the pipeline system is designed according to the operation bandwidth, so that the mapping efficiency of the chip hardware performance is improved, and the power consumption is reduced.

Description

Background Art

[0002] With the rapid development of artificial intelligence, the bottlenecks of existing computers with von Neumann architecture systems are gradually emerging. At present, the hardware running neural networks is roughly divided into two categories: deep learning accelerator chips and neuromorphic chips. In addition, Tsinghua University has developed hybrid brain-like chips that support heterogeneous fusion networks for general artificial intelligence. At present, most accelerators use a storage-computing separation architecture, that is, the traditional von Neumann architecture.

[0003] This type of von Neumann architecture chip uses increased transistor density to improve energy efficiency, such as the TPU from the first generation of 45nm to the fourth generation of 7nm, and on the other hand, uses near-memory design and domain-specific architecture design (DSA) to reduce power consumption. Due to the use of external storage, data requires a lot of movement between the computing unit and the storage unit, and read and write operations. The access power consumption caused by moving data will cause computing delays due to reading and writing. On the other hand, when the number of computing units gradually increases, the neural network needs to be finely split. Although the von Neumann architecture accelerator chip also has a multi-core architecture, most of them are serial processing from an algorithm perspective. The "von Neumann" based on the separation of storage and computing has a simple structure and is easy to use for high-speed computing, and the mapping method is simple, and resources are scheduled according to the time complexity of the task.

[0004] Due to the high degree of parallelism of deep neural networks, some deep learning accelerator chips are also beginning to adopt decentralized architectures. For example, Simba is decentralized at the chip level and uses storage and computing separation inside the chip. Graphcore's IPU distributed storage and computing integrated architecture reduces the storage wall effect through decentralized architecture. For example, Simba's 6-chip 36-core system. Graphcore's multi-core architecture has 1472 computing cores per chip. Tsinghua University's Tianji series chips, for example Figure 5 As shown in the figure, the 160 cores of Tianji X, under the multi-core architecture, this distributed parallel multi-core architecture can increase the concurrent access bandwidth within a single chip to up to 6000GB / s. The concurrent transmission bandwidth between chip cores can reach 512GB / s.

[0005] However, there is no fixed standard for decentralized design. In the fixed storage and computing resources, the chip design efficiency of Tianji series chips and Simba is relatively low. For example, in the mapping calculation of resnet50, the mapping efficiency is still less than 20%. The data of Graphcore's IPU computing system is not yet known.

[0006] For a chip with fixed storage and computing resources, a decentralized design, that is, a distributed multi-core architecture design, is adopted. The more cores there are, the greater the concurrent access bandwidth and inter-core transmission bandwidth of a single chip. When the read and write access bandwidth of the computing unit to the storage unit within a single computing core is fixed, for example, 32B / clc, the transmission access bandwidth is 8B / clc. Here, clc represents a clock. The number of MACs within a single core needs to be designed so that the computing power can be effectively exerted. However, there is currently no ultimate relationship or design method between the increase in parallel bandwidth and the computing speed.

[0007] References:

[0008] [1] S.Ma, J.Pei, W.Zhang, G.Wang, D.Feng, F.Yu, C.Song, H.Qu, C.Ma, M.Lu et al., "Neuromorphic computing chip with spatiotemporal elasticity for multi-intelligent-tasking robots," Science Robotics, vol.7, no.67, p.eabk2948, 2022.V.

[0009] [2] Pei Jing, Wang Song, Ma Cheng, Li Bowen, Yu Qiushuang, Xu Haizheng. Control method, device, computer equipment and storage medium of multiplier-adder. CN116400884A, 2023-07-07.

[0010] [3] Deng Lei. Research on computational models and key technologies of heterogeneous fusion brain-like computing platform[D]. Beijing: Tsinghua University, 2017.

[0011] [4] Wang Song. Research on performance optimization methods of Tianji X1 brain-like computing chip[D]. Beijing: Tsinghua University, 2021.

[0012] [5] Wang Guanrui. Research on efficient cross-paradigm brain-like computing chips and hardware platforms[D]. Beijing: Tsinghua University, 2021. Summary of the invention

[0013] The present application provides a decentralized method and system for a neuromorphic chip, so as to realize a decentralized design of a neuromorphic chip.

[0014] In a first aspect, a decentralized method for a neuromorphic chip is provided, comprising the following steps:

[0015] Design the number of MACs within a single computing core;

[0016] Designing the number of cores based on the number of MACs within the single computing core;

[0017] Based on the utilization of computing efficiency, the MEM capacity and MAC structure of a single core are designed to complete the decentralized design of the neuromorphic chip.

[0018] In the above technical scheme, the number of MACs inside a single computing core is designed; based on the number of MACs inside the single computing core, the number of cores is designed; based on the performance of computing efficiency, the MEM capacity and MAC structure of a single core are designed to complete the decentralized design of the neuromorphic chip; the decentralized architecture chip design is optimized, and the number of internal cores is designed; the number of MACs that is compatible with the pipeline system is designed according to the operating bandwidth, thereby improving the mapping efficiency of the chip hardware performance and reducing power consumption.

[0019] In a specific implementation scheme, the number of MACs within a single computing core is designed based on bandwidth operations and a pipeline system.

[0020] In a specific implementation scheme, based on bandwidth operation and pipeline system, the steps of designing the number of MACs in a single computing core include:

[0021] The number of MACs in the weighted direction is designed based on the read and write time determined by the access data bandwidth and the pipeline system characteristics;

[0022] Then design the parallelism of MAC according to the image size of the deep neural network;

[0023] Design the number of MACs inside a single computing core based on the size of each dimension.

[0024] In a specific implementation scheme, the water flow efficiency is guaranteed by using the product of the input depth direction and the output depth direction.

[0025] In a specific implementation scheme, the step of designing the number of cores based on the number of MACs within the single computing core is specifically as follows:

[0026] Based on the number of MACs inside the single computing core and the total computing power, the number of computing cores inside the chip is designed.

[0027] In a specific implementation scheme, the number of MACs within the single computing core is determined by the product of the corresponding weight part and the data parameter part.

[0028] In a specific implementation scheme, the total number of MACs required by the chip is determined by the performance indicator design and computing efficiency requirements of the benchmark network.

[0029] In a specific implementation scheme, the number of MACs for non-pipeline data is designed based on the size of the task being calculated.

[0030] In a specific implementation scheme, the storage capacity inside a single computing core is corrected, and the MEM storage capacity inside a single core and the structure of the MAC are estimated.

[0031] In the second aspect, a decentralized system of a neuromorphic chip is provided, comprising:

[0032] Single-core MAC number design module, used to design the number of MACs within a single computing core;

[0033] A core number design module, used to design the number of cores based on the number of MACs inside the single computing core;

[0034] The single-core capacity structure design module is used to design the MEM capacity and MAC structure of a single core based on the computational efficiency, thus completing the decentralized design of the neuromorphic chip.

[0035] In the above technical scheme, the number of MACs inside a single computing core is designed; based on the number of MACs inside the single computing core, the number of cores is designed; based on the performance of computing efficiency, the MEM capacity and MAC structure of a single core are designed to complete the decentralized design of the neuromorphic chip; the decentralized architecture chip design is optimized, and the number of internal cores is designed; the number of MACs that is compatible with the pipeline system is designed according to the operating bandwidth, thereby improving the mapping efficiency of the chip hardware performance and reducing power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a decentralized method for a neuromorphic chip provided in an embodiment of the present application;

[0037] Figure 2 A structural block diagram of a decentralized system of a neuromorphic chip provided in an embodiment of the present application;

[0038] Figure 3 Schematic diagram of different storage and computing densities of deep neural networks provided in embodiments of the present application, (a) functional core and (b) physical layout diagram of the entire chip;

[0039] Figure 4 A schematic diagram of the design of the number of MACs inside a single computing core provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the structure of the water flow system provided in the embodiment of the present application;

[0041] Figure 6 A schematic diagram of the degree of decentralization designed for an embodiment of the present application;

[0042] Figure 7 A schematic diagram of the coordinated design of the MAC structure and MEM capacity provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.

[0044] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0045] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0046] To facilitate understanding of the decentralized method and system of the neuromorphic chip provided in the embodiment of the present application, its application scenario is first explained. The decentralized method and system of the neuromorphic chip provided in the embodiment of the present application are used to realize the decentralized design of the neuromorphic chip. There is currently no fixed standard for decentralized design. In fixed storage and computing resources, the chip design efficiency of the Tianji series chips and Simba is relatively low. For example, in the mapping calculation of resnet50, the mapping efficiency is still less than 20%. The current data of the computing system of Graphcore's IPU is not yet known. For a chip with fixed storage and computing resources, a decentralized design, that is, a distributed multi-core architecture design, is adopted. The more the number of cores, the greater the concurrent access bandwidth and inter-core transmission bandwidth of a single chip. When the read and write access bandwidth of the computing unit to the storage unit inside a single computing core is fixed, for example, 32B / clc, the transmission access bandwidth is under 8B / clc. The clc here represents a clock. It is necessary to design the number of MACs inside a single core so that the computing power can be effectively exerted. However, there is currently no ultimate relationship or design method between the increase in parallel bandwidth and the computing speed. To this end, the present application provides a decentralized method and system for a neuromorphic chip to achieve a decentralized design of a neuromorphic chip. The following is a detailed description of the embodiment in conjunction with specific drawings.

[0047] refer to Figures 1 to 7 , Figure 1 A flowchart of a decentralized method for a neuromorphic chip provided in an embodiment of the present application; Figure 2A structural block diagram of a decentralized system of a neuromorphic chip provided in an embodiment of the present application; Figure 3 Schematic diagram of different storage and computing densities of deep neural networks provided in embodiments of the present application, (a) functional core and (b) physical layout diagram of the entire chip; Figure 4 A schematic diagram of the design of the number of MACs inside a single computing core provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of the water flow system provided in the embodiment of the present application; Figure 6 A schematic diagram of the degree of decentralization designed for an embodiment of the present application; Figure 7 A schematic diagram of the coordinated design of the MAC structure and MEM capacity provided in an embodiment of the present application.

[0048] exist Figure 1 In the embodiment of the present application, a decentralized method for a neuromorphic chip is provided, comprising the following steps:

[0049] Design the number of MACs within a single computing core;

[0050] Designing the number of cores based on the number of MACs within the single computing core;

[0051] Based on the utilization of computing efficiency, the MEM capacity and MAC structure of a single core are designed to complete the decentralized design of the neuromorphic chip.

[0052] In the above technical scheme, the number of MACs inside a single computing core is designed; based on the number of MACs inside the single computing core, the number of cores is designed; based on the performance of computing efficiency, the MEM capacity and MAC structure of a single core are designed to complete the decentralized design of the neuromorphic chip; the decentralized architecture chip design is optimized, and the number of internal cores is designed; the number of MACs that is compatible with the pipeline system is designed according to the operating bandwidth, thereby improving the mapping efficiency of the chip hardware performance and reducing power consumption.

[0053] Specifically, refer to Figures 1 to 7 , the specific steps include:

[0054] First, the number of MACs inside a single computing core is designed.

[0055] For neuromorphic chips of neural networks, the number of MACs can often be divided into the corresponding weight part and the data parameter part. The number of MACs is determined by the product of the two. Figure 4 As shown: the MAC array has 32 MACs in the corresponding weight part and 4 MACs in the corresponding data IA part, totaling 128 MACs.

[0056] Under a fixed read and write access operation bandwidth, if it is 32B / clc, for the neural network parallel computing pipeline mode, 32B of weight data is read in for each clock and calculated once at the same time. When the number of MACs is greater than 32 in the corresponding number of weights, such as 64, it is necessary to read in the weights twice and calculate once. At this time, the read-calculation ratio of the pipeline system is 2:1, and the calculation efficiency is not higher than 50%. When it is 128, the calculation ratio is 4:1. This will reduce the proportion of the chip's calculation time to the total time, and the calculation efficiency is no more than 25%. At this time, the calculation efficiency of the chip is low. Therefore, to ensure the efficiency of the pipeline system, when designing the number of MACs, the number corresponding to one side as the pipeline data is equal to the access bandwidth. For example, assuming that the image parameters are used as pipeline data, the data is read in once each time and calculated once at the same time, then the number of MACs corresponding to the image aspect under 32B / clc is 32.

[0057] The number of MACs in non-pipeline data takes into account the size of the computed task, such as ResNet and other networks, the front-end size is 56, 28, etc., and the back-end image size is mostly 7. Therefore, to ensure the efficiency of over-size matching, the parallelism should be 2, 4, 8, etc. to maximize the matching of the size of MAC and the size of the neural network.

[0058] Based on the above design mechanism, if the operating bandwidth is 64B / clc, the number of MACs is designed to be no more than 512.

[0059] Secondly, the number of cores is designed based on the internal number of a single computing core, the computing power requirements, or the performance indicators of a benchmark network, that is, the degree of decentralization of the chip for the von Neumann architecture. Figure 6 As shown: from one core with 128x256 MACs and a total storage capacity of 128x256KB of SRAM, through different degrees of decentralized design from 32 cores to 1024 cores, etc. Based on the demand for computing power or the performance index design and computing power efficiency requirements of a benchmark network, the total number of MACs required can be calculated, and the number of computing cores can be calculated based on the total number of MACs.

[0060] Finally, the computing efficiency of computing power is exerted under the fixed number of cores in a single chip. The MEM capacity and MAC structure corresponding to a single core can be designed to exert its mapping efficiency. By repeatedly correcting the storage capacity inside a single computing core, the MEM storage capacity and MAC structure inside a single core can be estimated. Figure 7 shown.

[0061] In this embodiment, the degree of decentralization design is implemented based on fixed storage and computing resources in a decentralized architecture that integrates storage and computing, the number of MACs is designed in a decentralized architecture that integrates storage and computing, and the optimal parallel access bandwidth and transmission bandwidth are implemented under a fixed access bandwidth to optimize the computing power of the chip.

[0062] In a specific implementation scheme, the number of MACs within a single computing core is designed based on bandwidth operations and a pipeline system.

[0063] In a specific implementation scheme, based on bandwidth operation and pipeline system, the steps of designing the number of MACs in a single computing core include:

[0064] The number of MACs in the weighted direction is designed based on the read and write time determined by the access data bandwidth and the pipeline system characteristics;

[0065] Then design the parallelism of MAC according to the image size of the deep neural network;

[0066] Design the number of MACs inside a single computing core based on the size of each dimension.

[0067] Specifically, the number and size of MACs of a single core are designed based on bandwidth operations and the pipeline system, including: designing the number of MACs in the weight direction, that is, the product of the input depth direction and the output depth direction, according to the read and write time determined by the access data bandwidth and the characteristics of the pipeline system to ensure high pipeline efficiency; then designing the parallelism of MAC according to the image size of the deep neural network; designing the number of MACs inside a single computing core according to the size of each dimension, thereby optimizing the design of the decentralized architecture chip.

[0068] In a specific implementation scheme, the water flow efficiency is guaranteed by using the product of the input depth direction and the output depth direction.

[0069] In a specific implementation scheme, the step of designing the number of cores based on the number of MACs within the single computing core is specifically as follows:

[0070] Based on the number of MACs inside the single computing core and the total computing power, the number of computing cores inside the chip is designed.

[0071] In a specific implementation scheme, the number of MACs within the single computing core is determined by the product of the corresponding weight part and the data parameter part.

[0072] In a specific implementation scheme, the total number of MACs required by the chip is determined by the performance indicator design and computing efficiency requirements of the benchmark network.

[0073] In a specific implementation scheme, the number of MACs for non-pipeline data is designed based on the size of the task being calculated.

[0074] In a specific implementation scheme, the storage capacity inside a single computing core is corrected, and the MEM storage capacity inside a single core and the structure of the MAC are estimated.

[0075] Specifically, according to the demand for computing power and computing power efficiency, the design scheme for the degree of decentralization, that is, the number of cores, includes: according to the number of MACs of a single computing core, and then according to the total computing power, the number of computing cores inside a chip is designed, and the MAC structure and MEM storage capacity are designed for computing efficiency. The mapping efficiency of chip hardware performance is improved and power consumption is reduced.

[0076] exist Figure 2 In the embodiment of the present application, a decentralized system of a neuromorphic chip is provided, including:

[0077] Single-core MAC number design module, used to design the number of MACs within a single computing core;

[0078] A core number design module, used to design the number of cores based on the number of MACs inside the single computing core;

[0079] The single-core capacity structure design module is used to design the MEM capacity and MAC structure of a single core based on the computational efficiency, thus completing the decentralized design of the neuromorphic chip.

[0080] In the above technical scheme, the number of MACs inside a single computing core is designed; based on the number of MACs inside the single computing core, the number of cores is designed; based on the performance of computing efficiency, the MEM capacity and MAC structure of a single core are designed to complete the decentralized design of the neuromorphic chip; the decentralized architecture chip design is optimized, and the number of internal cores is designed; the number of MACs that is compatible with the pipeline system is designed according to the operating bandwidth, thereby improving the mapping efficiency of the chip hardware performance and reducing power consumption.

[0081] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.

[0082] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.

[0083] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0084] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.

Claims

1. A decentralized method for a neuromorphic chip, characterized in that: The following steps are involved: Design the number of MACs within a single computing core; Designing the number of cores based on the number of MACs within the single computing core; Based on the utilization of computing efficiency, the MEM capacity and MAC structure of a single core are designed to complete the decentralized design of the neuromorphic chip.

2. The decentralization method of the neuromorphic chip according to claim 1, characterized in that: Based on bandwidth operations and pipeline systems, the number of MACs inside a single computing core is designed.

3. The decentralization method of the neuromorphic chip according to claim 2, characterized in that: Based on bandwidth operation and pipeline system, the steps to design the number of MACs in a single computing core include: The number of MACs in the weighted direction is designed based on the read and write time determined by the access data bandwidth and the pipeline system characteristics; Then design the parallelism of MAC according to the image size of the deep neural network; Design the number of MACs inside a single computing core based on the size of each dimension.

4. The decentralization method of the neuromorphic chip according to claim 3, characterized in that: The pipeline efficiency is guaranteed by using the product of the input depth direction and the output depth direction.

5. The decentralization method of the neuromorphic chip according to claim 4, characterized in that: The steps of designing the number of cores based on the number of MACs within the single computing core are as follows: Based on the number of MACs inside the single computing core and the total computing power, the number of computing cores inside the chip is designed.

6. The decentralization method of the neuromorphic chip according to claim 5, characterized in that: The number of MACs within the single computing core is determined by the product of the corresponding weight part and the data parameter part.

7. The decentralization method of the neuromorphic chip according to claim 6, characterized in that: The total number of MACs required by the chip is determined by the performance indicator design and computing efficiency requirements of the benchmark network.

8. The decentralization method of the neuromorphic chip according to claim 7, characterized in that: The number of MACs for non-pipeline data is designed based on the size of the task being computed.

9. The decentralization method of the neuromorphic chip according to claim 8, characterized in that: By correcting the storage capacity inside a single computing core, the MEM storage capacity inside a single core and the structure of MAC are estimated.

10. A decentralized system of neuromorphic chips, characterized in that: include: Single-core MAC number design module, used to design the number of MACs within a single computing core; A core number design module, used to design the number of cores based on the number of MACs inside the single computing core; The single-core capacity structure design module is used to design the MEM capacity and MAC structure of a single core based on the computational efficiency, thus completing the decentralized design of the neuromorphic chip.

Images