Artificial intelligence accelerator and design method

Abstract

The present application provides an artificial intelligence accelerator and a design method, wherein the artificial intelligence accelerator includes N computing cores and M cache cores forming at least one core stacking group, wherein the N computing cores and the M cache cores are arranged in a vertical direction, wherein each core stacking group includes a computing core and at least one cache core, wherein the cache cores are connected to the computing cores through vertical through holes, wherein an on-chip network is deployed on the computing cores, and adjacent core stacking groups are connected through the on-chip network; wherein the core stacking group includes a first number of storage and computing cores, wherein the storage and computing cores include a computing unit, a scheduling control unit, and a DMA write-back unit on the computing cores, and a cache unit and a DMA loading unit on the cache cores, wherein the scheduling control unit controls the computing unit, the DMA write-back unit, the cache unit, and the DMA loading unit to collaboratively perform data computing and caching. The present application can significantly increase the communication bandwidth between the on-chip cache and the computing unit, and significantly expand the on-chip cache capacity.

Description

Technical Field

[0001] The present application relates to the field of semiconductor technology, and in particular to an artificial intelligence accelerator and a design method. Background Art

[0002] In recent years, the amount of computation required for artificial intelligence models in areas such as computer vision, natural language processing, and speech has been growing rapidly. This trend has driven artificial intelligence (AI) accelerator hardware to focus on improving peak computing power. AI accelerators are a general term for processors that can run artificial intelligence model services, such as NVIDIA graphics processing units (GPUs) and Google tensor processing units (TPUs). Generally, they consist of several major components: tensor computing, miscellaneous computing, on-chip storage hierarchies and data paths, external storage and communication interfaces, etc.

[0003] The size of the recently popular Transformer-type models increases by an average of 240 times every two years, however, the on-chip memory of AI accelerator hardware is only increasing at a rate of 2 times per year. For emerging multi-modal large models and other applications, the hardware design that focused on peak computing power in the past has become stretched, and its storage capacity and bandwidth bottlenecks have become key issues limiting performance. The capacity and bandwidth challenges brought by large model businesses (also known as the memory wall problem) are essentially the problem that the storage capacity and transmission bandwidth at multiple levels lag far behind the hardware computing power.

[0004] With the vigorous development of 2.5D and 3D advanced packaging technologies, technical support is provided for achieving higher density in-package routing, which makes more design exploration possibilities possible in chip architecture / package co-design, and brings new opportunities to alleviate the memory wall problem of AI accelerators. Therefore, based on the support of advanced packaging technology, how to solve the performance bottleneck caused by the memory wall of large model business on planar AI accelerators has become a key issue. Summary of the invention

[0005] In view of the above problems, the embodiments of the present application provide an artificial intelligence accelerator and a design method that overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present application provides an artificial intelligence accelerator, comprising N computing core particles and M cache core particles forming at least one core particle stacking group, wherein the N computing core particles and the M cache core particles are arranged in a vertical direction, each core particle stacking group comprises a computing core particle and at least one cache core particle, the cache core particles are connected to the computing core particles through vertical through holes, an on-chip network is deployed on the computing core particles, adjacent core particle stacking groups are connected through the on-chip network of the computing core particles, and the values ​​of N and M are both greater than or equal to 1;

[0007] The core grain stacking group includes a first number of storage and computing cores, the storage and computing cores include a computing unit, a scheduling control unit and a direct memory access DMA write-back unit deployed on the computing core grain, and a cache unit and a DMA loading unit deployed on the cache core grain, the scheduling control unit is connected to the computing unit, the DMA write-back unit and the DMA loading unit, the DMA loading unit is connected to the cache unit, and the computing unit is connected to the cache unit and the DMA write-back unit;

[0008] The scheduling control unit is used to control the calculation unit, the DMA write-back unit, the cache unit and the DMA loading unit to collaboratively perform data calculation and cache.

[0009] In a second aspect, an embodiment of the present application provides an artificial intelligence accelerator design method, including:

[0010] Provide multiple target core particles and an input / output (IO) interface, wherein the multiple target core particles include N computing core particles and M cache core particles, the target core particles are provided with vertical through holes, the N computing core particles include a target computing core particle, the IO interface is deployed on the target computing core particle or the IO interface is connected to the target computing core particle, and the values ​​of N and M are both greater than or equal to 1;

[0011] The multiple target core particles are stacked vertically to form at least one core particle stacking group, each core particle stacking group includes a computing core particle and at least one cache core particle, the cache core particle is connected to the computing core particle through a vertical through hole, an on-chip network is deployed on the computing core particle, adjacent core particle stacking groups are connected through the on-chip network of the computing core particles, and at least one core particle stacking group shares the IO interface.

[0012] The technical solution of the embodiment of the present application decouples the computing unit and the cache unit of the AI ​​accelerator, designs computing cores and cache cores separately, and constructs a 3D high-bandwidth AI accelerator in the form of 3D core stacking. The on-chip network interconnection structure is expanded in the vertical dimension, and multiple cache cores are expanded to establish a globally accessible ultra-large on-chip cache pool. Multiple storage and computing cores are expanded in the planar dimension to build a large-scale AI computing power. Vertical through holes are set so that the cache cores provide ultra-high input and output bandwidth for the computing cores. Compared with traditional planar AI accelerators, the communication bandwidth between the on-chip cache and the computing unit can be significantly increased, and the on-chip cache capacity can be significantly expanded, thereby effectively solving the performance bottleneck problem caused by the memory wall of multi-modal large model services on planar AI accelerators. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram showing a stack of computing core particles and cache core particles provided in an embodiment of the present application;

[0014] Figure 2 A schematic diagram showing a storage and computing core provided in an embodiment of the present application;

[0015] Figure 3 A plan projection diagram of a 3D stacked structure provided in an embodiment of the present application is shown;

[0016] Figure 4 A schematic diagram showing the connection of various units in the storage and computing core provided in an embodiment of the present application;

[0017] Figure 5 A schematic diagram showing the connection between a matrix multiplication unit and a cache sub-unit provided in an embodiment of the present application;

[0018] Fig. 6A A schematic diagram showing the basic structure of the matrix multiplication unit calculation array;

[0019] Figure 6B A schematic diagram showing a matrix multiplication array corresponding to a matrix multiplication unit;

[0020] Figure 7 A schematic diagram showing a computing core deployed with an IO interface and multiple computing units according to an embodiment of the present application;

[0021] Figure 8 A schematic diagram showing a cache core provided by an embodiment of the present application deploying multiple cache units and a DMA loading unit;

[0022] Fig. 9 A schematic diagram of an artificial intelligence accelerator design method provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. The multiple in the embodiments of the present application can include two and more than two.

[0025] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0026] The present application embodiment provides an artificial intelligence accelerator, such as Figure 1 , Figure 2 and Figure 3 As shown, the artificial intelligence accelerator includes N computing core particles 11 and M cache core particles 12 forming at least one core particle stacking group 10, the N computing core particles 11 and the M cache core particles 12 are arranged in a vertical direction, each core particle stacking group 10 includes a computing core particle 11 and at least one cache core particle 12, the cache core particle 12 is connected to the computing core particle 11 through a vertical through hole, an on-chip network is deployed on the computing core particle 11, and adjacent core particle stacking groups 10 are connected through the on-chip network of the computing core particle 11, and the values ​​of N and M are both greater than or equal to 1;

[0027] The core stack group 10 includes a first number of storage and computing cores 13, the storage and computing core 13 includes a computing unit 131, a scheduling control unit 132 and a direct memory access DMA write-back unit 133 deployed on the computing core 11, and a cache unit 134 and a DMA loading unit 135 deployed on the cache core 12, the scheduling control unit 132 is connected to the computing unit 131, the DMA write-back unit 133 and the DMA loading unit 135, the DMA loading unit 135 is connected to the cache unit 134, and the computing unit 131 is connected to the cache unit 134 and the DMA write-back unit 133;

[0028] The scheduling control unit 132 is used to control the calculation unit 131 , the DMA write-back unit 133 , the cache unit 134 and the DMA loading unit 135 to collaboratively perform data calculation and cache.

[0029] The artificial intelligence accelerator provided in an embodiment of the present application includes N computing core particles 11 and M cache core particles 12 stacked in a vertical direction. The N computing core particles 11 and the M cache core particles 12 constitute at least one core particle stacking group 10, and each core particle stacking group 10 includes a computing core particle 11 and at least one cache core particle 12. Figure 1 The figure is a schematic diagram of a chip stacking group 10 composed of one computing chip 11 and two cache chips 12. The computing chip 11 is located at the bottom layer, and the two cache chips 12 are stacked in sequence above the computing chip 11. Figure 1 The arrows shown in the figure indicate that the cache core 12 is connected to the computing core 11. When N computing cores 11 and M cache cores 12 form at least two core stacking groups 10, the at least two core stacking groups 10 are arranged in sequence in the vertical direction to form a stacked form.

[0030] The computing core 11 and the cache core 12 are both provided with vertical through holes, and the cache core 12 is connected to the computing core 11 through the vertical through holes, so that the cores stacked in space are connected based on the vertical through holes. The vertical through holes can be through holes set based on through-silicon via technology (TSV). TSV technology has been gradually widely used in advanced packaging. It realizes vertical interconnection of chips by making vertical conduction between wafers. Compared with traditional packaging, it has high stacking density, small size, short communication distance, and can greatly reduce communication delay and power consumption.

[0031] Each computing core 11 is equipped with an on-chip network. Figure 1 and Figure 3 The small boxes in the figure can be regarded as nodes of the on-chip network. Since each chip stacking group 10 includes a computing chip 11, adjacent chip stacking groups 10 can be connected through the on-chip networks on the two computing chips 11 located in the two chip stacking groups 10. That is, in the same chip stacking group 10, at least one cache chip 12 is connected to the computing chip 11. For the cross-group situation, adjacent chip stacking groups 10 are connected through the on-chip networks deployed on the computing chips 11.

[0032] For any core stack group 10, it includes a first number of storage and computing cores 13, and the storage and computing core 13 includes a computing unit 131, a scheduling control unit 132, and a direct memory access (DMA) write-back unit 133 deployed on the computing core 11, and a cache unit 134 and a DMA loading unit 135 deployed on the cache core 12. Among them, the storage and computing core 13 has a computing function and a cache function, and can be regarded as a unit that integrates the functions of the computing core 11 and the cache core 12.

[0033] The scheduling control unit 132 is connected to the computing unit 131, the DMA write-back unit 133 and the DMA loading unit 135. The DMA loading unit 135 is connected to the cache unit 134. The computing unit 131 is connected to the DMA write-back unit 133 and the cache unit 134. The DMA loading unit 135 is used to load data into the cache unit 134. The computing unit 131 is used to obtain the data in the cache unit 134 for calculation and output the calculation result. The DMA write-back unit 133 is used to obtain the calculation result provided by the computing unit 131 and write the calculation result back to the on-chip network. The calculation result output by the computing unit 131 can also be provided to the cache unit 134 as input data of the cache unit 134. The scheduling control unit 132 is used to control the computing unit 131, the DMA write-back unit 133, the cache unit 134 and the DMA loading unit 135 to coordinate data calculation and caching, so that the core particles stacked in space cooperate with each other to complete data caching and calculation.

[0034] The above implementation scheme of the present application decouples the computing unit and the cache unit of the AI ​​accelerator, designs computing cores and cache cores respectively, and constructs a 3D high-bandwidth AI accelerator in the form of 3D core stacking. In the vertical dimension, the on-chip network interconnection structure is expanded, and multiple cache cores are expanded to establish a globally accessible ultra-large on-chip cache pool. In the planar dimension, multiple storage and computing cores are expanded to build large-scale AI computing power. Vertical through holes are set so that the cache cores provide ultra-high input and output bandwidth for the computing cores. Compared with traditional planar AI accelerators, the communication bandwidth between the on-chip cache and the computing unit can be significantly increased, and the on-chip cache capacity can be significantly expanded, thereby effectively solving the performance bottleneck problem caused by the memory wall of multi-modal large model services on planar AI accelerators.

[0035] The following describes the process of the scheduling control unit controlling the computing unit, the cache unit, and the DMA loading unit to work together. Figure 1 and Figure 2 As shown, the DMA loading unit 135 is connected to the scheduling control unit 132 and the on-chip network on the computing core 11 through a vertical through hole, and the computing unit 131 is connected to the cache unit 134 through a vertical through hole;

[0036] The DMA loading unit 135 loads the data provided by the on-chip network into the cache unit 134 through the vertical through hole, and sends a first notification message to the scheduling control unit 132 through the vertical through hole after the loading is completed; the scheduling control unit 132 is used to control the calculation unit 131 to obtain the data in the cache unit 134 through the vertical through hole, and perform calculations based on the obtained data to output the calculation results.

[0037] The DMA loading unit 135 arranged on the cache core 12 is connected to the scheduling control unit 132 and the on-chip network arranged on the computing core 11 through vertical through holes. Specifically, vertical through holes are set on both the cache core 12 and the computing core 11. The DMA loading unit 135 can be directly connected to the vertical through holes on the cache core 12, or it can be connected to the vertical through holes on the cache core 12 through the connection lines arranged on the end surface of the cache core 12. The vertical through holes on the cache core 12 are connected to the vertical through holes on the computing core 11 (such as aligning two vertical through holes to achieve direct connection), and the scheduling control unit 132 and the on-chip network on the computing core 11 are both connected to the vertical through holes on the computing core 11. At this point, the DMA loading unit 135 on the cache core 12 can be connected to the scheduling control unit 132 and the on-chip network through the vertical through holes on the cache core 12 and the vertical through holes on the computing core 11.

[0038] The DMA loading unit 135 obtains data provided by the on-chip network through the vertical through hole, and loads the obtained data into the matching cache unit 134, so as to store the data provided by the on-chip network through the cache unit 134. Since the cache unit 134 and the DMA loading unit 135 are both arranged on the cache core 12, the cache unit 134 and the DMA loading unit 135 are connected through a planar wiring.

[0039] After the DMA loading unit 135 obtains the data provided by the on-chip network through the vertical through hole and loads the data into the matching cache unit 134, it sends a first notification message to the scheduling control unit 132 through the vertical through hole, and the first notification message is used to indicate that the data obtained from the on-chip network has been successfully loaded into the cache unit 134. After the scheduling control unit 132 determines that the data has been successfully loaded into the cache unit 134 based on the first notification message, it controls the calculation unit 131 to interact with the cache unit 134 through the vertical through hole, obtain the data in the cache unit 134, and calculate and output the calculation result according to the data in the cache unit 134. The form in which the calculation unit 131 is connected to the cache unit 134 through the vertical through hole is similar to the form in which the DMA loading unit 135 is connected to the scheduling control unit 132 and the on-chip network through the vertical through hole, and will not be elaborated here.

[0040] Among them, the scheduling control unit 132 is connected to the computing unit 131 through planar wiring, and sends a second notification message to the computing unit 131; the computing unit 131 is used to send an acquisition request to the cache unit 134 through a vertical through hole according to the second notification message to obtain the data provided by the cache unit 134.

[0041] Since the scheduling control unit 132 and the computing unit 131 are both deployed on the computing core 11, the two are connected through planar wiring. After obtaining the first notification information sent by the DMA loading unit 135 through the vertical through hole, the scheduling control unit 132 sends a second notification information to the computing unit 131, and the second notification information is used to instruct to obtain data and start calculation. According to the second notification information, the computing unit 131 actively initiates an acquisition request to the cache unit 134 through the vertical through hole. The cache unit 134 responds to the acquisition request and provides the cached data to the computing unit 131 through the vertical through hole. After the computing unit 131 obtains the data provided by the cache unit 134, it performs calculation and obtains the calculation result.

[0042] In the above implementation scheme of the present application, the DMA loading unit arranged on the cache core obtains the data provided by the on-chip network of the computing core through the vertical through hole, stores the obtained data in the cache unit and notifies the scheduling control unit, the scheduling control unit notifies the computing unit to obtain the data and start the calculation, and the computing unit performs calculation after obtaining the data in the cache unit, thereby realizing the coordinated work of each unit under the scheduling of the scheduling control unit to cache and calculate the data.

[0043] The working process of the calculation unit is described in detail below. Figure 4 and Figure 5 As shown, the computing unit 131 includes a matrix multiplication unit 1311 and a miscellaneous computing unit 1312, and the cache unit 134 includes a first cache subunit 1341, a second cache subunit 1342 and a third cache subunit 1343;

[0044] The input end of the matrix multiplication unit 1311 is connected to the output ends of the first cache sub-unit 1341 and the second cache sub-unit 1342 through a vertical through hole, the output end of the matrix multiplication unit 1311 is connected to the input end of the third cache sub-unit 1343 through a vertical through hole, and the output end of the third cache sub-unit 1343 is connected to the input end of the miscellaneous calculation unit 1312 through a vertical through hole; the output end of the miscellaneous calculation unit 1312 is connected to the input ends of the first cache sub-unit 1341 and the second cache sub-unit 1342 through a vertical through hole, and is connected to the DMA write back unit 133 through planar wiring.

[0045] like Figure 1 , Figure 4 and Figure 5As shown, the computing unit 131 arranged on the computing core particle 11 includes a matrix multiplication unit 1311 and a miscellaneous computing unit 1312. The matrix multiplication unit 1311 is responsible for matrix multiplication operations, and the miscellaneous computing unit 1312 is responsible for non-matrix multiplication operations, such as vector, scalar, data transformation, etc. The cache unit 134 arranged on the cache core particle 12 includes a first cache subunit 1341, a second cache subunit 1342 and a third cache subunit 1343. The first cache subunit 1341 and the second cache subunit 1342 are input matrix caches of the matrix multiplication unit 1311, and the third cache subunit 1343 is a calculation result cache of the matrix multiplication unit 1311. At the same time, the third cache subunit 1343 also serves as an input cache of the miscellaneous computing unit 1312.

[0046] The input end of the matrix multiplication unit 1311 is connected to the output ends of the first cache sub-unit 1341 and the second cache sub-unit 1342. The matrix multiplication unit 1311 is used to obtain data in the first cache sub-unit 1341 and the second cache sub-unit 1342 to perform matrix multiplication operations. Since the matrix multiplication unit 1311 is deployed on the computing core 11 and the first cache sub-unit 1341 and the second cache sub-unit 1342 are deployed on the cache core 12, the matrix multiplication unit 1311 is connected to the first cache sub-unit 1341 and the second cache sub-unit 1342 through a vertical through hole.

[0047] The output end of the matrix multiplication unit 1311 is connected to the input end of the third cache sub-unit 1343 through a vertical through hole. After the matrix multiplication unit 1311 completes the matrix calculation based on the data provided by the first cache sub-unit 1341 and the second cache sub-unit 1342, the calculation result is transmitted to the third cache sub-unit 1343 through the vertical through hole to store the calculation result in the third cache sub-unit 1343.

[0048] The output end of the third cache sub-unit 1343 is connected to the input end of the miscellaneous calculation unit 1312 through a vertical through hole, and is used to provide the calculation result of the matrix multiplication unit 1311 to the miscellaneous calculation unit 1312, and the miscellaneous calculation unit 1312 performs calculation based on the acquired data under the scheduling of the scheduling control unit 132. That is, after the matrix multiplication unit 1311 completes the calculation, the scheduling control unit 132 notifies the miscellaneous calculation unit 1312 to start the calculation, and the miscellaneous calculation unit 1312 obtains the calculation result of the matrix multiplication unit 1311 stored in the third cache sub-unit 1343, and performs calculation according to the acquired data.

[0049] The output end of the miscellaneous calculation unit 1312 is connected to the input end of the first cache subunit 1341 and the second cache subunit 1342 through a vertical through hole, and is connected to the DMA write-back unit 133 through a plane wiring. After the miscellaneous calculation unit 1312 completes the calculation and obtains the output result, it transmits the output result to the first cache subunit 1341 and the second cache subunit 1342 through a vertical through hole as the input of the next matrix multiplication unit 1311 calculation; or, the output result is sent to the DMA write-back unit 133, and the DMA write-back unit 133 writes the output result back to the on-chip network. Whether the output result is provided to the first cache subunit 1341 and the second cache subunit 1342, or the output result is returned to the on-chip network, is determined by the calculation requirements. The scheduling control unit 132 performs scheduling after understanding the computing requirements. When the output result of the miscellaneous computing unit 1312 needs to continue matrix calculation, the scheduling control unit 132 schedules the output result to the first cache subunit 1341 and the second cache subunit 1342. If the output result of the miscellaneous computing unit 1312 does not need to continue matrix calculation, the scheduling control unit 132 controls the DMA write-back unit 133 to write the output result back to the on-chip network. Figure 4 In order to distinguish between planar data transmission and 3D data transmission, lines of different thicknesses are used to distinguish. The thin line path indicates that it is ordinary planar data transmission, and the thick line path indicates that it is high-bandwidth 3D vertical data transmission. Since only control signals are transmitted between the DMA loading unit 135 and the scheduling control unit 132, Figure 4 The connection between the two is represented by a thin line.

[0050] The matrix multiplication unit 1311 is used to perform matrix calculations. Taking [3, 3] x [3, 3] matrix multiplication as an example, the calculation form is as follows:

[0051]

[0052] Among them, c i,j =∑a i,k b k,j The basic structure of the matrix multiplication unit calculation array is as follows Fig. 6A As shown in Figure 1, a group of multiplication and addition units can be cascaded to complete the product and sum calculation of a group of numbers. Based on this structural expansion, a complete [3,3]x[3,3] matrix multiplication array can be obtained as follows: Figure 6BAs shown. Input the complete data of matrix A and matrix B to get the output result, which is the complete data of matrix C. In practice, the matrix multiplication unit calculation array can select a fixed size (for example, [256, 256] x [256, 256]), and matrix inputs of different shapes can be filled or blocked to complete the calculation. In this embodiment, the high-bandwidth input (matrix A, matrix B) and output (matrix C) required by the calculation unit are realized by 3D cache core particles through vertical through-hole interconnection. Compared with the traditional planar 2D structure, the bandwidth and capacity can be improved by one to two orders of magnitude.

[0053] In the above implementation scheme of the present application, a matrix multiplication unit is connected to two cache sub-units, and a matrix calculation is performed based on the data provided by the two cache sub-units to obtain a calculation result. After the calculation result is provided to the miscellaneous computing unit, the miscellaneous computing unit performs calculations based on the acquired data and provides the output result to the two cache sub-units for use in the next matrix operation or for writing back to the on-chip network, thereby realizing data calculation and caching by using the collaborative work of the computing unit and the cache unit.

[0054] In an optional embodiment of the present application, if Figures 1 to 5 As shown, the number of cache units corresponding to the storage and computing core 13 is the number of cache cores 12 corresponding to a single core stack group 10; when the core stack group 10 includes at least two cache cores 12, the computing unit 131 obtains data provided by at least two cache units 134 for calculation.

[0055] For any storage computing core 13, it includes a computing unit 131, a scheduling control unit 132 and a DMA write-back unit 133, and includes at least one cache unit 134 and at least one DMA loading unit 135. The number of cache units 134 included in the storage computing core 13 is adapted to the number of cache cores 12 included in a single core stack group 10. For example, if the core stack group 10 includes one cache core 12, the storage computing core 13 corresponding to the core stack group 10 includes one cache unit 134; if the core stack group 10 includes at least two cache cores 12, the storage computing core 13 corresponding to the core stack group 10 includes at least two cache units 134.

[0056] For different chip stacking groups 10, the number of cache chips 12 included therein can be the same or different. For example, adjacent chip stacking groups 10 each include one computing chip 11 and two cache chips 12; or, one chip stacking group 10 includes one computing chip 11 and two cache chips 12, and another chip stacking group 10 includes one computing chip 11 and one cache chip 12.

[0057] When the chip stacking group 10 includes at least two cache chips 12, the storage and computing core 13 includes at least two cache units 134, and the computing unit 131 is respectively connected to the at least two cache units 134, obtains data respectively provided by the at least two cache units 134, and performs a computing process according to the obtained data.

[0058] In the case where the core stack group 10 includes at least two cache cores 12 and the storage and computing core 13 includes at least two cache units 134, the computing unit 131 can read data or select data for calculation as a whole for at least two cache units 134. For example, the matrix multiplication unit 1311 performs a computing task for at least two cache units 134, and the computing task is: matrix A1*matrix B1=matrix C1, matrix A2*matrix B2=matrix C2, matrix A3*matrix B3=matrix C3; or, the computing task is: matrix A*matrix B=matrix C, where matrix A and matrix B are large matrices determined after fusion.

[0059] In the above implementation process, the number of cache units corresponding to the storage and computing core is determined based on the number of cache cores corresponding to the core stacking group. When the storage and computing core corresponds to at least two cache units, calculations are performed on the two cache units to obtain output results, thereby realizing calculations based on data provided by one or more adapted cache units.

[0060] In another embodiment of the present application, Figure 1 , Figure 2 , Figure 3 , Figure 7 and Figure 8 As shown, the first number is the total number of computing units 131 corresponding to a single computing core 11 and the total number of cache units 134 corresponding to a single cache core 12. Each computing unit 131 matches a scheduling control unit 132 and a DMA write back unit 133, and each cache unit 134 matches a DMA loading unit 135.

[0061] A single computing core 11 corresponds to a first number of computing units 131, a single cache core 12 corresponds to a first number of cache units 134, and a core stacking group 10 corresponds to a computing core 11 and at least one cache core 12. For a core stacking group 10, it includes a first number of storage and computing cores 13, and for each storage and computing core 13, it includes a computing unit 131 and at least one cache unit 134. Each computing unit 131 matches a scheduling control unit 132 and a DMA write-back unit 133, and each cache unit 134 matches a DMA load unit 135. For different core stacking groups 10, the corresponding first number values ​​can be the same or different. For example, the first number value corresponding to one core stacking group 10 is 16, and the first number value corresponding to another core stacking group 10 is 20.

[0062] Figure 7 as well as Figure 8 The diagram shows a situation where one computing core 11 corresponds to 16 computing units 131, one cache core 12 corresponds to 16 cache units 134, and each cache unit 134 matches a DMA loading unit 135, and the scheduling control unit and DMA write-back unit corresponding to the computing unit 131 are not shown in the figure.

[0063] Among them, Figure 1 As shown, for a chip stacking group 10, if it corresponds to at least two cache chips 12, the through-hole settings of the vertical through-holes corresponding to at least two cache chips 12 are the same, and the through-hole settings of the cache chips 12 and the computing chips 11 are the same; different chip stacking groups 10, the through-hole settings of the computing chips 11 can be exactly the same, or they can be different. For the different situations, it is necessary to ensure that the two computing chips 11 in the adjacent chip stacking groups 10 have at least one matching vertical through-hole, so that the two computing chips 11 are connected through the matching vertical through-holes. And because the through-hole settings of the computing chips 11 and the cache chips 12 in the same chip stacking group 10 are the same, for two adjacent chip stacking groups 10, the two computing chips 11 can be connected directly through their respective vertical through-holes, or they can be connected through their respective vertical through-holes and the vertical through-holes on the cache chips 12 between them.

[0064] The two computing core particles 11 in the adjacent core particle stacking group 10 are the first computing core particle and the second computing core particle, and the first computing core particle and the second computing core particle are connected through vertical through holes of their respective on-chip networks and vertical through holes on the cache core particle 12 arranged between the first computing core particle and the second computing core particle.

[0065] The first computing core and the second computing core in the adjacent core stacking group 10 can be directly adjacent or indirectly adjacent when stacked. In the case of direct proximity, there is no cache core 12 between the first computing core and the second computing core, and the first computing core and the second computing core are connected through the vertical through holes of their respective on-chip networks; in the case of indirect proximity, there is a cache core 12 between the first computing core and the second computing core, and the first computing core and the second computing core are connected through the vertical through holes of their respective on-chip networks and the vertical through holes on the cache core 12 arranged between the two.

[0066] In the above implementation scheme of the present application, the number of storage and computing cores corresponding to the chip stacking group is the number of computing units corresponding to a single computing chip and the number of cache units corresponding to a single cache chip. The storage and computing core includes a computing unit and at least one cache unit to realize the computing and caching functions. Multiple storage and computing cores can cooperate to build large-scale AI computing power. Adjacent computing chips are connected through vertical through holes, thereby realizing the connection between adjacent chip stacking groups, ensuring that multiple chip stacking groups can be connected when stacked.

[0067] In another optional embodiment of the present application, Figure 1 , Figure 3 and Figure 7 As shown, the artificial intelligence accelerator further includes: an input / output IO interface 14, wherein the IO interface 14 is an IO interface unit or an IO core particle, and at least one core particle stacking group 10 shares the IO interface 14;

[0068] In the case where the IO interface 14 is an IO core particle, the IO core particle is connected to the target computing core particle in a vertical direction through a vertical through hole, or the IO core particle is connected to the target computing core particle in a plane;

[0069] In the case where the IO interface 14 is an IO interface unit, the IO interface unit is deployed on the target computing core particle;

[0070] The target computing core is located in a first core stacking group, the target computing core is one of the N computing cores 11 , and the first core stacking group is one of at least one core stacking group 10 .

[0071] The first storage and computing core deployed in the first chip stacking group is connected to the IO interface 14 based on the on-chip network corresponding to the target computing chip, and communicates with the peripheral device through the IO interface 14; the second storage and computing core deployed in the second chip stacking group is connected to the on-chip network of the target computing chip based on the connection between the second chip stacking group and the first chip stacking group, is connected to the IO interface 14 based on the on-chip network of the target computing chip, and communicates with the peripheral device through the IO interface 14. The second chip stacking group is a chip stacking group different from the first chip stacking group.

[0072] The artificial intelligence accelerator also includes: (InputOutput, input and output) IO interface 14, IO interface 14 includes external storage interface, peripheral interface, chip interconnection interface and processor interface, etc. IO interface 14 can be in the form of an interface unit or an IO chip. When the IO interface 14 is in the form of an interface unit, the IO interface unit is directly deployed on the target computing chip; when the IO interface 14 is in the form of an IO chip, the IO chip is connected to the target computing chip, and the two can be planarly connected or spatially connected. For the case of planar connection, the connection between the IO chip and the target computing chip can be realized based on the packaging substrate silicon wafer interconnection. For the spatial connection, it can be connected in the vertical direction based on vertical through holes. Figure 3 as well as Figure 7 The diagram shows that the IO interface unit is arranged on the computing core particle.

[0073] The target computing core is one of the N computing cores 11, such as the bottom-most computing core 11. At least one core stacking group 10 shares an IO interface 14, so that the storage and computing cores 13 in different core stacking groups 10 communicate with peripheral devices based on the same IO interface 14.

[0074] It should be noted that, considering that the frontier advanced processes such as 3nm, 2nm, etc. have extremely high transistor density, but the IO interface intellectual property (Intellectual Property, IP) ecosystem is missing, the embodiments of the present application do not need to rely on the IO interface IP ecosystem, and support the use of frontier advanced processes to produce cache core particles with ultra-high transistor density that only contain cache units and simple logic. There is no need to lay out interfaces on the cache core particles, and mature advanced processes are used to produce computing or IO core particles and connect them to cache core particles, thereby achieving the goal of balancing performance and cost. It should be noted that IP refers to the design of circuit modules with independent functions in the chip, which is generally provided by professional functional IP suppliers. The so-called IP ecosystem of the process node refers to whether the IP options available under a certain process are rich.

[0075] The target computing core is located in the first core stacking group, which is one of at least one core stacking group 10, and the other core stacking groups different from the first core stacking group are all second core stacking groups. The storage and computing core 13 deployed in the first core stacking group is the first storage and computing core. Since the target computing core is located in the first core stacking group, the first storage and computing core can be directly connected to the IO interface 14 based on the on-chip network corresponding to the target computing core, and communicate with the peripheral device through the IO interface 14, so as to realize the interaction between the first storage and computing core and the peripheral device.

[0076] The storage and computing core 13 deployed in the second chip stacking group is the second storage and computing core. Since the adjacent chip stacking groups 10 can be connected based on the computing chip 11, for the second chip stacking group where the target computing chip does not exist, the second storage and computing core located in the second chip stacking group can be connected to the first chip stacking group based on the interconnection between the chip stacking groups 10, specifically connected to the on-chip network of the target computing chip in the first chip stacking group, connected to the IO interface 14 based on the on-chip network of the target computing chip, and communicate with peripheral devices through the IO interface 14.

[0077] In the above implementation scheme of the present application, at least one chip stacking group shares an IO interface, which can achieve the purpose of cost saving; by deploying an IO interface unit on a computing chip or setting an IO chip connected to the computing chip, based on the connection between the cache chip and the computing chip, and the connection between adjacent chip stacking groups, it is possible to achieve the purpose of balancing performance and cost while ensuring that each storage and computing core is connected to the IO interface.

[0078] The present application also provides an artificial intelligence accelerator design method, such as Fig. 9 As shown, including:

[0079] Step 901: provide multiple target core particles and an input / output (IO) interface, wherein the multiple target core particles include N computing core particles and M cache core particles, the target core particles are provided with vertical through holes, the N computing core particles include a target computing core particle, the IO interface is deployed on the target computing core particle or the IO interface is connected to the target computing core particle, and the values ​​of N and M are both greater than or equal to 1.

[0080] When designing an artificial intelligence accelerator, first provide multiple target cores and an IO interface. The multiple target cores include N computing cores and M cache cores, and each computing core is equipped with at least one cache core. Vertical through holes are deployed on the computing cores and the cache cores, and the cache cores are connected to the computing cores through the vertical through holes. The N computing cores include a target computing core, and the number of target computing cores is one. The IO interface is deployed on the target computing core or the target computing core is connected to the IO interface. The IO interface can be an IO interface unit deployed on the target computing core, or an IO core connected to the target computing core. N computing cores and M cache cores share one IO interface. The IO interface can include an external storage interface, a peripheral interface, a core interconnection interface, and a processor interface.

[0081] Step 902: vertically stack the multiple target cores to form at least one core stacking group, each core stacking group includes a computing core and at least one cache core, the cache core is connected to the computing core via a vertical through hole, an on-chip network is deployed on the computing core, adjacent core stacking groups are connected via the on-chip network of the computing core, and at least one core stacking group shares the IO interface.

[0082] After acquiring multiple target core particles, the multiple target core particles are stacked in the vertical direction to form at least one core particle stacking group, each core particle stacking group includes a computing core particle and at least one cache core particle, and the number of cache core particles corresponding to different core particle stacking groups may be the same or different. Since vertical through holes are provided on both the cache core particles and the computing core particles, the cache core particles can be connected to the computing core particles through the vertical through holes, and the cache core particles and computing core particles belonging to different core particle stacking groups and directly adjacent in space can also be connected through the vertical through holes. The vertical through holes can be through holes set based on through silicon via technology, or can be through holes set by other means.

[0083] An on-chip network is deployed on each computing core particle. Since each core particle stacking group includes a computing core particle, adjacent core particle stacking groups can be connected through the on-chip networks on the two computing core particles in the two core particle stacking groups. That is, in the same core particle stacking group, at least one cache core particle is connected to the computing core particle. For the cross-group situation, adjacent core particle stacking groups are connected through the on-chip network deployed on the computing core particles. Since only one target computing core particle is included in the N computing core particles, the IO interface is deployed on the target computing core particle or the IO interface is connected to the target computing core particle. After the core particles are stacked, the IO interface remains deployed on the target computing core particle or connected to the target computing core particle. Therefore, at least one core particle stacking group can share the IO interface, thereby achieving the purpose of cost saving.

[0084] Optionally, when multiple target core particles and an IO interface are provided, the method includes:

[0085] Providing N initial computing core particles and M initial cache core particles, wherein the initial computing core particles include a first number of computing units, a scheduling control unit, and a direct memory access (DMA) write-back unit, and the initial cache core particles include a first number of cache units and a DMA loading unit;

[0086] According to the vertical through-hole arrangement rule, vertical through-holes are arranged on the initial computing core particles and the cache core particles to form N computing core particles and M cache core particles;

[0087] Arrange an IO interface unit on a target computing core particle among the N computing core particles or set an IO core particle connected to the target computing core particle;

[0088] Among them, the IO interface is the IO interface unit or the IO core particle; the target computing core particle communicates with the peripheral device through the IO interface, and other computing core particles different from the target computing core particle are connected to the target computing core particle and communicate with the peripheral device through the IO interface.

[0089] The target core particles in the embodiment of the present application are core particles with vertical through holes. When providing the target core particles, N initial computing core particles and M initial cache core particles can be provided first. For the initial computing core particles, they include a first number of computing units, a scheduling control unit, and a DMA write-back unit. Each unit is connected by a planar wiring, and the DMA write-back unit is connected to the on-chip network for writing data back to the on-chip network. For the initial cache core particles, they include a first number of cache units and a DMA loading unit. The cache unit and the DMA loading unit are connected based on planar wiring.

[0090] For N initial computing core particles and M initial cache core particles, vertical through holes are set on the initial computing core particles and the cache core particles according to the vertical through hole layout rules to form N computing core particles and M cache core particles, and then an IO interface is set for the target computing core particles selected from the N computing core particles. The IO interface can be an IO interface unit arranged on the target computing core particle or an IO core particle connected to the target computing core particle. In the case where the IO interface is an interface unit, after the target computing core particle is selected, the IO interface unit is directly deployed on the core particle, and the IO interface unit is connected to the on-chip network of the target computing core particle. In the case where the IO interface is an IO core particle, after the target computing core particle is selected, an IO core particle connected to the target computing core particle is set, and the connection between the two can be a spatial connection based on a vertical through hole, or a planar connection based on a silicon wafer of a packaging substrate. By setting an IO interface unit on the target computing core particle or setting an IO core particle connected to the target computing core particle, an IO interface shared by at least one core particle stacking group is arranged.

[0091] Since the IO interface unit is arranged on the target computing chip or the target computing chip is connected to the IO chip, the target computing chip can directly communicate with the peripheral device through the IO interface; since the adjacent chip stacking groups are connected through the on-chip network of the computing chip, other computing chip particles different from the target computing chip can communicate with the peripheral device through the IO interface after connecting to the target computing chip.

[0092] The computing core particles and cache core particles belonging to the same core particle stacking group are provided with the same vertical through holes. For different core particle stacking groups, the through hole settings of the computing core particles can be exactly the same or different. For the different situations, it is necessary to ensure that the two computing core particles in the adjacent core particle stacking groups have at least one matching vertical through hole so that the two computing core particles can establish a connection through the matching vertical through holes. And because the through hole settings of the computing core particles and the cache core particles in the same core particle stacking group are the same, for two adjacent core particle stacking groups, the two computing core particles can be directly connected through their respective vertical through holes, or they can be connected through their respective vertical through holes and the vertical through holes on the cache core particles between them.

[0093] When adjacent chip stacking groups are connected through the on-chip network of computing chips, the adjacent chip stacking groups are connected through the vertical through holes connecting the on-chip network on the two computing chips and the vertical through holes on the cache chips between the two computing chips. That is, if the two computing chips in two adjacent chip stacking groups are in direct contact (there is no cache chip between the two computing chips), the two computing chips are connected through the vertical through holes connecting the on-chip network; if there is a cache chip between the two computing chips in two adjacent chip stacking groups, the two computing chips are connected through the vertical through holes connecting the on-chip network and the vertical through holes on the cache chips arranged between the two.

[0094] Optionally, vertically stacking the plurality of target core particles to form at least one core particle stacking group comprises: stacking the plurality of target core particles in a vertical direction in a preset stacking order to form at least one core particle stacking group based on a stacking rule that a computing core particle matches at least one cache core particle;

[0095] Among them, the core stacking group includes a first number of storage and computing cores, and the storage and computing core includes a computing unit, a scheduling control unit and a DMA write-back unit deployed on the computing core, and a cache unit and a DMA loading unit deployed on the cache core. The number of cache units corresponding to the storage and computing core is the number of cache cores corresponding to a single core stacking group.

[0096] When forming at least one chip stacking group by chip stacking, based on the stacking rule that one computing chip matches at least one cache chip, and each chip stacking group includes one computing chip and at least one cache chip, N computing chips and M cache chips are stacked in space to form at least one chip stacking group. After stacking, the vertical through holes on the cache chips and computing chips in the same chip stacking group are aligned to achieve the connection between the computing chip and the cache chip; for different chip stacking groups, if the through hole settings are exactly the same, the through holes can be fully aligned, and if the through hole settings are partially the same, the through holes can be partially aligned. That is, after the chips are stacked, the through holes that are matched in position can achieve through hole alignment.

[0097] Since the N computing core particles include the target computing core particle, the IO interface unit is deployed on the target computing core particle or the target computing core particle is connected to the IO core particles. After the core particles are stacked, the IO interface unit remains on the target computing core particle or the IO core particle remains connected to the target computing core particle. Therefore, the IO interface layout can be formed while the core particles are stacked.

[0098] For a chip stack group, it includes a first number of storage and computing cores, the storage and computing cores include computing units, scheduling control units and DMA write-back units deployed on the computing chip, and cache units and DMA loading units deployed on the cache chip. The number of cache units corresponding to the storage and computing core is the number of cache chip corresponding to the chip stack group.

[0099] The number of cache units included in the storage and computing core is compatible with the number of cache cores included in the core stacking group. For example, if the core stacking group includes one cache core, the storage and computing core corresponding to the core stacking group includes one cache unit; if the core stacking group includes at least two cache cores, the storage and computing core corresponding to the core stacking group includes at least two cache units.

[0100] The DMA loading unit on the cache core is connected to the scheduling control unit and the on-chip network on the computing core through vertical vias, the computing unit is connected to the cache unit through vertical vias, the scheduling control unit is connected to the DMA write-back unit and the computing unit through planar wiring, the computing unit is connected to the DMA write-back unit through planar wiring, and the cache unit and the DMA loading unit are connected through planar wiring. Although the computing unit is located on the same core as the on-chip network, they do not communicate directly.

[0101] For the interaction between the various units included in the storage and computing core, please refer to the description of the previous embodiment and will not be introduced in detail here.

[0102] By adopting the above-mentioned artificial intelligence accelerator design method to design an accelerator, the computing unit and cache unit of the accelerator can be decoupled, the computing core particles and cache core particles can be designed separately, and a 3D high-bandwidth accelerator with a 3D structure can be constructed in the form of 3D core particle stacking. The on-chip network interconnection structure can be expanded in the vertical dimension, and multiple cache core particles can be expanded to establish a globally accessible ultra-large on-chip cache pool. In the planar dimension, multiple storage and computing cores can be expanded to build large-scale AI computing power. Vertical through holes are set so that the cache core particles provide ultra-high input and output bandwidth for the computing core particles. Compared with traditional planar structure accelerators, the cache bandwidth and capacity are improved by one to two orders of magnitude, thereby effectively solving the performance bottleneck problem caused by memory walls in multi-modal large model businesses on planar AI accelerators.

[0103] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An artificial intelligence accelerator, characterized in that: The invention comprises N computing core particles and M cache core particles forming at least one core particle stacking group, wherein the N computing core particles and the M cache core particles are arranged in a vertical direction, each core particle stacking group comprises a computing core particle and at least one cache core particle, the cache core particles are connected to the computing core particles through vertical through holes, an on-chip network is deployed on the computing core particles, and adjacent core particle stacking groups are connected through the on-chip network of the computing core particles, and the values ​​of N and M are both greater than or equal to 1; The vertical through-hole is a through-hole set based on the through-silicon via technology, which realizes the vertical interconnection of chips by making vertical conduction between wafers; The core grain stacking group includes a first number of storage and computing cores, the storage and computing cores include a computing unit, a scheduling control unit and a direct memory access DMA write-back unit deployed on the computing core grain, and a cache unit and a DMA loading unit deployed on the cache core grain, the scheduling control unit is connected to the computing unit, the DMA write-back unit and the DMA loading unit, the DMA loading unit is connected to the cache unit, and the computing unit is connected to the cache unit and the DMA write-back unit; The scheduling control unit is used to control the calculation unit, the DMA write-back unit, the cache unit and the DMA loading unit to collaboratively perform data calculation and cache.

2. The artificial intelligence accelerator according to claim 1, characterized in that: The DMA loading unit is connected to the scheduling control unit and the on-chip network on the computing core grain through the vertical through hole, and the computing unit is connected to the cache unit through the vertical through hole; The DMA loading unit loads the data provided by the on-chip network into the cache unit through the vertical through hole, and sends first notification information to the scheduling control unit through the vertical through hole after the loading is completed; The scheduling control unit is used to control the calculation unit to obtain the data in the cache unit through the vertical through hole, and perform calculations according to the obtained data and output calculation results.

3. The artificial intelligence accelerator according to claim 2, characterized in that: The scheduling control unit is connected to the computing unit through planar wiring, and sends second notification information to the computing unit; The computing unit is used to send an acquisition request to the cache unit through the vertical through hole according to the second notification information, so as to acquire the data provided by the cache unit.

4. The artificial intelligence accelerator according to claim 2, characterized in that: The number of cache units corresponding to the storage and computing core is the number of cache cores corresponding to a single core stack group; In the case that the chip stack group includes at least two cache chips, the computing unit acquires data respectively provided by the at least two cache units for computing.

5. The artificial intelligence accelerator according to claim 2 or 4, characterized in that: The computing unit includes a matrix multiplication unit and a miscellaneous computing unit, and the cache unit includes a first cache subunit, a second cache subunit and a third cache subunit; The input end of the matrix multiplication unit is connected to the output ends of the first cache subunit and the second cache subunit through the vertical through hole, and the output end of the matrix multiplication unit is connected to the input end of the third cache subunit through the vertical through hole; The output end of the third cache subunit is connected to the input end of the miscellaneous computing unit through the vertical through hole; The output end of the miscellaneous calculation unit is connected to the input ends of the first cache subunit and the second cache subunit through the vertical through hole, and is connected to the DMA write-back unit through planar wiring.

6. The artificial intelligence accelerator according to claim 5, characterized in that: The output result of the miscellaneous calculation unit is transmitted to the first cache subunit and the second cache subunit through the vertical through hole, and the output result of the miscellaneous calculation unit is used as the input of the matrix multiplication unit; or The output result of the miscellaneous computing unit is transmitted to the DMA write-back unit, and the DMA write-back unit writes the output result of the miscellaneous computing unit back to the on-chip network.

7. The artificial intelligence accelerator according to claim 1, characterized in that: The first computing core particle is connected to the second computing core particle through vertical through holes of the respective on-chip networks and vertical through holes on the cache core particles arranged between the first computing core particle and the second computing core particle; The first computing core particle and the second computing core particle are two computing core particles in an adjacent core particle stacking group.

8. The artificial intelligence accelerator according to claim 7, characterized in that: It also includes: an input / output IO interface, wherein the IO interface is an IO interface unit or an IO core particle, and at least one core particle stacking group shares the IO interface; In the case where the IO interface is an IO core particle, the IO core particle is connected to the target computing core particle in a vertical direction through a vertical through hole, or the IO core particle is connected to the target computing core particle in a plane; In the case where the IO interface is an IO interface unit, the IO interface unit is deployed on the target computing core; The target computing core particle is located in a first core particle stacking group, the target computing core particle is one of the N computing core particles, and the first core particle stacking group is one of at least one core particle stacking group.

9. The artificial intelligence accelerator according to claim 8, characterized in that: The first storage computing core deployed in the first core grain stacking group is connected to the IO interface based on the on-chip network corresponding to the target computing core grain, and communicates with the peripheral device through the IO interface; The second storage and computing core deployed in the second chip stacking group is connected to the on-chip network of the target computing chip based on the connection between the second chip stacking group and the first chip stacking group, is connected to the IO interface based on the on-chip network of the target computing chip, and communicates with peripheral devices through the IO interface. The second chip stacking group is a chip stacking group different from the first chip stacking group.

10. The artificial intelligence accelerator according to claim 1 or 4, characterized in that: The first number is the total number of computing units corresponding to a single computing core particle and the total number of cache units corresponding to a single cache core particle; Each of the computing units matches one of the scheduling control units and one of the DMA write-back units, and each of the cache units matches one of the DMA loading units.

11. A method for designing an artificial intelligence accelerator, characterized in that: include: A plurality of target core particles and an input / output (IO) interface are provided, wherein the plurality of target core particles include N computing core particles and M cache core particles, the target core particles are provided with vertical through holes, the N computing core particles include a target computing core particle, the IO interface is deployed on the target computing core particle or the IO interface is connected to the target computing core particle, the values ​​of N and M are both greater than or equal to 1, wherein the IO interface is an IO interface unit or an IO core particle; The multiple target core particles are stacked vertically to form at least one core particle stacking group, each core particle stacking group includes a computing core particle and at least one cache core particle, the cache core particle is connected to the computing core particle through a vertical through hole, an on-chip network is deployed on the computing core particle, adjacent core particle stacking groups are connected through the on-chip network of the computing core particles, and at least one core particle stacking group shares the IO interface.

12. The method according to claim 11, characterized in that The provision of multiple target core particles and an input / output IO interface includes: Providing N initial computing core particles and M initial cache core particles, wherein the initial computing core particles include a first number of computing units, a scheduling control unit, and a direct memory access (DMA) write-back unit, and the initial cache core particles include a first number of cache units and a DMA loading unit; According to the vertical through-hole arrangement rule, vertical through-holes are arranged on the initial computing core particles and the cache core particles to form N computing core particles and M cache core particles; Arrange an IO interface unit on a target computing core particle among the N computing core particles or set an IO core particle connected to the target computing core particle; Among them, the IO interface is the IO interface unit or the IO core particle; the target computing core particle communicates with the peripheral device through the IO interface, and other computing core particles different from the target computing core particle are connected to the target computing core particle and communicate with the peripheral device through the IO interface.

13. The method according to claim 12, characterized in that The plurality of target core particles are stacked vertically to form at least one core particle stacking group, comprising: Based on a stacking rule that a computing core matches at least one cache core, stacking the plurality of target cores in a vertical direction according to a preset stacking order to form at least one core stacking group; Among them, adjacent chip stacking groups are connected through vertical through holes connecting the on-chip network on two computing chip particles and vertical through holes on the cache chip particles between the two computing chip particles; the chip stacking group includes a first number of storage and computing cores, and the storage and computing cores include computing units, scheduling control units and DMA write-back units deployed on the computing chip particles, as well as cache units and DMA loading units deployed on the cache chip particles, and the number of cache units corresponding to the storage and computing cores is the number of cache chip particles corresponding to a single chip stacking group.

Images