Three-dimensional integrated chip and construction method thereof, data processing method and electronic device

Abstract

The present invention relates to realizing a neural network system through three-dimensional integration technology, providing a large-capacity, high-bandwidth, low-latency, and low-power memory for numerous neural network computing cores of the neural network system, thereby effectively improving the performance of the neural network system and reducing the cost of the neural network system.

Description

Technical Field

[0001] The present invention relates to the field of memory, and in particular to a three-dimensional integrated chip and a construction method thereof, a data processing method and an electronic device. Background Art

[0002] The calculation of neural network system requires high bandwidth and high energy efficiency storage system. In the prior art, high bandwidth memory (HBM) unit is used as the memory of neural network calculation unit, such as Figure 1 As shown in . When the neural network computing unit is working, the data access high bandwidth memory HBM unit operates to obtain data. High bandwidth memory HBM unit can provide higher memory bandwidth than dynamic random access memory DRAM, but it also has a higher cost. In addition, the use of discrete memory in the neural network system will have higher latency and power consumption.

[0003] Therefore, it is urgent to solve the above technical problems in the prior art. Summary of the invention

[0004] The present invention aims to realize a neural network system through three-dimensional integration technology, and provide a large-capacity, high-bandwidth, low-latency, and low-power memory for the numerous neural network computing cores of the neural network system, thereby effectively improving the performance of the neural network system and reducing the cost of the neural network system.

[0005] According to a first aspect of the present invention, a three-dimensional integrated chip is provided, wherein the three-dimensional integrated chip comprises at least a memory unit and a neural network unit.

[0006] The memory unit includes at least two memory arrays, each of the at least two memory arrays having a corresponding first three-dimensional integrated port;

[0007] The neural network unit comprises at least two neural network sub-units, each of the at least two neural network sub-units comprises at least a second three-dimensional integrated port, at least one routing unit and at least one neural network computing core, wherein:

[0008] The second three-dimensional integrated port is connected to the corresponding first three-dimensional integrated port, thereby integrating the memory unit with the neural network unit three-dimensional matrix;

[0009] The at least one routing unit is configured to connect the at least two neural network sub-units, wherein the routing unit in the source neural network sub-unit is configured to enable the neural network computing core in the source neural network sub-unit to access the corresponding target access address based on the target access address in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection of the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit;

[0010] The at least one neural network computing core is configured to access the at least two storage arrays or the target access address, and implement a corresponding neural network computing function using the accessed data.

[0011] According to a preferred embodiment of the three-dimensional integrated chip of the present invention, the target access address is the address of the neural network computing core in the target neural network subunit, wherein

[0012] The routing unit in the source neural network sub-unit is configured to enable the neural network computing core in the source neural network sub-unit to access the neural network computing core in the target neural network sub-unit based on the address of the neural network computing core in the target neural network sub-unit in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection between the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit.

[0013] According to a preferred embodiment of the three-dimensional integrated chip of the present invention, the target access address is the address of the target storage array, wherein:

[0014] The routing unit in the source neural network sub-unit is configured to enable the neural network computing core in the source neural network sub-unit to access the target storage array based on the address of the target storage array in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection between the routing unit in the source neural network sub-unit, the routing unit in the neural network sub-unit corresponding to the target storage array, the second three-dimensional integrated port in the neural network sub-unit corresponding to the target storage array and the first three-dimensional integrated port of the target storage array.

[0015] According to a preferred embodiment of the three-dimensional integrated chip of the present invention, each of the at least two neural network sub-units further includes a memory controller, which is connected to the at least one neural network computing core and to the second three-dimensional integrated port, and the memory controller controls the at least one neural network computing core to perform storage access to the target storage array through the connection between the second three-dimensional integrated port and the first three-dimensional integrated port based on the address of the target storage array in the access instruction received from the at least one neural network computing core without passing through the at least one routing unit.

[0016] According to a preferred embodiment of the three-dimensional integrated chip of the present invention, each of the at least two neural network sub-units further includes a multi-way selection switch, which is connected to the at least one routing unit and the memory controller, and the multi-way selection switch is configured to selectively connect the neural network computing core to the at least one routing unit to access the target access address, or to connect the neural network computing core to the memory controller to access the target storage array.

[0017] According to a preferred embodiment of the three-dimensional integrated chip of the present invention, the neural network subunit further comprises:

[0018] A cache module, wherein the cache module is configured to convert the operating voltage of the neural network unit into the operating voltage of the memory unit or convert the operating voltage of the memory unit into the operating voltage of the neural network unit when the neural network unit accesses the memory unit for storage.

[0019] According to a preferred embodiment of the three-dimensional integrated chip of the present invention, the at least two neural network sub-units are connected into a bus, star, ring, tree, mesh or mixed topology through the at least one routing unit.

[0020] According to a preferred embodiment of the three-dimensional integrated chip of the present invention, the storage array is a dynamic random access memory array.

[0021] According to a second aspect of the present invention, a method for constructing a three-dimensional integrated chip is provided, wherein the three-dimensional integrated chip comprises at least a memory unit and a neural network unit, and the method for constructing comprises:

[0022] The memory cell is constructed, the memory cell comprising at least two memory arrays and at least two first three-dimensional integrated ports, wherein:

[0023] The storage array includes at least one storage bank;

[0024] Each memory bank corresponds to a first three-dimensional integrated port;

[0025] Constructing the neural network unit, the neural network unit includes at least two neural network sub-units, the neural network sub-unit includes at least a second three-dimensional integrated port, at least one neural network computing core and at least one router, wherein:

[0026] The second three-dimensional integrated port is connected to the at least one first three-dimensional integrated port;

[0027] The neural network computing core is configured to access the at least one storage body and implement a corresponding neural network computing function using the accessed data;

[0028] The routing unit is configured to connect the at least two neural network sub-units, wherein the routing unit in the source neural network sub-unit is configured to enable the neural network computing core in the source neural network sub-unit to access the corresponding target access address based on the target access address in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection between the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit.

[0029] According to a third aspect of the present invention, a data processing method for a three-dimensional integrated chip is provided, and the data processing method is based on a first three-dimensional integrated chip, and the method comprises: a neural network computing core in a source neural network sub-unit receives an access instruction; based on the access instruction, the neural network computing core in the source neural network sub-unit accesses a corresponding target access address through a routing unit in the source neural network sub-unit and a routing unit in a target neural network sub-unit; and the corresponding neural network computing function is implemented using the accessed data.

[0030] According to a fourth aspect of the present invention, there is provided an electronic device, comprising the three-dimensional integrated chip according to the first aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention will be more easily understood through the following description in conjunction with the accompanying drawings, in which:

[0032] Figure 1 The structure of a neural network system based on a high-bandwidth memory unit in the prior art is schematically shown.

[0033] Figure 2 An embodiment of a three-dimensional integrated chip according to the present invention is schematically shown.

[0034] Figure 3An embodiment of a neural network sub-unit accessing its own storage array in a three-dimensional integrated chip according to the present invention is schematically shown.

[0035] Figure 4 An embodiment of the internal connection of a three-dimensional integrated chip according to the present invention is schematically shown. DETAILED DESCRIPTION

[0036] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] Figure 2 An embodiment of a three-dimensional integrated neural network chip according to the present invention is schematically shown.

[0038] Figure 2 The three-dimensional integrated neural network chip shown includes a neural network wafer 200 (equivalent to a neural network unit) and a memory wafer 210 (equivalent to a memory unit). The neural network wafer 200 is a wafer used to implement the logic function of the neural network, and the memory wafer 210 is a wafer used to implement the memory function. The neural logic wafer 200 can realize the logical function of imitating the behavioral characteristics of animal neural networks and performing distributed parallel information processing. The neural logic wafer 200 has a very high computing speed, a strong associative ability, a strong adaptability, a strong fault tolerance and self-organizing ability, and can be applied to the information field (such as information processing, pattern recognition, etc.), the medical field (such as detection and analysis of biological signals, etc.), the economic field (such as market price forecasting, risk assessment, etc.), the control field, the transportation field, the psychology field, etc.

[0039] like Figure 2 As shown in , the neural network wafer 200 and the memory wafer 210 are stacked and integrated in the vertical direction. The “vertical direction” here refers to the thickness direction of the neural network wafer 200 and the memory wafer 210.

[0040] In addition, the memory wafer 210 and the neural network wafer 200 are bonded and connected by a three-dimensional heterogeneous integration method to achieve a three-dimensional heterogeneous integration of the neural network wafer 200 and the memory wafer 210. Specifically, the memory wafer 210 is bonded by a hybrid bonding technology ( Figure 2 It is schematically shown as ) is directly connected to the neural network wafer 200.

[0041] from Figure 2 It can also be seen that the neural network wafer 200 is connected to the substrate 220 through a bump process. The bump process includes but is not limited to BOPCOA, BOAC, and HOTROD known in the prior art.

[0042] In addition, Figure 2 In the description, neural network wafers and memory wafers are combined. It is known that wafers are the basic raw materials for manufacturing semiconductor devices, and extremely high-purity semiconductors are prepared into wafers through processes such as crystal pulling and slicing. The wafers undergo a series of semiconductor manufacturing processes to form extremely small circuit structures, which are then cut, packaged, and tested to become chips. The present invention is not limited to neural network wafers and memory wafers. In fact, the neural network wafer 200 can also be a neural network chip, and the memory wafer 210 can also be a memory chip. Preferably, the memory wafer 210 can be a dynamic random access memory wafer or a dynamic random access memory chip.

[0043] Figure 2 What is shown in the figure is a memory wafer 210 or a layer of memory wafer 210. It should be understood that in order to meet the bandwidth and storage capacity requirements, the memory wafer 210 can be more layers, including but not limited to 2 layers, 4 layers, 6 layers, etc. In addition, multiple memory wafers can be set in the same layer of the multi-layer memory wafer. For example, two memory wafers are set in the same layer of memory wafer layer, and the two memory wafers are independent of each other.

[0044] In addition, the multi-layer memory wafers can be respectively arranged on both sides of the neural network wafer, for example, one memory wafer can be arranged above the neural network wafer, and another memory wafer can be arranged below the neural network wafer. Thus, the storage access path between the neural network wafer and the memory wafer is shortened, and the storage access efficiency is improved.

[0045] Specifically, the memory wafers on both sides of the neural network wafer can be directly connected to the neural network wafer through hybrid bonding technology; and the neural network wafer is first connected to the memory wafer below through hybrid bonding technology, and then penetrates the through silicon via TSV of the memory wafer below and is then connected to the substrate through a bump process.

[0046] In the present invention, a neural network wafer is described, and the corresponding three-dimensional integrated chip obtained is a three-dimensional integrated neural network chip. The performance of the neural network system is limited by memory bandwidth, capacity, latency and power consumption. In the present invention, by providing large-capacity, high-bandwidth, low-latency, and low-power memory for the numerous neural network computing cores of the neural network system, the performance of the neural network system is effectively improved and the cost of the neural network system is reduced.

[0047] Figure 3 An embodiment of a neural network sub-unit accessing its own storage array in a three-dimensional integrated neural network chip according to the present invention is schematically shown.

[0048] The three-dimensional integrated neural network chip of the present invention includes several neural network subunits (such as Figure 4Each neural network subunit is able to access its own storage array. Figure 3 Such an embodiment is shown in .

[0049] exist Figure 3 In the embodiment, the neural network sub-unit 30 is stacked and integrated with the storage array 31 to which it belongs, and is preferably stacked and integrated by bonding, or more preferably stacked and integrated by hybrid bonding.

[0050] like Figure 3 As shown in , the neural network subunit 30 includes at least a neural network computing core 301, a memory controller 302 and a cache module 303.

[0051] The input and output three-dimensional heterogeneous integrated ports (i.e., hybrid bonded ports, schematically shown by two rows of "●" in the figure) in the cache module 303 are connected to the input and output three-dimensional heterogeneous integrated ports (i.e., hybrid bonded ports, schematically shown by two rows of "●" in the figure) in the storage array 304, thereby three-dimensionally heterogeneously integrating the neural network subunit 30 with the storage array 304. It should be understood that Figure 3 For illustration only, the input and output three-dimensional heterogeneous integration ports may also be located in other modules.

[0052] The cache module 330 is configured to perform voltage conversion, and is used to convert the operating voltage of the neural network sub-unit 30 into the operating voltage of the storage array 304, or to convert the operating voltage of the storage array 304 into the operating voltage of the neural network sub-unit 30 when the neural network sub-unit 30 accesses the storage array 304.

[0053] The neural network computing core 301 can directly access the storage array 304 to which it belongs (ie, the storage array 304 connected via its three-dimensional heterogeneous integrated port).

[0054] For example, when the neural network computing core 301 receives an access instruction, and the access address carried by the access instruction is the storage array 304 connected to its three-dimensional heterogeneous integrated port, the memory controller 320 accesses the storage array 304 based on the access address through the connection between the input and output three-dimensional heterogeneous integrated port in the cache module 303 and the input and output three-dimensional heterogeneous integrated port in the storage array 304.

[0055] Specifically, the access instruction includes, for example, a read instruction and a write instruction. When the access instruction is a read instruction, the memory controller 320 reads data from the storage array 304 based on the read instruction through the connection between the input and output three-dimensional heterogeneous integrated port in the cache module 303 and the input and output three-dimensional heterogeneous integrated port in the storage array 304. When the access instruction is a write instruction, the access instruction also includes the data to be written, and the memory controller 320 writes the data to be written into the storage array 304 in the upper left corner through the connection between the input and output three-dimensional heterogeneous integrated port in the cache module 303 and the input and output three-dimensional heterogeneous integrated port in the storage array 304 based on the write instruction.

[0056] Figure 4 An embodiment of the internal connection of a three-dimensional integrated neural network chip (ie, system on chip) according to the present invention is schematically shown.

[0057] exist Figure 4 In FIG. 4 , nine neural network subunits 40 are schematically shown. Figure 4 Each of the nine neural network sub-units 40 further includes a routing unit 404. The plurality of neural network sub-units 40 are interconnected through the routing units 404 located in the neural network sub-units 40 to form a system on a chip.

[0058] in addition, Figure 4 Each of the nine neural network sub-units 40 further includes a multiplexer switch 405. In each neural network sub-unit 40, the multiplexer switch 405 is connected to the routing unit 404 and the memory controller 402.

[0059] When the multi-way selection switch 405 is connected to the memory controller 402, the neural network computing core 401 in the neural network subunit 40 can directly access the storage array to which it belongs (i.e., the storage array connected through its three-dimensional heterogeneous integrated port). The steps can be referred to as above. Figure 3 as described.

[0060] When the multi-way selection switch 405 is connected to the routing unit 404, the neural network computing core in one neural network sub-unit can access the neural network computing core or storage array in another neural network sub-unit through one or more routing units.

[0061] Specifically, for example, when Figure 4 The neural network computing core in the neural network subunit in the upper right corner (hereinafter referred to as the "source neural network subunit") receives the access instruction, and the access address carried by the access instruction is Figure 4 The storage array (i.e., Figure 4When the neural network subunit in the lower right corner of the figure is connected to a storage array through its three-dimensional heterogeneous integrated port), the memory controller in the source neural network subunit accesses the memory based on the access address through the routing unit in the source neural network subunit (hereinafter referred to as the "source routing unit"), Figure 4 The access path formed by the routing unit (hereinafter referred to as the "intermediate routing unit") in the intermediate neural network sub-unit (hereinafter referred to as the "intermediate neural network sub-unit") in the rightmost row and the routing unit (hereinafter referred to as the "target routing unit") in the target neural network sub-unit accesses the storage array to which the target neural network sub-unit belongs.

[0062] For example, when Figure 4 The neural network computing core in the source neural network subunit shown receives an access instruction, and the access address carried by the access instruction is Figure 4 When the neural network computing core in the target neural network sub-unit is shown, the memory controller in the source neural network sub-unit accesses the neural network computing core in the target neural network sub-unit based on the access address through the access path formed by the source routing unit in the source neural network sub-unit, the intermediate routing unit in the intermediate neural network sub-unit, and the target routing unit in the target neural network sub-unit.

[0063] Figure 4 The memory network of the present invention includes but is not limited to bus, star, ring, tree, mesh and mixed topology structures.

[0064] The three-dimensional integrated chip proposed in the present invention is a memory chip or a memory wafer (e.g., ROM, SDRAM, RAM, DRAM, SRAM, FLASH, EPROM, EEPROM, CD-ROM or other optical disk storage device, magnetic disk storage device or other magnetic storage device) for storing data and / or computer code. The three-dimensional integrated chip may be or include a non-transient volatile memory or a non-volatile memory, etc.

[0065] The present application also proposes a data processing method for a three-dimensional integrated chip, the method comprising:

[0066] Step S1: The neural network computing core in the source neural network sub-unit receives an access instruction.

[0067] Step S2: Based on the access instruction, the neural network calculation core in the source neural network sub-unit accesses the corresponding target access address through the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit.

[0068] Step S3: Use the accessed data to implement the corresponding neural network calculation function.

[0069] Specifically, for example, when Figure 4 The neural network computing core in the neural network subunit in the upper right corner (hereinafter referred to as the "source neural network subunit") receives the access instruction, and the access address carried by the access instruction is Figure 4 The storage array (i.e., Figure 4 When the neural network subunit in the lower right corner of the figure is connected to a storage array through its three-dimensional heterogeneous integrated port), the memory controller in the source neural network subunit accesses the memory based on the access address through the routing unit in the source neural network subunit (hereinafter referred to as the "source routing unit"), Figure 4 The access path formed by the routing unit (hereinafter referred to as the "intermediate routing unit") in the intermediate neural network sub-unit (hereinafter referred to as the "intermediate neural network sub-unit") in the rightmost row and the routing unit (hereinafter referred to as the "target routing unit") in the target neural network sub-unit accesses the storage array to which the target neural network sub-unit belongs.

[0070] For example, when Figure 4 The neural network computing core in the source neural network subunit shown receives an access instruction, and the access address carried by the access instruction is Figure 4 When the neural network computing core in the target neural network sub-unit is shown, the memory controller in the source neural network sub-unit accesses the neural network computing core in the target neural network sub-unit based on the access address through the access path formed by the source routing unit in the source neural network sub-unit, the intermediate routing unit in the intermediate neural network sub-unit, and the target routing unit in the target neural network sub-unit.

[0071] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.It should be understood that the scope of the invention is defined by the claims.

Claims

1. A three-dimensional integrated chip, characterized in that: The three-dimensional integrated chip at least includes a memory unit and a neural network unit. The memory unit includes at least two memory arrays, each of the at least two memory arrays having a corresponding first three-dimensional integrated port; The neural network unit comprises at least two neural network sub-units, each of the at least two neural network sub-units comprises at least a second three-dimensional integrated port, at least one routing unit and at least one neural network computing core, wherein: The second three-dimensional integrated port is connected to the corresponding first three-dimensional integrated port, thereby integrating the memory unit with the neural network unit three-dimensional matrix; The at least one routing unit is configured to connect the at least two neural network sub-units, wherein the routing unit in the source neural network sub-unit is configured to enable the neural network computing core in the source neural network sub-unit to access the corresponding target access address based on the target access address in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection of the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit; The at least one neural network computing core is configured to access the at least two storage arrays or the target access address, and implement a corresponding neural network computing function using the accessed data; The target access address is the address of the neural network computing core in the target neural network subunit, The routing unit in the source neural network sub-unit is configured to enable the neural network computing core in the source neural network sub-unit to access the neural network computing core in the target neural network sub-unit based on the address of the neural network computing core in the target neural network sub-unit in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection between the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit.

2. The three-dimensional integrated chip according to claim 1, characterized in that: The target access address is the address of the target storage array, where: The routing unit in the source neural network sub-unit is configured to enable the neural network computing core in the source neural network sub-unit to access the target storage array based on the address of the target storage array in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection between the routing unit in the source neural network sub-unit, the routing unit in the neural network sub-unit corresponding to the target storage array, the second three-dimensional integrated port in the neural network sub-unit corresponding to the target storage array and the first three-dimensional integrated port of the target storage array.

3. The three-dimensional integrated chip according to claim 1, characterized in that: Each of the at least two neural network sub-units further includes a memory controller, which is connected to the at least one neural network computing core and to the second three-dimensional integrated port. The memory controller controls the at least one neural network computing core to perform storage access to the target storage array through the connection between the second three-dimensional integrated port and the first three-dimensional integrated port based on the address of the target storage array in the access instruction received from the at least one neural network computing core without passing through the at least one routing unit.

4. The three-dimensional integrated chip according to claim 3, characterized in that: Each of the at least two neural network sub-units further includes a multi-way selection switch, which is connected to the at least one routing unit and the memory controller, and the multi-way selection switch is configured to selectively connect the neural network computing core to the at least one routing unit to access the target access address, or to connect the neural network computing core to the memory controller to access the target storage array.

5. The three-dimensional integrated chip according to claim 1, characterized in that: The neural network subunit also includes: A cache module, wherein the cache module is configured to convert the operating voltage of the neural network unit into the operating voltage of the memory unit or convert the operating voltage of the memory unit into the operating voltage of the neural network unit when the neural network unit accesses the memory unit for storage.

6. The three-dimensional integrated chip according to claim 1, characterized in that: The at least two neural network sub-units are connected to form a bus, star, ring, tree, mesh or mixed topology through the at least one routing unit.

7. The three-dimensional integrated chip according to claim 1, characterized in that: The storage array is a dynamic random access memory array.

8. A method for constructing a three-dimensional integrated chip, characterized in that: The three-dimensional integrated chip includes at least a memory unit and a neural network unit, and the construction method includes: The memory cell is constructed, the memory cell comprising at least two memory arrays and at least two first three-dimensional integrated ports, wherein: The storage array includes at least one storage bank; Each memory bank corresponds to a first three-dimensional integrated port; Constructing the neural network unit, the neural network unit includes at least two neural network sub-units, the neural network sub-unit includes at least a second three-dimensional integrated port, at least one neural network computing core and at least one router, wherein: The second three-dimensional integrated port is connected to at least one first three-dimensional integrated port; The neural network computing core is configured to access the at least one storage body and implement a corresponding neural network computing function using the accessed data; The routing unit is configured to connect the at least two neural network sub-units, wherein the routing unit in the source neural network sub-unit is configured to implement access of the neural network computing core in the source neural network sub-unit to the corresponding target access address based on the target access address in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection of the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit; The target access address is the address of the neural network computing core in the target neural network sub-unit, and the routing unit in the source neural network sub-unit is configured to implement access of the neural network computing core in the source neural network sub-unit to the neural network computing core in the target neural network sub-unit based on the address of the neural network computing core in the target neural network sub-unit in the access instruction received from the neural network computing core in the source neural network sub-unit, at least through the connection between the routing unit in the source neural network sub-unit and the routing unit in the target neural network sub-unit.

9. A data processing method for a three-dimensional integrated chip, characterized in that: The data processing method is based on the three-dimensional integrated chip according to any one of claims 1 to 7, and the method comprises: The neural network computing core in the source neural network subunit receives the access instruction; Based on the access instruction, the neural network computing core in the source neural network subunit accesses the corresponding target access address through the routing unit in the source neural network subunit and the routing unit in the target neural network subunit; The accessed data is used to implement the corresponding neural network calculation function.

10. An electronic device, characterized in that: The electronic device comprises the three-dimensional integrated chip according to any one of claims 1-7.

Images