Memory interconnect architecture system and method

Abstract

The systems and methods of the present invention are used to efficiently and effectively incorporate processing power into internal memory. In one embodiment, a processing-in-memory (PIM) chip includes a memory array, logic components, and an interconnect network. The memory array is used to store information. In an exemplary implementation, the memory array includes memory cells and array peripheral components. The logic components are capable of processing the information stored in the memory array. The interconnect network is used to communicatively couple the logic components. The interconnect network may include interconnects, and a portion of the interconnects may be located in a metal layer region that is located above the memory array.

Description

Technical Field

[0001] This invention relates to the field of information processing and communication in internal memory. Background Technology

[0002] Many electronic technologies, such as digital computers, calculators, audio devices, video equipment, and telephone systems, enhance productivity and reduce costs in most business, scientific, educational, and entertainment fields by helping to analyze and transmit data and information. Electronic components can be used in many important applications (e.g., medical surgery, vehicle assistance, financial applications), and these activities often involve storing large amounts of information. The storage and retrieval of information can have a significant impact on system performance. The efficiency and effectiveness of internal memory operations depend on the internal memory configuration used.

[0003] Some conventional systems attempt to add processing power to internal memory. Many factors can influence the configuration of internal memory. There is a general preference for denser components, but the ability to achieve smaller components within the same semiconductor area has slowed significantly (e.g., according to Moore's Law, etc.). Furthermore, implementations with dedicated processors (e.g., CPUs, GPUs, etc.) and separate internal memory consume a significant amount of energy during access operations compared to the lower power consumption of implementations with in-memory processing (PIM) capabilities. Two-dimensional in-memory processing capability (2DPIM) typically reduces memory access latency, data movement energy, and manufacturing costs.

[0004] Adding processing power to internal memory also presents numerous challenges. Due to DRAM processing, the frequency and performance of a 2DPIM are significantly constrained by wiring / routing resources. DRAM arrays occupy a significant area within the PIM, but the top metal layer is typically not used (e.g., primarily only 3 out of 5 metal layers are used). The complexity of SoC design, the large number of components, and the substantial memory bandwidth requirements within the PIM often necessitate complex on-chip interconnects to support the corresponding scalability. However, these increased scalability requirements typically place significant stress and difficulty on component and interconnect placement within conventional chip design and architecture methodologies. Summary of the Invention

[0005] The systems and methods of the present invention are for efficiently and effectively adding processing power to internal memory. In one embodiment, a processing-in-memory (PIM) chip includes a memory array, logic components, and an interconnect network. The memory array is used to store information. In an exemplary implementation, the memory array includes: memory cells for storing information bits; and array peripheral components for controlling access to the memory cells. The memory cells are located in a first region (e.g., a first portion of the memory chip or die), while the array peripheral components are located in a second region (e.g., a second portion of the memory chip or die). The logic components are included in a third region (e.g., a third portion of the memory chip or die) and are capable of processing the information stored in the memory array. The interconnect network is used to communicatively couple the logic components. The interconnect network may include a first set of interconnects, and a portion of the first set of interconnects is located in a metal region above the memory array.

[0006] In one embodiment, the interconnect network includes a switch for routing information within the interconnect network. The switch may be included in a region having array peripheral components. In one exemplary implementation, the switch is included in a 2DPIM. The switch can be configured with other redundant components in the array peripheral components. In one embodiment, a second set of interconnects communicatively couples the memory cells and the memory array peripheral components. A portion of the second set of interconnects may be located in metal layers 1, 2, and 3, while a portion of the first set of interconnects may be located in metal layers 4 and 5. In one exemplary implementation, the internal memory may be configured as dynamic random access memory (DRAM). The array peripheral components may include: a column address decoder for decoding column addresses in the memory array; and a row address decoder for decoding row addresses in the memory array.

[0007] In one embodiment, a method of fabricating an in-memory processing (PIM) chip includes: forming a memory array in a memory array region of the PIM chip; forming logic components in a logic region of the PIM chip; and forming an interconnect network. The logic components may include processing components. The interconnect network includes interconnects, and a portion of these interconnects is located in a metal layer region that is also located above the memory array region. Forming the interconnect network may include forming a switch. The switch is coupled to the interconnects and is used to route information between the logic components. In an exemplary implementation, the memory array region may include memory cell portions and array periphery portions. The switch can be located in the periphery component portion of the memory array region. The switch may include various components (e.g., multiplexers and demultiplexers, buffers, inverters, etc.). A portion of the interconnects of the logic components is located in metal layers 4 and 5, and a portion of other interconnects of the memory array is located in metal layers 1, 2, and 3.

[0008] In one embodiment, the internal memory includes internal memory blocks, logic components, and an interconnect network. The internal memory blocks are used to store information. The logic components are used to process the information stored in the internal memory blocks. The interconnect network is used to communicatively couple the logic components. In one embodiment, the interconnect network includes interconnects, a portion of which is located in a metal region above the internal memory blocks. The communication network may include switches in the internal memory block region, and these switches are used to route information in the interconnect network. The logic components may include processing control components for performing various tasks (e.g., scheduling tasks, configuring registers, and handling global synchronization, accelerators for accelerating application processing, etc.). The interconnect network may include on-chip interconnects that communicatively couple accelerators to dynamic random access memory (DRAM) blocks and allow higher latency access than dedicated DRAM blocks. The interconnect network may include dedicated interfaces between accelerators and DRAM blocks, enabling high-bandwidth and low-latency DRAM access from the corresponding accelerator with higher priority than other blocks. In one exemplary implementation, the interconnect network includes on-chip interconnects forming on-chip networks (NoCs) with different topologies.

[0009] The present invention is provided to introduce, in a simplified form, the selection of concepts further described below in the detailed description. The present invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0010] The accompanying drawings are incorporated in and form a part of this specification to illustrate the principles of the invention by way of example, and are not intended to limit the invention to the specific implementations illustrated herein. Unless otherwise specifically indicated, the drawings are not drawn to scale. Embodiments of the technology are illustrated as examples and not as limitations in the figures, and similar reference numerals refer to similar elements in the drawings. Unless otherwise indicated, the drawings are not necessarily drawn to scale.

[0011] Figure 1 This is a block diagram of an exemplary internal storage system in one embodiment of the present invention, in which a portion of an interconnect network is located in an area above a memory array component.

[0012] Figure 2 This is a block diagram of an exemplary internal storage system in one embodiment of the present invention, in which a portion of the interconnect network is not located in an area above the memory array components.

[0013] Figure 3 This is a cut-out side view of an exemplary internal storage system in one embodiment of the present invention, showing a portion of the interconnect network located in the region above the memory array components.

[0014] Figure 4 This is a top view of an exemplary internal storage system located in an area above a memory array component, which is part of an interconnect network according to an embodiment of the present invention.

[0015] Figure 5 This is a top view of an exemplary internal storage system, where a portion of the interconnect network of one embodiment of the present invention is located in an area not above the memory array components.

[0016] Figure 6 This is a flowchart of an exemplary internal memory manufacturing method according to an embodiment of the present invention.

[0017] Figure 7 This is a block diagram of an exemplary internal memory interconnect architecture according to an embodiment of the present invention.

[0018] Figure 8 This is a block diagram of an exemplary portion of a PIM chip according to an embodiment of the present invention.

[0019] Figure 9 This is a block diagram of an exemplary computing system for storing and processing information according to an embodiment of the present invention.

[0020] Figure 10 This is a block diagram of an exemplary processing core configuration according to an embodiment of the present invention. Detailed Implementation

[0021] Reference will now be made in detail to embodiments of the present technology, examples of which are illustrated in the accompanying drawings. While the present technology will be described in conjunction with these embodiments, it should be understood that they are not intended to limit the present technology to these embodiments. Rather, the invention is intended to cover alternatives, modifications, and equivalents that may be included within the scope of the invention as defined by the appended claims. Furthermore, numerous specific details are set forth in the following detailed description of the present technology to provide a thorough understanding of the present technology. However, it should be understood that the present technology can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail to avoid unnecessarily obscuring aspects of the present technology.

[0022] The following embodiments of the present technology are presented according to routines, modules, logic blocks, and other notations for operating data within one or more electronic devices. The descriptions and representations herein are means of conveying the essence of its work most effectively to those skilled in the art. In this document, routines, logic blocks, and / or the like are generally conceived as self-consistent sequences of processes or instructions that result in desired outcomes. A process is a process that involves the physical manipulation of physical quantities. Typically, though not always, these physical manipulations take the form of electrical or magnetic signals that can be stored, transferred, compared, and otherwise manipulated within an electronic device. For convenience, and with reference to common usage, and with reference to embodiments of the present technology, these signals are referred to as data, bits, values, elements, symbols, characters, terms, numbers, strings, and / or the like.

[0023] However, it should be remembered that these terms will be interpreted as referring to physical manipulation and quantity, and are merely convenient labels, and will be further interpreted in light of terms commonly used in the art. Unless otherwise specifically stated as will be apparent from the following discussion, it should be understood that, through the discussion of this technology, the use of terms such as “receiving” and / or the like refers to the actions and processes of electronic devices, such as electronic computing devices that manipulate and transform data. Data is represented as physical (e.g., electronic) quantities within the logic circuits, registers, memories, and / or similar elements of the electronic device, and is transformed into other data similarly represented as physical quantities within the electronic device.

[0024] In this application, the use of antonymous conjunctions is intended to encompass conjunctions. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, references to “the” object or “a” object are intended to indicate one of a plurality of possible such objects. The use of terms such as “comprising,” “including,” “containing,” “including,” etc., specifies the presence of the stated element, but does not preclude the presence or addition of one or more other elements and / or groups thereof. It should also be understood that although the terms first, second, etc., may be used herein to describe various elements, such elements should not be limited by these terms. These terms are used herein to distinguish one element from another. For example, without departing from the scope of the embodiments, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element. It should also be understood that when an element is referred to as “coupled” to another element, it may be directly or indirectly connected to the other element, or there may be intermediate elements. In contrast, when an element is referred to as “directly connected” to another element, there are no intermediate elements. It should also be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive.

[0025] The systems and methods of this invention facilitate efficient and effective communication and processing of information in internal memory. In one embodiment, processing components and memory components may be included in the same semiconductor chip (e.g., DRAM, etc.). In an exemplary implementation, an on-chip interconnect architecture exists that couples the on-chip processing components to other components in the chip. The on-chip interconnect architecture may include interconnect networks for enabling improved communication between on-chip components. The interconnect network may include communication lines and switches. The interconnect network may include global routing in an upper metal layer above the DRAM array and switching / routing logic in the DRAM array region.

[0026] In one embodiment, an interconnect network communicatively couples on-chip processing components to other components on the chip. In one exemplary implementation, the communication network communicatively couples on-chip processing components to on-chip accelerator components. Portions of the interconnects may be located in the top metal layer of a DRAM process. The interconnects may be coupled to the top layer via vias. In one embodiment, portions of the interconnects utilize an upper metal layer (e.g., M4, M5, etc.) in the upper region of a PIM chip that includes internal memory block components. The region of the PIM chip that includes internal memory block components may include memory array components and array perimeter components. The region of the PIM chip that includes internal memory block components may also include other components relevant to the operation of the interconnect network (e.g., buffers, inverters, MUX, DeMUX, etc.).

[0027] Components associated with interconnect network operation may include communication repeater logic components, switches for global routing (e.g., on-chip interconnects, etc.). In one exemplary implementation, enhancer / relay components and switches / routers can be included in the DRAM array / peripheral region. The switches can be used to route data from initiator / source to destination / target via on-chip connections using different interconnect topologies. In one embodiment, the memory array block interconnects are located in the lower metal layers of the DRAM process (e.g., M1, M2, M3, etc.). In one exemplary implementation, the memory array block interconnects are located in the lower three metal layers, while the on-chip processing interconnects are located in the upper two metal layers.

[0028] The top metal layer provides routing configurations that facilitate increased drive power and improved electronic current (e.g., resulting in improved timing and performance). Interconnect networks can improve architectural scalability, timing, and performance. In one embodiment, area overhead occurs for the DRAM blocks, but the available routing space is significantly increased.

[0029] Figure 1This is a block diagram of an internal memory system 100 according to an embodiment of the present invention. The internal memory system 100 includes a CPU and controller 110, a connectivity component 120, a direct memory access (DMA) module 130, external peripheral components 140, DRAM blocks 131, 132, 133, and 134, and accelerators 141, 142, 143, and 144. The internal memory system 100 also includes network interconnects that communicatively couple the CPU and controller 110 to the accelerators 141, 142, 143, and 144. In one exemplary embodiment, interconnect 150 communicatively couples the CPU and controller 110 to the accelerator 142. A portion of interconnect 150 is located in a metal layer above a portion of the regions where DRAM blocks 131 and 132 are located.

[0030] The components of the internal memory system 100 cooperate to store information and provide on-chip processing of some of that information. In one embodiment, on-chip processing is considered to be in-memory processing (PIM). A central processing unit (CPU) 110 and a controller 110 direct various control operations (e.g., including task scheduling, register configuration, global synchronization, etc.). Accelerators 141 to 144 accelerate operations associated with various applications (e.g., deep neural networks (DNNs)). DRAM blocks 131 to 134 selectively store information. The DRAM blocks provide high bandwidth, high capacity, and low latency access. A connectivity component 120 is used to interface with a host (e.g., when the internal memory chip acts as an accelerator in the cloud). The connectivity component 120 is compatible with various protocols and configurations (e.g., PCIe, DDR4, etc.). An external peripheral interface component 140 is used to interface with external peripheral devices via various protocols (e.g., such as 2C, I2S, UART, USB, etc.). DMA component 130 is used to manage data transfers on the on-chip interconnect to free up CPU 110 and accelerators 141 to 144 to perform other computing-related workloads.

[0031] In one embodiment, the internal storage system 100 is a two-dimensional in-memory processing system-on-chip (2DPIM SoC). It should be understood that other components (not shown) may be included in the 2D PIM SoC (e.g., vector processor, video codec, audio DSP, etc.) depending on application requirements.

[0032] The interconnect network may include communication enhancers or relay logic components (e.g., buffers, inverters, etc.) in the DRAM array or peripheral logic component area of ​​the DRAM array. In one exemplary embodiment, the data relay logic component is included in the global routing connection path data channel or interconnect.

[0033] The interconnect network may include switches / routers. Switches / routers may use multiplexers / demultiplexers and FIFOs included in the DRAM array area or the DRAM peripheral logic area. In one embodiment, the switch / router utilizes different interconnect topologies to route data from initiator / source to destination / target via on-chip connections.

[0034] Figure 2 This is a block diagram of an internal memory system 200 according to an embodiment of the present invention. Internal memory system 200 is similar to internal memory system 100, except that the interconnect network lines and switches are located in areas not above the DRAM block locations. Internal memory system 200 includes a CPU and controller 210, a connectivity component 220, a direct memory access (DMA) module 230, external peripheral components 240, DRAM blocks 231, 232, 233, and 234, and accelerators 241, 242, 243, and 244. Internal memory system 200 also includes interconnect network lines that communicatively couple the CPU and controller 210 to the accelerators 241, 242, 243, and 244. In one exemplary implementation, interconnect line 250 communicatively couples the CPU and controller 210 to the accelerator 242. Interconnect line 250 is located in a metal layer and is located in a metal layer not above the DRAM blocks 231 and 232.

[0035] Figure 3 This is a side block view of an internal memory 300 according to an embodiment of the present invention. The internal memory 300 includes a memory array region 310, a logic component region 320, a logic component region 330, and an upper interconnect network region 350. In one embodiment, the upper interconnect network region 350 includes interconnects. A portion of the interconnect is included in metal layers 4 and 5 located in a region above the memory array region 310. The interconnect communicatively couples a portion of logic component 320 to a portion of logic component 330. In an exemplary embodiment, corresponding portions of metal layers 2 and 3 located in the regions above logic components 320 and 330 communicatively couple components included in logic components 320 and 330, and portions of metal layers 2 and 3 located in the region above the memory array 310 communicatively couple components included in the memory array 310.

[0036] Figure 4This is a block diagram of an internal storage system 400 according to one embodiment. The internal storage system 400 includes a CPU and controller 410, external peripheral components 320, DRAM arrays 430, 440, 450, and 470, and accelerators 481 and 482. DRAM arrays 430, 440, 450, and 470 include memory cell arrays and array peripheral components. The memory arrays include information storage cells that selectively hold characteristics or attributes corresponding to logical values ​​associated with information. The peripheral components include: a column address decoder for decoding column or X-axis addresses of the array; and a row address decoder for decoding row or Y-axis addresses of the array.

[0037] DRAM array 430 includes a memory cell array 411, a column or X-array dimension address decoder 432, and a row or Y-array dimension address decoder 433. DRAM array 440 includes a memory cell array 441, a column or X-array dimension address decoder 442, and a row or Y-array dimension address decoder 443. DRAM array 450 includes a memory cell array 451, a column or X-array dimension address decoder 452, and a row or Y-array dimension address decoder 453. DRAM array 470 includes a memory cell array 471, a column or X-array dimension address decoder 472, and a row or Y-array dimension address decoder 473.

[0038] The interconnect network may include interconnects and switches. Switches may be located in the array perimeter region. In one embodiment, a portion of the interconnects are located in the region above the array perimeter region, and a portion of the interconnects are located in the region above the memory cell array region.

[0039] The internal memory system 400 also includes an interconnect network that communicatively couples the CPU and controller 410 to accelerators 481 and 482. The interconnect network includes interconnects 491, 492, 493, 494, and 495, and switches 497 and 498. Interconnect 491 is communicatively coupled to the CPU and controller 410 and switch 497, which in turn is communicatively coupled to external peripheral components 420 and interconnect 493. Interconnect 493 is communicatively coupled to switch 497, which is communicatively coupled to interconnects 494 and 495. Interconnect 494 is communicatively coupled to arbitrator 481. Interconnect 495 is communicatively coupled to arbitrator 482.

[0040] In one embodiment, the switches in the internal storage system 400 are formed in a region that includes array peripheral components (e.g., array address decoding components, etc.). Switch 497 is located in column or X-array dimension address decoder region 432. Switch 498 is located in row or Y-array dimension address decoder region 433.

[0041] In one embodiment, interconnects may be located in a metal layer over a region including the memory array components. A portion of interconnect 491 is located in a metal layer over the array peripheral component region 432. A portion of interconnect 492 is located in a metal layer over the array peripheral component region 432, and a portion is located in a metal layer over the memory cell region 441. A portion of interconnect 493 is located in a metal layer over the array peripheral component region 432, and a portion of interconnect 493 is located in a metal layer over the aforementioned array peripheral component region 433. A corresponding portion of interconnect 494 is located in a metal layer over the array peripheral component regions 433, 453, 452, and the memory cell array 451. A corresponding portion of interconnect 495 is located in a metal layer over the array peripheral component regions 433, 453, 452, the memory cell array 471, and the array peripheral component region 472.

[0042] It should be understood that alternative configurations can also be implemented. In one embodiment, it is inconvenient to include a network switch in the array perimeter area, and the interconnects are not located above the memory array components. Figure 5 This is a block diagram of an internal memory system 500 according to one embodiment. The internal memory system 500 is similar to an internal memory system 500 except that the interconnect network lines and switches are located in areas not above the DRAM block locations. The internal memory system 500 includes a CPU and controller 510, external peripheral components 520, DRAM arrays 530, 540, 550, and 570, and accelerators 581 and 582. DRAM arrays 530, 540, 550, and 570 include memory cell arrays and array peripheral components. DRAM array 530 includes a memory cell array 511, a column or X-array dimension address decoder 532, and a row or Y-array dimension address decoder 533. DRAM array 540 includes a memory cell array 541, a column or X-array dimension address decoder 542, and a row or Y-array dimension address decoder 543. DRAM array 550 includes a memory cell array 551, a column or X-array dimension address decoder 552, and a row or Y-array dimension address decoder 553. The DRAM array 570 includes a memory cell array 571, a column or X array dimension address decoder 572, and a row or Y array dimension address decoder 573.

[0043] The internal memory system 500 also includes an interconnect network that communicatively couples the CPU and controller 510 to accelerators 581 and 582. The interconnect network includes interconnects 591, 592, 593, 594, and 595, and switches 597 and 598. Interconnect 591 is communicatively coupled to the CPU and controller 510 and switch 597, which in turn is communicatively coupled to external peripheral components 520 and interconnect 593. Interconnect 593 is communicatively coupled to switch 497, which is communicatively coupled to interconnects 594 and 595. Interconnect 594 is communicatively coupled to arbitrator 581. Interconnect 595 is communicatively coupled to arbitrator 582. Interconnects 591, 592, 593, 594, and 595 can be located in a metal layer above an area excluding the memory array components. Switches 507 and 598 can be included in areas other than the memory array region (e.g., logic component regions, etc.). In one embodiment, the interconnects are conductive traces or lines in the metal layers of a semiconductor chip.

[0044] It should be understood that the interconnect network can include various configurations. In one embodiment, the interconnect network includes interconnects and switches (e.g., similar to internal memory systems 400, 500, etc.). In one exemplary embodiment, some interconnects are included in a metal layer region above the memory array region, and some interconnects are included in other metal layer regions (e.g., not above the memory array region, etc.). Similarly, some interconnect network switches may be included in the array perimeter region, and some interconnect network switches may be included in other regions (e.g., non-memory array regions, logic component regions, etc.).

[0045] Figure 6 This is a flowchart of an internal storage fabrication method 600 according to one embodiment.

[0046] In block 610, a memory array is formed in a memory array region. In one embodiment, the memory array region includes a memory cell portion and a peripheral portion.

[0047] In block 620, a logical component is formed in the logical region. In one embodiment, the logical component includes a processing component.

[0048] In block 630, an interconnect network including interconnects is formed, wherein a portion of the interconnects is located in a metal layer region that is also located above the memory array region. Forming the interconnect network can include forming a switch. The switch can be coupled to the interconnects and can be used to route information between logical components. The switch can be located in a peripheral component portion of the memory array region. The switch can include various components (e.g., multiplexers, demultiplexers, buffers, inverters, etc.). A portion of the interconnects for the memory array can be located in metal layers 1, 2, and 3, and a portion of the interconnects for the logical components can be located in metal layers 4 and 5.

[0049] Figure 7 This is a block diagram of an internal memory system 700 according to one embodiment. The internal memory system 700 includes a CPU and controller 701, external peripheral components 702, direct memory access (DMA) 703, connectivity components 704, DRAM blocks 711, 712, 713, and 714, and accelerators 121, 122, 123, and 124. The internal memory system 700 also includes an interconnect network of on-chip interconnect components 790 (e.g., interconnects, routing switches, etc.). The on-chip interconnect components 790 are communicatively coupled to other components of the internal memory system. Compared to dedicated DRAM components, the on-chip interconnect components 790 allow accelerators to access other on-chip components with relatively high latency. In one exemplary embodiment, the on-chip interconnect components 790 allow different masters (e.g., CPU, accelerators, DMA, etc.) to access different slaves, such as DRAM and peripheral components. The on-chip interconnect components 790 may be part of an on-chip network (NoC) with different topologies (e.g., crossover switches, bus protocols, multilevel / multilayer buses / NoC, etc.). The on-chip interconnect component 790 may include an arbitrator to ensure that the accelerator has the highest priority to access the dedicated DRAM block.

[0050] The interconnect network may include a dedicated 1-2-1 interface (I / FA) between the accelerator and the DRAM block (e.g., 731, 732, 733, 734, 735, 737, 738, and 739). In one embodiment, the dedicated interface guarantees high-bandwidth and low-latency DRAM access from the corresponding accelerator. The dedicated interface can have higher priority compared to other blocks. In one exemplary embodiment, the dedicated interface (e.g., I / FA, etc.) may conform to various protocols (e.g., AXI, AHB, OCP, or even custom protocols, etc.). Another type of interface (e.g., 741, 742, 743, and 744) may be used to connect various components (e.g., CPU / control 702, DMA 703, etc.). Depending on the individual component design (e.g., APB for peripherals, AHB-lite for the CPU, etc.), other types of interfaces (e.g., I / FB) may be the same or different for different components.

[0051] The interconnect network may include interface units (IUs). The internal memory system 700 includes interface units 771, 772, 773, 774, 775, 776, 781, 782, 783, 784, 785, and 787. The IUs can be used to facilitate the coupling of various other interface components (e.g., 731, 733, 741, 757, etc.) to the on-chip interconnect component 790. In one embodiment, the interface unit converts signals from other components into internal protocols or interconnect components 790 (e.g., packet protocols for NoC or AHB-lite, pass-through, AHBIF connections to an AHB bus matrix, etc.).

[0052] Figure 8 This is a block diagram of an exemplary portion of a PIM chip 800 according to one embodiment of the present invention. The PIM chip 800 includes processing components 811, 812, and 813 in locations or regions 810 that are not part of the internal memory array region 82. In one exemplary implementation, the memory array region 820 is associated with an internal memory block. The memory array region 820 includes a memory array periphery region 830, a memory array periphery region 840, and memory array cells 821, 822, 823, and 824. In one embodiment, the memory array periphery region 830 includes row address decoding components 831 and 832, and the memory array periphery region 840 includes column address decoding components 841 and 842. The processing components 811, 812, and 813, the memory array cells 821, 822, 823, and 824, the row address decoding components 831 and 832, and the column address decoding components 841 and 842 can be formed as part of a front-end process on a production line.

[0053] The PIM chip 800 includes interconnects. In one embodiment, basic memory components memory array cells 821, 822, 823, and 824, row address decoding components 831 and 832, and column address decoding components 841 and 842 are coupled to interconnects in a lower metal layer. Row address decoding components 831 and 832 are coupled via interconnect 852. Column address decoding components 841 and 842 are coupled via interconnect 851. Interconnects 852 and 851 are located in metal layer 1. Column address decoding component 841 is communicatively coupled to memory cells 821 and 822 via interconnect 861, and column address decoding component 842 is communicatively coupled to memory cells 823 and 824 via interconnect 862. Interconnects 861 and 862 are in metal layer 2. Row address decoding component 831 is communicatively coupled to memory cells 821 and 822 via interconnect 871, and row address decoding component 832 is communicatively coupled to memory cells 823 and 824 via interconnect 872. Interconnects 871 and 872 are in metal layer 3. It should be understood that vias couple device components (e.g., processing component 811, memory array cell 821, row address decoding component 832, etc.) to metal layer 2 and above.

[0054] The PIM chip 800 also includes an interconnect network for processing components 811, 821, and 831. The interconnect network includes interconnects 881, 882, and 883, interconnects 891 and 892, vias 801 to 809, and a routing component or switch 815. Processing component 811 is coupled to switch 815 via vias 801, interconnects 881, 802, 891, and 803. Processing component 812 is coupled to switch 815 via vias 807, interconnects 882, 805, 892, and 804. Processing component 813 is coupled to switch 815 via vias 809, interconnects 883, and 808. Interconnects 891 and 892 are located in metal layer 4, and interconnects 881, 882, and 883 are located in metal layer 5. A portion of interconnects 891, 892, 881, 882, and 883 is located above memory array region 820. Switch 813 is located in column address decoding region 840.

[0055] Figure 9 This is a block diagram of an exemplary computing system 900 for storing and processing information according to one embodiment. In one embodiment, a PIM chip is included in internal memory 904 (e.g., similar to internal memory systems 100, 200, 400, 800, etc.). The computing system 900 may include a processor 902 and internal memory 904. In one implementation, the computing system 900 may be a server computer, a data center, a cloud computing system, a streaming service system, an Internet service provider system, a cellular service provider system, etc. The processor 902 may be a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a vector processor, a memory processing unit, etc., or a combination thereof. In one embodiment, the processor 902 may include communication interfaces (such as a peripheral component interface (PCIe4) 921 and an inter-integrated circuit (I2C) interface 933), an on-chip circuit tester (such as a Joint Test Action Group (JTAG) engine 923), a direct memory access engine 924, a command processor (CP) 925, and cores 931-934. The cores 931-934 can be coupled according to the directional ring bus configuration.

[0056] Still referencing Figure 9 Cores 931-934 can execute a set of computing device executable instructions to perform functions including, but not limited to, performing narrow-channel translation memory control methods. These functions can be executed on individual cores 931-934, distributed across multiple cores 931-934, or executed on a core along with other functions.

[0057] Figure 10This is a block diagram of an exemplary processing core configuration 1000 according to an embodiment of the present invention. In one embodiment, components of the processing core configuration 1000 participate in PIM operations. The processing core 1000 may include a tensor engine (TE) 1010, a pooling engine (PE) 1015, a memory copy engine (ME) 1020, a sequencer (SEQ) 1025, an instruction buffer (IB) 1030, local internal memory (LM) 1035, and a constant buffer (CB) 1040. The local internal memory 1035 can be pre-loaded with model weight values ​​and can store activation values ​​in use in real time. The constant buffer 1040 can store constants used for batch normalization, quantization, etc. The tensor engine 1010 can be used to accelerate fused convolutions and / or matrix multiplications. The pooling engine 1015 can support operations such as pooling, interpolation, and regions of interest. The internal memory copy engine 1020 can be used for data copying between cores and / or within cores, matrix transposition, etc. Tensor engine 1010, pooling engine 1015, and memory copying engine 1020 can operate in parallel. Sequencer 10210 can orchestrate the operations of tensor engine 1010, pooling engine 1015, memory copying engine 1020, local memory 10310, and constant buffer 1040 according to instructions from instruction buffer 1030. Processing core 1000 can provide efficient video encoding computation under the control of coarse-grained instructions for operation fusion. A detailed description of the exemplary processing unit core 1000 is not essential for understanding all aspects of this technology and will therefore not be described further herein.

[0058] In one embodiment, the PIM system and method can enhance the clock frequency and performance of logic blocks. Positioning the communication switch in the array periphery region and the interconnects in an upper metal layer above the memory array region achieves improved design scalability, improved routing timing, reduced routing area, and lower power consumption. The memory interconnect architecture facilitates local routing using more metal layers and has higher utilization. It should be understood that the proposed two-dimensional in-memory processing (2D PIM) architecture can be utilized in a variety of application environments, such as edge computing environments, cloud computing environments, etc.

[0059] For illustrative and descriptive purposes, the foregoing description has presented specific embodiments of the present technology. They are not intended to be exhaustive or to limit the technology to the precise forms disclosed, and it will be apparent that many modifications and variations are possible in light of the foregoing teachings. The embodiments have been chosen and described to best illustrate the principles of the present technology and its practical application, thereby enabling others skilled in the art to best utilize the present technology and various embodiments with various modifications as suited to the particular uses contemplated. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A Processing In-Memory (PIM) chip, the PIM chip comprising: A memory array for storing information, wherein the memory array includes: Memory unit, and Array peripheral components are used to control access to the memory units; Logic components for processing information stored in the memory array; and An interconnect network communicatively couples the logic components, wherein the interconnect network includes a first set of interconnects, and a portion of the first set of interconnects is located in a metal region above the memory array.

2. The PIM chip according to claim 1, wherein the interconnection network includes a switch for routing information in the interconnection network.

3. The PIM chip of claim 2, wherein the switch is included in the region having the array peripheral components.

4. The PIM chip of claim 2, wherein the switch is configured with other redundant components in the array peripheral components.

5. The PIM chip of claim 2, wherein a second set of interconnects communicatively couples the memory cell and the array peripheral components, wherein a portion of the second set of interconnects is located in metal layers 1, 2 and 3, and a portion of the first set of interconnects corresponding to the logic components is located in metal layers 4 and 5.

6. The PIM chip according to claim 2, wherein the array peripheral component comprises: A column address decoder, which is used to decode column addresses in the memory array; as well as A row address decoder is used to decode row addresses in the memory array.

7. A method for manufacturing a Process In-Memory (PIM) chip, the PIM chip manufacturing method comprising: A memory array is formed in the memory array region of the PIM chip; A logic component is formed in the logic region of the PIM chip, wherein the logic component includes a processing component for processing information stored in the memory array; as well as An interconnect network is formed to communicatively couple the logic components; the interconnect network includes interconnects, a portion of which is located in a metal layer region above the memory array region.

8. The PIM chip fabrication method of claim 7 further includes forming a switch in the interconnect network, wherein the switch is coupled to the interconnect and is used to route information between the logic components.

9. The PIM chip manufacturing method according to claim 8, wherein the memory array region includes a memory cell portion and a memory array peripheral component portion, and the switch is located in the memory array peripheral component portion of the memory array region.

10. The PIM chip manufacturing method according to claim 8, wherein the switch comprises a multiplexer and a demultiplexer.

11. The PIM chip fabrication method of claim 8, wherein a portion of the interconnects for the logic components are located in metal layers 4 and 5, and a portion of the other interconnects corresponding to the memory array are located in metal layers 1, 2 and 3.

12. A Processing In-Memory (PIM) chip, the PIM chip comprising: Internal memory blocks are used to store information; A logic component for processing information stored in the internal memory block; as well as An interconnect network for communicatively coupling the logic components, wherein the interconnect network includes interconnects, and a portion of the interconnects is located in a metal region above the memory block.

13. The PIM chip of claim 12, wherein the interconnect network includes a switch in the memory block region and the switch is used to route information in the interconnect network.

14. The PIM chip according to claim 12, wherein the logic component comprises: It handles control components to schedule tasks, configure registers, and manage global synchronization; as well as Accelerators are used to speed up application processing.

Images