Memory device and method of manufacturing the same

Abstract

A memory device may include a memory array, a first computing unit, and a second computing unit. The memory array may include a plurality of memory cells for storing weights of the neural network. The first computing unit may be configured to receive the stored weights from the plurality of storage units and generate a first partial sum according to the stored weights. The second computing unit may be configured to receive the stored weights and the first partial sum from the plurality of storage units, and generate a second partial sum according to the stored weights and the first partial sum. The second computing units may be sequentially connected to the first computing units. The embodiment of the invention also discloses a method for manufacturing the memory device.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of semiconductors, and more particularly, to memory devices and methods of manufacturing the same. Background Art

[0002] Memory devices are components of electronic systems that store data in a way that allows for rapid access and modification. Traditionally, memory devices are designed to store binary information in the form of "0"s and "1"s in a large number of memory cells. Due to manufacturing variations and design limitations, these cells often exhibit an unbalanced physical structure that results in differences in their electrical characteristics. Computing-in-memory (CIM) technology integrates processing power directly into the memory array, enabling faster data computation by reducing the distance data must travel between storage and processing units. Summary of the invention

[0003] An embodiment of the present disclosure provides a storage device, comprising: a storage array including a plurality of storage units for storing weights of a neural network; a first computing unit configured to receive the stored weights from the plurality of storage units and generate a first partial sum based on the stored weights; and a second computing unit configured to receive the stored weights and the first partial sum from the plurality of storage units and generate a second partial sum based on the stored weights, wherein the second computing unit is sequentially connected to the first computing unit.

[0004] Yet another embodiment of the present disclosure provides a memory device, comprising: a substrate; a memory cell array having a plurality of memory cell sub-arrays formed on the substrate and configured to store weights of a neural network; a first local accumulator formed on a first metal layer and configured to receive the stored weights from the memory cell sub-array and to generate a first partial sum based on the stored weights; a second local accumulator formed on the first metal layer and configured to receive the stored weights and the first partial sum from the memory cell sub-array and to generate a second partial sum based on the stored weights, wherein the second local accumulator is sequentially connected to the first local accumulator on the first metal layer.

[0005] Another embodiment of the present disclosure provides a method for manufacturing a memory device, comprising: providing a substrate; forming a memory cell array having a plurality of memory cell sub-arrays on a front side of the substrate; forming a first local accumulator on a first metal layer on the front side of the substrate; and forming a second local accumulator on the first metal layer, wherein the first local accumulator is sequentially connected to the second local accumulator on the first metal layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various aspects of the present invention will be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be noted that, in accordance with standard practice in the industry, the various components are not drawn to scale. In fact, the sizes of the various components may be arbitrarily increased or reduced for clarity of discussion.

[0007] Figure 1 A block diagram of a memory device 100 according to some embodiments of the present disclosure is shown.

[0008] Figure 2 Some embodiments of the present disclosure are shown Figure 1 Detailed schematic diagram of the memory device 100.

[0009] Figure 3 Some embodiments of the present disclosure are shown Figure 1 Detailed schematic diagram of the memory device 100.

[0010] Figure 4 Some embodiments of the present disclosure are shown Figure 2 Detailed schematic diagram of the memory device 100.

[0011] Figure 5 Some embodiments of the present disclosure are shown Figure 3 Detailed schematic diagram of the memory device 100.

[0012] Figure 6 According to some embodiments of the present disclosure, Figure 1 Flowchart of an example method of the memory device 100 .

[0013] Figure 7 is a flow chart of an example method for manufacturing the memory device 100 according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] The present invention provides many different embodiments or examples for realizing the different features of the present disclosure. Specific examples of components and arrangements are described below to simplify the present invention. Of course, these are merely examples and are not intended to limit the present invention. For example, in the following description, forming a first component above or on a second component may include an embodiment in which the first component and the second component are formed in direct contact, and may also include an embodiment in which an additional component may be formed between the first component and the second component so that the first component and the second component may not be in direct contact. In addition, the present invention may repeat reference numerals and / or characters in various examples. This repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or configurations discussed.

[0015] Furthermore, for ease of description, spatially relative terms such as "below," "beneath," "lower," "above," "upper," etc. may be used herein to describe the relationship of one element or component to another (or additional) elements or components as illustrated in the figures. The spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein should likewise be interpreted accordingly.

[0016] In traditional floorplanning of compute-in-memory (CIM) macros, design layout often poses significant challenges for the routing of accumulators (e.g., global and local accumulators). Such layouts often result in severe routing congestion as the electrical connection paths between various components become overly complex and intertwined. Congestion not only hinders signal integrity and may increase crosstalk between circuits, but also leads to inefficient usage of metal layers. This inefficiency increases the need for a higher number of metal layers or more complex metal stack configurations to accommodate dense routing requirements. This not only complicates the manufacturing process, but also increases production costs and may negatively impact the overall performance and scalability of the CIM system. Therefore, optimizing floorplanning to alleviate routing congestion and reduce metal layer usage is a key design consideration for improving CIM architectures.

[0017] In order to enhance the design of computing-in-memory (CIM) circuits, it is critical to address the challenge of wiring congestion, which can be alleviated by adopting a more streamlined layout plan that improves efficient routability. By reorganizing the layout, the path of signal transmission can be simplified, reducing the complexity and overlap of routes that cause congestion. In addition, this optimization work aims to reduce the reliance on multiple metal layers for local and global wiring, thereby not only simplifying the manufacturing process but also potentially reducing the physical thickness and manufacturing cost of the chip. The careful balance between circuit density and wiring simplicity can lead to a more scalable and cost-effective CIM design while significantly suppressing the number of metal layers required. The present disclosure not only alleviates wiring congestion, but also improves the overall performance and reliability of the CIM architecture.

[0018] Routability in the context of integrated circuit (IC) design refers to the ease with which electrical connections can be successfully and efficiently made between the various components on a chip during the layout phase. It measures the ease and effectiveness with which metal lines (also called "wiring") can be placed within the layers of an IC without causing signal integrity issues, manufacturing problems, or violating design rules. For example, a design with high routability means that there is enough space for all necessary connections with minimal risk of shorts, crosstalk, or other problems when routing is too dense or poorly organized. Factors that affect routability include the number of tracks within a metal layer, the complexity of the circuit, the number of layers available for routing, the precision of the manufacturing process, and the effectiveness of the design tools used to create the layout. Improving routability is critical to ensuring that semiconductor chips can be reliably manufactured at scale and to optimizing their performance and power efficiency. This is particularly important as semiconductor technology continues to advance and memory circuits become more complex and dense.

[0019] In a standard floorplan for compute-in-memory (CIM) macros, local accumulators are strategically placed between CIM bodies to facilitate multiply-accumulate (MAC) operations, which are the basis of the macro’s computational tasks. Despite the presence of positioning logic, these local accumulators operate by performing parallel accumulations, which can lead to a significant challenge: routing congestion. This congestion occurs because multiple parallel signals need to converge on the local accumulators, resulting in a dense and complex network of interconnects. As a result, a higher number of metal layers is often required to accommodate all the necessary routing paths. For example, the design may require up to 84 tracks for some routing and up to 70 tracks for others, illustrating the extensive use of metal layers required to maintain signal integrity and functionality in these dense routing environments. This not only increases the complexity of the chip design, but also affects the manufacturing process, potentially leading to higher costs and scalability issues.

[0020] The present disclosure provides various embodiments of memory devices that address these problems (e.g., routability). For example, as disclosed herein, the memory device includes a memory array, a first computing unit, and a second computing unit. The second computing unit is sequentially connected to the first computing unit. In some embodiments, the overall routing track can be optimized to 36 tracks using a fully sequential layout plan, and fewer metal layers can be used. In some embodiments, the overall routing track can be suppressed to 52 tracks using a hybrid (e.g., partially sequential and partially parallel layout plan) layout plan, and fewer metal layers can be used.

[0021] Figure 11 shows a block diagram of a memory device 100 according to some embodiments of the present disclosure. The memory device 100 may include a memory array 120, a first computing unit 140, and a second computing unit 160. In some embodiments, the memory device 100 may include a memory array 120, a first computing unit 140, a second computing unit 160, and a global computing unit 180.

[0022] The memory array 120 may include a plurality of memory cells. The plurality of memory cells may store the weights of a neural network. One or more peripheral circuits (not shown) may be located in one or more regions peripheral to or internal to the memory array 120. The memory cells and the peripheral circuits may be connected via word lines and / or complementary bit lines BL and BLB, and data may be read and written from the memory bit cells via the complementary bit lines BL and BLB. Different voltage combinations applied to the word lines and bit lines may define read, erase, or write (program) operations to the memory bit cells. In some embodiments, the memory array 120 architecture may incorporate various types of non-volatile or volatile memory technologies, including but not limited to static random access memory (SRAM), resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), and phase change random access memory (PCRAM).

[0023] Deep learning uses neural networks to achieve artificial intelligence. These networks consist of many interconnected processing nodes that facilitate machine learning by analyzing example data. For example, a system designed to recognize objects: it might process thousands of images of objects, such as trucks, to discern and learn the visual patterns that correspond to objects in new images. The structure of a neural network is usually layered, and data flows through the layers in a single forward direction. Each node in the network can have connections to multiple nodes in subsequent layers to which it sends data, as well as connections to multiple nodes in the previous layer from which it receives data.

[0024] In a neural network, nodes assign numerical values, called "weights," to their connections. Once activated, a node can multiply incoming data by this weight and sum the products of all its connections to produce a single digital output. If the output is below a certain threshold, the node can prevent it from entering the next layer. Conversely, if the output exceeds the threshold, the node can transmit the sum to the connected nodes in the next layer. In a deep learning system, the neural network model is stored in a memory, and computational logic in the processor performs a multiply-accumulate (MAC) calculation on the parameters (e.g., weights) stored in the memory. In some embodiments, the weights can be stored in multiple memory cells within the memory array 120.

[0025] In some embodiments, the first computation unit 140 may be configured to receive a plurality of inputs 122a, 122b from a plurality of storage cells 120. The plurality of inputs 122a, 122b may include at least one of storage weights and input activation vector elements from the plurality of storage cells 120. In some embodiments, the first computation unit 140 may be at least one of a local accumulator, a full adder, a half adder, a sum register, a partial sum register, and an accumulation circuit. In some embodiments, the weights (W) or the input activation vector elements may be stored in a subarray of the storage array 120. Each output of the subarray may be an input to the computation unit 140. For example, the first computation unit 140 may be configured to receive stored weights from a plurality of storage cells 120. In some embodiments, the first computation unit 140 is interposed between the subarrays of the storage cells 120. The first computation unit 140 and the subarrays may be connected on a plurality of local bit lines. In some embodiments, at least one additional computation unit may be interposed between the storage array 120 and the first computation unit 140. In some embodiments, the first computation unit 140 may be configured to generate the first partial sum 142 based on the stored weights 122a, 122b. In some embodiments, the first computation unit 140 may be configured to generate the first partial sum 142 based on the stored weights and the partial sum from at least one additional computation unit.

[0026] In some embodiments, the second calculation unit 160 may be configured to receive a plurality of inputs 122c, 122d from a plurality of storage cells 120. The second calculation unit 160 may be configured to receive a first partial sum 142 from the first calculation unit 140. The plurality of inputs 122c, 122d may include at least one of storage weights and input activation vector elements from the plurality of storage cells 120. In some embodiments, the second calculation unit 160 may be at least one of a local accumulator, a full adder, a half adder, a sum register, a partial sum register, and an accumulation circuit. In some embodiments, the weights (W) or the input activation vector elements may be stored in a subarray of the storage array 120. Each output of the subarray may be an input to the calculation unit 160. For example, the second calculation unit 160 may be configured to receive the stored weights from the plurality of storage cells 120. In some embodiments, the second calculation unit 160 is interposed between the subarrays of the storage cells 120. The second calculation unit 160 and the subarrays may be connected to a plurality of local bit lines. In some embodiments, at least one additional computation unit may be interposed between the storage array 120 and the second computation unit 160. In some embodiments, the second computation unit 160 may be configured to generate the second partial sum 162 based on the stored weights 122c, 122d and the first partial sum 142. In some embodiments, the second computation unit 160 may be configured to generate the second partial sum 162 based on the stored weights, the first partial sum 142, and the partial sum from the at least one additional computation unit.

[0027] In some embodiments, the first calculation unit 140 may generate the first partial sum 142 by multiplying the input activation vector elements with the weights stored in the subarray of the memory cell 120. The second calculation unit 160 may generate the second partial sum 162 by multiplying the input activation vector elements with the weights stored in the subarray of the memory cell 120. In some embodiments, each of the plurality of memory cells 120 may include a plurality of word lines. By accessing the subarray of the memory cell via the plurality of word lines, the product of the input activation vector elements on the plurality of word lines and the weights stored in the plurality of memory cells may be calculated.

[0028] In some embodiments, the second computing unit 160 can be connected sequentially to the first computing unit 140. In some embodiments, the first computing unit 140 and the second computing unit 160 can be connected sequentially in the same metallization / metal layer. In some embodiments, the first computing unit 140 can be directly interconnected to the second computing unit 160.

[0029] In some embodiments, the memory device 100 may employ a fully sequential floorplan. By employing a fully sequential floorplan in the design of the memory device, routing tracks may be used more optimally. By arranging the components in sequence, the total number of routing tracks required may be significantly reduced. This simplified approach facilitates a more organized, less crowded routing scheme, thereby allowing fewer metal layers to be used in the circuit. Additionally, due to fewer layers, the circuit length is shortened, which may improve signal integrity and may increase the operating speed of the circuit.

[0030] In some embodiments, the memory device 100 can implement a hybrid layout plan (e.g., a partially sequential layout plan and a partially parallel layout plan). By implementing a hybrid layout plan that combines sequential and parallel elements in an integrated circuit design, substantial improvements can be provided in terms of routability and layer usage. By selectively adopting sequential routing when feasible and parallel routing when necessary, the total number of routing tracks can be effectively reduced. This balanced approach alleviates the routing congestion typically associated with parallel design, while still maintaining the design compactness that a pure sequential layout plan may lack. As a result, the layout requires fewer metal layers, which can significantly reduce manufacturing complexity and cost. Therefore, this hybrid layout provides an effective way to optimize the wiring infrastructure of a chip, helping to simplify the manufacturing process and improve overall circuit performance.

[0031] In some embodiments, the global computing unit 180 may be configured to accumulate at least one partial sum 182 (e.g., the first partial sum 142 and / or the second partial sum 162) of the multiplication from the first computing unit 140 and the second computing unit 160. In some embodiments, the global computing unit 180 may be at least one of a global accumulator, a full adder, a half adder, a sum register, a partial sum register, and an accumulation circuit. In some embodiments, at least one additional computing unit may be between the global computing unit 180 and the second computing unit 160. In some embodiments, the global computing unit 180 may be sequentially connected to the first computing unit 140, the second computing unit 160, and the at least one additional computing unit.

[0032] Figure 2 Some embodiments of the present disclosure are shown Figure 1 Detailed schematic diagram of the memory device 100. Figure 4 Some embodiments of the present disclosure are shown Figure 2Detailed schematic diagram of a memory device 100 of the present invention. The memory device 100 may include a memory array 120, a first computing unit 140, a second computing unit 160, and a third computing unit 220. In some embodiments, the memory device 100 may include a memory array 120, a first computing unit 140, a second computing unit 160, a third computing unit 220, and a global computing unit 180. Figure 2 and Figure 4 The memory device 100 and Figure 1 For the sake of brevity, the specific operations of similar components that have been discussed in detail in the above paragraphs are omitted herein unless they are required to be introduced in conjunction with Figure 2 and Figure 4 The collaborative relationship of the components shown in Figure 2 and Figure 4 In FIG. 1 , it is assumed that 16 partial sums are accumulated from 16 CIM banks, each partial sum being 16 bits.

[0033] In some embodiments, the third calculation unit 220 may be configured to receive a plurality of inputs 122e, 122f from a plurality of storage cells 120. The third calculation unit 220 may be configured to receive a second partial sum 162 from the second calculation unit 160. The plurality of inputs 122e, 122f may include at least one of storage weights and input activation vector elements from a plurality of storage cells 120. In some embodiments, the third calculation unit 220 may be at least one of a local accumulator, a full adder, a half adder, a sum register, a partial sum register, and an accumulation circuit. In some embodiments, the weights (W) or the input activation vector elements may be stored in a subarray of the storage array 120. Each output of the subarray may be an input to the calculation unit 220. For example, the third calculation unit 220 may be configured to receive the stored weights from a plurality of storage cells 120. In some embodiments, the third calculation unit 220 is interposed between the subarrays of the storage cells 120. The third calculation unit 220 and the subarrays may be connected to a plurality of local bit lines. In some embodiments, at least one additional computation unit may be interposed between the storage array 120 and the third computation unit 220. In some embodiments, the third computation unit 220 may be configured to generate the third partial sum 222 based on the stored weights 122e, 122f and the second partial sum 162. In some embodiments, the third computation unit 220 may be configured to generate the third partial sum 222 based on the stored weights, the second partial sum 162, and the partial sums from the at least one additional computation unit.

[0034] In some embodiments, the third computing unit 220 can be connected to the first computing unit 140 and the second computing unit 160 sequentially. In some embodiments, the third computing unit 220, the first computing unit 140 and the second computing unit 160 can be connected sequentially in the same metallization / metal layer. In some embodiments, the third computing unit 220 can be directly interconnected to the second computing unit 160. In some embodiments, the memory device 100 can adopt a fully sequential layout plan (e.g., 140, 160, and 220). By adopting a fully sequential layout plan in the design of the memory device, the wiring tracks can be used more optimally. By arranging the components in sequence, the total number of wiring tracks required can be significantly reduced (e.g., 36 wiring tracks). This simplified approach helps to achieve a more organized and less crowded wiring scheme, allowing fewer metal layers to be used in the circuit. In addition, due to the fewer layers, the circuit length is shortened, which can improve signal integrity and may increase the operating speed of the circuit.

[0035] like Figure 2 and Figure 4As shown, the first storage unit 120a (e.g., CIM 0) can transmit / transmit the neural network data to the first local accumulator 140 using 16 wiring tracks. The second storage unit 120b (e.g., CIM 1) can transmit / transmit the neural network data to the first local accumulator 140 using 16 wiring tracks. The first local accumulator 140 can receive / compile the neural network data from the first storage unit 120a and the second storage unit 120b. The first local accumulator 140 can generate a first partial sum 142 and send the first partial sum 141 to the second local accumulator 160 by using 17 wiring tracks. The third storage unit 120c (e.g., CIM 2) can transmit / transmit the neural network data to the second local accumulator 160 using 16 wiring tracks. The fourth storage unit 120d (e.g., CIM 3) can transmit / transmit the neural network data to the second local accumulator 160 using 16 wiring tracks. The second local accumulator 160 can receive the neural network data from the third storage unit 120c and the fourth storage unit 120d. The second local accumulator 160 can generate a second partial sum 162. The second local accumulator 160 can send the second partial sum 162 to a subsequent component (e.g., a third local accumulator 200) by using 18 routing tracks. The third local accumulator 220 can receive neural network data from a storage unit. The third local accumulator 220 can generate a third partial sum 222. The third local accumulator 220 can send the third partial sum 222 to a subsequent component by using 19 routing tracks. In this case, for CIM 7 210, the maximum number of routing tracks in the system is 36 routing tracks. Compared to a conventional wiring layout planning design using up to 84 routing tracks, the present disclosure provides a memory device that reduces wiring congestion and minimizes metal layer usage.

[0036] In some embodiments, the first computing unit 140 can be sequentially connected to the second computing unit 160 by using routing tracks within the same metallization layer (e.g., metal layer N). In some embodiments, multiple metal layers (e.g., metal layers N+1, N+2) can be used. The first storage unit 120a and the second storage unit 120b can be in the same metal layer. The first storage unit 120a and the first computing unit 140 can be located in different metallization layers. For example, the first storage unit 120a and the second storage unit 120b can be formed in metal layer N. The first computing unit 140, the second computing unit 160, and the third computing unit 222 can be formed in material layer N+1.

[0037] Figure 3 Some embodiments of the present disclosure are shown Figure 1 Detailed schematic diagram of the memory device 100. Figure 5 Some embodiments of the present disclosure are shown Figure 3 Detailed schematic diagram of a memory device 100 of FIG. 1 . The memory device 100 may include a memory array 120, a first computing unit 140, a second computing unit 160, a fourth computing unit 320, and a fifth computing unit 340. In some embodiments, the memory device 100 may include a memory array 120, a first computing unit 140, a second computing unit 160, a fourth computing unit 320, a fifth computing unit 340, and a global computing unit 180. Figure 3 and Figure 5 The memory device 100 and Figure 1 For the sake of brevity, the specific operations of similar components that have been discussed in detail in the above paragraphs are omitted herein unless they are required to be introduced in conjunction with Figure 3 and Figure 5 The collaborative relationship of the components shown in Figure 3 and Figure 5 In FIG. 1 , it is assumed that 16 partial sums are accumulated from 16 CIM banks, each partial sum being 16 bits.

[0038] In some embodiments, the fourth computation unit 320 may be configured to receive a plurality of inputs 122a, 122b from a plurality of storage cells 120. The plurality of inputs 122a, 122b may include at least one of storage weights and input activation vector elements from the plurality of storage cells 120. In some embodiments, the fourth computation unit 320 may be at least one of a local accumulator, a full adder, a half adder, a sum register, a partial sum register, and an accumulation circuit. In some embodiments, the weights (W) or the input activation vector elements may be stored in a subarray of the storage array 120. Each output of the subarray may be an input to the computation unit 320. For example, the fourth computation unit 320 may be configured to receive stored weights from a plurality of storage cells 120. In some embodiments, the fourth computation unit 320 is interposed between the subarrays of the storage cells 120. The fourth computation unit 320 and the subarrays may be connected on a plurality of local bit lines. In some embodiments, at least one additional computation unit may be interposed between the storage array 120 and the fourth computation unit 320. In some embodiments, the fourth calculation unit 320 may be configured to generate a fourth partial sum 322 based on the stored weights 122a, 122b. In some embodiments, the fourth calculation unit 320 may be configured to generate a fourth partial sum 322 based on the stored weights and the partial sum from at least one additional calculation unit.

[0039] In some embodiments, the fifth computation unit 340 may be configured to receive a plurality of inputs 122c, 122d from a plurality of storage cells 120. The plurality of inputs 122c, 122d may include at least one of storage weights and input activation vector elements from the plurality of storage cells 120. In some embodiments, the fifth computation unit 340 may be at least one of a local accumulator, a full adder, a half adder, a sum register, a partial sum register, and an accumulation circuit. In some embodiments, the weights (W) or the input activation vector elements may be stored in each subarray of the storage array 120. Each output of the subarray may be an input to the computation unit 340. For example, the fifth computation unit 340 may be configured to receive the stored weights from the plurality of storage cells 120. In some embodiments, the fifth computation unit 340 is interposed between each subarray of the storage cell 120. The fifth computation unit 340 and the subarray may be connected on a plurality of local bit lines. In some embodiments, at least one additional computation unit may be interposed between the storage array 120 and the fifth computation unit 340. In some embodiments, the fifth calculation unit 340 may be configured to generate the fifth partial sum 342 based on the stored weights 122c, 122d. In some embodiments, the fifth calculation unit 340 may be configured to generate the fifth partial sum 342 based on the stored weights and the partial sums from at least one additional calculation unit.

[0040] In some embodiments, the first computing unit 140 may be connected in parallel to the fourth computing unit 320 and the fifth computing unit 340 to receive the fourth portion 322 and the fifth portion 342. The first computing unit 140 may receive the fourth portion 322 from the fourth computing unit 320 and the fifth portion 342 from the fifth computing unit 340. The first computing unit 140 may generate the first portion 142 based on the fourth portion 322 and the fifth portion 342. In some embodiments, the fourth computing unit 320 and the first computing unit 140 may be located in different metallization / metal layers. The fourth computing unit 322 and the fifth computing unit 340 may be in the same metallization / metal layer. In some embodiments, the fourth computing unit 320 may be interconnected with the first computing unit 140 using a through-hole structure 360. The fifth computing unit 340 may be interconnected to the first computing unit 140 via the through-hole structure 360.

[0041] In some embodiments, the second computing unit 160 may be configured to receive a plurality of inputs 122e, 122f, 122g, 122h from a plurality of storage units 120. The second computing unit 160 may be configured to receive a first partial sum 142 from the first computing unit 140. The plurality of inputs 122e, 122f, 122g, 122h may include at least one of a storage weight and an input activation vector element from a plurality of storage units 120. In some embodiments, the second computing unit 160 may be at least one of a local accumulator, a full adder, a half adder, a sum register, a partial sum register, and an accumulation circuit. In some embodiments, the weight (W) or the input activation vector element may be stored in a subarray of the storage array 120. Each output of the subarray may be an input to the computing unit 160. For example, the second computing unit 160 may be configured to receive the stored weights 122e, 122f, 122g, 122h from a plurality of storage units 120. In some embodiments, the second computing unit 160 is interposed between the subarrays of the storage unit 120. The second computation unit 160 and the sub-array may be connected on a plurality of local bit lines. In some embodiments, at least one additional computation unit is interposed between the memory array 120 and the second computation unit 160. In some embodiments, the second computation unit 160 may be configured to generate the second partial sum 162 based on the stored weights 122e, 122f, 122g, 122h and the first partial sum 142. In some embodiments, the second computation unit 160 may be configured to generate the second partial sum 162 based on the stored weights, the first partial sum 142, and the partial sum from at least one additional computation unit.

[0042] In some embodiments, second computing unit 160 may be sequentially connected to first computing unit 140. In some embodiments, first computing unit 140 and second computing unit 160 may be sequentially connected within the same metallization / metal layer (e.g., metal layer N, N+1, or N+2). In some embodiments, first computing unit 140 may be directly interconnected to second computing unit 160.

[0043] In some embodiments, the memory device 100 can implement a hybrid layout plan (e.g., a partially sequential layout plan and a partially parallel layout plan). By implementing a hybrid layout plan that combines sequential (e.g., the fourth computing unit 320, the fifth computing unit 340, and the first computing unit 140) and parallel (e.g., the first computing unit 40 and the second computing unit 160) elements in an integrated circuit design, substantial improvements can be provided in terms of routability and layer usage. By selectively adopting sequential routing when feasible and adopting parallel routing when necessary, the total number of routing tracks (e.g., 52 routing tracks) can be effectively reduced. This balanced approach alleviates the routing congestion typically associated with parallel design while still maintaining the design compactness that a pure sequential layout plan may lack. As a result, a layout requiring fewer metal layers can significantly reduce manufacturing complexity and cost. Therefore, this hybrid layout provides an effective method to optimize the routing infrastructure of a chip, helping to simplify the manufacturing process and improve overall circuit performance.

[0044] like Figure 3 and Figure 5As shown, the first storage unit 120a (e.g., CIM 0) can transmit / transmit the neural network data to the fourth local accumulator 320 using 16 wiring tracks. The second storage unit 120b (e.g., CIM 1) can transmit / transmit the neural network data to the fourth local accumulator 320 using 16 wiring tracks. The fourth local accumulator 320 can receive / compile the neural network data from the first storage unit 120a and the second storage unit 120b. The fourth local accumulator 320 can generate a fourth partial sum 322 and send the fourth partial sum 322 to the first local accumulator 140 by using 17 wiring tracks in different metal layers. The third storage unit 120c (e.g., CIM 2) can transmit / transmit the neural network data to the fifth local accumulator 340 using 16 wiring tracks. The fifth storage unit 120d (e.g., CIM 3) can transmit / transmit the neural network data to the fifth local accumulator 340 using 16 wiring tracks. The fifth local accumulator 340 may receive the neural network data from the third storage unit 120c and the fourth storage unit 120d. The fifth local accumulator 340 may generate a fifth partial sum 342. The fifth local accumulator 342 may send the fifth partial sum 342 to the first computing unit 140 by using 17 wiring tracks. The first local accumulator 140 may receive the neural network data from the storage unit 120. The first local accumulator 140 may generate a first partial sum 142 based on the fourth partial sum 322 and the fifth partial sum 342. The first local accumulator 140 may send the first partial sum 142 to a subsequent element (e.g., the second computing unit 160) by using 18 wiring tracks. The second local accumulator 160 may receive the first partial sum 142 and the neural network data from the storage unit. The second local accumulator 160 may generate a second partial sum 162. The second local accumulator 160 may send the second partial sum 162 to a subsequent component (e.g., the global accumulator 180) by using 19 wiring tracks. In this case, the highest number of routing tracks in the system is 52 routing tracks for CIM 6 310. Compared to conventional routing floorplanning designs using up to 84 routing tracks, the present disclosure provides a memory device that reduces routing congestion and minimizes metal layer usage.

[0045] In some embodiments, the first computing unit 140 may be connected to the fourth computing unit 320 and the fifth computing unit 340 in parallel by using a wiring track and a via structure 360. In some embodiments, the fourth computing unit 320 and the first computing unit 140 may be located in different metallization layers (e.g., metal layers N, N+1, and N+2). The fourth computing unit 320 and the fifth computing unit 340 may be in the same metallization layer (e.g., metal layer N). In some embodiments, the first computing unit 140, the fourth computing unit 320, and the fifth computing unit 340 may all be in the same metallization layer (e.g., metal layer N). In some embodiments, the first computing unit 140 may be connected to the second computing unit 160 sequentially by using a wiring track within the same metallization layer (e.g., metal layer N). In some embodiments, multiple metal layers (e.g., metal layers N+1, N+2) may be utilized. The first storage unit 120a and the second storage unit 120b may be in the same metal layer. The first storage unit 120a and the fourth computing unit 320 (or the fifth computing unit 340) may be located in different metallization layers. For example, the first storage unit 120a and the second storage unit 120b may be formed in metal layer N. The fourth computing unit 320 and the fifth computing unit 340 may be formed in metal layer N+1. The first computing unit 140 and the second computing unit 160 may be formed in metal layer N+2.

[0046] Figure 6 According to some embodiments of the present disclosure, Figure 1 600 of an example method of the memory device 100. It should be understood that in order to better understand the concepts of the present disclosure, Figure 6 is simplified. Therefore, it should be noted that Figure 6 Additional processes may be provided before, during, and after the methods, and some other processes may only be briefly described herein.

[0047] Reference now Figure 6 , operation 602 may provide a substrate. The substrate may have a front side and a back side opposite to each other. The substrate may be a semiconductor substrate, such as a bulk semiconductor, a semiconductor on insulator (SOI) substrate, etc., which may be doped (e.g., with a p-type or n-type dopant) or undoped.

[0048] Next, the method 600 proceeds to operation 604, i.e., forming a memory cell array 120 having a plurality of sub-memory cell arrays on the front side of the substrate. Each sub-array of the memory cell can be arranged adjacent to each other. In some embodiments, the sub-arrays of the memory array 120 can store weights (W) or input activation vector elements of the neural network. Each output of the sub-array can be an input to a computing unit.

[0049] Next, the method 600 proceeds to operation 606 to form a first local accumulator 140 on the first metal layer on the front side of the substrate. The first local accumulator 140 can generate a first partial sum 142 by multiplying the input activation vector element with the weight stored in the subarray of the memory cell 120.

[0050] Next, the method 600 proceeds to operation 608, and a second local accumulator 160 is formed on the first metal layer. The first calculation unit 140 may be at least one of a local accumulator, a full adder, a half adder, a summation register, a partial summation register, and an accumulation circuit. The first local accumulator 140 may be sequentially connected to the second local accumulator 160 on the first metal layer. The second calculation unit 160 may be at least one of a local accumulator, a full adder, a half adder, a summation register, a partial summation register, and an accumulation circuit. In some embodiments, the first calculation unit 140 may be directly interconnected to the second calculation unit 160. The second calculation unit 160 may be configured to receive multiple inputs from multiple storage units 120. The second calculation unit 160 may be configured to receive a first partial sum 142 from the first calculation unit 140. In some embodiments, the second calculation unit 160 may be configured to generate a second partial sum based on the stored weight and the first partial sum.

[0051] Next, method 600 may proceed to form a third local accumulator on the first metal layer. The third local accumulator may be sequentially connected to the first local accumulator and the second local accumulator on the first metal layer. The third computing unit may be at least one of a local accumulator, a full adder, a half adder, a summing register, a partial summing register, and an accumulation circuit. In some embodiments, the third computing unit may be sequentially connected to the first computing unit 140 and the second computing unit 160. In some embodiments, the third computing unit may be connected to the first computing unit 140 in parallel.

[0052] In some embodiments, method 600 may continue with the operation of forming an interconnect structure on the back side of the substrate. The interconnect structure may be connected to the memory cell array 120, the first local accumulator 140, and the second local accumulator 160 on the front side of the substrate. There may be a plurality of first via structures extending vertically from the back side of the substrate through the substrate to the front side. In some embodiments, the memory cell array 120, the first local accumulator 140, and the second local accumulator 160 may also be formed on the back side of the substrate. The first local accumulator 140 may be sequentially connected to the second local accumulator 160 within the interconnect structure.

[0053] Figure 7 700 is a flowchart of an example method 700 for manufacturing a memory device 100 according to some embodiments of the present disclosure. It should be understood that in order to better understand the concepts of the present disclosure, Figure 7 is simplified. Therefore, it should be noted that Figure 7 Additional processes may be provided before, during, and after the methods, and some other processes may only be briefly described herein.

[0054] Reference now Figure 7 , operation 705 may provide a substrate. The substrate may have a first side and a second side opposite to each other. The substrate may be a semiconductor substrate, such as a bulk semiconductor, a semiconductor on insulator (SOI) substrate, etc., and may be doped (e.g., with a p-type or n-type dopant) or undoped.

[0055] Next, the method 700 proceeds to operation 710, forming a plurality of first transistors and a second transistor on the first side of the substrate. The plurality of first transistors may be hardware components for storing data. Each of the plurality of first transistors may have p-type conductivity or n-type conductivity. In one aspect, the plurality of first transistors may be embodied as a semiconductor memory device or a memory cell array 120. In some embodiments, the second transistor may be used for a header device on the first side of the substrate. The second transistor may have p-type conductivity or n-type conductivity.

[0056] Next, the method 700 proceeds to operation 715, forming a metal structure (e.g., word lines and / or bit lines, interconnects) on the first side of the substrate. A plurality of first transistors and metal structures (e.g., word lines and / or bit lines) may be used for a plurality of memory cells 120 on the first side of the substrate. In some embodiments, a conductor structure may be formed on the first side of the substrate. The conductor structure may be configured to pass a power supply voltage to the first transistor. A plurality of via structures may be formed on the first side of the substrate. The via structure may be configured to electrically connect the source / drain terminal of the transistor to the conductor structure. In some embodiments, the first local accumulator 140 and the second local accumulator 160 may also be formed within the metal structure. The first local accumulator 140 may be sequentially connected to the second local accumulator 160 within the interconnect structure.

[0057] In the present disclosure, accumulators in a computation-in-memory (CIM) circuit are strategically placed in a sequential floorplanning / layout, enabling efficient aggregation of partial sums (PSUM). This methodical arrangement helps to ease routing congestion by simplifying the paths that signals must take, thereby reducing the complexity of the network. In addition, it reduces the reliance on multiple metal layers, facilitates a simpler, more cost-effective manufacturing process, and potentially improves signal integrity by shortening interconnect lengths.

[0058] The present disclosure also introduces a hybrid layout planning approach that merges sequential and parallel layout of accumulators that collect PSUMs. This design still aims to alleviate routing congestion by combining the advantages of a sequential system (reduced complexity and metal usage) with the higher connectivity of a parallel configuration. This hybrid model provides a balanced solution that can adapt to different design constraints and performance requirements, providing a general framework that maintains signal integrity and reduces crosstalk even in dense circuit architectures.

[0059] As used herein, the terms "approximately" and "approximately" generally refer to a value of a given quantity that may vary depending on a particular technology node associated with the subject semiconductor device. Based on the particular technology node, the term "approximately" may refer to a value of a given quantity that varies, for example, within a range of 10% to 30% of a value (e.g., +10%, ±20%, or ±30% of a value).

[0060] An embodiment of the present disclosure provides a storage device, comprising: a storage array including a plurality of storage units for storing weights of a neural network; a first computing unit configured to receive the stored weights from the plurality of storage units and generate a first partial sum based on the stored weights; and a second computing unit configured to receive the stored weights and the first partial sum from the plurality of storage units and generate a second partial sum based on the stored weights, wherein the second computing unit is sequentially connected to the first computing unit.

[0061] In some embodiments, the first computing unit and the second computing unit are located in the same metallization layer.

[0062] In some embodiments, the first computing unit is directly interconnected to the second computing unit.

[0063] In some embodiments, the storage device includes: a third computing unit configured to receive the stored weights and the second partial sum from the multiple storage units, and generate a third partial sum based on the stored weights, wherein the third computing unit is sequentially connected to the first computing unit and the second computing unit.

[0064] In some embodiments, the third computing unit and the first computing unit are located in the same metallization layer.

[0065] In some embodiments, the storage device includes: a fourth computing unit configured to receive the stored weights from the multiple storage units and generate a fourth partial sum based on the stored weights; a fifth computing unit configured to receive the stored weights from the multiple storage units and generate a fifth partial sum based on the stored weights, wherein the first computing unit is connected to the fourth computing unit and the fifth computing unit in parallel to receive the fourth partial sum and the fifth partial sum and generate the first partial sum.

[0066] In some embodiments, the fourth computing unit and the first computing unit are located in different metallization layers, and the third computing unit and the fifth computing unit are located in the same metallization layer.

[0067] In some embodiments, the fourth computing unit is interconnected to the first computing unit via a via structure, wherein the fifth computing unit is interconnected to the second computing unit via the via structure.

[0068] In some embodiments, the memory device includes: a global computation unit configured to accumulate partial sums of products from the first computation unit and the second computation unit.

[0069] In some embodiments, the first computation unit generates the first partial sum by multiplying an input activation vector element with a weight stored in a storage unit subarray.

[0070] In some embodiments, the second computation unit generates the second partial sum by multiplying the input activation vector elements with weights stored in the storage unit subarray.

[0071] In some embodiments, each of the plurality of memory cells includes a plurality of word lines, and products of input activation vector elements and weights stored in the plurality of memory cells are calculated by accessing a subarray of memory cells via the plurality of word lines.

[0072] An embodiment of the present disclosure provides a memory device, comprising: a substrate; a memory cell array having a plurality of memory cell sub-arrays formed on the substrate and configured to store weights of a neural network; a first local accumulator formed on a first metal layer and configured to receive the stored weights from the memory cell sub-array and to generate a first partial sum based on the stored weights; a second local accumulator formed on the first metal layer and configured to receive the stored weights and the first partial sum from the memory cell sub-array and to generate a second partial sum based on the stored weights, wherein the second local accumulator is sequentially connected to the first local accumulator on the first metal layer.

[0073] In some embodiments, the memory device includes: a third local accumulator formed on the first metal layer and configured to receive the stored weight and the second partial sum from the memory cell subarray, and to generate a third partial sum based on the stored weight, wherein the third local accumulator is sequentially connected to the first local accumulator and the second local accumulator on the first metal layer.

[0074] In some embodiments, the memory device includes: a fourth local accumulator formed on the second metal layer and configured to receive the stored weights from the memory cell subarray and generate a fourth partial sum based on the stored weights; a fifth computing unit formed on the second metal layer and configured to receive the stored weights from the memory cell subarray and generate a fifth partial sum based on the stored weights, wherein the first local accumulator is connected to the fourth local accumulator and the fifth local accumulator in parallel to receive the fourth partial sum and the fifth partial sum and generate the first partial sum.

[0075] In some embodiments, the fourth local accumulator and the first local accumulator are located in different metal layers, and the fourth local accumulator and the fifth local accumulator are located in the same metallization layer.

[0076] In some embodiments, the fourth local accumulator is interconnected to the first local accumulator via a via structure, and the fifth local accumulator is interconnected to the second local accumulator via the via structure.

[0077] An embodiment of the present disclosure provides a method for manufacturing a memory device, comprising: providing a substrate; forming a memory cell array having a plurality of memory cell sub-arrays on a front side of the substrate; forming a first local accumulator on a first metal layer on the front side of the substrate; and forming a second local accumulator on the first metal layer, wherein the first local accumulator is sequentially connected to the second local accumulator on the first metal layer.

[0078] In some embodiments, the method includes forming a third local accumulator formed on the first metal layer, wherein the third local accumulator is sequentially connected to the first local accumulator and the second local accumulator on the first metal layer.

[0079] The foregoing summarizes the features of several embodiments so that those skilled in the art can better understand aspects of the present disclosure. Those skilled in the art should understand that they can easily use the present disclosure as a basis for designing or modifying other processes and structures to achieve the same purposes and / or achieve the same advantages as the embodiments introduced herein. Those skilled in the art should also recognize that such equivalent constructions do not depart from the spirit and scope of the present disclosure, and that they can make various changes, substitutions and modifications without departing from the spirit and scope of the present disclosure.

Claims

1. A storage device, comprising: a memory array comprising a plurality of memory cells for storing weights of a neural network; a first calculation unit configured to receive the stored weights from the plurality of storage units and generate a first partial sum according to the stored weights; as well as A second calculation unit is configured to receive the stored weights and the first partial sum from the plurality of storage units and generate a second partial sum according to the stored weights, wherein the second calculation unit is sequentially connected to the first calculation unit.

2. The memory device according to claim 1, wherein: The first computing unit and the second computing unit are located in a same metallization layer.

3. The memory device according to claim 1, wherein: The first computing unit is directly interconnected to the second computing unit.

4. The memory device according to claim 1, comprising: A third computing unit is configured to receive the stored weights and the second partial sum from the plurality of storage units and generate a third partial sum according to the stored weights, wherein the third computing unit is sequentially connected to the first computing unit and the second computing unit.

5. The memory device according to claim 4, wherein: The third computing unit and the first computing unit are located in the same metallization layer.

6. The memory device according to claim 1, comprising: a fourth calculation unit configured to receive the stored weights from the plurality of storage units and generate a fourth partial sum according to the stored weights; A fifth computing unit is configured to receive the stored weights from the multiple storage units and generate a fifth partial sum based on the stored weights, wherein the first computing unit is connected in parallel to the fourth computing unit and the fifth computing unit to receive the fourth and fifth partial sums and generate the first partial sum.

7. A storage device comprising: substrate; a memory cell array having a plurality of memory cell sub-arrays formed on the substrate and configured to store weights of a neural network; a first local accumulator formed on a first metal layer and configured to receive the stored weights from the memory cell subarray and generate a first partial sum based on the stored weights; A second local accumulator is formed on the first metal layer and is configured to receive the stored weight and the first partial sum from the storage cell subarray, and to generate a second partial sum based on the stored weight, wherein the second local accumulator is sequentially connected to the first local accumulator on the first metal layer.

8. The memory device according to claim 7, comprising: A third local accumulator is formed on the first metal layer and is configured to receive the stored weight and the second partial sum from the storage cell subarray, and to generate a third partial sum based on the stored weight, wherein the third local accumulator is sequentially connected to the first local accumulator and the second local accumulator on the first metal layer.

9. A method for manufacturing a memory device, comprising: providing a substrate; forming a memory cell array having a plurality of memory cell sub-arrays on the front side of the substrate; forming a first local accumulator on a first metal layer on the front side of the substrate; as well as A second local accumulator is formed on the first metal layer, wherein the first local accumulator is sequentially connected to the second local accumulator on the first metal layer.

10. The method according to claim 9, comprising: A third local accumulator is formed on the first metal layer, wherein the third local accumulator is sequentially connected to the first local accumulator and the second local accumulator on the first metal layer.

Images