Scalable array architecture for in-memory computing

Abstract

The present invention provides a system, method, architecture, mechanism, and apparatus for realizing programmable or pre-programmed IMC computations via an array of configurable IMC cores interconnected by a configurable on-chip network, supporting scalable execution and dataflow of applications mapped to integrated in-memory computing (IMC) cores. [Solution] The IMC architecture implements an integrated IMC architecture on a semiconductor substrate that supports scalable execution and data flow of applications mapped to the IMC, and comprises an array of configurable IMC cores, such as an in-memory computing unit, which includes IMC hardware and optionally other hardware such as digital computing hardware, buffers, control blocks, configuration registers, digital-to-analog converters (DACs), and analog-to-digital converters (ADCs).

Description

Technical Field

[0001] Government Support This invention was made with government support under contract number NRO000 - 19 - C - 001 awarded by the United States Department of Defense and was made under Government support in accordance with 4. The Government has certain rights in this invention

[0002] Cross - Reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 62 / 970,309, filed on February 5, 2020 and this application is hereby incorporated by reference in its entirety into this specification

[0003] This disclosure generally relates to the fields of in - memory computing and matrix - vector multiplication

Background Art

[0004] This section is intended to introduce the reader to various aspects of the art that may be related to the various aspects of the invention described and / or claimed below and is not to be construed as an admission of prior art. This discussion is thought to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the invention Therefore, these descriptions should be read from this perspective and should not be read as an approval of the prior art and it should be understood that these descriptions are to be read from this perspective and not as an admission of prior art and should not be read as an approval of the prior art It is to be understood

[0005] Deep learning inference based on neural networks (NN) has been widely deployed and is powered by its epoch - making performance in cognitive tasks. However, it requires hardware acceleration for energy efficiency and throughput even though it has a flexible programmable architecture through which it can achieve hardware acceleration for energy efficiency and throughput The complexity (number of layers, number of channels) and diversity (network architecture) required for the neural network This promoted an increase in internal variables / representations.

[0006] The dominant operation in neural networks is typically matrix-vector multiplication (MV) involving high-dimensional matrices. M) This makes data storage and movement in the architecture the main challenge. However, MVM also requires that the hardware be explicitly arranged in a correspondingly two-dimensional array. This presents a structured data flow that motivates the accelerator architecture. Such architectures often involve a processing engine (PE) performing simple operations (multiplication). A systolic array that performs calculations (addition) and passes the output to an adjacent PE for further processing. This is called a spatial architecture, employing MVM computation and data flow mapping. It also provides support for different computational optimizations (e.g., sparsity, model compression). Based on various methods, many variations have been reported.

[0007] A recently gaining attention alternative architectural approach is in-memory computing (IMC). Therefore, while IMC can be viewed as a spatial architecture, PE is memory bits. It is a cell. IMC typically employs analog operations to constrain the number of bit cells. To adapt the computational functionality in the road (i.e., for area efficiency) and to maximize the energy Perform calculations with energy efficiency. Recent demonstrations of IMC-based NN accelerators are the most Compared to optimized digital accelerators, it offers approximately 10 times the energy efficiency (TOPS). Simultaneously achieved a computational density of 10 times greater (TOPS / mm2) and a power consumption of 10 times greater.

[0008] Such gains make IMC attractive, but recent demonstrations mainly show analog non-idealism. Several significant challenges arising from (variability, nonlinearity) also became apparent. Firstly, almost Which demonstrations are limited to small scale (less than 128Kb)? Secondly, analog non-idealism The use of advanced CMOS nodes has not been demonstrated when deterioration is expected. Thirdly, Integration into large-scale computing systems (architecture and software stack) is This is limited due to the difficulty in specifying functional abstractions for such analog operations.

[0009] Some recent studies have begun to explore system integration. For example, ISA was developed, An interface for domain-specific languages ​​was provided, but the application map The approach is limited to small inference models and hardware architectures (single bank). Meanwhile, functional specifications for IMC calculations were developed, but they were complex and involved many lines of code. The analog operations required for column IMCs are supported by the digital format of IMCs, which has reduced parallelism. And so it was avoided. Therefore, analog non-idealism is not a problem when scaling up practical neural networks. This architecture significantly hinders the utilization of the full potential of IMC. [Overview of the Initiative]

[0010] Various shortcomings in conventional technology are interconnected by a configurable on-chip network. This enables scalable execution and data transfer of applications mapped to the IMC core. Through an array of configurable IMC cores that support Toughflow, programmable or A system, method, or application that provides pre-programmed in-memory computation (IMC) operations. It is addressed by a mechanism, device, or apparatus.

[0011] For example, various embodiments are integrated in-memory computing (IMC) architectures that support scalable execution and data flow of applications mapped to the IMC and are configured to provide an integrated IMC architecture. The IMC architecture is implemented on a semiconductor substrate and includes an IMC hardware, and optionally, other hardware such as digital computing hardware, buffers, control blocks, configuration registers, digital-to-analog converters (DACs), analog-to-digital converters (ADCs), etc., as described in more detail below, and includes an array of configurable IMC cores, such as an in-memory computing unit (CIMU).

[0012] The array of configurable IMC cores / CIMUs is interconnected via an on-chip network or an on-chip network that includes an inter-CIMU network portion, and is configured to transmit input data and computed data (e.g., activation values in embodiments of neural networks) to / from other CIMUs or other structures inside or outside the CIMU array via respective configurable inter-CIMU network portions disposed therebetween, and is configured to transmit operand data (e.g., weights in embodiments of neural networks) to / from other CIMUs or other structures inside or outside the CIMU array via respective configurable operand loading network portions disposed therebetween. are configured as such.

[0013] Generally speaking, each of the IMC cores / CIMUs receives computation data from the inter- The system receives the data and performs matrix-vector multiplication (MVM) on the received computation data using CIMU. ) A configurable input vector for generating an output vector through processing. It has an input buffer.

[0014] Some embodiments describe a neural network having an array-based architecture. Equipped with a neural network accelerator, multiple calculations of the in-memory computing unit (CIMU) , arranged and interconnected using a highly flexible on-chip network, one CIM Even if the output of U is connected to or flows to the input of another CIMU or to multiple other CIMUs Often, the outputs of many CIMUs may be connected to the inputs of one CIMU. For example, the output of one MU may be connected to the output of another CIMU. The network is a single on-chip network, or multiple on-chip network components. Alternatively, it may be implemented as a combination of on-chip and off-chip network components. stomach.

[0015] One embodiment involves multiple configurable in-memory computing units that form an array of CIMUs. (CIMU) and transmit input data to the CIMU array, and computed data between CIMUs A configurable on-chip for transmitting and transmitting output data from a CIMU array. An integrated in-memory computing (IMC) architecture comprising a network, Supports scalable execution and data flow of applications mapped to C. It provides a configurable integrated IMC architecture.

[0016] One embodiment involves an application integrated into a configurable in-memory IMC architecture. This provides a computer implementation method that maps to computing (IMC) hardware, and IMC The software consists of multiple configurable in-memory computing units that form an array of CIMUs. CIMU) transmits input data to the CIMU array and transmits calculated data between CIMUs. A configurable on-chip network for transmitting and transmitting output data from the CIMU array. The method utilizes the parallelism and pipelining of IMC hardware, and is equipped with a network. By using this, IMC hardware is allocated according to the application calculations, resulting in high throughput. Generate IMC hardware allocation configured to provide application computing. The IMC generates the output data by assigning the IMC hardware to the IMC hardware. Minimize the distance between the hardware and the IMC hardware that processes the generated output data. Define it in a format that is easy to convert to a location within the CIMU array, and on-chip network The network is configured to route data between IMC hardware, This application may include NN. These various steps are discussed throughout this application. It can be implemented according to mapping techniques.

[0017] Additional objectives, advantages, and novel features of the present invention are partially described below. Furthermore, this will become clear to those skilled in the art through the following considerations, or will become clear through the implementation of the present invention. It may be obtained. The object and advantages of the present invention are the means and which are particularly pointed out in the appended claims. It can be achieved and attained through combinations.

[0018] The accompanying drawings incorporated herein and constituting part thereof illustrate embodiments of the present invention. This illustrates the general description of the present invention given above, and the detailed embodiments given below. Along with the explanation, it serves to describe the principle of the present invention. [Brief explanation of the drawing]

[0019] [Figure 1A] This diagram illustrates conventional memory access architectures and in-memory computing (IMC) architectures, which are useful for understanding this embodiment. [Figure 1B] This diagram illustrates conventional memory access architectures and in-memory computing (IMC) architectures, which are useful for understanding this embodiment. [Figure 2A] This diagram illustrates a capacitor-based high SNR charge domain SRAM IMC, which is useful for understanding this embodiment. [Figure 2B] This diagram illustrates a capacitor-based high SNR charge domain SRAM IMC, which is useful for understanding this embodiment. [Figure 2C] This diagram illustrates a capacitor-based high SNR charge domain SRAM IMC, which is useful for understanding this embodiment. [Figure 3A] This diagram schematically illustrates the 3-bit binary input vector and matrix elements. [Figure 3B] This illustrates a realized heterogeneous microprocessor chip, including a programmable heterogeneous architecture and the integration of software-level interfaces. [Figure 4A] This diagram illustrates a circuit diagram of an analog input voltage bit cell suitable for use in various embodiments. [Figure 4B] Figure 4A shows a circuit diagram of a multi-level driver suitable for providing an analog input voltage to the analog input bit cell. [Figure 5]This graphically illustrates the layer deployment by mapping multiple neural network layers to effectively form a pipeline. [Figure 6] This graphically represents pixel-level pipelining with input buffering for feature map rows. [Figure 7] This graphically illustrates the replication for throughput consistency in pixel-level pipelining. [Figure 8A] This diagram illustrates row underutilization and the mechanisms for addressing it, which are useful for understanding various embodiments. [Figure 8B] This diagram illustrates row underutilization and the mechanisms for addressing it, which are useful for understanding various embodiments. [Figure 8C] This diagram illustrates row underutilization and the mechanisms for addressing it, which are useful for understanding various embodiments. [Figure 9] This graphically illustrates examples of operations made possible by CIMU configurability via a software instruction library. [Figure 10] This graphically illustrates architectural support for spatial mapping within application layers such as neural network layers. [Figure 11] This graphically illustrates how to map NN filters to IMC banks, each having N rows and M columns in dimensions, by loading filter weights into memory as matrix elements, applying input activation values ​​as input vector elements, and calculating output pre-activation values ​​as output vector elements. [Figure 12] This diagram illustrates exemplary architectural support elements associated with IMC banks for layer and BPBS deployment. [Figure 13] This diagram illustrates a block diagram of an exemplary near-memory SIMD engine. [Figure 14]This diagram illustrates an exemplary LSTM layer mapping function that utilizes cross-element near-memory computation. [Figure 15] This graphically illustrates the mapping of a BERT layer that uses the generated data as a loaded matrix. [Figure 16] This diagram illustrates high-level block diagrams of scalable NN accelerator architectures based on IMC in several embodiments. [Figure 17] Figure 16 shows a high-level block diagram of a CIMU microarchitecture with a 1152×256 IMC bank suitable for use in the architecture shown in Figure 16. [Figure 18] This diagram illustrates a high-level block diagram of the segment used to acquire input from the CIMU. [Figure 19] This diagram illustrates a high-level block diagram of the segment that provides output to the CIMU. [Figure 20] This diagram illustrates a high-level block diagram of an exemplary switch block used to select which inputs are routed to which outputs. [Figure 21A] This diagram illustrates the layout of a CIMU architecture according to an embodiment implemented using 16nm CMOS technology. [Figure 21B] Figure 21A illustrates a full chip layout consisting of a 4x4 tiling of the CIMU. [Figure 22] This graphically illustrates the three stages of mapping software flows to an architecture, where the NN mapping flow is mapped to an 8x8 array of the CIMU. [Figure 23A] This illustrates the sample placement of layers from the pipeline segment. [Figure 23B] This illustrates sample routing from pipeline segments. [Figure 24]This diagram illustrates a high-level block diagram of a computing device suitable for use in performing functions according to various embodiments. [Figure 25] This illustrates a typical structure of an in-memory computing architecture. [Figure 26] This shows a high-level block diagram of an exemplary architecture according to an embodiment. [Figure 27] Figure 26 illustrates a high-level block diagram of an exemplary in-memory computing unit (CIMU) suitable for use in the architecture shown. [Figure 28] This figure shows a high-level block diagram of an input activation value vector reshaping buffer (IA BUFF) suitable for use in the architecture shown in Figure 2, according to the embodiment. [Figure 29] This figure illustrates a high-level block diagram of a CIMA read / write buffer suitable for use in the architecture shown in Figure 26, according to the embodiment. [Figure 30] This figure shows a high-level block diagram of a near-memory data path (NMD) module suitable for use in the architecture of Figure 26, according to the embodiment. [Figure 31] This figure shows a high-level block diagram of a direct memory access (DMA) module suitable for use in the architecture of Figure 26, according to an embodiment. [Figure 32A] Figure 26 illustrates high-level block diagrams of different CIMA channel digitization / weighting embodiments suitable for use in the architecture shown. [Figure 32B] Figure 26 illustrates high-level block diagrams of different CIMA channel digitization / weighting embodiments suitable for use in the architecture shown. [Figure 33] This diagram illustrates the flow chart of the method according to the embodiment. [Figure 34] This diagram illustrates the flow chart of the method according to the embodiment. [Modes for carrying out the invention]

[0020] The attached drawings are not necessarily to scale and illustrate various features illustrating the basic principles of the present invention. Please understand that we will be presenting somewhat simplified expressions. For example, various example compositions A set of operations disclosed herein, including specific dimensions, orientation, location, and shape of constituent elements. Specific design features are determined in part by the specific intended use and operating environment. Certain features of the exemplary embodiments are intended to facilitate visualization and clear understanding. These features are magnified or distorted compared to other features. In particular, thin features are, for example For clarity or illustrative purposes, it may be thickened.

[0021] Before the present invention is described in further detail, the present invention is not limited to the specific embodiments described. It is important to understand that this is not fixed and therefore, naturally, subject to change. The terminology used is solely for the purpose of describing specific embodiments, and the scope of the present invention is not limited to the appended information. It is also understood that, because it is limited only by the claims, it is not intended to be limited. I want to be treated that way.

[0022] If a range of values ​​is provided, unless otherwise explicitly indicated by the context, the upper limit of that range applies. Each intervening value between the upper and lower limits, up to one-tenth of the lower limit unit, and within the stated range. It should be understood that any other stated values ​​or intervening values ​​are incorporated into the present invention. The upper and lower limits of the smaller range are any specifically excluded limits within the range described. Subject to the condition that it is included independently in a smaller scope, the stated scope is , if it includes one or both of the restrictions, the scope of excluding one or both of the restrictions that are included. This is also included in the present invention.

[0023] Unless otherwise defined, all technical and scientific terms used herein are defined in accordance with the present invention. This has the same meaning as that generally understood by those skilled in the art in the field to which it belongs. Any methods and materials similar or equivalent to those described in the specification may also be used in carrying out or testing the present invention. It can be used for the following purposes, but a limited number of exemplary methods and materials are described herein. The singular forms "a" and "an" are used in this specification and the appended claims. ", and "the" refer to multiple objects unless otherwise explicitly indicated by the context. Please take note of this.

[0024] The following description and drawings are merely illustrative of the principles of the present invention. Therefore, those skilled in the art will be able to use them. Therefore, although not explicitly described or illustrated in this specification, the principle of the present invention is embodied in It should be understood that various arrangements within that range can be devised. Furthermore, all examples listed herein are primarily intended to help readers understand the principles and techniques of the present invention. To help understand the concepts to which the inventors contribute in order to further advance the technology, for educational purposes It is explicitly intended for this purpose only, and is not limited to such specifically enumerated examples and conditions. It should be interpreted that this does not mean that... In addition, when used herein, the use of "or" Unless otherwise indicated, the word is non-exclusive or (for example, "or otherwise") This refers to "or alternative cases." Also, some embodiments form new embodiments. Therefore, it can be combined with one or more other embodiments as described herein. The various embodiments are not necessarily mutually exclusive.

[0025] Many of the innovative teachings of this application are, in particular, described herein in the exemplary embodiments that are preferred. See [reference] for details. However, embodiments of this class are innovative as described herein. Please understand that this provides only a few examples of the many advantageous uses of the teachings. In general, The descriptions made in the patent application specification do not necessarily refer to any one of the various claimed inventions. This is not limiting. Furthermore, some descriptions apply to some inventive features, This may not apply to other features. A person skilled in the art who has obtained information from the teachings herein shall not use this information. You will recognize that this can be applied to various other technical fields or embodiments.

[0026] The various embodiments described herein are primarily programmable or pre-programmed. In-memory computation (IMC) operations performed, and the scheduling configured for in-memory computation. A system, method, architecture, or mechanism that provides a manageable data flow architecture. The subject is the structure or apparatus.

[0027] For example, various embodiments exist in terms of an integrated in-memory computing (IMC) architecture. , scalable execution and data flow of applications mapped to IMC It provides a configurable integrated IMC architecture that supports IMC architecture The component is mounted on a semiconductor substrate and the IMC hardware is described in more detail below. , and optionally, digital computing hardware, buffers, control blocks, configuration Diagrams, digital-to-analog converters (DACs), analog-to-digital converters (ADCs), etc. A configurable in-memory computing unit (CIMU) or other hardware, which includes other hardware components. It features an array of IMC cores.

[0028] The configurable IMC core / CIMU array includes the inter-CIMU network portion. Interconnected via a chip network, each configurable chip placed between them Other CIMUs within or outside the CIMU array via the inter-CIMU network portion Input data and computed data (e.g., neural network) to / from a CIMU or other structure. The activity value in the embodiment of the twerk is transmitted, and the twerk is disposed between them. Through each configurable operand loading network portion, the CIMU array Operand data to / from other CIMUs or other structures inside or outside the CIMU array (e.g.) It is configured to transmit weights in an embodiment of a neural network.

[0029] Generally speaking, each IMC core / CIMU receives computational data from the inter-CIMU network. The system receives the data and performs matrix-vector multiplication (MVM) on the received computation data using CIMU. ) A configurable input vector to be constructed in order to generate an output vector through processing It has an input buffer.

[0030] The additional embodiments described below are independent of or in combination with the embodiments described above. A scalable dataflow architecture for in-memory computing, ideal for combined use. This targets the texture.

[0031] Various embodiments show that multiplication is digital, while accumulation is analog and bitwise This is achieved by short-circuiting the charges from capacitors localized in a given area. By transitioning to load domain computation, we address analog non-idealism. These capacities The sita relies on geometric parameters that are well controlled by advanced CMOS technology, and therefore And semiconductor devices, for example (transistors, resistive memory), have much larger wires. It allows for shape and small variations (e.g., process, temperature). The groundbreaking scale of Schott's fully parallel IMC bank (e.g., 2.4 Mb), and furthermore Large computing systems (e.g., heterogeneous programmable architectures, software) Integration into software libraries becomes possible, demonstrating practical neural networks (e.g., 10 layers). ru.

[0032] Improvements to these embodiments result in high energy efficiency when running state-of-the-art neural networks. The architectural scale of the IMC bank required to maintain throughput Addressing the issue. These improvements employ a proven approach to charge domain IMC. Therefore, in order to scale up the IMC while maintaining such efficiency and throughput We will develop an architecture and associated mapping approach.

[0033] Basic trade-offs of IMC IMC performs analog calculations and moves raw data, and moves the calculation results. By amortizing in dynamic terms, we derive gains in energy efficiency and throughput. This leads to a fundamental trade-off, and ultimately to scaling up the architecture. This shapes the challenges of application mapping.

[0034] Figure 1 shows a conventional memory access architecture and which are useful for understanding this embodiment. This diagram illustrates the in-memory computing (IMC) architecture. In particular, Figure 1 The diagrammatic representation first shows the IMC (Figure 1B), which separates memory and computation in a conventional (digital) way. ) Compared with the memory access architecture (Figure 1A), then the spatial digital architecture We illustrate the trade-offs by expanding the insights for comparison with Cha.

[0035]

number

number

number

number

[0036] However, a significant trade-off is that the traditional architecture uses a single building block (BL / BLb). While accessing the data, IMC

number

number

number

[0037] The digital space architecture loads operands into PE and reuses data. By taking advantage of short-range communication opportunities (i.e., between PEs), memory access and data Reduce data movement. Typically, the computational cost of multiplication by multiplication (MAC) operations is dominant. IMC once again introduces a trade-off between energy efficiency and throughput versus SNR. In this case, analog calculations enable efficient MAC calculations, but also the subsequent analog data This raises the need for digital conversion (ADC). On the other hand, there are many analog MAC operations (i.e. (And high row parallelism) amortizes the overhead of the ADC, while on the other hand, more MAC The calculations increase the analog dynamic range and degrade the signal-to-noise ratio (SNR).

[0038] The trade-off between energy efficiency and throughput versus SNR is a key factor in computing systems. The main limitations were imposed on scaling up and integrating MC. Regarding scaling up, ultimately... This results in extremely low computational precision, and the energy / throughput benefits that can be derived from row parallelism are also detrimental. Limit the gains. Regarding integration in computing systems, noisy computations are architecture Robust abstractions required for interface design and software It limits the ability to form. Previous efforts regarding integration in computing systems have focused on row parallelism. It has been necessary to limit it to 4 lines or 2 lines. Charge domains, as described below. Analog computing overcomes this, resulting in a substantial increase in row parallelism (4608 rows) and heterogeneity. This led to both integration into the architecture and a high level of development. Row parallelism is advantageous in terms of energy efficiency and throughput, but it is also a limitation of neural networks. This limits the hardware granularity for mapping and necessitates the specialized strategies explored in this study. Let's assume that.

[0039] High SNR SRAM-based charge domain IMC Our previous research showed that the bit cell output signal modulates the resistance of the internal device by The process shifts from current-domain calculations, which deal with the resulting current, to charge-domain calculations. Here, the bit cell output signal is the charge stored in the capacitor. Resistance is due to the material and Depending on the characteristics of the device, and especially in advanced nodes, it can result in substantial process and temperature changes. Although it tends to exhibit dynamic behavior, capacitance depends on geometric characteristics, and in advanced CMOS technology... It can be controlled very well.

[0040] Figure 2 shows a capacitor-based high SNR charge domain, which is useful for understanding this embodiment. This illustrates a schematic representation of SRAM IMC. In particular, the schematic representation in Figure 2 shows charge doming Logical representation of the in-calculation (Figure 2A), schematic representation of the bit cell (Figure 2B), and 2.4 Figure 2C shows an image of the realization of the Mb integrated circuit.

[0041] Figure 2A illustrates an approach to charge domain calculations. Each bit cell It takes binary input data xn / xbn and binary stored data am,n / abm,n Perform multiplication with . Treat binary 0 / 1 data as -1 / +1, this is digital This is an XNOR operation. The binary output result is then used to represent a charge on the local capacitor. It is then stored. Next, the charge from all bit cell capacitors in the row is short-circuited to each other. Accumulation is implemented by generating an analog output ym. Digital binary multiplication is This avoids analog noise sources and ensures complete linearity (the two levels are perfectly aligned to the line). (Suitable for) On the other hand, capacitor-based charge accumulation is due to its superior matching and temperature stability. This avoids noise and ensures high linearity (an essential characteristic of capacitors).

[0042] Figure 2B illustrates an SRAM-based bit cell circuit. It shows a typical six-bit configuration. In addition to the transistor, two additional PMOS transistors are used depending on the XNOR condition. Adopted for passith charging, two additional NMOS / PMOS transistors are used to accumulate charge. Adopted outside the bit cell for multiplication (to pre-discharge all capacitors after accumulation) Therefore, a single additional NMOS transistor is required for the entire column. (Extra bit cell) While transistors impose an 80% reported area overhead, local capacity Pasita is laid out using metal wiring above the bit cell, thus exceeding the area limit. No load is imposed. The dominant source of capacitor non-idealism can be a mismatch, and computational noise Allows more than 100k row parallelism until the is equivalent to the minimum analog signal isolation. This means that the largest size already reported for the IMC bank is (2.4Mb). This enables and overcomes the significant limitations of the SNR trade-off that previously restricted IMC. (Figure 2C).

[0043] Charge domain IMC operations involve binary input vectors and matrix elements, but they are multiplied Extend to include tiny elements.

[0044] Figure 3A schematically illustrates the 3-bit binary input vector and matrix elements. This is achieved through bit-parallel / bit-serial (BPBS) computation. The matrix element bits of a number are mapped to parallel columns, while the elements of multiple input vectors are mapped to parallel columns. It is provided serially. Then, each of the column calculations is calculated based on energy overhead and area overhead. Digitized using an 8-bit ADC selected to balance with the bar head. The digitized column output is then bit-weighted appropriately in the digital domain (bits After applying the fitting, the sum is finally calculated. This approach uses two's complement representation and It supports both a special number representation optimized for bitwise XNOR calculations and a standard number representation.

[0045] The analog dynamic range of column calculations is calculated by an 8-bit ADC (256 levels). Because it can be larger than the dynamic range that is ported, BPBS calculations are standard This results in a different type of rounding than typical integer calculations. However, both IMC columns and ADCs Precise charge domain calculations are performed within the scope of architectural and software abstractions. This enables robust modeling of the effects. Figure 3B shows a programmable heterogeneous architecture and software level A picture of a realized heterogeneous microprocessor chip, including interface integration. This illustrates the concept. Current research focuses on apps for efficient and scalable execution. We are developing a heterogeneous IMC architecture driven by communication mapping. This technology is extended by using the BPBS approach, as described below. The fundamental need for high row parallelism for energy efficiency and throughput in IMC This overcomes the hardware granularity constraints that arise from this.

[0046] Figure 4A shows a circuit diagram of an analog input voltage bit cell suitable for use in various embodiments. This illustrates the analog input voltage bit cell design in Figure 4A, which is drawn above with respect to Figure 2B. Can be used instead of the indicated digital input (digital input voltage level) bit cell design The bit cell design in Figure 4A has two digital voltage levels in the input vector element (for example) This allows multiple voltage levels to be applied, rather than just VDD and GND. It is configured as follows. In various embodiments, the use of the bit cell design in Figure 4A is used in BPBS. This enables a reduction in the number of cycles, thereby providing corresponding benefits in throughput and energy. This results in a multi-value voltage from a dedicated power supply (e.g., x0, x1, x2, x3, and x By providing b0, xb1, xb2, xb3), it is possible to use lower voltage levels. As a result, additional energy reductions are achieved.

[0047] The bit cell circuit illustrated in Figure 4A has a switch-free coupling structure according to the embodiment. It is depicted as such. Other variations of this circuit are also possible within the context of the disclosed embodiments. Please note that this is possible. The bit cell circuit is formed by (MN1~3 / MP1~2) The data W / Wb stored within the 6-transistor cross-coupled circuit and the input data This enables the implementation of either an XNOR or AND operation between IA / IAb. For example, for XNOR operations, after reset, IA / IAb are driven in a complementary manner, Pull up / pull down the bottom plate of the gal capacitor according to IA XNOR W. This can be done. On the other hand, for AND operations, after reset, only IA is used. Move (and keep IAB low) and the bottom plate of the local capacitor IA AND It can be made to pull up / pull down according to W. Advantageously, this The structure involves a series pull-up / pull-down configuration resulting between all coupled capacitors. The total switching energy of the capacitor is reduced due to the charging structure, and the output node This enables the reduction of the effects of switch charge injection errors caused by the elimination of coupled switches. .

[0048] Multi-valued driver Figure 4B is suitable for providing an analog input voltage to the analog input bit cell in Figure 4A. This shows the circuit diagram of a multi-level driver. The multi-level driver 1000 in Figure 4B has eight It is depicted as providing a level of output voltage, but actually provides any number of output voltage levels. It is used to support processing any number of bits of the input vector elements in each cycle. Please note that the actual voltage level of the dedicated power supply may be affected by off-chip control. It can be fixed or selectable. For example, this means that multiple bits of the input vector element are +1 / - The XNOR calculation required when selecting to be 1, or multiple input vector elements It is necessary when the bits are selected to be 0 / 1 as in the standard two's complement form. It may be useful to construct an AND operation within a bit cell. In this case, an XNOR operation The calculation is x3, x2, x1, x0, to uniformly cover the input voltage range of VDD~0V. While it requires the use of xb0, xb1, xb2, and xb3, AND calculations are performed using VD. x3, x2, x1, and x0 are used to uniformly cover the input voltage range from D to 0V. Furthermore, it is necessary to set xb0, xb1, xb2, and xb3 to 0V. The embodiment can be modified as needed to provide numerical formats for XNOR calculations, AND calculations, etc. A dedicated power supply, such as a port, can be configured as off-chip / external control, and a multi-value driver is proposed. It may be provided.

[0049] The current from each power source is reduced accordingly, and the power grid density of each power source is also reduced accordingly. Therefore, it becomes possible to avoid requiring additional power grid wiring resources, and thus a dedicated voltage Please note that this can be easily provided. One of several applications The challenge is when many IMC columns must be driven (i.e., a single driver) This may necessitate multi-level repeaters, such as when the number of IMC columns driven exceeds the circuit's capacity. In this case, the digital input vector bits, in addition to the analog driver / repeater output, Routing can occur across the IMC array. Therefore, the number of levels is... The selection should be based on resource availability.

[0050] In various embodiments, a bit cell is depicted, and there are two 1-bit input operands. The value is represented by one of binary zero (GND) or binary one (VDD). This operand is obtained by multiplying the bit cell by another 1-bit value, and this is the bit cell One of these two voltage levels into the sampling capacitor associated with it It brings about memory. All the capacitors in the column containing that bit cell are connected to each other, When the stored values ​​of these capacitors (i.e., the charge stored in each capacitor) are collected, The resulting cumulative charge is the result of multiplying all the bit cells in the sequence of bit cells. It provides a voltage level that represents the cumulative sum.

[0051] Various embodiments use n-bit operands and represent n-bit operands. The use of bit cells is planned such that the voltage level always includes one of n different voltage levels. This is illustrated. For example, a 3-bit operand is represented by eight different voltage levels. Obtain. When that operand is multiplied by a bit cell, it is applied to the memory capacitor, The resulting charge is obtained from n different voltages during the cumulative phase (short-circuiting the capacitor row). It is something that can have levels. In this way, a more accurate and flexible system Therefore, the multi-level driver in Figure 4 provides such accuracy / flexibility. It is used in various embodiments. Specifically, in response to an n-bit operand, n bits One of the voltage levels is selected and coupled to a bit cell for processing. Selected by decoding multiple bits of the operand or input vector element. A multi-level driver employing a dedicated voltage power supply enables multi-level input vector element signaling. To be served.

[0052] Scalable IMC challenges IMC is a scalable mapping of neural networks that arises from its basic structure and trade-offs. This presents three notable challenges: (1) matrix loading cost, (2) Essential coupling between data storage and computing resources, and (3) large row parallelism These are large number of column dimensions, each of which will be discussed below. This discussion is a general convolution. NN (CNN) benchmark (characteristically excluded from analysis due to a small number of input channels) The first layer (which is provided) is used with 8-bit precision to provide the application context. Table I (Example: Scalable Application Mapping for CNN Benchmarks) (Examples of some IMC challenges) and Algorithm 1 (Execution in a typical CNN) This information is taken from (example pseudocode for loops). [Table 1]

number

[0053] Matrix loading cost. As mentioned above regarding the basic trade-offs, IMC is, While it reduces memory read and computation costs (energy, delay), IMC is a memory write It does not reduce loading costs. This is an overall benefit in running the complete application. This could potentially degrade the benefits. The common approach in reported demonstrations is, This involves loading matrix data and holding it statically in memory. However, this This is illustrated by the numerous model parameters in the first row of Table I, which represent the required amount of memory and From the perspective of both the reproduction required to ensure sufficient utilization, as described below, and This makes it impractical for practical-scale applications.

[0054] The essential coupling between data storage and computing resources. Combining memory and computation. This imposes constraints on IMC in allocating computing resources along with memory resources. Accept. The data involved in practical neural networks are both large (first row of Table I), and memory lithography This could potentially place a substantial burden on the system, and the calculation requirements can vary widely. For example, each MAC operation with weights is determined by the number of pixels in the output feature map. As illustrated in the second row of Table I, this varies significantly from layer to layer. Unless the computation is evenly distributed through a topping strategy, it could lead to a significant loss of utilization. There is.

[0055] Large column dimensions for row parallelism. As mentioned above regarding the basic trade-offs, MC derives its gain from a high level of row parallelism. However, high row parallelism The large number of column dimensions required to enable this reduces the granularity of mapping matrix elements. Table I As illustrated in the third line, the size of the CNN filter is within the application and in the app. It varies widely between both applications. For layers with small filters, filter weights Forming it into a matrix and mapping it to a large IMC column results in low utilization and row parallelism. This leads to a deterioration of the benefits derived from sex.

[0056] To illustrate, we will map a CNN to show how the above challenges manifest themselves. Two common strategies are then considered. CNNs, as shown in Algorithm 1, nested It is necessary to map the loops. The mapping to hardware is the loop Choosing a p-ordering method and considering space (expansion, replication) and time for parallel hardware. This involves scheduling in a blocking manner.

[0057] Static mapping to IMC. Much of the current IMC research is primarily based on relatively high matrix loads. To avoid the cost of static mapping, the entire CNN is statically mapped to hardware. (That is, loops 2, 6-8) are being considered (the first issue mentioned above). Two approaches As analyzed in Table II, this is due to a very low utilization rate and / or very This is likely to lead to significant hardware requirements. The first approach is to assign one weight to each weight. Simply map to the IMC bit cell, and furthermore, the IMC column has various values ​​across the layer. Assume that the filter has a different number of dimensions to fit perfectly (i.e., (Ignoring the utilization loss from the third problem mentioned above). This means that each weight is equal in amount. The MAC is assigned to the device, but is set by the number of pixels in the output feature map. Because the number of operations varies widely, it leads to low utilization (the second problem mentioned above). Alternatively, The second approach assigns weights to multiple IMC bit cells according to the number of operations required. Perform the mapping and replication. Again, ignore the utilization loss from the third challenge mentioned above. This allowed for a high utilization rate, but a very large amount of IMC hardware... A is required. This may be practical for very small neural networks, but it is not practical. This is not feasible for neural networks of a certain size. [Table 2]

[0058] Therefore, to map a CNN loop with non-static weight mapping... A more thorough strategy must be considered, and therefore, there will be a weight loading cost. (The first challenge mentioned above). This is because most NVM technologies are limited by the number of write cycles. Facing limitations, this raises further technical challenges when using NVM in IMC. I'll leave it at that.

[0059] Layer-by-layer mapping to IMC. A common approach used in digital accelerators. The approach involves mapping the CNN to each layer (i.e., unfolding loops 6-8). Yes. This means that as the number of operations with each weight equalizes, the second problem mentioned above becomes It provides an easy way to deal with it. However, to achieve high throughput within the accelerator... The high level of parallelism that is frequently adopted raises the need for replication in order to ensure high utilization rates. Yes. The main challenge here is the high weight loading cost (the first challenge mentioned above).

[0060] For example, expanding loops 6-8 and duplicating the filter weights with multiple PEs is an input This allows for parallel processing of feature maps. However, here, the stored weight Each of these is involved in fewer MAC operations by the replication coefficient. Therefore, The relative total cost of loading (the first challenge mentioned above) is high compared to MAC calculations. Yes. Digital architectures are often feasible, but this occurs for the following two reasons. Therefore, this is a problem for IMC. (1) Very high hardware density This leads to significant weight duplication to maintain utilization, and therefore, matrix loading If the number of operations increases significantly, and (2) the cost of MAC operations decreases, the matrix loading cost This becomes dominant, significantly reducing the gain at the full application level.

[0061] Generally speaking, layer-by-layer mapping requires data buffering. In either CIMU, the next layer is not currently mapped to the mapping. Then, the layer deployment mapping is performed so that the data progresses within the pipeline to the next layer in the CIMU. This refers to the currently mapped mapping. Layer-by-layer mapping and layer-expand mapping. Both are supported in various embodiments.

[0062] IMC's scalable application mapping Various embodiments are based on two ideas, namely (1) high utilization of parallel hardware To achieve this, we expand the layered loop (loop 2) and (2) from the BPBS calculation. Leveraging the appearance of additional loops and adopting an app for scalable mapping I plan to introduce a roach. These ideas are described further below.

[0063] Layered expansion. This approach involves expanding loops 6-8. However, each hardware unit Reduces the number of operations involving the weights and loads, and is a multi-parallel hardware approach. Instead of manufacturing, parallel hardware is used to map multiple neural network layers.

[0064] Figure 5 shows how to map multiple NN layers so that the pipeline is effectively formed. This graphically illustrates the layer development by which. As described below, various practical In this configuration, filters within the NN layer are mapped to one or more physical IMC banks. If more IMC banks than can be physically supported are needed for a particular layer, Loop 5 and / or Loop 6 is blocked, and the filters of the NN layer are then mapped in time. This allows for scalability of both the NN input and NN output channels that can be supported. It becomes possible. On the other hand, to map the next layer, more than can be physically supported. If a certain IMC bank is required, loop 2 is blocked, and the layer then takes time. It is mapped to the target. This leads to the pipeline segment of the NN layer and supports This enables scalability of the NN depth, which can be reduced. However, such NN Layered pipelines present two challenges in terms of latency and throughput.

[0065] Regarding latency, the pipeline introduces a delay in the generation of the output feature map. Due to the deep properties of N, some latency inherently occurs. However, In traditional layer-by-layer mapping, all available hardware is immediately utilized. Expanding layer loops effectively delays the use of hardware in later layers. Pipeline loading occurs only at startup, but involves a wide variety of latency sensitivities. This becomes a significant concern when prioritizing small-batch inference in applications. The state uses an approach referred to herein as pixel-level pipelining, Reduce latency.

[0066] Figure 6 shows a pixel-level pipelining with input buffering for feature map rows. It is a way to graphically represent the . Specifically, it is a pixel-level piping. The goal is to start processing subsequent layers as quickly as possible. Feature map pixels This represents the smallest granularity data structure processed through the pipeline. Therefore, given Pixels consisting of parallel output activation values ​​calculated from the hardware executing the layer are used for the next layer. It is immediately provided to the hardware on which it will run. In CNNs, the il×jl filter kernel is Since it requires that a corresponding number of pixels be available for the calculation, single Some pipeline latency must occur that exceeds the pixel latency. This avoids the high cost of moving intercalation activity values ​​to a global buffer. This raises the need for a local line buffer near the IMC. The complexity of buffering. To facilitate this, various embodiments of the application of pixel-level pipe lining Roach receives feature map pixels row by row, as illustrated in Figure 6. This fills the input line buffer.

[0067] Regarding throughput, pipelining requires throughput matching between CNN layers. The necessary operations depend on both the number of weights and the number of operations per weight, and the width between layers. It changes widely. As mentioned above, IMC is essential for data storage and computing resources. This combines with hardware allocation to handle computational scaling based on the number of weights. This is provided. However, the calculation per weight depends on the number of pixels in the output feature map. This is determined, and the output feature map itself varies widely (second row of Table I).

[0068] Figure 7 shows the replication for throughput matching in pixel-level pipelining. It is a graphical representation, and fewer operations in layer l+1 (for example, larger (This is due to large convolutional striding) which requires replication to layer l. As illustrated in Figure 7, throughput matching is the number of output feature map pixels (layers). l has four times the output pixels of layer l+1) within the mapping of each CNN layer It requires duplication. Otherwise, if the number of output pixels is smaller, pipeline This will likely result in a loss of utilization due to the stall.

[0069] As discussed above, replication involves operations with each weight stored in parallel hardware. Reduce the number. This is a problem in IMC, and the lower cost of MAC calculations Maintain a large number of operations for each stored weight to amortize the matrix loading cost. This requires... However, in reality, the replication required for throughput consistency is... It has been found to be acceptable for two reasons. Firstly, such replication does not affect all layers. It is not performed uniformly, but rather explicitly depends on the number of operations per weight. However, the hardware used for replication still incurs matrix loading costs. This can be effectively amortized. Secondly, the large number of copies will not take up the entire physical IMC bank. This leads to the use of the following layers: Enforce a new pipeline segment with merger and replication requirements. Therefore, replication The quantity is self-regulating depending on the amount of hardware.

[0070] Algorithm 2 is a bit-parallel / bit-serial (BPB) algorithm with various embodiments. S) Exemplary pseudocode for the execution loop in a CNN using computation is shown.

number

[0071] BPBS expansion. As mentioned earlier, a high number of column dimensions to maximize the gain from IMC. The need for this is that when used to map smaller filters, it results in a loss of utilization. This brings about the following. However, BPBS calculation is performed as shown in Algorithm 2, input activity Two additional loops corresponding to the processing of the value bits and the weight bits. This effectively generates the desired effect. Expand these loops to increase the amount of column hardware used. It can be added.

[0072] Figure 8 shows a schematic representation of insufficient row utilization and row utilization, which are useful for understanding various embodiments. This illustrates the mechanism for addressing the shortage. Specifically, Figure 8 shows the row utilization rate. This paper illustrates the challenges and the results of expanding the BPBS calculation loop to increase IMC column utilization. It is.

[0073] Figure 8A shows an example where a small filter occupies only one-third of the IMC column, resulting in an underutilized row. This graphically depicts the foot problem. Assuming a 4-bit weight, BPBS A The approach employs four parallel columns for each filter. Two alternative mapping applications By adopting Roach, the utilization rate can be increased to over 0.33. The first approach is As illustrated in Figure 8b, two adjacent columns are merged into one. However, the original column Because different matrix element bit positions correspond to the bits from the higher positions, the bits from the higher positions correspond to the bits from the higher positions. The input vector provided serially must be replicated in a column with Inari weighting The elements are simply duplicated in the same way. This ensures proper capacitance charge short circuits during column accumulation calculations. This will be secured.

[0074] Figure 8A graphically illustrates the effective utilization rate of the columns. Specifically, the column meters The process has two limitations. First, merging bits from the position of the higher matrix element... The replication required for this purpose leads to a high physical utilization rate, but the effective utilization rate is somewhat low. However, the effective utilization rate of the column in Figure 8B is only 0.66, and more columns have corresponding bi It is further restricted because it is merged with binary-weighted copies. Secondly, binary weighting Due to the need for deletion and duplication, the dimensionality requirement of the columns is exponential, along with the number of columns being merged. It increases functionally. This limits the cases in which column merging can be applied.

[0075] For example, two columns can only be merged if the original utilization rate is <0.33, and the original If the utilization rate is <0.14, the three columns can be merged, and if the original utilization rate is <0.0 For example, four columns can only be merged if the value is 7. Duplicates and shifts. The second approach is illustrated in Figure 8C. Specifically, the matrix elements are duplicated, It is shifted and requires an additional IMC column. In this case, the two input vector bits are parallel. It is provided to the column, and the higher bits are shifted to the matrix element. Unlike column merging, Duplication and shifting result in high effective utilization, equal to physical utilization. Furthermore, columns The dimensionality requirement does not increase exponentially with utilization, thus avoiding duplication and shifting. This becomes applicable in more cases. The main limitation is that the middle column achieves high utilization. On the other hand, columns that face either edge show a decrease in utilization, and the first and last columns, This is limited to the original utilization level, as shown in 8C. Nevertheless, Significant utilization gains can be achieved using various embodiments for weight precisions of 4 to 8 bits. It will be done.

[0076] Multi-level input activation values. The BPBS scheme optimizes the energy and throughput of IMC calculations. It scales along with the number of bits in the input vector applied serially. The multi-value driver Figure 4 is discussed above.

[0077] Figure 9 shows the configurability of the CIMU via a software instruction library. This graphically illustrates a sample of the calculation. In addition to the temporal mapping of the NN layer, Furthermore, the architecture provides extensive support for spatial mapping (loop unfolding). Given the high hardware density / parallelism of IMC, this would result in state replication between engines. Therefore, beyond typical replication strategies that result in excessive state loading overhead, HW Provides a wide range of mapping options for utilization. Supports spatial mapping of NN layers. To perform the IMC calculation, a method for receiving and sequencing input activation values. Various approaches have been presented, including the configuration of input buffers and shortcut buffers. This is made possible by: (1) high bandwidth input for fully coupled layers, (2) convolution (3) Reduced bandwidth input and line buffering for the layer, memory extension layer For the purpose of feedforward and regression input, and output element calculation, (4) NN and show Parallel input and buffering of activation values ​​in the Tocut path, as well as summation of activation values. A wide range of other Activity value reception / sequencing approach, and configurable parameters for the above approach. Sexuality is supported.

[0078] Figure 10 shows how to reduce data swap / movement overhead and scale the NN model. - For both enabling flexibility and within the application layer such as the NN layer This graphically illustrates the architectural support for spatial mapping. For example, The output tensor depth (number of output channels) is determined by the OCN route of the input activation values ​​to multiple CIMUs. It can be extended by ting. The input tensor depth (number of input channels) is determined by the number of adjacent CIs. It can be extended via short, high-bandwidth face-to-face connections between MU outputs, and a third CI By summing the partial prior activity values ​​from two CIMUs using MU, further expansion is achieved. This allows for efficient scaling up of layer calculations (a wide range of NN benchmarks). By mapping the IMC, it enables the balance of IMC core dimensions, In this case, coarse particle size benefits the parallelism and energy of IMC, while fine particle size is efficient. This brings benefits to rational computational mapping.

[0079] General Considerations of Modular IMC for Scalability Both layer unfolding and BPBS unfolding introduce important architectural challenges. In layer unfolding the main challenge is the need to support diverse dataflows and computations between the layers of the NN application which leads to the need for architectural configurability that can be generalized for current and future NN designs. In contrast, within a single NN layer, the MVM operation is dominant and the compute engine benefits from a relatively fixed dataflow involved (although various optimizations that exploit attributes such as sparsity are being looked at). Examples of the dataflow and compute configurability required between layers are discussed below.

[0080] In BPBS unfolding, in particular, duplication and shifting affect the bit-level sequencing of operations on the input activations and pose additional complexity for throughput matching ([[]]column merging, fixation of input activations to bit-level computations, preservation of sequencing for pixel-level pipelining[[[]]). More generally, various levels of input activation quantization are employed between layers and thus, if this requires different numbers of IMC cycles, this has to be considered within the replication approach discussed above for throughput matching in the pixel-level pipeline.

[0081] Figure 11 maps the NN filter to an IMC bank where each bank has dimensions of N rows and M columns by loading the filter weights as matrix elements into the memory and applying the input activations as input vector elements to compute the output pre-activations as output vector elements ​​​​​​​​​​This graphically illustrates the method of loading into memory. Specifically, Figure 11 shows how to load into memory. Load the Luta weights as matrix elements for the IMC bank, and input activation values ​​into the input vector This illustrates how to apply it as a component and calculate the output pre-activation value as an output vector element. Each bank processes an input vector with N rows and M columns (i.e., an input vector with N dimensions). It is depicted as having an output vector of dimension M.

[0082] IMC implements MVM in the following format:

number

[0083] This corresponds to weight-fixed mapping. Alternative mappings such as fixed input are also possible. Yes, in this case the input activation value is stored in the IMC bank, and the filter weight is the input vector Ru

number

number

[0084] Architecture support Following the basic approach of mapping NN layers to an IMC array, various embodiments Accordingly, various microarchitecture support can be provided around the IMC bank. .

[0085] Figure 12 shows an exemplary architecture associated with the IMC bank for layering and BPBS deployment. This diagram illustrates a block diagram showing example texture support elements.

[0086] Input line buffering for convolution. In pixel-level piping... The pixel output activation value is generated by one IMC module and the next module... It is sent to [the appropriate destination]. Furthermore, in the BPBS approach, incoming activity values ​​are processed all at once. However, convolution involves calculations on multiple pixels at once. This is related to IMC The input has a configurable buffer with support for various stride step sizes. A ring is required. There are various ways to do this, but the approach in Figure 12 is (in Figure 6) Buffer some rows of the input feature map corresponding to the height of the illustrated convolutional kernel. The row width supported by the buffer is required to process the input feature map within the vertical segment (e.g., by performing blocking on loop 4). The height / width of the kernel supported by the buffer is an important architectural design parameter, but the trend of 3×3 primary kernels can be utilized to build larger kernels. Such buffering allows the incoming pixel data to be provided to the IMC one bit at a time, processed one bit at a time, and transmitted one bit at a time (in accordance with the output BPBS operation).

[0087] The input line buffer can also support the acquisition of input pixels from various IMC modules by having additional input ports from the on-chip network. This enables the assignment of multiple input IMC modules and equalizes the number of operations performed by each IMC module within the pipeline, allowing the throughput alignment required at the pixel level pipelining.

[0088]

[0089] The inventors have developed a method that allows incoming (received) pixels to be suitably buffered. Please note that various techniques are being considered. The approach depicted in Figure 12 is shown in Figure 7. Assign different input ports to different vertical segments of each row in the format shown.

[0089] Near-memory element-wise computation. From IMC hardware running one NN layer to the next N To directly feed data to IMC hardware running N layers, activation functions are required. Operations on individual elements such as batch normalization, scaling, and offset setting, as well as batch normalization, scaling, and offset setting. For operations on small element groups such as rings, integrated near-memory arithmetic (NMC) is used. ) is required. Generally, such operations are programmable at a higher level than MVM. It requires capability and involves less input data.

[0090] Figure 13 illustrates a block diagram illustrating an exemplary near-memory computing SIMD engine. Specifically, Figure 13 shows the integration at the IMC output (i.e., ADC (Continued) Depicts a programmable single-instruction multiple-data (SIMD) digital engine. The exemplary embodiment shown has two SIMD controllers, one of which is One is for parallel control of BPBS near-memory computation, and the other is for arithmetic near-memory computation. This is for parallel control of computations. Generally, it involves combining it with a SIMD controller. The NMC shown may include and / or other such controllers. The data is grouped into eight blocks, each containing an IMC column and columns in various ways. To configure it, eight parallel computing channels (A / B, and 0-3) are provided. Nell includes a local arithmetic logic unit (ALU) and a register file (RF). To address throughput and layout pitch consistent with IMC calculations, four columns It is multiplexed between them. In general, other architectures can also be adopted. In addition, nonlinear relationships A lookup table (LUT)-based implementation is shown for a number of cases. It can be used as an activation function. Here, a single LUT is used across all parallel computing blocks. The bits of the shared LUT entry are broadcast serially between compute blocks. Next, each calculation block selects the desired entry and determines the bit precision of the entry. Bits are received serially over several corresponding cycles. This means that each parallel Controlled via LUT clients (FSMs) within the calculation block, and routing is broadcast. By sacrificing some features, we avoid the area cost of having a LUT for every computation block. do.

[0091] Near-memory cross-element computation. Generally, the computation is performed on individual output elements from an MVM computation. It is needed not only for processing but also between output elements. For example, long-term short-term memory (LSTM), This includes recurrent units with turntables (GRUs) and transformer networks. The near-memory SIMD engine in Figure 10 performs subsequent digital operations between adjacent IMC columns. It also supports reduction operations (adders, multiplier trees) between all columns.

[0092] As an example, output elements from different MVM operations are combined through element-wise calculations. In order to map LSTM, GRU, etc., the corresponding output vector elements are adjacent. The matrix is ​​made available for near-memory cross-element calculations in adjacent rows, with different I It can be mapped to an interleaved IMC column.

[0093] Figure 14 shows an example LSTM layer mapping function that utilizes cross-element near-memory computation. This illustrates the diagrammatic representation. Specifically, as illustrated in Figure 14, 2-bit weights Typical LSTM layer mapping to CIMU in the example (Bw=2). GRU It follows a similar mapping. Four MVM operations are performed to generate each output yt. intermediate output

number

number

number

[0094] In various embodiments, each CIMU is, this is, inside the CIMU, outside the CIMU, and / or C Each near memory, programmable, may be contained within a separate element in the array, including the IMU. It is associated with a single-instruction multiple-data (SIMD) digital engine. The digital engine uses input buffer data for inclusion within the feature vector map. Combining shortcut buffer data and / or output feature vector data, Alternatively, it is suitable for use in chronological alignment. Various embodiments are SIMD Enables computation across / between parallelized computation paths of an engine.

[0095] Shortcut buffering and merging. Pixel-level pipelining, Spanning between NN layers ensures that pipeline latency matches the latency of the NN path. Therefore, the shortcut path requires special buffering. In Figure 12, two To ensure that the data flow and latency of the path are aligned, such as the shortcut path The furling is incorporated along with the IMC input line buffering of the calculated NN path. It is possible that there are multiple overlapping shortcut passes (for example, UN The number of such buffers (as in et) is an important architectural parameter. Yes, however, using a buffer available from any IMC bank for this purpose This gives flexibility in mapping such overlapping shortcut paths. Yes, it is possible. The final sum of the shortcut paths and the NN-calculated paths is as shown. Supported by feeding shortcut buffer output to near-memory SIMD. The shortcut buffer is used to access input ports in a similar manner to the input line buffer. It can be ported. However, typically in CNNs, shortcut connections are used. The layers it passes through maintain a fixed number of output pixels, enabling the final pixel-level summation. This leads to a fixed number of operations between layers, typically when the IMC module is 1 This results in being fed by one IMC module. The exception to this is U-Net, which potentially makes additional input ports in the shortcut buffer beneficial. include.

[0096] Input feature map depth expansion. The number of IMC rows limits the input feature map depth that can be processed. This necessitates depth expansion through the use of multiple IMC banks. Deep input within a segment. If multiple IMC banks are used to process the channel, Figure 10 shows the segmentation. Includes hardware for adding the preceding segment to the subsequent IMC bank. The comment data is stored in the local input buffer and shortcut buffer between the output channels. The data is then provided in parallel. The parallel segment data is then customized between the two buffer outputs. The additions are performed together via an adder. To perform such additions, the IMC bank is used. By cascading, arbitrary depth expansion can be performed.

[0097] The adder output is fed to near-memory SIMD for further element-level calculations and cross-referencing. It enables elemental computation (e.g., activation functions).

[0098] On-chip network interface for weight loading. Input vector data for receiving input vector data from the network (i.e., input for MVM computation) In addition to the force interface, it also receives weight data from an on-chip network. An interface for (i.e., for storing matrix elements) may be included. This allows the matrix generated from MVM operations to be used in IMC-based MVM operations. This makes it possible, and this can be applied to various things, such as mapping transformer networks. This is useful in applications. Specifically, Figure 15 shows how the generated data is loaded. Used as a matrix, the matrix of the bidirectional encoder representation from the transformer (BERT) layer This graphically illustrates PING. In this example, the input vector X and the generated matrix are shown. Both Yi and 1 are loaded to the IMC module through the weight loading interface. It is done. The on-chip network is a single on-chip network, and multiple on As an on-chip network portion, or a combination of on-chip and off-chip network portions It may be implemented as a combination.

[0099] Scalable IMC architecture Figure 16 shows several embodiments of scalable NN acceleration based on IMC. This illustrates the high-level block diagram of the architecture. Specifically, Figure 16 is This illustrates a scalable NN accelerator based on IMC, and the surrounding IMC bank Integrated microarchitecture support for application mapping However, modules that enable scaling up the architecture through tiling and interconnection. Form a circle.

[0100] Figure 17 shows a 1152×256 IMC bank suitable for use in the architecture of Figure 16. This diagram illustrates a high-level block diagram of a CIMU microarchitecture having [the specified component]. In other words, the overall architecture is illustrated in Figure 16, but in that architecture An integrated IMC bank and an in-memory computing unit (CIMU) suitable for use with A module with microarchitecture support is shown in Figure 17. The researchers found that benchmark throughput, latency, and energy are related to the number of tiles. It was determined that it scales to (throughput / latency should scale proportionally). (And the energy remains virtually constant.)

[0101] As illustrated in Figure 16, the array-based architecture comprises the following: (1) 4x4 array in-memory compute unit (CIMU) core, (2) on-chip network between cores (3) Network (OCN), (4) Off-chip interface and control circuit, and (5) An additional weight buffer with a dedicated weight loading network for the CIMU.

[0102] As illustrated in Figure 17, each CIMU may include the following: (1) In-memory measurement (2) Flexible element units, an IMC engine for MVM, referred to as Computational Array (CIMA) NMC digital SIMD with a custom instruction set for the calculation of (3) wide Buffering and control circuits to enable NN dataflow. Each CIMU core , provides a high level of configurability and (application) for interfacing with the compiler Software instructions (for assigning / mapping instructions, such as NNs, to architectures) It can be abstracted into a software library, and therefore, in this case, instructions can be added in the future. It can also be done this way. In other words, the library uses elements mult / add, h(●)activat ion, (N-step convolutional stride+MVM+bat ch norm.+h(●)activation+max.pool), (dense Includes single / fused instructions such as +MVM.

[0103] OCN is a routing channel within a network in / out block and a dissociation It consists of a switch block that provides flexibility through its architecture, and the OCN is configurable. Working with capable CIMU input / output ports, it optimizes the data structure to and from the IMC engine. Optimize and improve locality of data across MVM dimensions and tensor depth / pixel index To maximize the OCN routing channel, while providing sufficient density, repeaters / To mitigate the insertion of pipeline-FFs, bidirectional wiring pairs may be included.

[0104] Using the IMC architecture, we create a neural network (NN) accelerator. It is possible to implement this, and multiple calculations in an in-memory computing unit (CIMU) are very flexible. The chip network is used to arrange and interconnect the chips, and the output of one CIMU is connected to another It may be connected to or flowed into the input of a CIMU or to multiple other CIMUs, and many CI The output of the MU may be connected to the input of one CIMU, and the output of one CIMU may be connected to another It may be connected to the output of the CIMU, for example. The on-chip network is a single As an on-chip network, as multiple on-chip network components, or on-chip It may be implemented as a combination of the pre- and off-chip network portions.

[0105] Referring to Figure 17, the CIMU data is transmitted through one of the following two buffers. Received from OCN. (1) Input buffer that provides data to CIMA in a configurable manner, and (2) Bypass CIMA and element-wise in a separate and / or converged NN activity value path A shortcut buffer that provides data directly to NMC digital SIMD for calculations. a. The central block is a mixed signal N (rows) x M (columns) for multi-bit element MVM (example). For example, it is a CIMA consisting of 1152 (rows) x 256 (columns) IMC macros. In its implementation form, CIMA uses metal fringe capacitors for fully row / column parallel computing. A transformation is employed. Each multiplication bit cell (M-BC) takes the input activation value data (IA / I 1-bit digital multiplication (XNOR) with Ab) and stored weight data (W / Wb) The capacitor is driven by (AND). This drives the power between the M-BC capacitors in the row. A load redistribution is triggered, and the inner product between binary vectors on the computation line (CL) is given. This means that multiplication is digital and accumulation is defined by high lithographic precision. Because it involves only the capacitor being processed, low computational noise (nonlinearity, variability) is generated. The 8-bit SAR ADC digitizes CL and bit-parallel / bit-serial (B P / BS) calculation enables extension to multi-bit activation values / weights, in which case, The weight bits are mapped to a parallel column, and the activation value bits are input serially. Therefore, each column is subjected to digital bit shifting (for appropriate binary weighting) and The sum of the ADC outputs is achieved using the simple multi-bit vector inner product. It performs the binary vector dot product. Digital BP / BS operations are performed using a dedicated NMC BPB. This is done in the S SIMD module, for 1-8 bit weight / activation values. Optimizable and programmable element-level operations (e.g., arbitrary activation functions) This is performed in the NMC CMPT SIMD module.

[0106] In the overall architecture, each CIMU moves activation values ​​between CIMUs. On-chip network (active value network), as well as from embedded L2 memory to CI On-chip network for transferring weights to the MU (weight loading interface) It is surrounded by (a). This is used in Coarse-Grained Reconstructible Arrays (CGRA). It has similarities to the architecture used, but provides a core that offers high-efficiency MVM and NN. It includes calculations of the element units that are the target of the acceleration.

[0107] Various options exist for implementing on-chip networks. (Figures 16-17) This approach involves routing segments aligned with the CIMU, and outputting from that CIMU. This enables the acquisition and / or provision of input to the CIMU. Then, the data generated from any CIMU is transmitted to any CIMU and any number of CIMUs. It can be routed to. The implementation adopted for the purposes described herein.

[0108] Various embodiments are integrated in-memory computing (IMC) architectures, and the IMC Supports scalable execution and data flow of mapped applications. The aim is to create a configurable IMC architecture that allows for multiple configurable integrated in-memory components. A computation unit (CIMU), comprising multiple configurable CIs that form an array of CIMUs. MU and input operands are transmitted from the input buffer to the CIMU, so input is transmitted between the CIMUs. To transmit operands, to transmit calculated data between CIMUs, and CIMU A configurable on-chip network for transmitting calculated data from the output buffer to the output buffer. It is equipped with the following.

[0109] Each CIMU receives computation data from the on-chip network and the received computation The data is processed using matrix-vector multiplication (MVM) by CIMU, and the output vector is calculated. Associated with an input buffer for constructing an input vector to generate pre-calculated data. It is being done.

[0110] Each CIMU ensures that the alignment of data flows across multiple CIMUs is maintained. According to the flow map, calculation data is received from the on-chip network, and the received calculations The computation data is given a time delay, and the delayed computation data is directed towards the next CIMU or output. It is associated with a shortcut buffer for transferring data from the input buffer. At least some of them are from the on-chip network or shortcut buffers. The received computation data may be configured to have a time delay. Data flow map This provides pixel-level pipelining to offer pipeline latency matching. It can be supported.

[0111] The time delays imposed by the shortcut buffer and input buffer are absolute time delays. Delay, predetermined time delay, time delay determined with respect to the size of the input calculation data, CIMU Time delay determined regarding the estimated computation time, control received from the data flow controller Responding to your signals, control signals received from other CIMUs, and events occurring within the CIMU. and include at least one of the control signals generated by the CIMU.

[0112] In some embodiments, each input buffer of multiple CIMUs in an array of CIMUs And at least one of the shortcut buffers performs pipeline latency matching. To provide a data flow map that supports pixel-level pipelining Thus, it is constructed.

[0113] The CIMU array also includes, among its respective input buffers and shortcut buffers... A parallel computing hardware configured to process input data received from at least one of the following sources. It may include hardware.

[0114] At least a subset of CIMU is an application mapped to IMC. Includes an operandloading network portion configured according to the data flow of the sequence. It can be associated with the on-chip network portion. Applications mapped to IMC The process involves a configured CIMU running at a given layer, with the following parallel output calculated data: Neurals mapped to IMC so as to be provided to a configured CIMU that runs in layers The network (NN) includes the parallel output calculated data, and each NN feature is... Forms a pixel.

[0115] The input buffer processes the input NN feature map data according to the selected stride step. It can be configured to transfer the data to parallel computing hardware within the CIMU. The NN performs convolution It may include a neural network (CNN), and the input buffer is the size of the CNN kernel. Used to buffer several rows of the input feature map corresponding to the size or height. It can be done.

[0116] Each CIMU uses iterative barrel shifting along with a column weighting process to create a single barrel shifter. Bit-parallel bit-serial (BP) is a process that follows the bit calculation and then accumulates the results. BS) Configured to perform matrix-vector multiplication (MVM) according to the calculation process. It may include an in-memory computing (IMC) bank.

[0117] Figure 18 shows that data on several parallel routing channels is taken from adjacent CIMUs. A mark to choose whether to obtain or provide from a previous network segment. By employing a ticplexer, the segment for acquiring input from the CIMU is high-resolution. This illustrates a bell block diagram.

[0118] Figure 19 shows how data from several parallel routing channels is fed to adjacent CIMUs. By employing a multiplexer to select whether or not to supply, the CIMU is output This diagram illustrates the high-level block structure of the segments that provide the necessary resources.

[0119] Figure 20 shows a multiplexed circuit for selecting which input is routed to which output. Exemplary examples employing (and optionally, flip-flops for pipe lining) This shows a high-level block diagram of a switch block. The number of parallel routing channels is an architectural parameter, and this is the desired number. Between classes of neural networks, complete routing possibility (between all points) or high probability You can choose to ensure routableness.

[0120] In various embodiments, L2 memory reduces access costs and networking complexity. To reduce the size, the CIMUs are positioned along the top and bottom, and each CIMU is divided into separate blocks. The amount of embedded L2 cache is determined by the architecture, which is appropriately selected for the application. This is a parameter, and this quantity is, for example, a typical NN model in the application of interest. It can be optimized for the number of parameters. However, a separate batch for each CIMU Splitting into locks results in additional buffering due to replication within the pipeline segment. L2 requires processing. Based on the benchmark used in this study, a total of 35MB of L2 is required. It is used. Depending on the application, other configurations or larger or smaller sizes may be used. That is appropriate.

[0121] Each CIMU, as described above, consists of an IMC bank, a near-memory computing engine, and data It is equipped with a buffer. The IMC bank is selected to be a 1152 x 256 array, In this case, 1152 optimizes the mapping of 3x3 filters with a maximum depth of 128. It is selected for the following. The number of dimensions of the IMC bank is determined by the amortization of the energy overhead of the peripheral circuits and It is selected to balance the amortization of area overhead.

[0122] Discussion of several embodiments The various embodiments described herein are formed using multiple CIMUs, and C To allow data to flow between IMUs and be processed in an efficient manner by CIMUs. Placing data in a location, and timing the mapped neural network (or other application). Data processed by the CIMU (or bypass specific CIMU) to maintain columns A variety of configurable / programmable modules targeting things like delaying An array-based architecture (array) that is operationally enhanced through the use of or all of the It provides 1-dimensional, 2-dimensional, 3-dimensional...n-dimensional solutions as needed / desired. Advantageously, various embodiments allow n-dimensional CIMU arrays to communicate over a network. This allows for the development of neural networks (NNs) and CNNs of various sizes and complexities, where matrix multiplication is a crucial solution component. , and / or other problem spaces such that a scale can be obtained from various embodiments Enables durability.

[0123] Generally speaking, a CIMU is an in-memory computing array (CIMA), and exemplarily speaking... CIMA provides programmable in-memory computation capabilities such as matrix-vector multiplication. Various configuration registers are used to configure the bit cell CIMA, including various It has structural elements. In particular, a typical CIMU multiplies the input matrix X by the input vector A. The task is to generate the output matrix Y. CIMU is an in-memory computing array (CI MA)310, Input activation value vector reshaping buffer (IA BUFF)320, Sparsity / AND logic controller 330, memory read / write interface Face 340, line decoder / WL driver 350, multiple A / D converters 360, and near It is depicted as including a Memory Computation Multiplication Shift Accumulation Data Path (NMD) 370.

[0124] Regardless of how they are implemented, each CIMU described herein is a CIM On-chip network for moving activation values ​​between U (in the case of NN implementation, activation values On-chip networks such as networks, and architectural trade-offs Regarding this, as mentioned above, in order to incorporate the weights and move them from L2 memory to CIMU On-chip networking (e.g., weight loading interface) We are surrounded.

[0125] As described above, the activity value network is an activity value network in various embodiments. This is interpreted as an I / O data transfer network, an inter-CIMU data transfer network, etc. To enable this, calculation input and output data are transmitted from CIMU to CIMU and between CIMUs. It has a configurable / programmable network for transmitting. Therefore, this These terms refer to configurable / programmable data transfer to and from a CIMU. They are used interchangeably to some extent, encompassing the network.

[0126] As mentioned above, the weight loading interface or network is the CIMU It includes a configurable / programmable network for loading operands internally, It may also be referred to as an operandloading network. Therefore, these The term refers to a configurable process that loads operands such as weighting coefficients into a CIMU. To some extent, to include a functional / programmable interface or network It is used interchangeably.

[0127] As mentioned above, the shortcut buffer is located within or outside the CIMU. It is depicted as being associated with the IMU. The shortcut buffer is also N Applications that are mapped to shortcut buffers, such as N and CNN Depending on the circumstances, it may be used as an array element.

[0128] As mentioned above, near memory programmable single instruction multiple data (SIMD) digital The Tal engine (or near memory buffer or accelerator) is located within the CIMU or It is depicted as being associated with the CIMU, such as an external CIMU. Near memory Programmable Single Instruction Multiple Data (SIMD) Digital Engine (or Near Memory Engine) Buffers (or accelerators) are also used in NNs, CNNs, etc. Depending on the application mapped to the array, it may be used as an array element. .

[0129] In some embodiments, the above-described input buffer is also used in convolutional neural networks and the like. CI in a configurable manner that provides configurable shifting that corresponds to riding Please note that you may provide data to CIMA within MU. To implement nonlinear calculations, we map inputs to outputs according to various nonlinear functions. A lookup table is provided individually to the SIMD digital engine of each CIMU. or may be shared across multiple SIMD digital engines of the CIMU (e.g.) For example, the implementation of a parallel lookup table for a nonlinear function. In this way, each SIM The D digital engine selectively assigns specific bits appropriate to its SIMD digital engine. The SIMD digital engine can process the data from the location of the lookup table. It will be broadcast in between.

[0130] Architecture Evaluation - Physical Design IMC-based N compared to conventional spatial accelerators composed of digital PE The evaluation of the N accelerator will be carried out. In both designs, bit-precision scalability T is possible, but fixed-point 8-bit calculation is assumed. CIMU, Digital P E, on-chip network block, and embedded L2 array up to 16n physical design It is implemented using mCMOS technology.

[0131] Figure 21A shows the CIMU architecture of an embodiment implemented with 16nm CMOS technology. Figure 21B shows a layout diagram of the 4× provided in Figure 21A. This diagram illustrates the layout of a full chip consisting of four tiling CIMUs. The mixed signal properties of the kitech are due to the fully custom transistor-level design, as well as the standard cell architecture. Both RTL design and synthesis and APR are required. Functional verification is performed at the RTL level. This means that the operating model of the IMC bank is It is necessary to adopt this, and the operating model itself is a Spectre (equivalent to SPICE) simulator. It is verified through a ration.

[0132] Architecture Evaluation - Energy and Speed ​​Modeling The physical design of IMC-based architectures and digital architectures involves parasitic capacities. Based on post-layout extraction, it enables robust energy and speed modeling. Speeds are, respectively (from both STA and Spectre simulations), I Achievable clock cycles for MC-based architectures and digital architectures The frequency is parameterized as FCIMU and FPE. The energy is parameterized as follows: It will be lameterized. ● Input buffer (EBuff). This is the input buffer and shortcut buffer. Energy in the CIMU required to write and read input activation values ​​from / —. ● IMC (EIMC). This is an IMC bank (using 8-bit BPBS calculations). This is the energy in the CIMU required for the MVM calculation. ● Near-memory calculation (ENMC). This is required for near-memory calculation of all IMC column outputs. This is the energy at CIMU, as it is believed. ● On-chip network (EOCN). This allows activity value data to be moved between CIMUs. This is the energy in an IMC-based architecture for achieving this. ● Processing Engine (EPE). This performs 8-bit MAC operations and outputs data to adjacent PEs. This is the energy used in digital PE for movement. ● L2 readout (EL2). This is an IM for reading weight data from L2 memory. Energy in both C-based architectures and digital architectures. . ● Weight Loading Network (EWLN). This network loads weight data from L2 memory. IMC-based architecture and digital for moving to CIMU and PE respectively This is the energy in both aspects of the barrel architecture. ●CIMU weight loading (EWL, CIMU). This is used to write weight data. This is the energy in the CIMU. ●PE weight loading (EWL,PE). This is a digital loading method for writing weight data. This is the energy in Tal PE.

[0133] Architecture evaluation - Neural network mapping and execution To evaluate the impact of scaling up the architecture, IMC-based architecture Different physical chip regions are considered for comparison with digital architectures. The regions correspond to 4x4, 8x8, and 16x16 IMC banks. For benchmarking purposes Therefore, a common CNN set is adopted to achieve energy efficiency, throughput, and latency. The metrics of the measurement are evaluated for both small batch sizes (1) and large batch sizes (128). do.

[0134] Figure 22 illustrates how the NN mapping flow is mapped to an 8x8 array of the CIMU. The three stages of mapping software flow to architecture are graphically represented. This is illustrated in Figure 23A, which shows the sample placement of layers from the pipeline segment. This illustrates the process, and Figure 23B shows sample routing from pipeline segments. This illustrates the concept.

[0135] Specifically, the benchmark maps each architecture via a software flow. It is mapped. In the case of an IMC-based architecture, the software flow mapping is This involves three stages, namely allocation, placement, and routing, as shown in Figure 22.

[0136] The allocation is based on the filter mapping, layering, and BPBS expansion described above. This corresponds to allocating CIMUs to NN layers in different pipeline segments.

[0137] The placement is such that the CIMUs assigned to each pipeline segment are within the architecture. This corresponds to mapping to a CIMU location (as depicted in Figure 23A). This is the activation value network segment required between the transmission and reception of the CIMU. A simulated annealing algorithm is employed to minimize the effects. The sample arrangement of the layers from the plane segment is shown in Figure 23A.

[0138] Routing is performed by CIMUs (for example, on-chip networks that form inter-CIMU networks). Routines within the on-chip network move activation values ​​between network sections. This corresponds to configuring Grease Resources. This is under routing resource constraints, CI Minimize the activation value network segment required between MU transmission and reception. Dynamic programming is employed for this purpose. Sample routines from pipeline segments. The ng is shown in Figure 23B.

[0139] Following each stage of the mapping flow, the functionality is verified using the behavioral model, and this This is also verified against RTL design. After three stages, configuration data is output, and configuration data The data is loaded into an RTL simulation for final design verification. (Operational model) It is cycle-accurate, and based on the modeling of the above parameters, energy - Enables the characterization of speed.

[0140] In the case of a digital architecture, the application mapping flow is hardware Using replication to maximize utilization, with typical layer-by-layer mapping. Here again, Cycl Using a highly accurate behavioral model, we verify the functionality based on the above modeling. , perform energy and velocity characterization.

[0141] Architecture scalability evaluation - energy, throughput, and latency analysis Compared to digital architectures, the energy efficiency of IMC-based architectures This is increasing. In particular, batch sizes of 1 and 128, respectively, are increasing across the entire benchmark. Regarding the IMS, in an IMC-based architecture, the gain is 12-25 times and 17- A gain of 27 times is achieved. This means that the matrix loading energy is effectively compensated for. This suggests that the column utilization improved as a result of the layering and BPBS deployment.

[0142] Compared to digital architectures, the throughput of IMC-based architectures is It has improved. In particular, across the entire benchmark, batch sizes of 1 and 128 respectively have improved. Regarding the scalar, in an IMC-based architecture, a gain of 1.3 to 4.3 times and 2. A gain of 2 to 5.0 times has been achieved. Throughput gain is not as large as energy efficiency gain. It won't work. The reason is that layering maps the later layers in each pipeline segment. This effectively reduces the loss utilization rate of the IMC hardware used for tracking. In fact, this effect is amortized for small batch sizes where pipeline loading delays are offset. This is most noticeable, and is somewhat small for large batch sizes. However, even for large batches... To avoid overlap of convolution kernels across different inputs, the inputs Some delay is required in CNNs to clear the iPline.

[0143] Compared to digital architectures, IMC-based architectures have lower latency. It is being reduced. The reductions observed track throughput gain and follow the same rationale.

[0144] Architecture Scalability Assessment - Impact of Layer Deployment and BPBS Deployment To analyze the benefits of layered deployment, IMCA with layer-by-layer mapping compared to layered deployment. - The ratio of the total amount of weight loading required in the architecture is considered. Layer expansion is particularly As the architecture scales up, this brings about a substantial reduction in weight loading. This has been determined by the inventors. More specifically, from 4x4, 8x8 to 1 In IMC bank scaling to 6x16, weight loading is performed on a layer-by-layer basis. (Batch size 1) accounts for 28%, 46%, and 73% of the average total energy. Weight loading accounts for only 23% of the average total energy in layered deployment (batch size of 1). In contrast, these account for 24% and 27%, respectively, enabling much better scalability. Furthermore, due to the significantly higher energy of MVM compared to IMC, conventional layer-by-layer mapping The ng is acceptable in the digital architecture, and the average total energy (1 batch size) They account for 1.3%, 1.4%, and 1.9%.

[0145] To analyze the benefits of BPBS deployment, factors that reduce the proportion of unused IMC cells were examined. This involves column merging (as physical and effective utilization gains) as well as duplication and shifting. Both are shown in Figure 18. As can be seen, the ratio of unused bit cells is significant. Significant reduction has been achieved. The combined average bit cell of column merging, duplication, and shifting. The effective utilization rates were 82.2% and 80.8%, respectively.

[0146] Figure 24 is suitable for use in implementing various control elements or parts thereof, and is a figure Regarding the functions associated with the various elements described herein, etc. A high-level block diagram of a computing device suitable for use in performing the specified functions is shown. It is what it is.

[0147] For example, the NN and application mapping tools described above, as well as various A The application program is a general-purpose computing application as depicted herein with respect to Figure 24. It can be implemented using kitech.

[0148] As depicted in Figure 24, the computing device 2400 includes a processor element 2402 (for example) If a central processing unit (CPU) or other suitable processor is used, memory 2404 (for example, RAM (RAM), read-only memory (ROM), etc., collaborative modules Process 2405 and various input / output devices 2406 (e.g., communication module) Includes (wires, network interface modules, receivers, transmitters, etc.).

[0149] The functions described and mentioned herein are, for example, general-purpose computers, one or more specific functions Using an ASIC (Application-Specific Integrated Circuit), or any other hardware equivalent, the hardware It is understood that this can be implemented in the form of a software or in a combination of software and hardware. It will be understood. In one embodiment, the cooperative process 2405 loads into memory 2404 It may be performed by processor 2402 to implement the functions discussed herein. Therefore, collaborative process 2405 (including related data) is computer-readable. It can be stored in a storage medium, such as RAM memory, a magnetic or optical drive, or a disk. ru.

[0150] The computing device 2400 depicted in Figure 24 is a functional element or as described herein. A general architecture and mechanism suitable for implementing the functional elements described in the document. It will be understood that this provides the capability.

[0151] Some of the steps discussed herein are, for example, various method steps. It can be implemented in hardware as a circuit that works in cooperation with the processor to perform this task. It is intended that the functions / elements described herein are computer program products and It may be implemented in this way, and when a computer instruction is processed by a computing device, The calculation is performed so that the methods or techniques described in the specification are invoked or otherwise provided. To adapt the operation of the device. Instructions for calling the method of the present invention are provided on a fixed medium or Tangible and non-temporary computer-readable media such as removable media or memory devices Whether stored in the body or in the memory of a computing device that operates according to instructions good.

[0152] Various embodiments are mappings and designs associated with the embodiments described herein. Computer implementation tools configured for testing, operation, and / or other functions, applications The aim is to create application programs, systems, etc. For example, using the computing device shown in Figure 24. Using this, an integrated in-memory computing (IMC) architecture as described herein It may provide a computer implementation method for mapping applications, neural networks, or other functions. ru.

[0153] As mentioned above regarding Figures 22-23, the software flow or application Mapping NN or other functions to IMC hardware / architecture is Generally, it involves three stages: allocation, placement, and routing. Allocation is Based on the aforementioned filter mapping, layering, and BPBS deployment, different patterns are used. This corresponds to assigning CIMUs to NN layers in the iPline segment. The arrangement is as follows: CIMUs assigned in the iPline segment are physical CIMU locations within the architecture This corresponds to mapping to a sequence. Routing is handled by CIMU (e.g., CIM). To move activation values ​​between the on-chip network portion that forms the inter-U network. This corresponds to configuring routing resources within the on-chip network.

[0154] In a broad sense, these computer implementation methods are for the desired / target application, Accepts input data describing a neural network or other function, and the desired / target application Program an IMC architecture that enables NN or other functions. It can generate output data in a form suitable for the configured use in the response. This is by default. The IMC architecture, or the target IMC architecture (or a part thereof) It may be provided as such.

[0155] The computer implementation method is the desired / target application, NN, or input date. Characterize and define other functions from the perspectives of operation, operation sequencing, output data, etc. To describe or render computation graphs, data flow representations, high / mid / low-level descriptors, etc. Various known tools and techniques may be employed.

[0156] The computer implementation method is characterized by allocating IMC hardware as appropriate. The applications, NNs, or other functions defined, defined, or described by IMC Mapping to the architecture and running the IMC hardware - In a manner that substantially maximizes input and energy efficiency (e.g., IMC hardware) Various aspects of computational parallelism and pipelining, etc., as discussed herein It can be configured to do so (by using techniques). Computer implementation methods The method involves using a tiled array of in-memory computing hardware to perform neural networks. Mapping to, in-memory, specific calculations required by neural networks The process of allocating computing hardware, and the allocated in-memory computing hardware Performing placement to a specific location within a tiled array (optional) Specifically, the arrangement involves in-memory computing hardware that provides a specific output and specific inputs. (When configured to minimize the distance to the in-memory computing hardware that benefits from it), An optimization method (e.g., simulated annealing) is employed to minimize such distances. This involves configuring the available routing resources and performing in-memory computing hardware The output from is forwarded to the input of in-memory computing hardware in a tiled array. To achieve routing between the deployed in-memory computing hardware, Minimizing the total amount of routing resources required, and / or such routing We employ optimization methods (e.g., dynamic programming) to minimize programming resources. It is configured to take advantage of some or all of the features described herein, such as That's fine.

[0157] Figure 34 illustrates a flowchart of the method according to the embodiment. Specifically, Figure 34 This maps applications to an integrated in-memory computing (IMC) architecture. A computer implementation method, wherein the IMC architecture comprises multiple configurable in-memory Computational Unit (CIMU), comprising multiple configurable C units that form an array of CIMUs. Input data is transmitted between the IMU and the CIMU array, and calculated data is transmitted between the CIMUs. Furthermore, a configurable on-chip network for transmitting output data from the CIMU array. This describes a computer implementation method that includes a workpiece.

[0158] The method in Figure 34 applies to the IMC architecture discussed above, or This is a computation graph, dataflow map, and / or a computation graph suitable for use when programming neural networks. Or it targets the generation of other mechanisms / tools. This method generally applies as described above. Then, various configurations, mappings, optimizations, and other steps are performed. In particular, this method Steps to allocate IMC hardware according to the computational requirements of the application or NN. IMC hardware that generates output data and IMC that processes the generated output data The assigned IMC hardware is configured in a way that minimizes the distance to the hardware. The steps include defining the placement of the IMC hardware to a location within the IMC core array, and the IMC hardware The step of configuring an on-chip network to route data between A and B, Steps to configure the power / output buffer, shortcut buffer, and other hardware. , the development of BPBS discussed above (e.g., duplication and shifting, column replication, other techniques) Steps to apply, steps to apply replication optimization, steps to apply stratified optimization, steps to apply spatial optimization These are described as step-by-step, time optimization steps, pipeline optimization steps, etc. Various calculations, optimizations, and decisions may be implemented using arbitrary logical sequences to reach a solution. It may be repeated to reach or be repeated, where the data flow map is IM It may be generated for use in programming the C architecture.

[0159] One embodiment involves an application integrated into a configurable in-memory IMC architecture. This provides a computer implementation method that maps to computing (IMC) hardware, and IMC The software consists of multiple configurable in-memory computing units that form an array of CIMUs. CIMU) transmits input data to the CIMU array and transmits calculated data between CIMUs. A configurable on-chip network for transmitting and transmitting output data from the CIMU array. The method utilizes the parallelism and pipelining of IMC hardware, and is equipped with a network. By using this, IMC hardware is allocated according to the application calculations, resulting in high throughput. Generate IMC hardware allocation configured to provide application computing. The IMC generates the output data by assigning the IMC hardware to the IMC hardware. Minimize the distance between the hardware and the IMC hardware that processes the generated output data. Define it in a format that is easy to convert to a location within the CIMU array, and on-chip network The network is configured to route data between IMC hardware, This application may include NN. Various steps are discussed throughout this application. It can be implemented according to the ping technique.

[0160] By using the various mapping and optimization techniques described herein, Various modifications may be made to the computer implementation method. For example, the application, NN, Alternatively, the function calculates parallel output data, which then forms each NN feature map pixel. In cases such as the one described above, the parallel output data of a CIMU configured to run at a given layer is then calculated as follows: It may be mapped to the IMC so as to be provided to a configured CIMU that runs in layers. Furthermore, the computational piping is performed in a given layer rather than in subsequent layers, resulting in a large number of configured By allocating CIMUs, we can compensate for more computation time in a given layer than in the next layer. It may be supported.

[0161] The functions described and mentioned herein are, for example, general-purpose computers, one or more specific functions Using an ASIC (Application-Specific Integrated Circuit), or any other hardware equivalent, the hardware It is understood that this can be implemented in the form of a software or in a combination of software and hardware. It will be understood. Some of the steps discussed herein are, for example, In order to execute various steps, the hardware contains circuits that work in cooperation with the processor. It is intended to be installed. The parts of the functions / elements described herein are computer It may be implemented as a program product, and computer instructions are processed by computing devices. When applied, the methods or techniques described herein are invoked or otherwise provided. The operation of the computing device is adapted so that it can be done. The instruction for calling the method of the present invention is, A tangible and non-temporary computer such as a fixed medium, removable medium, or memory It is stored on a readable medium or in the memory of a computing device that operates according to instructions. It may also be used.

[0162] With respect to the various figures, the systems, methods, apparatus, mechanisms, techniques, and so forth described herein Various modifications may be made to these parts, and such modifications are within the scope of the present invention. It is intended that there be a step or While a specific order of placement of functional elements is presented, various other steps or functional elements are also presented. The order / arrangement may be used within the context of various embodiments. Furthermore, modifications to the embodiments may be made. While each modification may be discussed individually, various embodiments may use multiple modifications simultaneously or sequentially. You may do so, and you may also use compound modifications, etc.

[0163] Specific systems, devices, methodologies, and mechanisms have been disclosed as discussed above. Without departing from the inventive concept of this specification, many modifications other than those already described may be made. It should be obvious to those skilled in the art that this is possible. Therefore, the subject of the present invention is This disclosure is not limited to the intent of this disclosure. Furthermore, in interpreting this disclosure, all The term should be interpreted as broadly as possible to match the context. In particular, "preparing / The usage of "comprises" and "comprising" A word refers to an element, component, or step that exists, is used, or is explicitly used. Elements that indicate they can be combined with other elements, components, or steps that are not referenced. It should be interpreted as referring non-exclusively to components or steps. In addition, this The references listed in this specification are also part of this application, and their entirety is fully contained herein. It is incorporated by reference, as described.

[0164] Exemplary IMC Core / CIMU Discussion Various embodiments of the IMC core or CIMU may be used within the context of various embodiments. Good. Such an IMC core / / CIMU is built around an in-memory computing accelerator. Integrating feasibility and hardware support, necessary for scaling to practical applications This enables programmability and virtualization. Generally, in-memory computation involves matrix computing. Matrix multiplication is implemented, where matrix elements are stored in a memory array and vector elements are stored in a memory array. It is broadcast in parallel via (i). Some embodiments of the embodiment are as follows: The aim is to enable the programmability and configurability of the architecture.

[0165] In-memory computation typically involves either matrix elements, vector elements, or both. This is accompanied by a 1-bit representation. This means that memory stores data in independent bit cells. In contrast, broadcasting involves different bits between the bits required for multibit calculations. This is because it is performed in a parallel, homogeneous manner without providing binary weighted coupling. In this invention, the extension to multi-bit matrix and vector elements is bit parallel / bit This is achieved via a serial (BPBS) scheme.

[0166] To enable common computational operations that often involve matrix-vector multiplication, it is highly structured. It includes a possible / programmable near-memory compute data path. This is both in-memory. The calculations required to extend Mori calculations from bit-level calculations to multi-bit calculations This makes it possible, and generally speaking, it means that in-memory computation is no longer constrained to the 1-bit representation inherent in it. Supports multibit operations that cannot be performed. Programmable / configurable, and multibit Since the calculation of analog is more efficient in the digital domain, in this invention, analog A digital conversion is performed following an in-memory calculation, and in certain embodiments, it is configurable. The data path is multiplexed across eight ADC / in-memory compute channels, but other multiplexing Ratios can be used. This can also be used to support multi-bit matrix elements. The BPBS scheme aligns well with the implementation, and the embodiment provides support for operands up to 8 bits. To be served.

[0167] Input vector sparsity is common in many linear algebra applications. The present invention integrates support to enable sparsity control proportional to energy. This means that the broadcasting of bits from the input vector corresponding to zero-value elements This is achieved by masking the ng (such masking is bit serial This is done for all bits in the broadcast process. Energy and computational energy within the memory array are saved.

[0168] Internal bit unit computation architecture for in-memory computation and typical microprogram Considering the external digital word architecture of the sesser, the input vector passes through it. The provided computing interface, through which matrix elements are written and read Data reshaping hardware is used for both of the memory interfaces being used. ru.

[0169] Figure 25 illustrates a typical structure of an in-memory computing architecture. (Note) In-memory array consisting of rearrays (which may be based on standard bit cells or modified bit cells) The calculation involves two additional "vertical" signal sets, namely (1) the input line and (2) the cumulative It involves calculation lines. Referring to Figure 25, a two-dimensional array of bit cells is depicted, and multiple Each of the in-memory computation channels 110 of the number is a common cumulative sum of each channel of bit cells. Lines and bit lines (columns), and their respective input lines and word lines (rows) It can be seen that each associated bit cell has its own column. As illustrated in Figure 25. This simply shows the row / column relationships within the context of an array of bit cells, such as a two-dimensional array of bit cells. To that end, the columns and rows of signals are denoted herein as being "perpendicular" to each other. Please note that the term “vertical” as used herein does not refer to any particular geometric shape. It is not intended to convey any academic relationship.

[0170] The input / bit and cumulative / bitset of the signal are based on existing signals in memory (e.g., watt The lines (drive lines, bit lines) may be physically combined with or separate from the lines. To implement column-vector multiplication, the matrix elements are first loaded into memory cells. Then, multiple input vector elements (and in some cases all of them) are applied at once via the input line. This means that local calculations, typically some form of multiplication, are performed in memory. This is performed in each bit cell. Then, the result of the calculation is carried out on the shared accumulation line. It is moved. In this way, the cumulative line is activated by multiple input vector elements. This represents the calculation result across the bit cells. This means that the bit cell can multiply the bit line at once. Standard memory access is accessed via and activated by a single word line. These are in contrast.

[0171] As mentioned above, in-memory computation has several important attributes. Firstly, computation is Typically, it is analog. This is because the constraint structure of memory and bit cells is simple. This requires a more complex computational model than what is possible with digital switch-based abstraction. This is for the purpose of: Secondly, local operations in bit cells are typically performed in bit cells This involves calculations using a 1-bit representation stored in memory. This means that the bit cells in a standard memory array are , because they do not join each other in any binary weighting scheme, and any such join is This must be achieved by a method of accessing / reading bit cells from the surrounding area. The extension of in-memory computation proposed in this invention is described below.

[0172] Extensions to near-memory and multi-bit computation. In-memory computation uses matrix vectors in a way that conventional digital acceleration lacks. While it has the potential to handle multiplication, a typical computation pipeline surrounds matrix-vector multiplication. It will involve a wide range of other operations. Typically, such operations are conventional digital operations. Despite being adequately addressed by ceration, such acceleration Placing the in-memory computing hardware near the in-memory computing hardware improves parallelism and high throughput. Put (and therefore the need for high communication bandwidth in the way), and related to in-memory computation High value in an appropriate architecture for dealing with common computational patterns. It may have. Many of the surrounding operations will preferably be performed in the digital domain. From there, analog-to-digital conversion via ADC follows each of the in-memory calculation accumulation lines. It is included and therefore called an in-memory computation channel. The main challenge is that each in-memory The invention involves integrating ADC hardware into the pitch of the Mori calculation channel, but in this invention The appropriate layout approach taken makes this possible.

[0173] By introducing an ADC following each computing channel, each bit parallel / bit Supports multi-bit matrix and vector elements via serial (BPBS) computation. This enables efficient methods for extending in-memory computation. Bit-parallel computation is This involves loading different matrix element bits into different in-memory computation sequences. Then, different The ADC output from the resulting column is bit-shifted appropriately to represent the corresponding bit weighting. Then, digital accumulation is performed across all columns to obtain the multi-bit matrix element calculation result. On the other hand, bit serial computation uses the next output corresponding to the subsequent input vector bits. Before digital accumulation, the ADC output is stored each time, and the stored output is appropriately bitwise calculated. This involves applying each bit of the vector element at once. Analog and digital Such a BPBS approach that enables hybridization with calculations is a BPBS approach. The approach overcomes the access costs associated with conventional memory operations, while analog (1 The high-efficiency, low-precision regime of (bits) is replaced by the high-efficiency, high-precision regime of digital (multi-bit) systems. Since they are used together, it is extremely efficient.

[0174] A wide range of near-memory computing hardware can be considered, but the current implementation of the present invention Details of the hardware integrated in such a multi To facilitate the physical layout of the bit digital hardware, eight in-memory components are used. The computation channel is multiplexed across each near-memory computation channel. This enables in-memory computation. This highly parallel computing achieves throughput that matches the high-frequency operation of digital near-memory computing. It should be noted that this becomes possible (highly parallel analog in-memory computing is digital near-memory computing). (Operates at a lower clock frequency than Mori calculations). Then, each near-memory calculation channel , digital barrel shifter, multiplier, accumulator, and lookup table (LUT) and This includes implementation methods for fixed nonlinear functions. In addition, it is associated with near-memory computing hardware. A configurable finite state machine (FSM) controls the computation through hardware. They are combined.

[0175] Input interfacing and bit scalability control To integrate in-memory calculations with a programmable microprocessor, internal bits Unit operations and representations are performed using external multiplication tables adopted in typical microprocessor architectures. It must be properly interfaced with the cibit representation. Therefore, the data is The shaping buffer is an input vector interface and memory read / write interface. The matrix elements are contained in both interfaces and stored in a memory array. Details of the design adopted in the form are described below. Data reshaping buffer This enables bit-width scalability of input vector elements, while in-memory computation The maximum bandwidth for data transfer to the software is between external memory and other architectural blocks. Maintained between blocks. The data reshaping buffer enters element by element of the input vector. It receives incoming parallel multibit data and outputs parallel single bit data for all vector elements. It consists of a register file that functions as a line buffer providing bit data.

[0176] In addition to word-level / bit-level interfacing, it is also applied to the input vector. Hardware support for convolution operations is also included. Such operations are convolutional. This is particularly noticeable in neural networks (CNNs). In this case, matrix-vector multiplication is, It will only run on a subset of the new vector elements that need to be provided (other input vectors) The toll elements are stored in a buffer and simply shifted appropriately. This results in high sloop. The bandwidth constraints for acquiring data to the in-memory computing hardware are relaxed. In embodiments of the present invention, a suitable bit serial sequence of multibit input vector elements The convolutional support hardware that must perform the single operation has output readout, In a dedicated buffer, data is appropriately shifted for possible convolutional striding. It will be implemented in [location / platform].

[0177] Control of dimensionality and sparsity For programmability, the hardware must address two additional considerations. (1) The dimension of a matrix / vector can be variable between applications, 2) In many applications, vectors will be sparse.

[0178] Regarding the number of dimensions, in-memory computing hardware often uses tiling arrays. Enable / disable the selected portion and use only the energy at the desired dimensional level in the application. Integrate control to consume ghee. However, in the BPBS approach, the input vector The number of dimensions has a significant impact on computational energy and SNR. Regarding SNR, each ingredient In bitwise calculations in Mori computation channels, each input (provided on the input line) and Assuming that calculations between data stored in bit cells result in a 1-bit output, The number of possible different levels on the calculation line is equal to N+1, where N is the input vector. This is a dimensional number. This suggests the need for a log2(N+1) bit ADC. However, ADCs have an energy cost that scales strongly with the number of bits. Therefore, in order to reduce the relative contribution of ADC energy, in the ADC, It is beneficial to support N which is large but smaller than log2(N+1) bits. As a result of doing this, the signal-to-quantized noise ratio (SQNR) of the computation is standard. Unlike fixed-precision calculations, the accuracy decreases with the number of ADC bits. Therefore, various... Support the application level dimensions and SQNR requirements with corresponding energy consumption. To achieve this, hardware support for a configurable number of input vector dimensions is essential. For example, if a reduced SQNR is acceptable, then input vector segments with a large number of dimensions If the system should be supported, while a high SQNR must be maintained, Lower-dimensional input vector segments are combined from different in-memory computation banks. The inner product result from multiple possible input vector segments should be supported. (Therefore, in particular, the number of input vector dimensions is ideally consistent with standard fixed-precision arithmetic.) To ensure sufficient computation, the signal can be reduced to a level set by the number of ADC bits. The hybrid analog / digital approach taken in this invention is This makes it possible. That is, by masking the input vector elements and broadcasting only the desired number of dimensions. The cast can be filtered. This allows the input vector to be filtered in proportion to its dimensionality. This saves broadcast energy and bit cell computation energy.

[0179] Regarding sparseness, the same masking approach can be applied to the entire bit serial operation. Prevents broadcasting of all input vector element bits corresponding to zero-value elements. This is possible. The BPBS approach adopted contributes particularly to this. Note that this is because the expected number of non-zero elements is a factor in sparse linear algebra applications. While this provides much insight, the input vector dimension can become large. Therefore, The BPBS approach allows for increasing the number of input vector dimensions, while cumulative The number of levels that need to be supported on the line is within the ADC resolution. This ensures a high computational SQNR. The expected number of non-zero elements is... However, it is not possible to support a variable number of actual non-zero elements, which can differ for each input vector. , remains indispensable. This means that masking hardware is simply a given vector Count the number of zero-value elements, and then the corresponding offset is calculated as the digital value after BPBS calculation. Since it must be applied to the final inner product result of the domain, hybrid analog This can be easily achieved with a digital approach.

[0180] Exemplary Integrated Circuit Architecture Figure 26 illustrates a high-level block diagram of an exemplary architecture according to an embodiment. Specifically, the exemplary architecture in Figure 26 is an example of various embodiments described herein. To test this, VLSI fabrication techniques using specific components and functional elements are employed. It was implemented as an integrated circuit. Different components (e.g., larger or more powerful CP) Further embodiments having U, memory elements, processing elements, etc., are within the scope of this disclosure. It will be understood that this is intended by the inventors.

[0181] As illustrated in Figure 26, the architecture 200 has a central processing unit (CPU) 2 10 (for example, a 32-bit RISC-V CPU), Program Memory (PMEM) 22 0 (for example, 128KB program memory), data memory (DMEM) 230 (for example) , 128KB data memory), external memory interface 235 (for example, exemplary, Access one or more 32-bit external memory devices (not shown) and thereby Boot loader module 240 (configured to expand accessible memory) (For example, configured to access an 8KB off-chip EEPROM (not shown)) (Includes various configuration registers 255, and according to the embodiments described herein, Computational memory units (CIs) configured to perform arithmetic calculations and various other functions. MU)300, Direct Memory Access (DMA) mode with various configuration registers including 265 Joule 260, and a universal asynchronous receiver / transmitter for receiving / transmitting data. UART module 271, General-purpose input / output (GPIO) module 273, It is equipped with various support / peripheral modules such as Timer 274. Other elements not shown may also be included in the architecture 200 of Figure 26, for example. For example, SoC component modules (not shown).

[0182] The CIMU300 is very well suited for matrix-vector multiplication, but not for other types of calculations / The calculation may be more preferably performed by a non-CIMU computing device. Therefore, various In this embodiment, the selection of a computing device for a specific calculation and / or function is more efficient. The CIMU300 and near memory can be controlled to provide computational functionality. A proximity coupling is provided.

[0183] Figure 27 shows an exemplary in-memory computing unit suitable for use in the architecture of Figure 26. This shows the high-level block diagram of (CIMU)300. The following explanation is for Figure 26. Architecture 200, and suitable examples for use within the context of Architecture 200. Regarding the exemplary CIMU300.

[0184] Generally speaking, the CIMU300 is an in-memory computing array (CIMA), for example. CIMA provides programmable in-memory computation capabilities such as matrix-vector multiplication. To provide this, the CIMA of the bit cell is configured via various configuration registers, It has various structural elements. In particular, the exemplary CIMU300 has an input vector in the input matrix X. A 590kb, 16-bank CIM tasked with generating the output matrix Y by multiplying by A. It is structured as U.

[0185] Referring to Figure 27, CIMU300 is an in-memory computing array (CIMA)310, Input activation value vector reshaping buffer (IA BUFF) 320, sparsity / A ND logic controller 330, memory read / write interface 340, A row decoder / WL driver 350, multiple A / D converters 360, and near-memory computation multiplication. It is depicted as including Shift Cumulative Data Path (NMD) 370.

[0186] The exemplary In-Memory Computing Array (CIMA) 310 is a 4x4 clock-gateable array. A 256×(3×3×2) array is arranged as a 64×(3×3×64) in-memory computing array. 56) Equipped with an in-memory computing array, and therefore a total of 256 in-memory computing cells It has a channel (e.g., a memory column) and in this case supports an in-memory compute channel. Therefore, it also includes 256 ADC360s.

[0187] The IA BUFF320, as an example, receives a sequence of 32-bit data words. These 32-bit datawords are high-dimensional vectors suitable for processing by CIMA310. It works to reshape the sequence into 32-bit, 64-bit, or any other. The width of the data word is the available or selected number of spaces for calculation in the memory array 310. The memory array 310 itself may be reshaped to fit the size, and the high-dimensional number Configured to perform operations on a bit, 2-8 bits, 1-8 bits, or several bits This includes elements of other sizes, and applies them in parallel across the entire array. Please note: The matrix-vector multiplication operations described herein apply to the entire CIMA310. Although depicted as being used, in various embodiments only a portion of CIMA310 It should also be noted that it is used. Furthermore, in various other embodiments, CIMA310 and The associated logic circuits are arranged so that the parallel part of the matrix is ​​determined by each part of the CIMA310. Adapted to provide interleaved matrix-vector multiplication operations that are processed simultaneously. It can be done.

[0188] In particular, the IA BUFF320 processes a sequence of 32-bit data words using CIMA3. It is added to 10 at once (or in at least larger chunks) and is bit-serialized. Reshape it into a highly parallel data structure that can be appropriately sequenced by an expression. For example, 8 BE A 4-bit calculation with metric elements is a high-dimensional number of n-bit data elements with more than 2000 elements. It can be associated with a vector. IA BUFF320 forms this data structure.

[0189] As described herein, the IA BUFF320 exemplifies a 32-bit data It receives the input matrix X as a tower sequence and according to the size of the CIMA310. The sequence of received data words is resized / rearranged, and as an example, 2303 words. It is configured to provide a data structure containing n-bit data elements. Each of the three n-bit data elements, along with its respective masking bit, is IA B The signal is transmitted from UFF320 to the sparsity / AND logic controller 330.

[0190] The sparsity / AND logic controller 330 exemplifies 2303 n bits Receive the data element and its respective masking bit, and zero-value data element (each (As indicated by the masking bit) is propagated to the CIMA310 for processing. It is configured to call a sparsity function in response. In this way, CIMA The energy required for processing such bits by 310 is conserved.

[0191] During operation, the CPU210 transmits data to the PMEM via a direct data path implemented in a standard manner. Load 220 and boot loader 240. CPU 210 is implemented in a standard format. Direct data path access to DMEM230, IA BUFF320, and memory readout. You may access the write buffer 340. All of these memory modules / buffers The fa, CPU 210, and DMA module 260 are connected by the AXI bus 281. The chip configuration module and other peripheral modules are routed to the AXI bus 281. It is grouped by the APB bus 282 attached as a B. CPU210 It is configured to write to PMEM220 via AXI bus 281. DMA Module 260 includes DMEM230, IA BUFF320, and memory read / write. Access buffer 340 and NMD370 through a dedicated data path, and DMA code All other accessible memos via the AXI / APB bus, such as per controller 265. It is configured to access the space. CIMU300 is the BPBS matrix mentioned above. Perform vector multiplication. Further details of these and other embodiments are provided below.

[0192] Therefore, in various embodiments, CIMA receives vector information and matrix vectors Perform multiplication and provide a digitized output signal (i.e., Y=AX) The calculation is performed in bit-serial bit-parallel (BSBP) format, which is a composite matrix. To provide multiplication functionality, the results may be further processed by other calculation functions as appropriate. Generally speaking, the embodiments described herein provide an in-memory computing architecture. This in-memory computing architecture provides the sequence of received data words. Reshaping configured to shape and form a large-scale parallel bit-unit input signal. A buffer and an in-memory computation (CIM) array, via a first CIM array dimension to receive large-scale parallel bit-unit input signals, and via a second CIM array dimension A CIM array of bit cells configured to receive one or more cumulative signals, Each of the multiple bit cells associated with the cumulative signal provides its own output signal. A CIM array and analog data form each CIM channel configured in such a way. A digital converter (ADC) circuit that processes multiple CIM channel output signals, Analog digital configured to provide a sequence of multibit output words Using a single-bit internal circuit and signal in the analog-to-digital converter (ADC) circuit and CIM array A control circuit configured to perform multibit calculations on input and cumulative signals. It is configured to provide a path and a sequence of multi-bit output words as a result of the calculation. It includes a near-memory computation path.

[0193] Memory map and programming model CPU210 is the IA BUFF320 and memory read / write buffer 340 Because they are configured to access directly, these two memory spaces are particularly Regarding structured data such as array / matrix data, from the perspective of a user program, and In terms of latency and energy, it is similar to DMEM230. In various embodiments When the in-memory computation feature is activated or partially activated, memory read The write buffers 340 and CIMA310 can be used as regular data memory. ru.

[0194] Figure 28 shows an input activation value vector suitable for use in the architecture of Figure 26, according to an embodiment. A high-level block diagram of the Truly Shaping Buffer (IA BUFF) 320 is shown. The depicted IA BUFF320 accepts input with 1-bit to 8-bit element precision. It supports activity value vectors and, in various embodiments, can accommodate other accuracies. According to the bit serial flow mechanism discussed, all elements of the input activation value vector Certain bits are broadcast to the CIMA310 at once for matrix-vector multiplication operations. It is stopped. However, the high degree of parallelism of this operation is most important for the elements of the high-dimensional number input activation value vector. It is necessary to provide high bandwidth and low energy consumption, otherwise, in memory meters The advantages of computational throughput and energy efficiency will not be utilized. To achieve this, the input activation value reshaping buffer (IA BUFF) 320 is as follows: It may be constructed in such a way that in-memory calculations are performed on a 32-bit microprocessor. It can be integrated into the (or other bit width) architecture, thereby corresponding to the 32-bit architecture. Hardware for data transfer is utilized to its fullest extent for highly parallel internal organization of in-memory computations. It will be done.

[0195] Referring to Figure 28, the IA BUFF320 is an input vector with 1 to 8 bits of bit precision. It receives a 32-bit input signal that may contain the Tor element. Therefore, a 32-bit input The signals are first stored in a 4x8 bit register 410, and there are a total of 24 of these (this These registers are referred to as registers 410-0 to 410-23 in the specification. 410 consists of eight register files, each with 96 columns (register file 420-0 (It is written as ~420-8) and provides those contents, with a maximum of 3 × 3 × 256 = 230 An input vector with 4 dimensions has its elements arranged in a parallel column. This is 8 bits For input elements, 96 parallel outputs are generated across one of the 420 register files. This is done by the 24 4x8 bit registers 410 provided, for a 1 bit input element. , providing 1536 parallel outputs across all 8 register files 420, 24 This is done by 4x8 bit registers 410 (or by other bit-precision intermediate configurations) (This is performed). The height of each register file sequence is 2 × 4 × 8 bits, and all input vectors When a Tor element is loaded, each input vector (with element precision of up to 8 bits) It becomes possible to store data in four segments, enabling double buffering. On the other hand, if only one-third of the input vector elements are loaded (i.e., a stride of 1) In CNN, one of every four register file columns acts as a buffer, and the other three It becomes possible to forward propagate data from the column to the CIMU for computation.

[0196] Therefore, of the 96 columns output by each register file 420, each Only 72 can be selected by this circular barrel shifting interface 430. Yes, it gives a total of 576 outputs at once across 8 register files of 420. The output is one of the four input vector segments stored in the register file. This corresponds to sparsity / AND logic control within a 1-bit register. Four cycles are required to load all input vector elements into the RA330. .

[0197] To take advantage of the sparsity in the input activation value vector, CPU210 or DMA2 While 60 writes to the reshaping buffer 320, mask bits are used for each data element. This is generated. The masked input activation value is used in the charge-based computational calculations of CIMA310. This prevents calculations, thereby saving computational energy. The mask vector is also SRA. Stored in the M block, and similar to the input activation value vector, but in a 1-bit representation, It will be accomplished.

[0198] Using a 4-3 barrel shifter 430, perform VGG-style (3x3 filter) CNN computation. It supports the following: When moving on to the next filtering operation (convolution reuse), the input activation Only one of the three value vectors needs to be updated, and this allows the energy to be nodal This reduces throughput.

[0199] Figure 29 shows a CIMA readout suitable for use in the architecture of Figure 26, according to an embodiment. This shows the high-level block diagram of the write buffer 340. The IMA read / write buffer 340 is, exemplarily, a 768-bit wide static random It is organized as access memory (SRAM) block 510, while the depicted CPU The word width is 32 bits in this example, and the read / write buffer 340 is... It is used to interface between them.

[0200] The described read / write buffer 340 has a 768-bit write register 5 It includes 11 and 768-bit read registers 512. Read / write buffer A340 generally refers to the cache across a wide range of SRAM blocks in CIMA310. It works similarly, but with some differences in detail. For example, the read / write buffer 34 0 is written back to CIMA310 only when CPU210 writes to a different line, Reading a different row does not trigger a write back. The read address is the write register. When matched with the tag, it is not a read from CIMA310, but a write register 5 Eleven modification bytes (indicated by the contamination bit) are bypassed to read register 512. It will be done.

[0201] Cumulative line analog-to-digital converter (ADC). Cumulative line from CIMA310 is Each has an 8-bit SAR ADC that matches the pitch of the in-memory compute channel. To conserve memory, a finite state machine (F) controls the bit cycling of the SAR ADC. The SM is shared among the 64 ADCs required for each in-memory compute tile. The M control logic consists of 8+2 shift registers, and includes reset, sampling, and It generates a pulse that cycles through the subsequent 8-bit decision phase. The shift register pulse is Broadcast to 64 ADCs, buffered locally, and local ratio Trigger the comparator decision and store the corresponding bit decision in the local ADC code register. , used to trigger the following capacitor DAC configuration. High-precision metal oxide metal (M OM) Using caps allows for a small size capacitor array for each ADC. good.

[0202] Figure 30 shows a near-memory device suitable for use in the architecture of Figure 26, according to one embodiment. This shows a high-level block diagram of the NMD module 600, along with other features. Digital near-memory computing can be employed. As shown in Figure 30, The NMD Module 600 supports multibit matrix multiplication via the BPBS scheme. This shows the digital calculation data path after the ADC output.

[0203] In a particular embodiment, 256 ADC outputs are used for 8 groupings of digital computation flows. It is organized into a loop. This enables support for matrix element configurations of up to 8 bits. Therefore, the NMD module 600 contains 32 identical NMD units. The NMD unit offers a choice of eight ADC outputs 610 and corresponding bias 621. Multiplexer 610 / 620, multiplier 622 / 623, shift count 624 and cumulative count 8-bit unsigned input for subtracting zistor, global bias, and mask count. and an adder 631 having a 9-bit signed input, a neural network task A signed adder 632 for calculating the cal bias, and a solid adder for performing scaling. The constant-point multiplier 633 calculates the exponent of the multiplicand and the different bits of the weight elements. Barrel shifter 634 for performing shifts, 32-bit signed for performing cumulative operations Adder 635, 1, 2, 4, and 8-bit configurations to support weights 32-bit signed cumulative register 640, and neural network application It consists of a ReLU unit 650 for the system.

[0204] Figure 31 shows a direct memo suitable for use in the architecture of Figure 26, according to an embodiment. This diagram shows the high-level block diagram of the Reaccess (DMA) module 700. The DMA module 700 shown exemplifies the process of switching between different hardware resources. Two channels to support simultaneous data transfer, and DMEM and IA respectively. BUFF, CIMU R / W BUFF, NMD results, and 5 from / to the AXI4 bus. It features two independent data paths.

[0205] Bit-parallel / bit-serial (BPBS) matrix-vector multiplication Multibit MVM

number

[0206] Bitwise AND is a standard two's complement operation for multi-bit matrices and input vector elements. It can support representation. This means that the digitized output can be used in other column calculations. Before adding to the force, in the digital domain after ADC, the most significant bit (MSB) element This involves appropriately applying the corresponding negative sign to the column calculation.

[0207] Bitwise XNOR requires a slight modification of the numerical representation. That is, the element bit The LSB maps to +1 / -1 instead of 1 / 0, and to properly represent zero. We need two bits with weights. This is done as follows: First, each B Decomposes a bit operand (standard two's complement representation) into a B+1 bit signed integer. Example For example, y is B+1 plus / minus 1 bit.

number

number

[0208] When a bit of a 1 / 0 value maps to a mathematical value of +1 / -1, the bitwise integer Mori calculation multiplication may also be implemented via logical XNOR operation. Therefore, the input vector M-BC, which performs a logical XNOR using the differential signal of the column element, is a digital calculation from column calculations. Signed multibit multiplication is performed by bit-weighting and adding the converted output. It can be made possible.

[0209] AND-based M-BC multiplication and XNOR-based M-BC multiplication offer two options. This presents the idea that by using appropriate numerical representations in logical operations possible with M-BC, other Options are also possible. Such alternatives would be beneficial. For example, XNOR-based M-BC multiplication is preferred for binary (1-bit) calculations, while AND-based M-BC multiplication is... To facilitate integration within digital architectures, and to enable more standard numerical representations... Furthermore, the two approaches produce slightly different signal-to-quantization noise ratios (SQNRs). Therefore, this can be selected based on application needs.

[0210] Heterogeneous computing architecture and interface The various embodiments described herein are bit cells (or multiplicative bit cells, M-BC). ) drives the output voltage corresponding to the calculation result to the local capacitor, charge domain i This involves different forms of in-memory computation. Next, it involves the acquisition of data from the in-memory computation channel (column). The japashitas are coupled, and accumulation occurs through charge redistribution. As mentioned above, These capacitors are simply in close proximity to each other and are therefore coupled via an electric field, such as through wiring. Therefore, using specific geometric shapes that are very easy to replicate using VLSI processes, etc. It can be formed in this way. Therefore, the local bit cell formed as a capacitor is While storing a charge that represents 1 or zero, the number of charges of these capacitors or bit cells By adding everything locally, the core operation in matrix-vector multiplication is multiplication and This allows for various implementation methods for cumulative / totaling functions.

[0211] The various embodiments described above are advantageous in that they utilize an improved bitcell-based architecture. It provides a computing engine and platform. Matrix-vector multiplication is standard. , one that cannot be efficiently performed by digital processing or digital acceleration It is an operation. Therefore, this one type of in-memory computation is used in existing digital designs. It offers enormous advantages in comparison. However, various other types of operation use digital design. It is executed efficiently.

[0212] Various embodiments include these bit cell-based architectures, computing engines, and plastics. To form heterogeneous computing architectures, etc., more conventional To connect / interface to digital computing architectures and platforms This mechanism is intended to enable bit cell architecture processing (for example, matrix vector While those calculations well suited to the Torr process are processed as described above, traditional Those other computational operations that are well suited to computer processing are traditional computer architectures It is processed via kucha. That is, various embodiments are described herein at an advanced level. It provides a computational architecture that includes a column processing mechanism, which is more conventional than digital computation. It is connected to multiple interfaces so that it can be externally coupled to the architecture. In this way, the digital computing architecture can be directly connected to the in-memory computing architecture. It can be efficiently aligned, two can be placed close together, and data movement between them can be overridden. - Head can be minimized. For example, machine learning applications can be 80% to 9 It may include 0% matrix-vector calculations, but even so, it is one of the other types of calculations / operations. 0% to 20% is executed and left unfinished. The in-memory computation discussed herein is the architecture By combining the technique with the more conventional near-memory computation, the result is The system offers exceptional configurability for performing many types of processing. Therefore, various embodiments, in conjunction with the in-memory calculations described herein, can achieve near- The aim is to perform digital computation in memory.

[0213] The in-memory operations discussed herein are large-scale parallelisms, but are single-bit operations. Yes. For example, in a bit cell, only one bit can be stored: 1 or zero. The signal that drives the cell is typically an input vector (i.e., a 2D vector raised to power). In arithmetic operations, each matrix element is multiplied by each vector element. Vector elements are digital. It is also placed on a signal that is only 1 bit, and as a result, the vector elements are similar It is 1 bit.

[0214] Various embodiments use a bit-parallel / bit-serial approach to matrix / bit Extends the ctor from a single-bit element to a multi-bit element.

[0215] Figures 8A-8B show CIMA channel digital suitable for use in the architecture of Figure 26. This illustrates high-level block diagrams of embodiments with different weightings. Specifically, Figure 32A shows digital binary weighting similar to that described above for various other figures. Figure 32B illustrates the embodiment of Figure 32A and / or this embodiment. This enables the use of fewer analog-to-digital converters than in other embodiments described in the specification. To achieve this, various circuit elements have been modified in the form of analog binary weighting and summation. This illustrates the concept.

[0216] As discussed earlier, various embodiments involve in-memory computation (CIM) of bit cells. (i) is arranged on a large scale through the first CIM array dimension (e.g., rows of a 2D CIM array). To receive bitwise input signals in a column, and to receive a second CIM array dimension (e.g., 2 Configured to receive one or more cumulative signals via a column of the CIM array D, common Multiple bit cells associated with the cumulative signal (for example, depicted as a sequence of bit cells) Each of the CIM channels is configured to provide its own output signal. The intention is to form a wire. The analog-to-digital converter (ADC) circuit consists of multiple CIs. The M-channel output signal is processed to produce a sequence of multi-bit output words. The control circuit is configured to provide a single bit internal circuit and signal to the CIM array. It is configured to perform multibit calculations on the input signal and the cumulative signal. Therefore, the near-memory calculation path involved in enabling the control circuit to operate is the calculation result It can be configured to provide a sequence of multi-bit output words.

[0217] Referring to Figure 32A, the digital binary weighting and summation that perform the ADC circuit function are shown. Embodiments are depicted. In particular, the 2D CIMA810A is in the first (row) dimension ( That is, matrix input values ​​are received (through multiple buffers 805) and the second (column) dimension is used. Upon receiving the input value, the CIMA810A performs calculations according to the control circuit (not shown). It provides various channel output signals (CH-OUT).

[0218] The ADC circuit in Figure 32A outputs the CIM channel output signal CH- for each CIM channel. Each ADC760 is configured to digitize the OUT, and the digitized Each binary weight is assigned to the CIM channel output signal CH-OUT, and then Therefore, it is configured to form a part of each of the multibit output words 870. The shift register 865 is provided.

[0219] Referring to Figure 32B, analog binary weighting and summation are performed to execute the ADC circuit function. An embodiment is depicted. In particular, the 2D CIMA810B has a first (row) dimension ( That is, matrix input values ​​are received (through multiple buffers 805) and the second (column) dimension is used. Upon receiving the input value, the CIMA810B performs calculations according to the control circuit (not shown). It provides various channel output signals (CH-OUT).

[0220] The ADC circuit in Figure 32B includes switches 815-1, 815-2, etc. within the CIMA810B. It provides four controllable (or pre-configured) banks, shaping the CIMA810B The resulting capacitors are coupled and / or separated, thereby controlling one or more channels. Implement the analog binary weighting scheme for each group, and channel subgroup Each of these is the weighting of the CIM channel output signals for each subset of the CIM channels. The resulting analog sum is digitized, thereby producing each of the multi-bit output words. A single output so that only one ADC860B is needed to form this section. Provides a signal.

[0221] Figure 33 illustrates a flowchart of the method according to the embodiment. Specifically, Figure 33 Method 900 is an input matrix / vector that is a bit parallel / bit serial approach The architecture, system, etc. described herein is scaled to be computationally expandable. This covers various processing operations that are implemented.

[0222] In step 910, the matrix and vector data are placed in appropriate memory locations. It will be done.

[0223] In step 920, each of the vector bits (from MSB to LSB) is processed sequentially. Specifically, the MSB of the vector is multiplied by the MSB of the matrix, and the MSB of the vector The matrix's MSB-1 is multiplied by the vector's MSB, and the matrix's MSB is multiplied by the matrix's MSB-2, etc. This is done until the MSB of the vector is multiplied by the LSB of the matrix. Then the result obtained The analog charge result is obtained by digitally multiplying each of the vector multiplications from MSB to LSB. The result is obtained by performing a recalculation, and this result is latched. This process is performed using the vector MSB~LSB. The vector MSB multiplies each of the MSB to LSB elements of the matrix up to a number of times. This pattern repeats for vectors up to the LSB, starting with -1, then vector MSB-2, and so on.

[0224] In step 930, the bits are shifted to apply the appropriate weighting, and The results are added together. In some of the embodiments in which analog weighting is used Note that the shifting operation in step 930 is unnecessary.

[0225] Various embodiments are used within circuits to store data in high-density memory. This enables stable and robust computation. Furthermore, various embodiments allow for memory bit cell cycles. By enabling higher density for roads, the computation engine and The platform will be advanced. The density will be due to a more compact layout. and highly aggressive design rules used in memory circuits (i.e., push rules) This can increase both due to improved layout compatibility using various implementations. This significantly improves the performance of processors for machine learning and other linear algebra applications.

[0226] We have disclosed bit cell circuits that can be used within an in-memory computing architecture. The approach is very stable within the circuit used to store data in high-density memory / Enables robust computation. Disclosed for robustness in memory computation. This approach allows for a higher density of memory bit cell circuits than known approaches. This makes it possible. The density is due to the more compact layout, and the memory cycles Layers using highly aggressive design rules (i.e., push rules) for roads This can increase due to both improved compatibility with the OUTSIDE and other factors. The disclosed devices are standard It can be manufactured using CMOS integrated circuit processing.

[0227] Partial list of disclosed embodiments Various embodiments are specified in the claims. However, those and other aspects of the subset are specified in the following numbered clauses.

[0228] 1. Integrated in-memory computing (IMC) architecture It is a technology, and the IMC architecture is an application that is mapped to IMC. It can be configured to support the data flow of and has multiple configurable in-memory components. It is a Compute-In-Memory Unit (CIMU), It comprises a plurality of configurable CIMUs that form an array of CIMUs, and the CIMUs are Through each configurable inter-CIMU network portion located between them, the CIMU To transfer activity values ​​to / from other CIMUs or other structures inside or outside the CIMU, and Each configurable operand loading network portion is located between them Through this, weights are transmitted to / from other CIMUs or other structures inside or outside the CIMU. An integrated IMC architecture configured in such a way.

[0229] 2. Each CIMU receives computation data from the inter-CIMU network, and the received The computational data is processed using matrix-vector multiplication (MVM) by CIMU to produce output feature vectors. A configurable input buffer for constructing an input vector for generating a condition The integrated IMC architecture described in Section 1.

[0230] 3. Each CIMU is configurable to receive computational data from the inter-CIMU network. Each CIMU has an input buffer and performs matrix-vector multiplication (M) on the received computation data. Item 1: The input vector is configured to generate the output feature vector through VM processing. The integrated IMC architecture described above.

[0231] 4. Each CIMU calculates data from the inter-CIMU network according to the data flow map. The system receives the data, adds a time delay to the received computation data, and then processes the delayed computation data. Associated with a configurable shortcut buffer for transfer to CIMU, The integrated IMC architecture described in Clause 2 or 3.

[0232] 5. Each CIMU receives computation data from the inter-CIMU network, and the received computation The data is given a time delay, and the delayed computational data is transferred to a configurable input buffer. Clause 2 or 3 associated with a configurable shortcut buffer for sending The integrated IMC architecture described above.

[0233] 6. Each CIMU will use the least of its respective input buffer and shortcut buffer. A parallel computing hardware configured to process input data received from at least one source. An integrated IMC architecture as described in Clause 2 or 3, including (a).

[0234] 7. Each CIMU shortcut buffer aligns the data flow between multiple CIMUs. In order to maintain this, the system described in clause 4 or 5 is configured according to the data flow map. Integrated IMC architecture.

[0235] 8. Each shortcut buffer of multiple CIMUs within the CIMU array is a pipe. Supports pixel-level pipelining to provide line latency matching. The integrated IMC architecture described in Clause 4 or 5 is configured according to the data flow map. Cha.

[0236] 9. The time delay imposed by the CIMU shortcut buffer is the absolute time delay. , predetermined time delay, time delay determined with respect to the size of the input calculation data, CIMU's predetermined time delay Time delay determined in relation to the calculation time, control received from the data flow controller In response to signals, control signals received from other CIMUs, and events occurring within the CIMU, The clause includes at least one of the control signals generated by the CIMU, in accordance with clause 4 or 5. The integrated IMC architecture described.

[0237] 10. Each configurable input buffer can be used for inter-CIMU networks or shortcuts. A time delay may be applied to the calculation data received from the fa, as per clause 4, 5, or 6. The integrated IMC architecture described above.

[0238] 11. The time delay imparted by the configurable input buffer of the CIMU is the absolute time delay. Delay, predetermined time delay, time delay determined with respect to the size of the input calculation data, CIMU Time delay determined regarding the estimated computation time, control received from the data flow controller Responding to your signals, control signals received from other CIMUs, and events occurring within the CIMU. The following provisions of Clause 10 include at least one of the control signals generated by the CIMU. Integrated IMC architecture.

[0239] 12. CIMU, inter-CIMU network portion, and operand loading network At least a subset of the work portion is the data of the application mapped to IMC. An integrated IMC architecture according to claim 1, configured according to Toughflow.

[0240] 13. CIMU, inter-CIMU network portion, and operand loading network At least a subset of the work portion is a neural network (NN) layer to IMC. The `sapping` method is configured according to the data flow of the layer, thereby executing on a given layer. The parallel output activation values ​​calculated by the configured CIMU are then executed in the next layer. The CIMU is provided with the parallel output activation values, which shape each NN feature map pixel. The integrated IMC architecture described in Clause 9.

[0241] 14. The configurable input buffer is configured according to the selected stride step, and the input NN The feature map data is configured to be transferred to the parallel computing hardware within the CIMU. The integrated IMC architecture described in Clause 13.

[0242] 15. NN includes Convolutional Neural Networks (CNNs), and input line buffering a buffers several rows of the input feature map corresponding to the size of the CNN kernel. The integrated IMC architecture described in Clause 14 is used for this purpose.

[0243] 16. Each CIMU uses iterative barrel shifting in conjunction with the column weighting process. A bit-parallel bit-serial process is performed after a single-bit calculation, followed by a result accumulation process. (BPBS) is configured to perform matrix-vector multiplication (MVM) according to the calculation process. An integrated IMC bank as described in Clause 2 or 3, comprising an in-memory computing (IMC) bank. Kitekcha.

[0244] 17. Each CIMU uses iterative column merging along with a column weighting process to create a single bit The calculation is performed followed by a result accumulation process (BPBS) in a bit-parallel bit-serial format. The input is configured to perform matrix-vector multiplication (MVM) according to the calculation process. An integrated IMC architecture as described in Clause 2 or 3, comprising a memory computation (IMC) bank. .

[0245] 18. Each CIMU is an in-memory computation (IMC) bank, and the elements of the IMC bank Bit parallel bit serial (BP) is assigned using the BPBS expansion process. BS) Configured to perform matrix-vector multiplication (MVM) according to the calculation process. An integrated IMC architecture as described in Clause 2 or 3, comprising an IMC bank.

[0246] 19. The IMC bank element implements the MVM using the replication and shifting process. The integrated IMC architecture described in Clause 18 is further configured to perform the following actions.

[0247] 20. Each CIMU has its own near memory, programmable single instruction multiple data (S Associated with the SIMD digital engine, the SIMD digital engine uses an input buffer For inclusion in data, shortcut buffer data, and / or feature vector maps. Suitable for use in combinations or temporal arrangements of output feature vector data. The integrated IMC architecture described in item 4 or 5.

[0248] 21. At least a portion of the CIMU maps the input to the output according to multiple nonlinear functions. Each lookup table associated with the nonlinear function output data is linked to the lookup table for the lookup. However, the terms provided to the SIMD digital engine associated with each CIMU are The integrated IMC architecture described in section 20.

[0249] 22. At least a portion of the CIMU maps the input to the output according to multiple nonlinear functions. The nonlinear function output data is associated with a parallel lookup table for the purpose of looking up the data. Provided to the SIMD digital engine associated with each CIMU, under Clause 20 The integrated IMC architecture described.

[0250] 23. An in-memory computing (IMC) architecture, wherein the IMC architecture has This is an IMC architecture for mapping neural networks (NNs). An on-chip array of in-memory computing units (CIMUs), wherein the CMIU has a map It can be logically constructed as an element within the layers of the NN being generated, and each CIMU output activation value is Supports each portion of the dataflow associated with the mapped neural network. Each feature vector is included and the parallel output is computed by a CIMU running on a given layer. The force activation values ​​form feature map pixels in the on-chip array of the CIMU, On-chip activation configured to transmit CIMU output activation values ​​between adjacent CIMUs. A value network, where parallel output activity is computed by a CIMU running in a given layer. The values ​​form a feature map pixel in an on-chip activation value network, Weights are loaded into adjacent CIMUs, and weight loading is performed between adjacent CIMUs. On-chip operand loading network for communication via the interface An IMC architecture that includes the following features.

[0251] 24. Computation inputs and outputs are connected to a single chip network via a configurable on-chip network. For in-memory calculations that are passed from one in-memory calculation block to the next in-memory calculation block. The above clauses will be modified as needed to provide a data flow architecture. One of the following.

[0252] 25. An in-memory computing module receives input from multiple in-memory computing modules. For in-memory computing, which can do so and provide output to multiple in-memory computing modules. The above clauses will be modified as needed to provide a data flow architecture. Either one.

[0253] 26. To enable inputs and outputs to flow between modules in a synchronized manner. Appropriate buffering is provided to the input or output of the in-memory computing module. To provide a dataflow architecture for in-memory computation, modify as needed. Any of the above clauses will be corrected.

[0254] 27. In the output channel of a specific pixel in the output feature map of a neural network Corresponding parallel data is processed from one in-memory computation block to the next in-memory computation block. To provide a data flow architecture that is passed to, it will be modified as needed, Any of the following clauses.

[0255] 28. The neural network weights are associated with memory sequences corresponding to different output channels. Furthermore, in-memory computations, where matrix elements are stored in memory, and neural network computations are performed. Any of the above clauses, which may be modified as necessary to provide a way to map mosquito.

[0256] 29. In-memory computations in which matrix elements stored in memory can be modified throughout the computation process. To provide a way to map neural network computations to computation hardware. or any of the above clauses, as modified as necessary.

[0257] 30. Matrix elements stored in memory are used in multiple in-memory computation modules or locations In memory computing hardware that can store data in memory, neural network calculations are mapped to the memory. Any of the above clauses, to be modified as necessary to provide a method of implementation.

[0258] 31. Multiple neural network layers are mapped simultaneously (layer deployment) in memo This provides a way to map neural network computations to recalculation hardware. Any of the above clauses, as amended as necessary.

[0259] 32. Different matrix element bits are mapped to the same column (BPBS expansion), bit single The in-memory computing hardware that performs the operations of the order of magnitude is mapped to the neural network computation. Any of the above clauses, to be modified as necessary to provide a method of implementation.

[0260] 33. Higher-order bits are duplicated to enable appropriate analog weighting (column merging). ) Provides a way to map multiple matrix element bits to the same column, as needed. Any of the above clauses, which will be amended accordingly.

[0261] 34. Elements are duplicated and shifted, and higher-order input vector elements are shifted elements In rows that have multiple matrix element bits provided (duplicated and shifted) in the same column, Any of the above clauses, to be modified as necessary to provide a method of loading.

[0262] 35. Performs bitwise operations, but multiple input vector bits are multi-level (analog). The neural network is simultaneously provided as a signal to the in-memory computing hardware. Any of the above clauses will be modified as necessary to provide a way to map calculations. Reka.

[0263] 36. The multi-level driver selects by decoding multiple bits of the input vector elements. This provides a method for multi-value input vector element signaling that employs a dedicated voltage power supply. Any of the above clauses, to be modified as necessary.

[0264] 37. To provide a multi-level driver in which a dedicated power supply can be derived from an off-chip source (for example) (and to support XNOR calculations and numerical formats for calculations) as needed. Any of the above clauses will be amended accordingly.

[0265] 38. In-memory meters that achieve scale-up by arranging modular tiles in groups. To provide a modular architecture for calculations, it will be modified as needed. Any of the following clauses.

[0266] 39. In-memory modules connected by a configurable on-chip network. It will be modified as needed to provide a modular architecture for computation. Any of the above clauses.

[0267] 40. A module may include any one or combination of the modules described herein. To provide a modular architecture for in-memory computing, as needed Any of the above clauses will be amended.

[0268] 41. Controls for properly configuring modules and providing appropriate localized control. Any of the above clauses, which may be modified as necessary to provide the necessary configuration logic. .

[0269] 42. Provide an input buffer for receiving data calculated by the module. Any of the above clauses, as modified as necessary.

[0270] 43. Input data delay to properly synchronize data flow through the architecture The above clauses will be modified as necessary to provide a buffer for providing Is it off?

[0271] 44. The above will be modified as needed to provide local near-memory computation. Any of the clauses.

[0272] 45. Another method for synchronizing data flow within a module or through the architecture. As an individual module, it will be modified as needed to provide a buffer, as described above. Any of the items.

[0273] 46. ​​Located near in-memory computing hardware, output data from in-memory computing It provides near-memory digital computing that offers programmable / configurable parallel computing. Any of the above clauses, as amended as necessary.

[0274] 47. Provide a computation data path between parallel output data paths to enable different in-memory computation output To provide computation between forces (for example, between adjacent in-memory computation outputs), as needed. Any of the above clauses, as amended accordingly.

[0275] 48. Reduce data between all parallel output data paths in a hierarchical format down to a single output. The above clauses will be modified as necessary to provide a computational data path for this purpose. Either one.

[0276] 49. In-memory calculation output (e.g., shortcut buffer, input buffer and show In addition to computation units between the input buffer and other sources, input is obtained from auxiliary sources. The above clauses shall be modified as necessary to provide a computational data path that can be used. Either one.

[0277] 50. Provides near-memory digital computation employing instruction decoding, and also in memory Hardware shared between parallel data paths applied to output data from recalculations Any of the above clauses, modified as necessary to control the market.

[0278] 51. Compositable / controllable multiplication / division, addition / subtraction, bitwise shifting, etc. To provide a near-memory data path that provides the calculations, it will be modified as needed. Any of the following clauses.

[0279] 52. Local registers for intermediate calculation results (scratchpad) and parameters. The above clauses shall be modified as necessary to provide a near memory data path. Either one.

[0280] 53. Arbitrary nonlinearity between parallel data paths via a shared lookup table (LUT). Any of the above clauses, which may be modified as necessary to provide a way to compute the function. .

[0281] 54. Lookup table with local decoder for LUT decoding (LUT) Provides sequential bitwise broadcasting of bits as needed. Any of the above clauses, which will be amended accordingly.

[0282] 55. Provide an input buffer located near the in-memory computing hardware, To provide storage for input data processed by Mori computing hardware, as needed. Any of the above clauses will be amended accordingly.

[0283] 56. Reuse of data for in-memory computations (e.g., required for convolution operations) The above will be modified as needed to provide input buffering that enables its use. Any of the clauses.

[0284] 57. Buffer rows of the input feature map (across rows and across multiple rows). (This enables the reuse of convolutions in two dimensions of the filter kernel) Any of the above clauses, as modified as necessary to provide a fairing.

[0285] 58. The input may be provided from multiple different sources. To provide input buffering that allows data to be obtained from the input port, Any of the above clauses, as amended accordingly.

[0286] 59. For example, one method is to buffer data from different input ports. It may be arranged in different vertical segments of the row, and from multiple different input ports The above will be modified as needed to provide multiple different ways of arranging the data. Any of the clauses.

[0287] 60. Input data is provided to in-memory computing hardware in multiples of the clock frequency. It will be modified as needed to provide the ability to access data from the server. Any of the following clauses.

[0288] 61. Near the in-memory computing hardware, or the tie of the in-memory computing hardware They are located in separate locations within a ringed array, but not necessarily in memory computing hardware. To provide additional buffering as needed, instead of directly providing data to the software. Any of the above clauses will be amended accordingly.

[0289] 62. To ensure that data from different in-memory computing hardware can be properly synchronized ( For example, as in the case of shortcut connections in neural networks, data To provide appropriate delay, additional buffering will be provided as needed. Any of the above clauses will be corrected.

[0290] 63. Reuse of data for in-memory computations (e.g., required for convolution operations) It provides additional buffering to enable the use of rows in the input feature map, optionally buffering rows. Filtering, two filter kernels (crossing rows and intersecting multiple rows) To provide input buffering that enables the reuse of convolutions in dimensions, as needed. Any of the above clauses will be amended accordingly.

[0291] 64. The input may be provided from multiple different sources so that incoming data can be provided from multiple different sources. To provide additional buffering that allows data to be obtained from the input port, Any of the above clauses, as required.

[0292] 65. For example, one method is to buffer data from different input ports. It may be arranged in different vertical segments of the row, and from multiple different input ports The above will be modified as needed to provide multiple different ways of arranging the data. Any of the clauses.

[0293] 66. Obtain matrix elements stored in bit cells via an on-chip network. To provide an input interface for in-memory computing hardware, as needed. Any of the above clauses will be amended accordingly.

[0294] 67. Enables the use of the same on-chip network for input vector data. To provide an input interface for matrix element data, it will be modified as needed. Any of the following clauses.

[0295] 68. Input buffering and additional buffers close to the in-memory computing hardware, The above clauses are modified as necessary to provide computing hardware during the period. Reka.

[0296] 69. Provides parallel computation between input buffering and output from additional buffering. The above clauses will be modified as necessary to provide computing hardware capable of performing such tasks. Any of the items.

[0297] 70. Provides computation between input buffering and additional buffering output. The above clauses will be modified as necessary to provide the computing hardware that can perform the task. Reka.

[0298] 71. Computing hardware, wherein the output of the computing hardware is in-memory computing hardware Modify as needed to provide computing hardware that can feed into the market. Any of the above clauses.

[0299] 72. Computing hardware, wherein the output of the computing hardware is in-memory computing hardware Computation hardware that can feed into near-memory computing hardware following the ware Any of the above terms, to be provided, and modified as necessary.

[0300] 73. A segment with parallel routing channels surrounds a CIMU tile. We provide an on-chip network with a r-type structure between in-memory computing tiles. Any of the above clauses, as amended as necessary.

[0301] 74. Each takes input from in-memory computing hardware and / or in memory It has several routing channels that can provide output to computing hardware. The above clauses will be modified as necessary to provide an on-chip network. Either one.

[0302] 75. Data originating from any in-memory computing hardware is tiled into Any other in-memory computing hardware within the ray, and potentially multiple different in It has routing resources that can be used to provide memory computing hardware. Any of the above terms, which may be modified as necessary to provide a chip network mosquito.

[0303] 76. In-memory computing hardware provides data to routing resources, Alternatively, data can be retrieved from routing resources via multiplexing across routing resources. To provide an on-chip network implementation, it will be modified as necessary. Any of the above clauses.

[0304] 77. Connections between routing resources are switches at the intersection of routing resources. To provide an on-chip network implementation method that is carried out via a connecting block. or any of the above clauses, as modified as necessary.

[0305] 78. Complete switching between intersecting routing resources, or intersecting routes Switching can provide a subset of the complete switching between resources. Any of the above clauses, modified as necessary to provide a block.

[0306] 79. Neural networks in memory computing hardware tiled To provide software for mapping to I, it will be modified as needed, above Any of the following clauses.

[0307] 80. In-memory computing hardware for specific calculations required by neural networks It will be modified as needed to provide a software tool that performs the allocation of shares. or any of the above clauses.

[0308] 81. Identification of allocated in-memory computing hardware within a tiled array. To provide software tools to perform the deployment to the location as needed, Any of the above clauses will be amended.

[0309] 82. In-memory computing hardware that provides a specific output and an in-memory computing hardware that obtains a specific input. The memory computing hardware and the software are configured to minimize the distance between them. Any of the above clauses may be modified as necessary to provide software tools.

[0310] 83. What is an optimization method for minimizing such distances (e.g., simulated annealing)? The above clauses will be modified as necessary to provide the adopted software tools. Either one.

[0311] 84. Configure available routing resources and in-memory computing hardware Transfer the output from A to the input of in-memory computing hardware in a tiled array. Any of the above terms will be modified as necessary to provide the necessary software tools. mosquito.

[0312] 85. Required to achieve routing between deployed in-memory computing hardware. To provide a software tool that minimizes the total amount of routing resources. or any of the above clauses, as modified as necessary.

[0313] 86. Optimization methods to minimize such routing resources (e.g., dynamic We provide software tools that employ programming, and modify them as needed. Any of the above clauses.

[0314] With respect to the various figures, the systems, methods, apparatus, mechanisms, techniques, and so forth described herein Various modifications may be made to these parts, and such modifications are within the scope of the present invention. It is intended that there be a step or While a specific order of placement of functional elements is presented, various other steps or functional elements are also presented. The order / arrangement may be used within the context of various embodiments. Furthermore, modifications to the embodiments may be made. While each modification may be discussed individually, various embodiments may use multiple modifications simultaneously or sequentially. You may also use compound modifications, etc. When used herein, "or" and The term is nonexclusive or (for example, "or otherwise") unless otherwise indicated. It will be understood that this refers to the use of "or alternatively."

[0315] Various embodiments incorporating the teachings of the present invention are shown and described in detail herein, Those skilled in the art can still easily devise many other various embodiments that incorporate these teachings. This is possible. Therefore, the above applies to various embodiments of the present invention. Other and further embodiments of the present invention may be devised without departing from its basic scope. That's good too.

Claims

1. Integrated In-Memory Computing (IMC) Architecture The IMC architecture is mapped to the IMC architecture. Designed to support scalable execution and data flow of applications. It is possible, Multiple configurable in-memory computing units ( CIMU) comprising a plurality of CIMUs forming an array of CIMUs, Input data is transmitted to the array of the CIMU, and calculated data is transmitted between the CIMUs. and a configurable on-chip network for transmitting output data from the CIMU array. An integrated IMC architecture that includes a work environment.

2. Each CIMU receives computation data from the on-chip network, and the received The calculated data is then processed by the CIMU using matrix-vector multiplication (MVM) to produce an output vector. The input vector for the MVM processing is configured to generate calculated data including the toll. The integrated IMC architecture according to claim 1, comprising an input buffer for performing the following.

3. Each CIMU ensures that the alignment of data flows between multiple CIMUs is maintained. According to the flow map, the on-chip network receives computation data, and the receiving The calculated data is given a time delay, and the delayed calculation is directed towards the next CIMU or output. Associated with a shortcut buffer for transferring calculation data, as described in claim 2. Integrated IMC architecture.

4. Each CIMU has at least one of its input buffers and shortcut buffers. Parallel computing hardware configured to process input data received from another source The integrated IMC architecture described in claim 2, including the integrated IMC architecture described in claim 2.

5. The input buffer and short of each of the plurality of CIMUs in the array of CIMUs At least one of the cut buffers provides pipeline latency matching. Configured according to a dataflow map that supports pixel-level pipelining. The integrated IMC architecture according to claim 3.

6. The time delay provided by the CIMU shortcut buffer is an absolute time delay. , a predetermined time delay, a time delay determined with respect to the size of the input calculation data, the CIMU The time delay determined regarding the estimated computation time is received from the data flow controller. Control signals, control signals received from another CIMU, and the occurrence of events within the CIMU. The request includes at least one of the control signals generated by the CIMU in response to the request. The integrated IMC architecture described in Requirement 3.

7. At least some of the aforementioned input buffers receive from the on-chip network, Alternatively, configure to add a time delay to the calculation data received from the shortcut buffer. The integrated IMC architecture according to claim 3, which may be possible.

8. The time delay provided by the CIMU's input buffer is an absolute time delay, a predetermined time Interval delay, time delay determined with respect to the size of the input calculation data, and the expected calculation of the CIMU. Time delay determined in relation to time, control signals received from the data flow controller, In response to control signals received from another CIMU, and the occurrence of events within the CIMU The following is a claim that includes at least one of the control signals generated by the CIMU: Integrated IMC architecture.

9. At least a subset of the CIMU is an application mapped to the IMC Includes an operand loading network portion configured according to the data flow of the application. The integrated IMC architecture according to claim 8, associated with the on-chip network portion Tech.

10. The application mapped to the IMC is configured to run at a given layer. The parallel output calculated data from the CIMU is provided to the configured CIMU that will run in the next layer. It includes a neural network (NN) that is mapped to the IMC so as to The parallel output calculated data forms each NN feature map pixel, according to claim 9. The integrated IMC architecture described.

11. The input buffer maps the input NN features according to the selected stride step. The data is configured to be transferred to the parallel computing hardware within the CIMU, The integrated IMC architecture described in item 10.

12. The NN includes a convolutional neural network (CNN), and the input buffer is , buffer a certain number of rows of the input feature map corresponding to the size of the CNN kernel. The integrated IMC architecture according to claim 11, used for the purpose of:

13. Each CIMU uses iterative barrel shifting in conjunction with a column weighting process to create a single barrel shifting process. Bit-parallel bit-serial (BP) The BS (Brainstorming System) is configured to perform matrix-vector multiplication (MVM) according to the calculation process. The integrated IMC architecture according to claim 2, comprising an in-memory computing (IMC) bank. 。

14. Each CIMU uses iterative column merging along with a column weighting process for single-bit computations. Bit-parallel bit-serial (BPBS) calculations are performed, followed by a results accumulation process. In-memory configuration configured to perform matrix-vector multiplication (MVM) according to the process. The integrated IMC architecture according to claim 2, comprising a Computation (IMC) bank.

15. Each CIMU is an in-memory computing (IMC) bank, and the elements of the IMC bank are Bit-parallel bit-serial (BPB) assigned using the BPBS expansion process. S) I configured to perform matrix-vector multiplication (MVM) according to the calculation process The integrated IMC architecture according to claim 2, comprising an MC bank.

16. The IMC bank elements will perform MVM using the replication and shifting process. The integrated IMC architecture according to claim 15, further comprising the above.

17. Each CIMU has its own near memory, programmable single instruction multiple data (SIMD) ) Associated with the digital engine, the SIMD digital engine is an input buffer For inclusion in the data, shortcut buffer data, and / or feature vector map. Suitable for use in combinations or temporal arrangements of output feature vector data, claim The integrated IMC architecture described in item 15.

18. At least a portion of the CIMU maps the input to the output according to a plurality of nonlinear functions. Includes each lookup table for the lookup, and the nonlinear function output data is the same Provided to the SIMD digital engine associated with each CIMU, The integrated IMC architecture described in 15.

19. At least a portion of the CIMU maps the input to the output according to a plurality of nonlinear functions. The nonlinear function output data is associated with a parallel lookup table for the purpose of the lookup, The billing is provided to the SIMD digital engine associated with each CIMU. The integrated IMC architecture described in item 15.

20. Each input includes a multibit input, and each multibit input value corresponds to the respective voltage level The IMC architecture according to claim 1, as represented by [the specified method].

21. Integrated In-Memory Computing (IMC) Architecture The IMC architecture is mapped to the IMC architecture. Supports scalable execution and data flow of neural networks (NNs). It can be configured to An array of CIMUs logically configured as elements within the layers of the NN to be mapped Forming multiple configurable in-memory computing units A CIMU, where each CIMU is associated with the mapped NN. It provides a calculated data output representing each part of the vector within the Taflow, and in a given layer The parallel output computed data of the CIMU being executed forms multiple feature map pixels. A configurable CIMU, Input data is transmitted to the array of the CIMU, and calculated data is transmitted between the CIMUs. and a configurable on-chip network for transmitting output data from the CIMU array. The workpiece is such that the on-chip network is between the CIMUs and between the CIMUs On-chip operand loader for transmitting operands via this interface An integrated IM comprising a configurable on-chip network including a wing network. C architecture.

22. The mapping of neural network computations to in-memory computing hardware is It operates to perform operations on a bit-by-bit basis, and multiple input vector bits are provided simultaneously. , represented via a selected voltage level of an analog signal, as described in claim 21 - Kittech.

23. The multi-level driver transmits an output signal from one of several selected voltage sources, and the power Claim 2, the pressure source is selected by decoding multiple bits of the input vector elements. The IMC architecture described in 1.

24. Each input includes a multibit input, and each multibit input value corresponds to the respective voltage level The IMC architecture according to claim 20, as represented by [the specified method].

25. Configurable in-memory computing (in-memory.com) in an integrated IMC architecture. (IMC) A computer that maps applications to hardware. A method for implementing a computer, wherein the IMC hardware performs in-memory computing (Compute-I Multiple configurable CIMUs forming an array of n-Memory Units (CIMUs) Then, input data is transmitted to the array of the CIMU, and calculated data is transmitted between the CIMUs. , and a configurable on-chip network for transmitting output data from the CIMU array. The method comprises a twerk and, Using the parallelism and pipelining of IMC hardware, application computing IMC hardware is allocated accordingly to provide high-throughput application computing. To generate an IMC hardware assignment configured to do so, IMC hardware that generates output data, and I that processes the generated output data The IMC hardware is allocated in a way that minimizes the distance between it and the MC hardware. Defining the placement of the wear in the array of the CIMU, The on-chip network is used to route the data between IMC hardware. A computer implementation method that includes configuring such a configuration.

26. The application mapped to the IMC is configured to run at a given layer. The parallel output calculated data from the CIMU is provided to the configured CIMU that will run in the next layer. It includes a neural network (NN) that is mapped to the IMC so as to Claim 25, where the parallel output computed data forms each NN feature map pixel. The computer implementation method described above.

27. The computational pipelining is more than the configured CIMU that performs in the next layer. Allocate a number of CIMUs configured to run in the given layer, and calculate in the next layer This is supported by compensating for computation times greater than the computation time in the given layer. The computer implementation method described in item 25.

Images