Computing processing system in block memory with random clock interleaving and output binding

Abstract

The invention relates to a computing processing system in block memory with random clock interleaving and output binding. The first in-memory computation (IMC) processing block and the second IMC processing block store computation weight data for in-memory computation operations performed in response to the feature data. The first IMC processing block is clocked by a first clock signal to control execution of a computing operation in the memory, and the second IMC processing block is clocked by a second clock signal to control execution of a computing operation in the memory. The clock tree generates a first clock signal and a second clock signal. In response to the random number, the clock tree applies a random interleaving to the timing of the first clock signal and the second clock signal. The binding circuit matches and binds the first calculation output and the second calculation output. In response to the random number, the binding circuitry takes into account a timing offset between the first and second computational outputs due to a random interleaving of timings applied to the first and second clock signals.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 650,202, filed May 21, 2024, which is incorporated by reference herein. TECHNICAL FIELD

[0003] Embodiments herein relate to an in-memory compute processing system comprising a plurality of in-memory compute processing blocks, and in particular to random clock staggering and output binding for use with these in-memory compute processing blocks. BACKGROUND

[0004] An in-memory compute (IMC) processing block stores information in bit cells of a memory array and performs computations at the bit cell level. One example of a computation performed by an IMC processing block is a multiply and accumulate (MAC) operation in which an input array of numbers (also referred to as feature or coefficient data (FD)) is multiplied by an array of compute weights (WD) stored in memory, and the products are added together to produce an output array of numbers (CMP).

[0005] By performing these computations in memory at the bit cell level, an IMC processing block does not need to move data back and forth between a memory device and a compute device. Thus, the limitations associated with data transfer bandwidth between devices are eliminated, and computations can be performed with lower power consumption.

[0006] An IMC processing block includes circuitry that utilizes a memory array formed of a plurality of memory cells arranged in a matrix format. Each memory cell is programmed to store a bit of compute weight data WD (also referred to as kernel data) for an in-memory compute operation. In one implementation, each bit of compute weight data has a logical “1” value or a logical “0” value, for example, represented by a logical state programmed into the memory cell.

[0007] Often, the compute weight data is very valuable proprietary information. Malicious parties often attempt to extract the compute weight data using extraction techniques known in the art, referred to as side-channel attacks, which are used to assess power consumption of a processing system including one or more IMC processing blocks during operation. In implementations where the weights remain stationary over long periods of operation and have a particular sparsity, the compute weight data is even more susceptible to side-channel attacks. There is a need in the art to provide protection for a processing system against side-channel attacks to decode details of the compute weight data (e.g., sparsity and stationarity) stored in the memory array of each included IMC processing block. SUMMARY

[0008] In one embodiment, a circuit includes: a first in-memory compute (IMC) processing block configured to store first compute weight data for an in-memory compute operation and configured to receive first feature data for the in-memory compute operation and to receive a first clock signal, the first IMC processing block generating a first compute output in response to execution of the in-memory compute operation; a second IMC processing block configured to store second compute weight data for an in-memory compute operation and configured to receive second feature data for the in-memory compute operation and to receive a second clock signal, the second IMC processing block generating a second compute output in response to execution of the in-memory compute operation; a clock tree configured to generate the first clock signal and the second clock signal, wherein the clock tree applies a random skew to timing of the first clock signal and the second clock signal in response to a random number; and a binding circuit configured to match and bind the first compute output and the second compute output, wherein the binding circuit accounts for a timing offset between the first compute output and the second compute output due to the random skew applied to timing of the first clock signal and the second clock signal in response to the random number.

[0009] In one embodiment, a method includes: storing first compute weight data for an in-memory compute operation in a first in-memory compute (IMC) processing block; storing second compute weight data for an in-memory compute operation in a second IMC processing block; applying first feature data for the in-memory compute operation to the first IMC processing block; applying second feature data for the in-memory compute operation to the second IMC processing block; clocking the first IMC processing block with a first clock signal to control execution of the in-memory compute operation by the first IMC processing block resulting in a first compute output; clocking the second IMC processing block with a second clock signal to control execution of the in-memory compute operation by the second IMC processing block resulting in a second compute output; generating the first clock signal and the second clock signal to have a random skew in timing controlled by a random number; and binding the first compute output and the second compute output in response to the random number, wherein the binding includes matching to account for a timing offset between the first compute output and the second compute output due to the random skew of the first clock signal and the second clock signal.

[0010] In one embodiment, a circuit comprises: a first in-memory compute (IMC) processing block group, wherein the first IMC processing block group comprises a first plurality of IMC processing blocks, each of the first plurality of IMC processing blocks is configured to store compute weight data for an in-memory compute operation and is configured to receive feature data for the in-memory compute operation, wherein the first plurality of IMC processing blocks of the first IMC processing block group receive a first clock signal, the first plurality of IMC processing blocks generate a first compute output in response to execution of the in-memory compute operation, the first IMC processing block group further comprises a first binding circuit configured to bind the first compute output to generate a first block group compute output; a second IMC processing block group, wherein the second IMC processing block group comprises a second plurality of IMC processing blocks, each of the second plurality of IMC processing blocks is configured to store compute weight data for an in-memory compute operation and is configured to receive feature data for the in-memory compute operation, wherein the second plurality of IMC processing blocks of the second IMC processing block group receive a second clock signal, the second plurality of IMC processing blocks generate a second compute output in response to execution of the in-memory compute operation, the second IMC processing block group further comprises a second binding circuit configured to bind the second compute output to generate a second block group compute output; a clock tree configured to generate the first clock signal and the second clock signal, wherein the clock tree applies a random skew to timing of the first clock signal and the second clock signal in response to a random number; and a third binding circuit configured to match and bind the first block group compute output and the second block group compute output, wherein the third binding circuit accounts for a timing offset between the first block group compute output and the second block group compute output due to the random skew applied to timing of the first clock signal and the second clock signal in response to the random number.

[0011] In one embodiment, a method includes storing compute weight data for an in-memory compute operation in a first plurality of in-memory compute (IMC) processing blocks arranged to form a first IMC processing block group; storing compute weight data for the in-memory compute operation in a second plurality of IMC processing blocks arranged to form a second IMC processing block group; applying feature data for the in-memory compute operation to the first plurality of IMC processing blocks; applying feature data for the in-memory compute operation to the second plurality of IMC processing blocks; clocking the first plurality of IMC processing blocks within the first IMC processing block group with a first clock signal to control execution of the in-memory compute operation by the first plurality of IMC processing blocks to produce a first compute output; binding the first compute output to generate a first block group compute output; clocking the second plurality of IMC processing blocks within the second IMC processing block group with a second clock signal to control execution of the in-memory compute operation by the second plurality of IMC processing blocks to produce a second compute output; binding the second compute output to generate a second block group compute output; generating the first clock signal and the second clock signal to have a random interleaving in timing controlled by a random number; and binding the first block group compute output and the second block group compute output in response to the random number, wherein the binding includes matching to account for a timing offset between the first block group compute output and the second block group compute output due to the random interleaving of the first clock signal and the second clock signal.

[0012] In one embodiment, an in-memory compute (IMC) processing block is configured to store compute weight data for an in-memory compute operation. A clock tree is configured to generate a clock signal to apply to the IMC processing block to control execution of the in-memory compute operation. The clock tree applies a random interleaving to timing of the clock signal in response to a random number. The random interleaving of timing applied to the block processing operation randomizes a power mode in which the in-memory compute system processes operations even in the case that the stored compute weight data is stable and exhibits sparsity. BRIEF DESCRIPTION OF DRAWINGS

[0013] For a better understanding of the embodiments, reference will now be made, purely by way of example, to the accompanying drawings in which:

[0014] Figure 1 is a block diagram of an in-memory compute processing system including a plurality of in-memory compute processing blocks;

[0015] Figure 2 is a timing diagram illustrating an example operation of the processing system of Figure 1

[0016] Figure 3 ​is a timing diagram illustrating example operations of the processing system of Figure 1

[0017] Figure 4 is a schematic diagram of an analog in-memory computing block used in the processing system of Figure 1

[0018] Figure 5 is a circuit diagram of a 6T static random access memory (SRAM) cell

[0019] Figure 6 is a circuit diagram of an 8T SRAM cell

[0020] Figure 7 is a schematic diagram of a digital in-memory computing block used in the processing system of Figure 1

[0021] Figure 8 is a block diagram of a memory-in-computing processing system including a plurality of groups of in-memory computing processing blocks

[0022] Figure 9 shows a block diagram of a group of blocks

[0023] Figure 10 is a timing diagram illustrating example operations of the processing system of Figure 8

[0024] Figure 11 is a timing diagram illustrating example operations of the processing system of Figure 8 DETAILED DESCRIPTION

[0025] Reference is now made to Figure 1 which shows a block diagram of a memory-in-computing processing system 10. The processing system 10 includes a plurality of in-memory computing (IMC) processing blocks 12. For example, the IMC processing blocks 12 can be arranged in an array format having one or more block rows and a plurality of block columns (or a plurality of block rows and one or more block columns). Figure 1 The arrangement of the IMC processing blocks 12 for the processing system 10 is illustrated by way of example only as including a single block row including a plurality of IMC processing blocks 12, with each IMC processing block 12 being located in a block column.

[0026] ​​​​​The in-memory computation processing operation performed by each IMC processing block 12 depends at least on the computation weights or kernel data (WD) stored in the memory array of the IMC processing block 12, the feature or coefficient data (FD) input to the IMC processing block 12, and the clock signal CLKin input to the IMC processing block 12. One or more pulses in the pulse train of the clock signal CLKin control the timing of the in-memory computation processing operation at each IMC processing block 12 to access the computation weights or kernel data (WD) and multiply the accessed computation weights or kernel data (WD) by the feature or coefficient data (FD) to generate computation output (CMP) data for multiplication and accumulation (MAC) operations.

[0027] exist Figure 1 In the architectural context shown, index r represents a block row, and index c represents a block column. Therefore, the weights or kernel data WD are calculated. rc Typically, this represents IMC processing block 12 located at block row r and block column c within processing system 10. rc The memory array stores data used for in-memory computational processing operations. Additionally, feature or coefficient data FD... rc Typically, this refers to IMC processing block 12 located at block row r and block column c within processing system 10. rc The input data for in-memory computation processing operations. Additionally, the block computation output (CMP) is... rc This typically indicates that block 12 is processed by the IMC at block row r and block column c within processing system 10. rc The generated output data from computational processing operations in memory. Additionally, the clock signal CLKin. rc Typically, this refers to IMC processing block 12 located at block row r and block column c within processing system 10. rc The input clock.

[0028] The processing system 10 also includes an output bonding circuit 16, which is configured to output from each IMC processing block 12. rc The in-memory computation processing operation is executed, and the received block computed output (CMP) is output. rc And it binds the received block computation data to generate decision outputs (decisions) for in-memory computation operations. In this case, each IMC processes 12 blocks. rc These are configured to generate partial computational outputs that contribute to the final result (e.g., decision-making). For example, this final output could represent computational values ​​for a layer or a specific sub-tensor geometry. These IMC processing blocks 12 are configured according to the graphical topology used to process the data. rc They can be arranged sequentially or aligned in a parallel configuration. Block 12 is processed by IMC. rcThe operations performed by the binding circuit 16 between them represent the stagnation domain and the co-scheduled execution pipeline group. In the operations performed, the IMC processing block 12... rc The various outputs (despite timing offsets) are matched and bound to each other through a binding operation performed by the binding circuit 16.

[0029] Applied to the corresponding IMC processing block 12 rc Various clock signals CLKin rc It is generated by clock tree circuit 20 based on the master clock signal CLKmstr. However, for various clock signals CLKin... rc There is no fixed (i.e., unchanging) timing relationship. Clock tree circuit 20 receives a random number (RN) generated by random number generator (RNG) circuit 22. In response to the received random number RN, clock tree circuit 20, for example, combines the execution of each computational operation in memory, to apply random interleaving to various clock signals CLKin. rc The timing relationship. For example, this random interleaving can be achieved by interleaving various clock signals CLKin. rc One or more randomly selected clock signals can be phase-shifted. For example, this random interleaving can be achieved by skipping various clock signals CLKin. rc This random interleaving can be achieved using clock pulses from one or more randomly selected clock signals. For example, this random interleaving can be achieved using various clock signals CLKin. rc This is achieved by adding clock pulses to one or more randomly selected clock signals.

[0030] Due to its application in various clock signals CLKin rc The random interleaving of timing relationships in IMC processing block 12 rc CMP output for block computation rc The generated timing will contain corresponding random interleaving. To account for the timing offset introduced by the random interleaving of timing relationships, and to ensure the correct matching and binding of the received computation data to generate the decision output (decision) for in-memory computation operations, a random number RN is also applied to the output binding circuit 16. In response to the received random number RN, the output binding circuit 16 can apply the corresponding random interleaving to the timing process to pass through the IMC processing block 12. rc Collect, match, and bind computational outputs (CMP) rc In fact, the output bonding circuit 16 will correctly match and bond to the IMC processing block 12. rc The received computational data for the corresponding but time-shifted in-memory computation operation.

[0031] By considering Figure 2The specific example of the timing diagram shown above can better illustrate the above. For a given pulse 30 of the master clock signal CLKmstr, the clock tree circuit 20 responds to the random number (RN) generated by the random number generator (RNG) circuit 22 and applies it to the IMC processing block 12. 11 clock signal CLKin 11 Generate corresponding pulse 32 11 For use in IMC processing block 12 12 clock signal CLKin 12 Generate corresponding pulse 32 12 For use in IMC processing block 12 13 clock signal CLKin 13 Generate corresponding pulse 32 13 Note that in pulse 32 11 32 12 and 32 13 The timing of the leading edge contains random interleaving (see figure 34), where the random interleaving depends on the generated random number (RN) and is implemented as a phase shift. Therefore, each IMC processing block 12 rc The timing of in-memory computation operations will include corresponding random interleaving 36 (in-memory computation performance indicated by the dotted-dash arrow), and will be handled by IMC processing block 12. 11 12 12 and 12 13 The computation output CMP of the in-memory computation operation performed. 11 CMP 12 and CMP 13 The timing of the presentation will contain corresponding random interleaving 37. Using random numbers (RN) generated by the random number generator (RNG) circuit 22, the output binding circuit 16 will control the output from the IMC processing block 12 respectively. 11 12 12 and 12 13 Receive 38 11 38 12 and 38 13 Matching calculation output CMP 11 CMP 12 and CMP 13 The timing is adjusted to ensure correct data matching and binding, thereby generating a decision output (Decision). In this way, the calculated output CMP is generated in response to the initial pulse 30 of the master clock signal CLKmstr. 11 CMP 12 and CMP 13 They are correctly matched and bound together for processing to produce decision output (decision).

[0032] As another example, considerFigure 3 The timing diagram is shown. Clock tree circuit 20 receives the pulse train 42 of the master clock signal CLKmstr, and for each clock signal CLKin... rc Output pulse train 44. However, clock tree circuit 20, in response to the random number (RN) generated by random number generator (RNG) circuit 22, randomly suppresses (i.e., skips) the clock signal CLKin. rc A clock pulse from one or more clock signals (as shown by reference numeral 46 in the figure). This is generated in the clock tree circuit 20 for application to the IMC processing block 12. 11 clock signal CLKin 11 Used for application in IMC processing block 12 12 clock signal CLKin 12 and for use in IMC processing block 12 13 clock signal CLKin 13 In this example, the logic state of a bit of the random number (RN), or the logic state of a bit of the signal generated by decoding the random number (RN), will specify whether the clock tree circuit 20 should selectively suppress (i.e., skip) the included clock pulses. In this example case, for the application to IMC processing block 12 12 clock signal CLKin 12 The pulse 46 in the clock pulse exhibits random suppression (i.e., skipping), thereby introducing a timing offset (or interleaving) 34 at the leading edge of the clock pulse, which controls the timing of the execution of computational operations in memory. Therefore, relative to the processing block 12 by the IMC... 11 and 12 13 The performance of in-memory computation operations is determined by IMC processing block 12. 12 The performance of in-memory computation operations will have a corresponding timing offset (or interleaving) of 36 (because IMC processing block 12). 12 (In-memory computation will be performed in response to pulse 47 following skipped pulse 46), where the performance of in-memory computation is indicated by the dotted-dash arrow. Therefore, for block 12 processed by IMC... 11 12 12 and 12 13 The in-memory computation operation performed, relative to the computation output CMP 11 and CMP 13 Calculate the output CMP 12 The presentation will have a corresponding timing offset of 37. Using a random number (RN) generated by the random number generator (RNG) circuit 22, the output binding circuit 16 will control the output from the IMC processing block 12. 11 12 12 and 12 13 Receive 38 11 3812 and 38 13 Matching calculation output CMP 11 CMP 12 and CMP 13 The timing is adjusted to ensure correct data matching and binding, thereby generating a decision output (Decision). In this way, the calculated output CMP is generated in response to the initial pulse 30 of the master clock signal CLKmstr. 11 CMP 12 and CMP 13 They are correctly matched and bound together for processing to produce decision output (decision).

[0033] It should be recognized that the operations described herein (which introduce controls for the processing of IMC block 12) 11 The random interleaving of timing for in-memory computation operations provides a protection mechanism that makes it more difficult for power-based side-channel attacks to successfully identify the stored computation weight data (WD). In fact, the clock signal CLKin... rc The random interleaving of the leading edges of the pulses (reference numeral 34) and the relative timing offset will result in a random power waveform for the processing system 10, which is combined with the process of accessing computational weights or kernel data (WD) and multiplying the accessed computational weights or kernel data (WD) by feature or coefficient data (FD) to generate data for block computation output (CMP). This is applied to the clock signal CLKin. rc The random interleaving or offset (reference numeral 34) minimizes electromagnetic interference (EMI) and manages clock transient current consumption. This reduces the likelihood of successfully processing the system's current profile to recover the stored weighted data (e.g., this data may be stable and exhibit sparsity).

[0034] Now for reference Figure 4 It shows a schematic diagram of a simulated IMC processing block 110, which can be used as, for example... Figure 1 One or more IMC processing blocks 12 in system 10 rc Block 110 utilizes memory circuitry including a static random access memory (SRAM) array 112, which consists of standard 6 TSR RAM memory cells 114 arranged in a matrix format with N rows and M columns (see [link to SRAM]). Figure 5 This can be achieved by using a standard 8T memory cell (see [link]). Figure 6) or SRAM with similar functionality and topology. Each memory cell 114 is programmed to store bits of a compute weight or kernel data (WD) for an in-memory compute operation. In this case, an in-memory compute operation is understood to be a form of high-dimensional matrix vector multiplication (MVM) that supports multi-bit weights stored in multiple bit cells of the memory. A group of bit cells (in the case of multi-bit weights) can be considered as a virtual synapse element. Each bit of a compute weight has a logical "1" or logical "0" value.

[0035] Each SRAM cell 114 includes a word line WL and a pair of complementary bit lines BLT and BLC. An 8T type SRAM cell also includes a read word line RWL and a read bit line RBL. Cells 114 in a common row of the matrix are connected to each other by a common word line WL (and, in an 8T type implementation, by a common read word line RWL). Cells 114 in a common column of the matrix are connected to each other by a common pair of complementary bit lines BLT and BLC (and, in an 8T type implementation, by a common read bit line RBL). Each word line WL, RWL is driven by a word line drive circuit 116, which can be implemented as a CMOS drive circuit (e.g., a series connected pair of p-channel and n-channel MOSFET transistors forming a logic inverter circuit). Word line signals applied to the word lines and driven by the word line drive circuits 116 are generated in accordance with feature data input to the in-memory compute block 110 and controlled by a row controller circuit 118. Column processing circuit 120 senses analog signals on the paired complementary bit lines BLT and BLC (and / or read bit line RBL) of M columns, converts the analog signals to digital signals, performs digital computations on the digital signals, and generates a compute output CMP for the in-memory compute operation.

[0036] It will be appreciated that the block 110 can instead use different types of memory cells, e.g., any form of bit cell, storage element, or synapse element.

[0037] Although Figure 4 Not explicitly shown in FIG. 1, it will be appreciated that the block 110 also includes conventional row decoding, column decoding, and read-write circuitry known to those skilled in the art to be used in conjunction with writing data bits (e.g., compute weight data) to and reading data bits from the SRAM cells 114 of the memory array 112. This operation is referred to as a conventional memory access mode and is distinct from the analog in-memory compute operation described above.

[0038] The row controller circuit 118 receives feature data (FD) for an in-memory compute operation and, in response thereto, performs the following functions: selecting word lines WL <n-1>(or read word lines RWL<0> to RWL <n-1>which word lines in the analog memory are to be simultaneously accessed (or actuated) during the in-memory compute operation and also used to control the application of pulse signals to the word lines in accordance with the in-memory compute operation. Figure 4 The simultaneous actuation of all N word lines with pulse word line signals is illustrated by way of example only, and it will be appreciated that the in-memory compute operation can utilize less than the simultaneous actuation of all rows of the SRAM array. The analog signals on a given pair of complementary bit lines BLT and BLC (or the analog information on the read bit line RBL in an 8T implementation) depend on the logical state of the bits of the compute weight stored in the corresponding column of memory cells 114 and the width(s) of the pulse word line signals applied to these memory cells 114.

[0039] The control circuitry controls the performance of the analog in-memory compute operation in response to the received clock signal CLKin.

[0040] Figure 4 The illustrated implementation shows one example in the form of pulse width modulation (PWM) for the applied word line signals for the in-memory compute operation in accordance with the received feature data. The use of PWM or periodic pulse modulation (PTM) for the applied word line signals is a common technique for in-memory compute operations based on the linearity of vectors for multiply-accumulate (MAC) operations. The pulse word line signal format can be further evolved to encode pulse trains to manage block sparsity of the feature data for the in-memory compute operation. It can thus be appreciated that any set of encoding schemes for the applied word line signals can be used when driving multiple word lines simultaneously. Further, in a simpler implementation, it can be appreciated that all of the applied word line signals in the simultaneous actuation can instead have the same pulse width.

[0041] Reference is now made to Figure 7 which shows a block diagram of a digital IMC processing block 210 that can be used as one or more of the IMC processing blocks 12 in the system 10 of Figure 1 rc The block 210 is implemented using a memory circuit that includes a static random access memory (SRAM) array 212 formed of a plurality of SRAM memory cells 214 arranged in a matrix format having N rows and M columns. Each memory cell 214 is programmed to store a data bit. To support the digital in-memory compute processing, the data stored in the memory array 212 includes compute weights or kernel data (WD). In this case, the digital in-memory compute operation is understood to be a form of high-dimensional matrix vector multiplication (MVM) that supports multi-bit weights stored in multiple bit cells of the memory. The groups of bit cells (in the case of multi-bit weights) can be viewed as virtual synapse elements. Each data bit stored in the memory array (whether user data or weight data) has a logical "1" or logical "0" value.​

[0042] Each SRAM memory cell 214 may include, for example: Figure 5 The 6TB memory cell shown is an alternative to a standard 8TB memory cell (see [link]). Figure 6 Or an SRAM with similar functionality and topology. It should be understood that block 210 can be replaced with different types of memory cells, such as any form of bit cell, storage element, or synaptic element that produces deterministic reads arranged in an array. As a non-limiting example, consider using non-volatile memory (NVM) cells, such as, for example, magnetoresistive RAM (MRAM) cells, flash memory cells, phase-change memory (PCM) cells, or resistive RAM (RRAM) cells. In the following discussion, the focus is on the implementation using 8T-type SRAM cells 214, but this is done as a non-limiting example, and it should be understood that any suitable memory element (e.g., binary (two-level) memory element or m-level (multi-level) memory element) can be used.

[0043] Each cell 214 includes a word line WL, a pair of complementary bit lines BLT and BLC, a read word line RWL, and a read bit line RBL. SRAM memory cells in the common rows of the matrix are interconnected via the common word line WL and the common read word line RWL. Each word line (WL and / or RWL) is driven by word line driver circuitry 216 during read and write operations using word line signals generated by row decoder circuitry 218. SRAM memory cells in the common columns of the matrix spanning the entire array 212 are interconnected via a common pair of complementary (write) bit lines BLT and BLC. Array 212 is divided into P subarrays 2130 to 213. P-1 Each subarray 213 includes M columns and N / P rows of memory cells 214. The SRAM memory cells in the common columns of each subarray 213 are connected to each other via local read bit lines RBL.

[0044] P local read bit lines RBL0 from subarray 213 for column x in array 212 <x>To RBL P-1 <x>complementary bit line pair BLT for column x in array 212 <x>and BLC <x>to column input / output (I / O) circuitry 220(x). Here, x = 0 to M-1. Data input ports (D) of column I / O circuitry 220 receive input data (user or weight data) to be written into SRAM memory cells 214 in the column over a pair of complementary bit lines BLT, BLC in response to assertion of a word line signal in a regular memory access mode of operation. Data output ports (Q) of column I / O circuitry 220 generate output data read from SRAM memory cells 214 in the column over read bit line RBL in response to assertion of a read word line signal in the regular memory access mode of operation. In addition, column I / O circuitry 220 also includes P subarray data output ports R0 to R P-1 to generate output data read from memory cells 214 on local read bit line RBL of the corresponding subarray 2130 to 213 P-1 in response to simultaneous assertion of multiple read word line signals (one read word line signal per subarray 213) in a digital in-memory compute mode of operation. Digital compute processing circuitry 223 performs digital computations on the output data from subarray data output ports R in accordance with received feature data (FD) and generates compute output CMP for the in-memory compute operation. Processing circuitry 223 can implement computation logic for digital signal processing in a variety of ways, including: full support for Boolean operations (XOR, XNOR, AND, OR, etc.) and vector operations depending on system and application requirements; accumulation pipelined operations with support for vector multiplication in memory; and matrix vector multiplication pipelined operations with output from memory as one vector of a multiply and accumulate (MAC) function. It should be noted that processing circuitry 223 is an integral part of digital in-memory compute circuitry 210.

[0045] The computation logic for digital signal processing performed by processing circuitry 223 is closely integrated with input / output circuitry and subarray data output ports R0 to R P-1 to support utilization of wide (e.g., P times) vector access. This solution brings many figure of merit (FOM) advantages, including: implementation of multiple word access in the same cycle can amortize the common logic switching capability within the SRAM when wide vector access occurs; use of subarrays 213 can reduce bit line switching power consumption (i.e., where P word lines are asserted in parallel to access P corresponding subarrays); support for both regular memory access mode of operation and digital in-memory compute mode of operation with opportunity to switch between regular memory access mode of operation and digital in-memory compute mode of operation; and improved on / off current ratio on the same bit line, which is a key issue when the circuitry system is implemented using fully depleted silicon-on-insulator (FDSOI) technology where positive body bias is actively used.

[0046] It should be noted that in accordance with the conventional memory access mode of operation, the block 210 presents a conventional SRAM interface through the data input port D and the data output port Q. In response to the applied memory address (Addr), the circuitry supports read (via the data output port Q) and write (via the data input port D) access to a single row of memory cells 214 in the array 212 through the selected assertion of a single word line WL or RWL. The circuitry also supports, in accordance with the digital in-memory compute mode of operation, simultaneous read access (via the subarray data output ports R0 through R P-1 The subarray processing interface is provided. In response to the applied memory address (Addr), the circuitry supports simultaneous read access (via the data output ports R0 through R P-1 ) to a single row of memory cells 214 in each of the subarrays 2130 through 213 P-1 ) can be decoded to select multiple word lines (one word line per subarray 213) for assertion, or multiple addresses can be decoded to select multiple word lines (one word line per subarray 213) for assertion. The use of multiple subarrays 213 in this mode enables parallelism to support very extensive compute processing access without sacrificing density. Advantageously, this digital in-memory compute mode of operation leverages the resources of a conventional SRAM design with modified control, decoding, and input / output circuitry (as will be discussed in detail herein) to enable parallel access in the digital in-memory compute mode of operation, and switches between the conventional memory access mode of operation and the digital in-memory compute mode of operation through additional control as needed by the system application. This architecture brings parallelism with the use of push rule bit cells, enabling high density / compute density when configured for the in-memory compute mode of operation. Although the foregoing provisions, other bit cell types can also be used as noted above.

[0047] The control circuitry 219 controls the mode of operation of the circuitry within the block 210 in response to the logic state of the control signal IMC and the received clock signal CLKin. When the control signal IMC is in a first logic state (e.g., logic low), the block 210 operates in accordance with the conventional memory access mode of operation (for writing data from the data input port D to the memory array, or reading data from the memory array to the data output port Q). Conversely, when the control signal IMC is in a second logic state (e.g., logic high), the block 210 operates in accordance with the digital in-memory compute mode of operation (for reading weight data from the memory array to the subarray data output ports R).

[0048] When the block 210 is operated in a regular memory access mode of operation, in response to the clock signal CLKin, the row decoder circuit 218 decodes the received address (Addr) to selectively actuate only one word line WL (during a write) or one read word line RWL (during a read) of the entire array 212 with a word line signal pulse to access a corresponding one of the rows of memory cells 214. During a write, the logical state of the data at the input port D is written by the pair of complementary bit lines BLT, BLC to the single row of memory cells coupled with the accessed word line WL by the column I / O circuit 220. During a read, the logical state of the data stored in the single row of memory cells coupled with the accessed word line WL is output from the read bit line RBL to the column I / O circuit 220 for output at the data output port Q.

[0049] When the block 210 is operated in a digital in-memory compute mode of operation, in response to the clock signal CLKin, the row decoder circuit 218 decodes the received address (Addr) to selectively (simultaneously) actuate one read word line RWL in each subarray 213 in the memory array 212 with a word line signal pulse to access a corresponding row of memory cells 214 in each subarray 213. The logical state of the weight data stored in the row of memory cells in each subarray 213 coupled with the accessed read word line RWL is output from the read bit line RBL0 <x>To RBL P-1 <x>to column I / O circuitry 220 for output at corresponding subarray data output ports RO to R P-1

[0050] It should be noted that the output of each subarray 213 can be considered as one sub-sensor / tensor for processing operations. Further, the output of multiple subarrays 213 can be grouped as larger tensors. The grouping of subarray outputs can be across columns, across rows, or both. Such processing is supported by the configuration and operation of processing circuitry 223.

[0051] Reference is now made to Figure 8 which shows a block diagram of an in-memory compute processing system 50. Processing system 50 includes a plurality of in-memory compute (IMC) processing block groups 52. IMC processing block groups 52 can be arranged, for example, in an array format having one or more group rows and a plurality of group columns (or a plurality of group rows and one or more group columns). Figure 8 The arrangement of IMC processing block groups 52 of processing system 50 is illustrated by way of example only to include a single group row of a plurality of IMC processing block groups 52, with each IMC processing block group 52 located in a group column. Each IMC processing block group 52 includes one or more in-memory compute (IMC) processing blocks 12. For example, the IMC processing blocks 12 within each IMC processing block group 52 can be arranged in an array format having one or more block rows and one or more columns. Figure 8 The arrangement of IMC processing blocks 12 within each block group 52 is illustrated by way of example only to include a single block row of a plurality of IMC processing blocks 12, with each IMC processing block 12 located in a block column. Figure 9 An example implementation of a block group 52 is shown in

[0052] By way of reference, Figure 1 An implementation of processing system 10 of Figure 8 A special case of an implementation of processing system 50 of

[0053] The in-memory compute processing operations performed by each IMC processing block 12 depend at least on compute weight or kernel data (WD) stored in a memory array of the IMC processing block 12, feature or coefficient data (FD) input to the IMC processing block 12, and a clock signal CLKin input to the IMC processing block 12. One or more pulses of the clock signal CLKin control the timing of the in-memory compute processing operations at each IMC processing block 12 to access the compute weight or kernel data (WD) and multiply the accessed compute weight or kernel data (WD) by the feature or coefficient data (FD) to generate data for a compute output (CMP) of a multiply and accumulate (MAC) operation.

[0054] In Figure 8 ​and Figure 9 In the illustrated architectural context, the index r denotes a block row within a given tile 52, and the index c denotes a block column within the given tile 52. Thus, the weight or kernel data WD rc generally denotes the IMC processing block 12 rc located at block row r and block column c within a given tile 52 rc generates for an in-memory compute operation. Further, the compute output CMP rc generally denotes the output data of an in-memory compute operation generated by the IMC processing block 12 RC located at block row r and block column c within a given tile 52 RC generates for an in-memory compute operation. Further, the clock signal CLKin rc generally denotes the input clock applied to each IMC processing block 12 RC located at block row R and block column C within a tile 52 RC generates for an in-memory compute operation. Further, the tile compute output GCMP rc generally denotes the output data at block row R and block column C of a tile 52 RC generated by binding the compute output CMP RC generated by the IMC processing block 12 rc located at block row r and block column c within a given tile 52 rc .

[0055] Each tile 52 includes an output binding circuit 58 configured to receive the compute output CMP rc from the in-memory compute operation performed by each IMC processing block 12 rc within the tile, and to bind the received data to generate the tile compute output GCMP RC for the tile 52 RC . In this case, each IMC processing block 12 rc is configured to generate a partial compute output that contributes to an intermediate result. For example, the intermediate output can represent a layer or a particular sub-tensor geometry’s computed values. Depending on the graph topology used to process the data, these IMC processing blocks 12 rc may be arranged in a sequence or aligned in a parallel configuration. The operations performed by the binding circuit 58 between the IMC processing blocks 12 rc represent a stall domain and a co-scheduled execution pipeline group. In the operations performed, the IMC processing blocks 12 rc The various outputs of the IMC processing blocks 12 RC within a block group 52 rc receive the same clock signal CLKin RC The block group computation outputs GCMP rc are (substantially) presented at the same time for binding by the circuit 58.

[0056] The processing system 50 further comprises an output binding circuit 56 configured to receive the block group computation outputs GCMP RC from the block group 52 RC and to bind the received data to generate a decision output (decision) for the in-memory computation operation. In this case, each block group 52 RC is configured to generate partial computation outputs that contribute to the final result (e.g., the decision). For example, the final output can represent a computation value of a layer or a particular sub-tensor geometry. Depending on the graph topology for processing the data, these block groups 52 RC may be arranged in a sequence or aligned in a parallel configuration. The operations performed by the binding circuit 56 between the block groups 52 RC represent a stall domain and a co-scheduled execution pipeline group. Among the operations performed, the various outputs of the block groups 52 RC are bound to each other by the binding circuit 56, albeit with a timing offset. In contrast to the timing operations of the IMC processing blocks 12 RC within a block group 52 rc , each block group 52 RC receives its respective clock signal CLKin RC and, thus, the block group computation outputs GCMP RC may be presented at different times for binding by the circuit 56, and this timing offset has to be accounted for in order to correctly match and bind the block group computation outputs GCMP RC .

[0057] The various clock signals CLKin RC applying to the corresponding block groups 52 RC are generated by a clock tree circuit 20 from a master clock signal CLKmstr. However, there is no fixed (i.e., invariant) timing relationship for the various clock signals CLKin RC . The clock tree circuit 20 receives a random number (RN) generated by a random number generator (RNG) circuit 22. In response to the received random number RN, the clock tree circuit 20 applies a random skew to the timing relationship of the various clock signals CLKin RC , e.g., in connection with the execution of each in-memory computation operation. For example, such a random skew can be applied by shifting the phase of the various clock signals CLKin RC This can be achieved by applying a phase shift to one or more randomly selected clock signals. For example, this random interleaving can be achieved by skipping various clock signals CLKin. RC This random interleaving can be achieved using clock pulses from one or more randomly selected clock signals. For example, this random interleaving can be achieved using various clock signals CLKin. RC This is achieved by adding clock pulses to one or more randomly selected clock signals.

[0058] Due to its application in various clock signals CLKin RC The random interleaving of timing relationships in block group 52 RC GCMP is calculated for block groups. RC The generated timing will contain corresponding random interleaving. To address this issue and ensure the correct matching and binding of received data to generate the decision output (decision) for in-memory computation operations, a random number RN is also applied to the output binding circuit 56. In response to the received random number RN, the output binding circuit 56 can apply the corresponding random interleaving to the timing process to collect and match data from block group 52. RC GCMP block group computation output RC .

[0059] Figure 8 and Figure 9 The operation of system 50 is similar to that of... Figure 2 and Figure 3 The timing diagram shown Figure 1 The operation of System 10 in the example. Figure 10 and Figure 11 As shown in the timing diagram, the main difference is that interleaving (offset) 34, 36, and 37 are applied to block group 52. RC Overall processing operations and block group computation output GCMP RC Matching and binding are shown as indicated by the dashed arrows.

[0060] U.S. Patent Application Publications Nos. 2024 / 0071439 and 2024 / 0112728 are incorporated herein by reference.

[0061] The foregoing description provides a complete and informative description of exemplary embodiments of the invention through exemplary and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the art when read in conjunction with the accompanying drawings and appended claims, given the foregoing description. Nevertheless, all such and similar modifications to the teachings of the invention will still fall within the scope of the invention as defined in the appended claims.< / x> < / x> < / x> < / x> < / x> < / x>

Claims

1. A circuit comprising: a first in-memory computing (IMC) processing block configured to store first computation weight data for an in-memory computation operation and configured to receive first feature data for the in-memory computation operation and to receive a first clock signal, the first IMC processing block generating a first computation output in response to execution of the in-memory computation operation; a second IMC processing block configured to store second computation weight data for an in-memory computation operation and configured to receive second feature data for the in-memory computation operation and to receive a second clock signal, the second IMC processing block generating a second computation output in response to execution of the in-memory computation operation; a clock tree configured to generate the first clock signal and the second clock signal, wherein the clock tree applies a random skew to timing of the first clock signal and the second clock signal in response to a random number; and a binding circuit configured to match and bind the first computation output and the second computation output, wherein the binding circuit accounts for a timing offset between the first computation output and the second computation output due to the random skew to timing of the first clock signal and the second clock signal in response to the random number.

2. The circuit of claim 1, wherein the random skew to timing of the first clock signal and the second clock signal comprises a phase shift between the first clock signal and the second clock signal.

3. The circuit of claim 1, wherein the random skew to timing of the first clock signal and the second clock signal comprises skipping of clock pulses among the first clock signal and the second clock signal.

4. The circuit of claim 1, wherein the random skew to timing of the first clock signal and the second clock signal comprises addition of clock pulses among the first clock signal and the second clock signal.

5. The circuit of claim 1, further comprising a random number generator configured to generate the random number in conjunction with the execution of the in-memory computation operation.

6. The circuit of claim 1, wherein each of the first IMC processing block and the second IMC processing block is implemented as one of an analog IMC processing block or a digital IMC processing block.

7. A method comprising: storing first computation weight data for an in-memory computation operation in a first in-memory computing (IMC) processing block; storing second computation weight data for an in-memory computation operation in a second IMC processing block; applying first feature data for the in-memory computation operation to the first IMC processing block; applying second feature data for the in-memory computation operation to the second IMC processing block; clocking the first IMC processing block with a first clock signal to control execution of the in-memory computation operation by the first IMC processing block, resulting in a first computation output; clocking the second IMC processing block with a second clock signal to control execution of the in-memory computation operation by the second IMC processing block, resulting in a second computation output; and binding the first computation output and the second computation output in response to a random number, wherein the binding accounts for a timing offset between the first computation output and the second computation output due to a random skew to timing of the first clock signal and the second clock signal in response to the random number. clocking the second IMC processing block with a second clock signal to control execution of the in-memory compute operation by the second IMC processing block to produce a second compute output; generating the first clock signal and the second clock signal to have a random interleaving in timing controlled by a random number; and binding the first compute output and the second compute output in response to the random number, wherein the binding includes matching to account for a timing offset between the first compute output and the second compute output due to the random interleaving of the first clock signal and the second clock signal.

8. The method of claim 7, wherein generating the first clock signal and the second clock signal to have the random staggering comprises: applying a phase shift between the first clock signal and the second clock signal.

9. The method of claim 7, wherein generating the first clock signal and the second clock signal to have the random staggering comprises: skipping a clock pulse among the first clock signal and the second clock signal.

10. The method of claim 7, wherein generating the first clock signal and the second clock signal to have the random staggering comprises: adding a clock pulse among the first clock signal and the second clock signal.

11. The method of claim 7, wherein the in-memory compute operation performed by each of the first IMC processing block and the second IMC processing block is one of an analog IMC processing operation or a digital IMC processing operation.

12. A circuit comprising: a first in-memory compute (IMC) processing block group, wherein the first IMC processing block group includes a first plurality of IMC processing blocks, each of the first plurality of IMC processing blocks configured to store compute weight data for an in-memory compute operation and configured to receive feature data for the in-memory compute operation, wherein the first plurality of IMC processing blocks of the first IMC processing block group receive a first clock signal, the first plurality of IMC processing blocks generate a first compute output in response to execution of the in-memory compute operation, the first IMC processing block group further including a first binding circuit configured to bind the first compute output to generate a first block group compute output; a second IMC processing block group, wherein the second IMC processing block group includes a second plurality of IMC processing blocks, each of the second plurality of IMC processing blocks configured to store compute weight data for an in-memory compute operation and configured to receive feature data for the in-memory compute operation, wherein the second plurality of IMC processing blocks of the second IMC processing block group receive a second clock signal, the second plurality of IMC processing blocks generate a second compute output in response to execution of the in-memory compute operation, the second IMC processing block group further including a second binding circuit configured to bind the second compute output to generate a second block group compute output; a clock tree configured to generate the first clock signal and the second clock signal, wherein the clock tree applies a random interleaving to timing of the first clock signal and the second clock signal in response to a random number; and a binding circuit configured to bind the first compute output and the second compute output in response to the random number, wherein the binding includes matching to account for a timing offset between the first compute output and the second compute output due to the random interleaving of the first clock signal and the second clock signal. a third binding circuit configured to match and bind the first tile group compute output and the second tile group compute output, wherein the third binding circuit accounts for a timing offset between the first tile group compute output and the second tile group compute output due to the random staggering of timing for the first clock signal and the second clock signal in response to the random number.

13. The circuit of claim 12, wherein the random staggering of timing applied to the first clock signal and the second clock signal comprises: a phase shift between the first clock signal and the second clock signal.

14. The circuit of claim 12, wherein the random staggering of timing applied to the first clock signal and the second clock signal comprises: a skipping of a clock pulse among the first clock signal and the second clock signal.

15. The circuit of claim 12, wherein the random staggering of timing applied to the first clock signal and the second clock signal comprises: an addition of a clock pulse among the first clock signal and the second clock signal.

16. The circuit of claim 12, further comprising a random number generator configured to generate the random number in conjunction with the performance of the in-memory compute operation.

17. The circuit of claim 12, wherein the processing blocks in each of the first plurality of IMC processing blocks and the second plurality of IMC processing blocks are implemented as one of analog IMC processing blocks or digital IMC processing blocks.

18. A method comprising: storing compute weight data for an in-memory compute operation in a first plurality of in-memory compute (IMC) processing blocks arranged to form a first IMC processing tile group; storing compute weight data for an in-memory compute operation in a second plurality of IMC processing blocks arranged to form a second IMC processing tile group; applying feature data for the in-memory compute operation to the first plurality of IMC processing blocks; applying feature data for the in-memory compute operation to the second plurality of IMC processing blocks; clocking the first plurality of IMC processing blocks within the first IMC processing tile group with a first clock signal to control performance of the in-memory compute operation by the first plurality of IMC processing blocks to produce a first compute output; binding the first compute output to generate a first tile group compute output; clocking the second plurality of IMC processing blocks within the second IMC processing tile group with a second clock signal to control performance of the in-memory compute operation by the second plurality of IMC processing blocks to produce a second compute output; binding the second compute output to generate a second tile group compute output; generating the first clock signal and the second clock signal to have a random staggering in timing controlled by a random number; and binding the first tile group compute output and the second tile group compute output in response to the random number, wherein the binding includes matching to account for a timing offset between the first tile group compute output and the second tile group compute output due to the random staggering of the first clock signal and the second clock signal.

19. The method of claim 18, wherein generating the first clock signal and the second clock signal to have the random staggering comprises: applying a phase shift between the first clock signal and the second clock signal.

20. The method of claim 18, wherein generating the first clock signal and the second clock signal to have the random staggering comprises: skipping a clock pulse among the first clock signal and the second clock signal.

21. The method of claim 18, wherein generating the first clock signal and the second clock signal to have the random staggering comprises: adding a clock pulse among the first clock signal and the second clock signal.

22. The method of claim 18, wherein the in-memory compute operations performed by each of the first and second pluralities of IMC processing blocks are one of analog or digital IMC processing operations.

23. An in-memory compute system, comprising: in-memory compute (IMC) processing blocks configured to store compute weight data for in-memory compute processing operations; a clock tree configured to generate clock signals to apply to the IMC processing blocks to control performance of the in-memory compute processing operations; wherein the clock tree applies random staggering to timing of the clock signals in response to a random number; wherein the random staggering applied to performance of in-memory compute processing operations by the IMC processing blocks randomizes power modes of the in-memory compute system during processing operations in a case where the stored compute weight data is stationary and exhibits sparsity.

24. The system of claim 23, wherein the random staggering comprises phase shifting applied between the clock signals.

25. The system of claim 23, wherein the random staggering comprises selective skipping of clock pulses in the clock signals.

26. The system of claim 23, wherein the random staggering comprises selective addition of clock pulses in the clock signals.

27. The system of claim 23, further comprising a random number generator configured to generate the random number in conjunction with the performance of the in-memory compute processing operations.

28. The system of claim 23, wherein each of the IMC processing blocks is implemented as one of an analog or digital IMC processing block.

Images