A programmable activity adaptive dual-mode inference accelerator and method

Abstract

The application provides a programmable activity adaptive dual-mode inference accelerator and method, relates to the technical field of neural networks, and the accelerator comprises a dual-mode front end configured to selectively work in an index-driven mode or a pre-computation mode according to a current processing data activity level, and normalize a generated request in the selected mode into a request in a unified format; a unified lookup table back end configured to receive the normalized request, and perform unified lookup table reading and accumulation operation based on the normalized request; and a cross-layer data transfer unit configured to directly provide an output of a current layer to a subsequent layer in the form of an executable encoded stream, so that the dual-mode front end of the subsequent layer directly parses and processes the encoded stream; the current layer and the subsequent layer are both inference calculation layers in a neural network. The application solves the problem that the prior art cannot simultaneously consider the efficiency of two scenes when facing a dramatic fluctuation of activity between layers in a network.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a programmable activity-adaptive dual-mode inference accelerator and method. Background Technology

[0002] Spiking neural networks (SNNs), hailed as the third generation of neural networks, transmit information between neurons through discrete spiking events. Unlike traditional continuous-valued artificial neural networks, the binary nature of spiking signals inherently gives them an advantage in event-driven computation, demonstrating great potential in low power consumption and biological plausibility. Early research on SNNs focused primarily on shallow structures and simple neuromorphic tasks. The non-differentiability of spiking activation functions posed a significant obstacle to training deep networks. The introduction of alternative gradient learning methods allowed researchers to effectively optimize network parameters using gradient backpropagation while maintaining the spiking communication mechanism, thus paving the way for the practical application of deep SNNs. Subsequently, the introduction of residual connection structures further improved the training stability and inference accuracy of deep SNNs.

[0003] In recent years, the architecture of spiking neural networks has undergone significant expansion. Transformer-style spiking network backbones have been proposed, introducing self-attention mechanisms and large-scale meta-architecture into the spiking domain. Simultaneously, the application scope of spiking neural networks has expanded from traditional image classification to fields such as object detection. These new models not only incorporate traditional sparse convolution operations but also widely adopt feedforward network modules, residual paths, attention-related operators, and integer-count-based spiking coding. This evolution has placed entirely new demands on hardware accelerators: accelerators must be able to handle a wider variety of operator types and cope with drastic changes in activity between different layers.

[0004] In practical applications, the internal activity of modern spiking neural network models exhibits highly non-uniform characteristics. The causes of this fluctuation are structural and predictable: channel expansion in feedforward modules drastically reduces the firing rate of hidden layers; residual addition operations push the activity of subsequent layers into denser regions; and thermometer-encoded counting codes introduce a significant pre-concentration effect over time. This dramatic fluctuation in inter-layer activity presents a fundamental challenge to hardware accelerator design. In low-activity regions, where most pulse positions are zero, index-driven sparse execution is suitable, concentrating hardware computing resources on effective pulse events. However, in high-activity regions, the metadata traversal of sparse indexes and irregular front-end scheduling control themselves become execution bottlenecks, making regularized dense execution paths less efficient.

[0005] Research on hardware acceleration for spiking neural networks has been conducted in multiple directions by both academia and industry. One approach is sparsity-aware accelerator design, which leverages the sparsity of spiking activity to avoid ineffective computations or uses spatiotemporal load imbalance as an optimization objective for scheduling. Another approach is dense pipeline design, which focuses on high-throughput regularized convolution execution, improving inference speed by optimizing data flow and backend utilization. A third approach is code-aware design, which optimizes hardware latency, power consumption, and memory access through specific spiking coding schemes. In recent years, accelerators for spiking Transformers have emerged as a new research direction. These accelerators attempt to cover multiple layer types and different density regions through unified processing units, dual-engine coverage architectures, or token-time binding strategies.

[0006] However, the existing solutions mentioned above are typically optimized around a dominant execution style, failing to treat the choice between sparse and rule-based execution as a programmable strategy. Specifically, sparsity-aware accelerators perform poorly in high-activity layers because metadata traversal, index scheduling, and irregular front-end control logic themselves become execution bottlenecks, with hardware utilization decreasing significantly as activity increases. Dense pipeline accelerators waste significant bandwidth and computational resources in low-activity layers because they perform the same operation at all pulse locations, regardless of whether a valid pulse event is generated at that location. While dual-engine or heterogeneous core designs attempt to cover both sparse and dense scenarios simultaneously, they require physically replicating high-capacity computing backends, incurring significant resource overhead, and data migration between different engines introduces additional latency. Summary of the Invention

[0007] This application provides a programmable activity-adaptive dual-mode inference accelerator and method to solve the problem that existing technologies cannot balance the efficiency of both scenarios when faced with drastic fluctuations in activity between layers within a network.

[0008] In a first aspect, this application provides a programmable activity-adaptive dual-mode inference accelerator, the accelerator comprising: A dual-mode frontend is configured to selectively operate in index-driven mode or pre-compute mode based on the activity level of the currently processed data, and to normalize requests generated in the selected mode into requests of a uniform format. A unified lookup table backend is configured to receive the normalization request and perform unified lookup table reading and accumulation operations based on the normalization request. A cross-layer data transfer unit is configured to directly provide the output of the current layer to the subsequent layer in the form of an executable encoded stream, so that the dual-mode front end of the subsequent layer can directly parse and process the encoded stream; the current layer and the subsequent layer are both inference computation layers in a neural network.

[0009] In some implementations, the dual-mode front end includes: An index-driven scheduler is configured to parse the executable encoded stream, extract index information of active channels, and generate a request including a lookup table address based on the index information in the index-driven mode. A pre-computation scheduler is configured to read dense pulse words in the pre-computation mode, group the dense pulse words into groups according to a preset channel group granularity, and generate a corresponding pattern summation lookup address based on each pulse pattern group, thereby generating the request.

[0010] In some implementations, the normalization request includes fields for instructing the unified lookup table backend operation, the fields including at least a bank enable signal, a lookup table address, a lane valid mask, a programmable shift amount, and a state buffer address.

[0011] In some implementations, the unified lookup table backend includes multiple physical banks, each of which stores a contribution vector table; The unified lookup table backend is configured to read contribution vectors from the corresponding physical banks according to the bank enable signal in the normalization request, and perform lane-based reduction, programmable shift, and accumulation operations on the read contribution vectors.

[0012] In some implementations, the cross-layer data transfer unit is further configured to encode the output pulses of the current layer into a block-row-coordinate encoded stream; the encoded stream includes a row header indicating the intra-block row index and block label, and a coordinate data packet indicating the column label and active channel index.

[0013] In some implementations, the cross-layer data transmission unit further includes: A halo-aware dual-view DMA descriptor generator is configured to simultaneously generate a storage descriptor and a load descriptor for the same encoded stream; the storage descriptor covers the output range generated for the current block, and the load descriptor covers the complete input window of the target block and its adjacent blocks required for subsequent layer computation.

[0014] In some implementations, the dual-mode front end is also configured to switch between the index-driven mode and the pre-computed mode based on an offline-calibrated activity threshold or an activity level statistically analyzed during online runtime.

[0015] In some implementations, the channel group granularity preset in the pre-computation scheduler is configurable, and the channel group granularity includes a group of 2 channels, a group of 4 channels, or a group of 8 channels.

[0016] In some implementations, the unified lookup table backend further includes: A programmable shifter configured to apply a shift amount corresponding to a binary weighted slice to a reduced contribution vector to achieve bit-plane scheduling of a spiking neural network model employing counting encoding.

[0017] Secondly, this application also provides a programmable activity-adaptive dual-mode inference method, applicable to any of the programmable activity-adaptive dual-mode inference accelerators mentioned above, the method comprising: Depending on the current activity level of the data being processed, selectively operate in either index-driven mode or pre-compute mode, and normalize requests generated in the selected mode into requests of a uniform format. Receive the normalization request, and perform a unified lookup table read and accumulation operation based on the normalization request; The output of the current layer is provided directly to the successor layer in the form of an executable encoded stream, so that the successor layer can directly parse and process the encoded stream; both the current layer and the successor layer are inference computation layers in the neural network.

[0018] This application has the following advantages: By flexibly switching between index-driven mode and pre-computation mode through a dual-mode front-end, the accelerator always selects the optimal execution path under the current activity conditions, significantly improving hardware utilization.

[0019] Both modes share the same unified lookup table backend, avoiding the duplication of computational logic. The resource increment of the dual-mode frontend is controlled to less than 6% of the total LUT and trigger resources.

[0020] By using block-row-coordinate encoding streams and halo-aware DMA mechanisms, the output of sparse layers can be directly loaded as executable input by subsequent layers without the need to reconstruct dense tensors, thus achieving continuous and efficient index field execution across layers.

[0021] It supports bit-plane scheduling and efficiently processes counting coding models; the same FPGA image can cover multiple tasks such as image classification and object detection without resynthesis, offering high deployment flexibility. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a programmable activity-adaptive dual-mode inference accelerator according to this application; Figure 2 This is a schematic diagram of the dual-mode front end in a programmable activity-adaptive dual-mode inference accelerator according to this application; Figure 3 This is a schematic diagram of dual-mode front-end lookup table addressing; Figure 4 A schematic diagram comparing the timing of block pipelines under different execution modes; Figure 5 A diagram illustrating the unified backend pipeline for lookup tables; Figure 6 This is a flowchart of a programmable activity-adaptive dual-mode inference method according to this application. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0026] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0027] See Figures 1 to 5 As can be seen, this embodiment provides a programmable activity-adaptive dual-mode inference accelerator, the accelerator comprising: A dual-mode frontend is configured to selectively operate in either index-driven mode or pre-compute mode based on the activity level of the currently processed data, and to normalize requests generated in the selected mode into requests of a uniform format.

[0028] Specifically, in this embodiment, the dual-mode front-end is key to the accelerator's adaptive activity level. It integrates two parallel scheduling paths: an index-driven scheduler and a pre-computation scheduler. In practice, the dual-mode front-end determines the activity level based on the current data being processed. This activity level can be understood as the sparsity of the pulse signal in the current layer or time step.

[0029] For example, when only a very small number of neurons in a layer are firing pulses (i.e., low activity), the dual-mode frontend selects the index-driven mode. In this mode, the index-driven scheduler parses the encoded stream from the previous layer, extracts which channels are active (i.e., firing pulses), and then generates computation requests only for these active channels. Conversely, if the current layer has high activity and almost all channels are firing pulses, the dual-mode frontend switches to pre-computation mode. In this case, the pre-computation scheduler directly reads the dense pulse words, groups them according to a preset channel group granularity (e.g., 4 channels per group), and then generates a pattern summation lookup address for each group to generate computation requests. Regardless of the mode selected, the dual-mode frontend normalizes the generated requests, converting them into a unified format before sending them to the backend.

[0030] The accelerator also includes: A unified lookup table backend is configured to receive the normalization request and perform unified lookup table reading and accumulation operations based on the normalization request.

[0031] Specifically, in this embodiment, the unified lookup table backend is the part that performs the actual calculations. It consists of multiple physical banks (e.g., four), each storing a contribution vector table. When a normalization request is received from the dual-mode frontend, the backend parses the fields in the request, such as the bank enable signal, to determine which banks to read data from. The unified lookup table backend reads the corresponding contribution vectors and performs lane-based reduction (adding contribution vectors from corresponding positions in different banks), programmable shifting (for subsequent bit-plane scheduling), and accumulation operations on these vectors to finally obtain the calculation result. Importantly, the backend is completely indifferent to whether the request comes from the index-driven mode or the pre-computation mode; its processing path for all normalization requests is completely consistent.

[0032] The accelerator also includes: A cross-layer data transfer unit is configured to directly provide the output of the current layer to the subsequent layer in the form of an executable encoded stream, so that the dual-mode front end of the subsequent layer can directly parse and process the encoded stream; the current layer and the subsequent layer are both inference computation layers in a neural network.

[0033] Specifically, in this embodiment, the cross-layer data transfer unit is responsible for efficiently passing the computation results of the current layer to the next layer. It does not revert the computation results to traditional dense tensors, but instead encodes them into a special encoded stream that can be directly executed by the next layer. Specifically, it encodes the output pulses of the current layer into a block-row-coordinate format encoded stream. This encoded stream contains a row header and coordinate data packets. The row header indicates the row index and block label within the block, while the coordinate data packets specify the column label and the specific active channel index. In this way, the next layer's dual-mode front-end (especially its index-driven scheduler) can directly parse this encoded stream and begin processing immediately, without any decompression or format conversion steps.

[0034] Furthermore, in some embodiments, the dual-mode front end includes: An index-driven scheduler is configured to parse the executable encoded stream, extract index information of active channels, and generate a request including a lookup table address based on the index information in the index-driven mode.

[0035] Specifically, in this embodiment, the index-driven scheduler is dedicated to index-driven mode. Its core component is an index decoder. Upon receiving the block-row-coordinate encoded stream from the upper layer, the index decoder parses it. It first reads the row header to obtain the currently processed row index and block label information. Then, it reads the coordinate data packet, extracting the column label and the channel index that is active in that column. These active channel indices are typically 6-bit values ​​that can be directly used as the address of the lookup table. Based on this index information, the index-driven scheduler generates a computation request containing the lookup table address. For example, if a coordinate data packet indicates that channels 3, 13, 32, and 40 are active, the scheduler will generate four requests, each pointing to the contribution vector of these four channels in the lookup table.

[0036] The dual-mode front end also includes: A pre-computation scheduler is configured to read dense pulse words in the pre-computation mode, group the dense pulse words into groups according to a preset channel group granularity, and generate a corresponding pattern summation lookup address based on each pulse pattern group, thereby generating the request.

[0037] Specifically, in this embodiment, the pre-computation scheduler is dedicated to the pre-computation mode. In this mode, it receives uncompressed, dense 64-bit pulse words. It first groups these dense pulse words according to a preset channel group granularity. For example, with 4 channels per group, the 64-bit pulse word is divided into 16 groups. Each group's 4-bit binary number represents the pulse state (0 or 1) of each channel within that group; this 4-bit binary number itself is an address between 0 and 15. The pre-computation scheduler uses this address to look up a pre-programmed pattern summation lookup table. In this lookup table, address m stores the pre-computed sum of the contribution vectors of all active channels within the group. For example, if a group's pulse pattern is 1010 (binary), corresponding to address 10, then the lookup table returns the sum of the contribution vectors of channels 0 and 3 within that group. Based on this, the scheduler generates a request containing the pattern summation lookup address.

[0038] Furthermore, in some embodiments, the normalization request includes fields for instructing the unified lookup table backend operation, the fields including at least a bank enable signal, a lookup table address, a lane valid mask, a programmable shift amount, and a state buffer address.

[0039] Specifically, in this embodiment, the normalization request is a structured data packet that contains at least the following key fields: Bank enable signal: This is a multi-bit signal used to indicate which physical banks(s) in the unified lookup table backend need to be activated. For example, a 4-bit enable signal [b3, b2, b1, b0] with a value of 1010 indicates that bank3 and bank1 need to participate in the current calculation. This allows the frontend to flexibly enable some banks according to the specific needs of the request, thereby saving power.

[0040] Lookup Table Address: This is an address field that specifies from which location in the activated bank the contribution vector is read. In index-driven mode, this address is the index of a single active channel; in pre-computed mode, this address is the entry address of the pattern summation subtable. The backend does not care about the semantics of the address; it is only responsible for reading data based on the address.

[0041] Lane validity mask: This is a mask corresponding to the hardware parallelism (e.g., 64 lanes). Each bit corresponds to one lane and indicates whether the data on that lane is valid. This is very useful when handling padding operations or when the contribution of certain channels does not need to be accumulated, as it can force the contribution of invalid lanes to zero.

[0042] Programmable shift amount: This is a numerical field that controls the number of bits used to arithmetically shift the reduced contribution vector left or right. This mechanism is fundamental to bit-plane scheduling, used to apply the correct scaling to pulse slices with different binary weights.

[0043] State Buffer Address: This is an address field that specifies the exact location in the membrane state buffer where the result of this calculation should be accumulated. This ensures that the incremental contributions from different requests are correctly accumulated to the corresponding neuron states.

[0044] Furthermore, in some embodiments, the unified lookup table backend includes multiple physical banks, each of which stores a contribution vector table; The unified lookup table backend is configured to read contribution vectors from the corresponding physical banks according to the bank enable signal in the normalization request, and perform lane-based reduction, programmable shift, and accumulation operations on the read contribution vectors.

[0045] Specifically, in this embodiment, the unified lookup table backend consists of multiple physical banks. In a typical implementation, these four banks are placed close to the processing unit to shorten critical path latency. Each bank stores a contribution vector table. The contents of this table are not fixed but are dynamically programmed by the compiler each time the weight context is switched, based on the current computation mode (index-driven or pre-computed).

[0046] When the backend receives a normalization request, its processing flow is as follows: Reading the contribution vector: The backend first parses the bank enable signal in the request. Based on this signal, it only reads data from the selected physical banks. Each bank returns a contribution vector with a width equal to the number of hardware lanes (e.g., 64 lanes). Banks that are not enabled are not read from, thus saving power.

[0047] Applying the lane valid mask: The read contribution vector is bitwise ANDed with the lane valid mask in the request. For bits in the mask that are 0, their corresponding contribution values ​​are set to zero. This step is used to handle invalid channels or padding data to ensure the accuracy of the calculation.

[0048] Lane-based reduction: The contribution vectors from different banks, after being masked, are fed into a four-way reduction tree based on lane. This reduction tree sums the contribution values ​​returned by all enabled banks that are located on the same lane, ultimately producing a merged contribution vector.

[0049] Programmable shift: The merged contribution vector then enters a programmable shifter. The shifter performs an arithmetic shift operation on this vector according to the programmable shift amount requested. This is crucial for processing spiking neural networks that use counting codes, as it allows for the correct weights to be applied to different binary bit planes.

[0050] Accumulation: Finally, the shifted vector is accumulated in the membrane state buffer at the location specified by the state buffer address. This process is managed by a read-modify-write lifecycle, controlled by the first / last flags in the request, ensuring the correctness of the accumulation operation.

[0051] Furthermore, in some embodiments, the cross-layer data transmission unit is also configured to encode the output pulse of the current layer into a block-row-coordinate encoded stream; the encoded stream includes a row header for indicating the row index and block label within the block, and a coordinate data packet for indicating the column label and active channel index.

[0052] Specifically, in this embodiment, the hardware encoder in the cross-layer data transmission unit scans the dense pulse words corresponding to each 64-channel slice in the output pulse buffer after the post-processing unit completes pulse generation. It does not store these pulse words in their original dense format, but rather compresses and encodes them into a block-row-coordinate encoded stream. This encoded stream consists of a two-level data structure: Line header: Each line header corresponds to a row in the output feature map. It contains two key pieces of information: Intra-block line index: Indicates the vertical position of the line within the currently processed block.

[0053] Block label: A 2-digit label used to identify the block number to which the row belongs (modulo 4). This label is crucial for subsequent cross-layer halo processing.

[0054] The line header may also contain a count field indicating how many coordinate data packets follow the line.

[0055] Coordinate data packet: Each coordinate data packet corresponds to one or more column positions in a row. It contains the following information: Column label: Indicates the starting position of the column described by this data packet.

[0056] Active Channel Index: This is the core data, listing the index values ​​of the channels that are active (i.e., have emitted pulses) in this column. Typically, a data packet can carry up to eight 6-bit active channel indices.

[0057] End-of-coordinate marker: A flag indicating whether this packet is the last packet in the current row.

[0058] When the number of active channels in a column exceeds 8, the encoder will generate multiple coordinate data packets to describe the same column until the last data packet sets the coordinate end marker high.

[0059] Furthermore, in some embodiments, the cross-layer data transmission unit further includes: A halo-aware dual-view DMA descriptor generator is configured to simultaneously generate a storage descriptor and a load descriptor for the same encoded stream; the storage descriptor covers the output range generated for the current block, and the load descriptor covers the complete input window of the target block and its adjacent blocks required for subsequent layer computation.

[0060] Specifically, in this embodiment, when processing conformal convolution with a stride of 1, the input window required to compute an output block is typically larger than the output block itself, requiring extensions of several rows to the top and bottom. These extended rows are called halo regions. The data in these halo rows is actually stored in the encoded stream of adjacent input blocks. To address this issue, this embodiment introduces a halo-aware dual-view DMA descriptor generator.

[0061] This generator will generate two different DMA descriptors for the same encoded stream simultaneously: Storage Descriptor: This descriptor defines which area of ​​off-chip DDR memory the output calculated by the current layer should be written back to. Its coverage is the output range generated by the current block itself, excluding any halo areas.

[0062] Load Descriptor: This descriptor is prepared for the successor layer. It defines which data the successor layer needs to load from off-chip DDR memory during computation. Its coverage includes not only all rows of the current block, but also the halo rows at the end of the previous block and the halo rows at the beginning of the next block, thus forming a complete halo-equipped input window.

[0063] When the successor layer begins computation, it does not use a storage descriptor; instead, it uses a load descriptor to initiate a DMA transfer. By using the load descriptor, the successor layer directly obtains a continuous sequence of encoded lines containing the necessary halo information. Then, the index-driven frontend of the successor layer uses the block label field in the encoded stream header to identify the positional relationship of each line relative to its target block (e.g., whether it belongs to the previous, current, or next block), and converts the line index between the coordinate spaces of different blocks through simple bit operations.

[0064] Furthermore, in some embodiments, the dual-mode front end is also configured to switch between the index-driven mode and the pre-computed mode based on an offline-calibrated activity threshold or an activity level statistically analyzed during online runtime.

[0065] Specifically, in this embodiment, the mode switching strategy of the dual-mode front-end is not static, but can be flexibly configured according to the actual situation. This embodiment provides two feasible switching methods: Offline-calibrated activity thresholding: This is a static, compile-time determined strategy. Before deploying the neural network model to the accelerator, the compiler performs inference simulations on the network using a validation dataset. During this process, the compiler calculates the burst rate (i.e., activity) for each layer (and even each time step). Based on these statistics, the compiler can set one or more activity thresholds for each layer. For example, a threshold can be set such that when the activity of a layer is below the threshold, the compiler configures that layer in index-driven mode; when the activity is above the threshold, it is configured in pre-computation mode. This configuration information is embedded in the final instruction stream, and the accelerator directly reads the instructions at runtime without needing to perform real-time judgments.

[0066] Activity based on online runtime statistics: This is a dynamic, runtime-determined strategy. In this mode, the dual-mode front-end itself or nearby hardware monitors the activity of the currently processed data in real time. For example, during the loading phase of each block, the number of non-zero bits in the input pulse word can be counted to calculate the current activity. This real-time statistic is then compared to a preset hardware threshold, and based on the comparison result, it is immediately determined whether the current block uses index-driven mode or pre-computed mode. This method can respond more accurately to instantaneous changes in activity, but requires additional hardware counters.

[0067] Furthermore, in some embodiments, the channel group granularity preset in the pre-computation scheduler is configurable, and the channel group granularity includes a group of 2 channels, a group of 4 channels, or a group of 8 channels.

[0068] Specifically, in this embodiment, in pre-computation mode, the pre-computation scheduler divides the input dense burst word (e.g., 64 bits) into several groups, each group containing a certain number of channels; this number is called the "channel group granularity." This embodiment explicitly states that this granularity is not fixed but configurable. Specifically, it can be set to the following common values: A set of 2 channels: 64-bit pulse words are divided into 32 groups. Each 2-bit pulse pattern corresponds to a 4-entry lookup subtable.

[0069] A set of 4 channels: 64-bit pulse words are divided into 16 groups. Each 4-bit pulse pattern corresponds to a 16-entry lookup subtable.

[0070] An 8-channel array: 64-bit pulse words are divided into 8 groups. Each 8-bit pulse pattern corresponds to a lookup sub-table with 256 entries.

[0071] This configurability is achieved by setting a programmable register in the pre-computation scheduler. The compiler or runtime system can select the optimal channel group granularity based on factors such as the LUTRAM capacity on the target FPGA platform and the activity distribution characteristics of the deployed network. For example, if LUTRAM resources are limited, a group of 2 channels can be selected to reduce the sub-table size; if the network activity is extremely high and maximum computational throughput is desired, a group of 8 channels can be selected to further reduce the number of traversal cycles.

[0072] Furthermore, in some embodiments, the unified lookup table backend further includes: A programmable shifter configured to apply a shift amount corresponding to a binary weighted slice to a reduced contribution vector to achieve bit-plane scheduling of a spiking neural network model employing counting encoding.

[0073] Specifically, in this embodiment, in addition to containing multiple physical banks and reduction logic, the unified lookup table backend also integrates a programmable shifter. This shifter is located after the lane-based reduction tree and before the accumulator. Its core function is to apply an arithmetic shift operation to the contribution vector obtained after reduction.

[0074] The key role of shifters lies in processing spiking neural network models that employ counting encoding (such as thermometer encoding). In these models, the output of a neuron is not a single pulse, but an integer value (count). To efficiently process this integer count on binary-weighted hardware, it can be decomposed into multiple binary-weighted slices, each slice being a binary pulse plane. For example, a count ranging from 0 to 15 can be decomposed into four binary planes with weights of 1, 2, 4, and 8.

[0075] When the accelerator processes these bit planes, the computation for all bit planes reuses the same unified lookup table backend, the only difference being the shift amount set in the programmable shifter. For a bit plane with a weight of 1, the shift amount is 0; for a bit plane with a weight of 2, the shift amount is 1; for a bit plane with a weight of 4, the shift amount is 2, and so on. In this way, the contribution of each bit plane is correctly assigned to the order of magnitude matching its binary weight before being accumulated into the membrane state buffer.

[0076] See Figure 6 It can be seen that this embodiment also provides a programmable activity-adaptive dual-mode inference method, the method comprising: S1, based on the activity level of the current data being processed, selectively operates in either index-driven mode or pre-compute mode, and normalizes the requests generated in the selected mode into requests of a uniform format.

[0077] Specifically, in this embodiment, the accelerator's dual-mode front end acquires the activity level of the data to be processed. This activity level can be obtained in advance through offline calibration or calculated online at runtime. Based on this activity level, the dual-mode front end makes a decision: if the activity level is low, it selects the index-driven mode; if the activity level is high, it selects the pre-computation mode.

[0078] In index-driven mode, the frontend parses the executable code stream from the upper layer and extracts the index information of the active channels. Then, it generates a series of requests only for these active channels, each request containing an address pointing to the contribution vector table.

[0079] In pre-computation mode, the front end directly reads the dense pulse words and groups them according to a preset channel group granularity (e.g., 4 channels per group). Then, for each group, it generates a pattern summation lookup address based on the pulse pattern within the group, and generates a request accordingly.

[0080] Regardless of the mode used, the final generated request will be normalized to a uniform format, which includes fields such as bank enable, lookup table address, lane mask, shift amount, and state buffer address.

[0081] The method further includes: S2, receive the normalization request, and perform a unified lookup table read and accumulation operation based on the normalization request.

[0082] Specifically, in this embodiment, the normalization request is sent to the unified lookup table backend. Upon receiving the request, the backend processes it according to the exact same procedure, regardless of whether the request originates from index mode or pre-computation mode. The specific process includes: reading the contribution vector from the corresponding physical bank based on the bank enable signal; applying a lane effective mask to mask invalid data; merging the contributions from multiple banks using a reduction tree based on the lane; shifting the merged contribution vector using a programmable shifter (for bit plane scheduling); and finally, accumulating the result into the membrane state specified by the state buffer address.

[0083] The method further includes: S3, the output of the current layer is directly provided to the successor layer in the form of an executable encoded stream, so that the successor layer can directly parse and process the encoded stream; the current layer and the successor layer are both inference computing layers in the neural network.

[0084] Specifically, in this embodiment, after the current layer's computation is complete, the cross-layer data transfer unit intervenes. Instead of writing the computation result out as a dense tensor, it encodes it into a special encoded stream (such as a block-row-coordinate format) that can be directly executed by subsequent layers. Simultaneously, the halo-aware dual-view DMA mechanism generates a dedicated loading descriptor for this encoded stream, ensuring that subsequent layers can obtain the complete input window of the target block and its adjacent blocks required for computation in one go when loading data. In this way, the dual-mode front-end of the subsequent layer can directly parse and process this encoded stream without any format conversion, thereby initiating the next round of inference loop.

[0085] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the discussion in some embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the contents of this disclosure, thereby enabling those skilled in the art to better utilize the embodiments.

Claims

1. A programmable activity-adaptive dual-mode inference accelerator, characterized in that, The accelerator includes: A dual-mode frontend is configured to selectively operate in index-driven mode or pre-compute mode based on the activity level of the currently processed data, and to normalize requests generated in the selected mode into requests of a uniform format. A unified lookup table backend is configured to receive the normalization request and perform unified lookup table reading and accumulation operations based on the normalization request. A cross-layer data transfer unit is configured to directly provide the output of the current layer to the subsequent layer in the form of an executable encoded stream, so that the dual-mode front end of the subsequent layer can directly parse and process the encoded stream; the current layer and the subsequent layer are both inference computation layers in a neural network.

2. The programmable activity-adaptive dual-mode inference accelerator according to claim 1, characterized in that, The dual-mode front end includes: An index-driven scheduler is configured to parse the executable encoded stream, extract index information of active channels, and generate a request including a lookup table address based on the index information in the index-driven mode. A pre-computation scheduler is configured to read dense pulse words in the pre-computation mode, group the dense pulse words into groups according to a preset channel group granularity, and generate a corresponding pattern summation lookup address based on each pulse pattern group, thereby generating the request.

3. A programmable activity-adaptive dual-mode inference accelerator according to claim 2, characterized in that, The normalization request includes fields for instructing the backend operations of the unified lookup table, the fields including at least a bank enable signal, a lookup table address, a lane valid mask, a programmable shift amount, and a state buffer address.

4. A programmable activity-adaptive dual-mode inference accelerator according to claim 1, characterized in that, The unified lookup table backend includes multiple physical banks, and each physical bank stores a contribution vector table. The unified lookup table backend is configured to read contribution vectors from the corresponding physical banks according to the bank enable signal in the normalization request, and perform lane-based reduction, programmable shift, and accumulation operations on the read contribution vectors.

5. A programmable activity-adaptive dual-mode inference accelerator according to claim 1, characterized in that, The cross-layer data transmission unit is also configured to encode the output pulses of the current layer into a block-row-coordinate encoded stream; the encoded stream includes a row header indicating the row index and block label within the block, and a coordinate data packet indicating the column label and active channel index.

6. A programmable activity-adaptive dual-mode inference accelerator according to claim 5, characterized in that, The cross-layer data transmission unit further includes: A halo-aware dual-view DMA descriptor generator is configured to simultaneously generate a storage descriptor and a load descriptor for the same encoded stream; the storage descriptor covers the output range generated for the current block, and the load descriptor covers the complete input window of the target block and its adjacent blocks required for subsequent layer computation.

7. A programmable activity-adaptive dual-mode inference accelerator according to claim 1, characterized in that, The dual-mode front end is also configured to switch between the index-driven mode and the pre-computed mode based on an offline-calibrated activity threshold or an activity level statistically analyzed during online runtime.

8. A programmable activity-adaptive dual-mode inference accelerator according to claim 2, characterized in that, The channel group granularity preset in the pre-computation scheduler is configurable, and the channel group granularity includes a group of 2 channels, a group of 4 channels, or a group of 8 channels.

9. A programmable activity-adaptive dual-mode inference accelerator according to claim 1, characterized in that, The unified lookup table backend also includes: A programmable shifter configured to apply a shift amount corresponding to a binary weighted slice to a reduced contribution vector to achieve bit-plane scheduling of a spiking neural network model employing counting encoding.

10. A programmable activity-adaptive bimodal inference method, the method being applicable to any one of the programmable activity-adaptive bimodal inference accelerators described in claims 1 to 9, characterized in that, The method includes: Depending on the current activity level of the data being processed, selectively operate in either index-driven mode or pre-compute mode, and normalize requests generated in the selected mode into requests of a uniform format. Receive the normalization request, and perform a unified lookup table read and accumulation operation based on the normalization request; The output of the current layer is provided directly to the successor layer in the form of an executable encoded stream, so that the successor layer can directly parse and process the encoded stream; both the current layer and the successor layer are inference computation layers in the neural network.

Images