Bit slice accelerator for efficient convolutional neural network reasoning

Abstract

The invention provides a bit slice accelerator for efficient convolutional neural network reasoning, and belongs to the field of neural network reasoning hardware. The bit slice accelerator comprises a lookup table (LUT)-based multiplier which comprises a bit operation analysis unit (BMAU) and a bit level multiplier unit; and the BMAU manages bit-level multiplication operation by utilizing an LUT mechanism and performs bit field division on input data, and the bit-level multiplier unit processes groups of continuous multiplier bits after bit field division so as to realize bit-level data processing. According to the bit slice accelerator provided by the invention, through the multiplier based on the lookup table LUT, the BMAU of the integrated bit level and the bit level multiplier unit are included, and operation of different bit widths is supported; data sparsity can be effectively utilized in DNN reasoning, and reasoning efficiency and energy efficiency in a limited environment are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network inference hardware, and in particular to a bit-slicing accelerator for efficient convolutional neural network inference. Background Art

[0002] Deep Neural Networks (DNNs) have demonstrated excellent computational efficiency in various current artificial intelligence applications and can significantly reduce computational requirements. Currently, the most advanced DNNs still need to perform billions of operations and require a large amount of storage space to save activation values ​​and weights. To address these challenges, existing technologies have gradually turned their attention to customized bit-slicing accelerators to improve computing performance and energy efficiency. Bit-slicing processing achieves more efficient computing by classifying multiplication operations into four categories: invalid operations, redundant operations, basic operations, and repeated operations. These innovations collectively aim to optimize the deployment of DNNs so that they can be more widely used in a variety of constrained environments. DNNs use variable precision for different convolutional layers to effectively reduce memory and computing costs, so DNN accelerators that can adapt to this precision change are needed. Bit-slicing accelerators improve processing efficiency by exploiting data sparsity at the bit level while maintaining computational accuracy. On the one hand, when the input is detected to be zero, energy consumption is reduced by gating the computing unit, but this method cannot effectively improve latency or throughput; on the other hand, some architectures improve latency and throughput by exploiting the sparsity in the compressed input, but usually only support the sparsity of a single operand and cannot handle the sparsity of two operands at the same time. Previous DNN inference accelerators have shown insufficient efficiency in dealing with variable precision scenarios because they generally use fixed-precision computing units. The variable precision requirements between different levels require accelerators with sufficient flexibility to reduce memory and computing overhead. Therefore, solving these challenges is crucial to improving the efficiency and adaptability of DNN inference accelerators under heterogeneous precision conditions.

[0003] To address this issue, a potential solution is to more effectively combine value-level sparsity and bit-level sparsity to reduce the number of non-contributing bits in deep learning models. However, bit-level sparsity varies between computational units, resulting in a large amount of ineffective computation during parallel processing. Therefore, precision scalable accelerators developed through algorithm-hardware co-design represent a promising approach to address this irregularity. Quantization of deep learning models with more than 6.7 billion parameters faces the challenge of maintaining accuracy during inference. This makes it particularly important to ensure computational efficiency while maintaining model accuracy during inference. The algorithm design should manage the irregular distribution of zero bits in the operands, while the hardware simplifies the control logic and develops efficient sparse indexing mechanisms. Algorithm-hardware co-design ensures wider applicability and minimizes computational overhead. It can be seen that the existing technology has the following problems in DNN reasoning: 1) Resource and energy consumption limitations. On hardware platforms with limited resources and energy consumption, it is difficult to effectively utilize the data sparsity in DNN, which limits the improvement of reasoning efficiency and energy efficiency; 2) Lack of flexibility. Traditional hardware accelerators mostly use fixed-precision computing units, which lack the ability to adapt to different bit widths and sparse operations, and cannot flexibly configure computing resources to meet the needs of different network layers. Summary of the invention

[0004] The purpose of an embodiment of the present invention is to provide a bit-slice accelerator for efficient convolutional neural network reasoning, which can solve the above-mentioned problems existing in DNN reasoning.

[0005] In order to achieve the above-mentioned object, an embodiment of the present invention provides a bit-slicing accelerator for efficient convolutional neural network inference, wherein the bit-slicing accelerator includes a multiplier based on a lookup table LUT, and the multiplier based on the lookup table LUT includes a bit operation analysis unit BMAU and a bit-level multiplier unit. The bit operation analysis unit BMAU manages the bit-level multiplication operation by using the LUT mechanism, and performs bit segment division on the input data, and the bit-level multiplier unit processes the grouping of continuous multiplier bits after the bit segment division to realize bit-level data processing.

[0006] Optionally, the multiplier based on the lookup table LUT configures the operation data as follows: counting the non-zero bits in the most significant bit MSB and the least significant bit LSB in the input data, and storing the counting result in a trigger; interfacing with the LUT address generator, generating partial products, and calculating through the LUT, and storing the calculation results in the trigger; determining the bit path based on the non-zero bits in the MSB and the LSB.

[0007] Optionally, the bit path is determined based on the non-zero bits in the most significant bit MSB and the least significant bit LSB, including: when the MSB and the LSB are both zero, skipping the calculation process and setting the output to zero; when the LSB is non-zero, performing a specified bit shift operation to ensure that the input data is aligned; when the MSB is non-zero, processing the input data to ensure effective participation in the multiplication operation; when the MSB and the LSB are both non-zero, calculating and summing multiple partial products to generate a final product.

[0008] Optionally, the multiplier bit operation analysis unit BMAU based on the lookup table LUT is also used to optimize the process of computing overhead when processing sparse and irregular matrices. Through parallel computing and pipeline operations, the PE bit slice calculation unit processes non-zero elements and merges partial products through reduction operations. The addition operation is started in each clock cycle and allocated to the digital signal processing unit DSP to improve the computing speed and resource utilization.

[0009] Optionally, the PE bit slice calculation unit processes non-zero elements through parallel calculation and pipeline operation, and merges partial products through reduction operation, including: determining the size of the convolution kernel, identifying non-zero elements, and grouping them into multiple data blocks according to the positions of the non-zero elements; marking the multiple data blocks, and loading each data block to their respective designated positions; after integrating the multiple data blocks into a complete convolution kernel, preparing for calculation processing. Each PE bit slice calculation unit performs multiplication calculation on the loaded data according to a predetermined operation, and in this process, the generated partial products are subsequently merged through reduction operation.

[0010] Optionally, the addition operation is started in each clock cycle and allocated to the digital signal processing unit DSP, including: a core addition unit adds two operands and sets a completion flag; in a parallel manner, the addition operation can be started in each clock cycle, and the addition operation in the parallel stage is allocated to the DSP.

[0011] Optionally, the PE bit slice computation unit is further configured to: utilize the PE bit slice computation unit column template function to process a column addition operation by initializing input, managing intermediate results, and ensuring full pipelining; and utilize the top-level function to coordinate the addition process, including managing input data flow, calling the column addition pipeline, and storing the final result.

[0012] Optionally, the PE bit slice computing unit is further configured to: adopt data multiplexing and streamlined configuration to ensure that the matrix data is adapted to the PE bit slice computing unit array, and realize data flow and parallel processing through two computing stages.

[0013] Optionally, the data multiplexing and streamlined configuration is adopted to ensure that the matrix data is adapted to the PE bit slice calculation unit array, and the data flow and parallel processing are realized through two calculation stages, including: dividing the matrix data into preset slice units to utilize the locality of the data and optimize the memory bandwidth; adapting the matrix data to the PE bit slice processing unit array so that the data is transmitted through the PE bit slice calculation unit in two calculation stages. Each PE bit slice calculation unit calculates a partial sum and passes the data to the next PE bit slice calculation unit.

[0014] Optionally, the PE bit slice calculation unit is further configured to: use a loop operation to handle the iterative process of traversing the slice unit within multiple cycles, excluding the stride factor, wherein a new operation is started in each cycle; control logic to ensure that the calculation results are stored in the output matrix; use the analysis of the dimension of the matrix data to adaptively divide the irregular matrix; the PE bit slice calculation unit array selects an appropriate slice size to support parallel computing.

[0015] Through the above technical solution, an embodiment of the present invention provides a bit-slicing accelerator for efficient convolutional neural network reasoning, and the bit-slicing accelerator includes a multiplier based on a lookup table LUT. The multiplier of the lookup table LUT includes an integrated bit-level bit operation analysis unit BMAU and a bit-level bit-level multiplier unit to support operations of different bit widths. The embodiment of the present invention can effectively utilize data sparsity in deep neural network reasoning, and significantly improve the reasoning efficiency and energy efficiency in resource and energy-constrained environments. At the same time, adaptive sparse operations and flexible bit width support enable the embodiment of the present invention to flexibly adapt to the computing needs of different row layers, greatly improving the adaptability and overall performance of the hardware accelerator.

[0016] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings: Figure 1 is a schematic diagram of the structure of a bit-slicing accelerator for efficient convolutional neural network inference provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a bit-slicing accelerator for efficient convolutional neural network inference; Figure 3 is a schematic diagram of an example bit-slicing computation unit; Figure 4is a schematic diagram of an example dynamic computation array. DETAILED DESCRIPTION

[0018] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0019] As mentioned above, data sparsity is introduced in deep neural networks (DNNs), which significantly improves the efficiency of resource and energy-constrained hardware platforms. However, in order to fully utilize data sparsity and further improve energy efficiency, these DNN models require specialized hardware accelerator support. An embodiment of the invention provides a bit-slicing accelerator for efficient convolutional neural network inference, which can support adaptive sparse operations between DNN layers.

[0020] Figure 1 is a schematic diagram of the structure of a bit-slicing accelerator for efficient convolutional neural network inference provided by an embodiment of the present invention. Figure 1 The bit slice accelerator may include a multiplier based on a lookup table LUT, and the multiplier based on the lookup table LUT may include a bit operation analysis unit BMAU and a bit-level multiplier unit. The bit operation analysis unit BMAU manages the bit-level multiplication operation by using the LUT mechanism, and performs bit segment division on the input data, and the bit-level multiplier unit processes the grouping of continuous multiplier bits after the bit segment division to realize bit-level data processing.

[0021] An embodiment of the present invention provides a bit-slicing accelerator for efficient convolutional neural network reasoning, the bit-slicing accelerator comprising a multiplier based on a lookup table LUT. The multiplier of the lookup table LUT comprises an integrated bit-level bit manipulation analysis unit (BMAU) and a bit-level multiplier unit to support operations of different bit widths, providing technical support for improving the energy efficiency and adaptability of DNN reasoning in a resource-constrained environment.

[0022] The multiplier based on the lookup table LUT in the embodiment of the present invention can be configured to operate data as follows: count the non-zero bits in the most significant bit MSB and the least significant bit LSB in the input data, and store the counting result in a trigger; interface with the LUT address generator to generate partial products, and calculate through the LUT, and store the calculation results in the trigger; determine the bit path according to the non-zero bits in the MSB and LSB.

[0023] Please refer to Figure 2For example, the multiplier based on the lookup table LUT provided in the embodiment of the present invention can support 4-bit, 8-bit, 16-bit and 32-bit logical operations, and by combining the BMAU and the bit-level multiplier unit, the computational efficiency of the multiplier can be improved. By way of example, the BMAU uses the LUT mechanism to manage the bit-level multiplication operation, divides the data into 2N bit segments, and adopts a simplified addressing scheme with an address range of 0 to 2N-1. The bit-level multiplier unit can process the grouping of 2N consecutive multiplier bits (b[2N-1:0]), thereby achieving efficient data processing. Taking the 8-bit multiplier (b[7:0]) as an example, the BMAU counts the non-zero bits in the most significant bit (MSB) and the least significant bit (LSB). These count values ​​are then stored in a flip-flop (FF) with an index of [2N / 2-1:0]. Next, a partial product is generated by interfacing the LUT address generator with the BMAU. The shared bit-slice computation unit can be integrated in 4-bit, 8-bit, 16-bit and 32-bit LUT-based multipliers.

[0024] In the implementation, the BMAU can count the non-zero bits in the most significant bit (MSB) and the least significant bit (LSB) in the input data and store the count results in the flip-flops. The BMAU interfaces with the LUT address generator to generate partial products and calculate them through the LUT. According to the non-zero bits in the MSB and LSB, the BMAU optimizes the calculation path to improve efficiency. For example, if both the MSB and LSB are zero, the calculation will be skipped and the output will be zero; if the LSB is non-zero, a shift operation will be performed to ensure data alignment; if the MSB is non-zero, the data is ensured to participate in the calculation effectively.

[0025] In a preferred embodiment of the present invention, determining the bit path (i.e., determining the operation path of the non-zero bits) based on the non-zero bits in the most significant bit MSB and the least significant bit LSB may include: when both the MSB and the LSB are zero, skipping the calculation process and setting the output to zero; when the LSB is non-zero, performing a specified shift operation to ensure that the input data is aligned; when the MSB is non-zero, processing the input data to ensure effective participation in the multiplication operation; when both the MSB and the LSB are non-zero, calculating and summing multiple partial products to generate a final product.

[0026] Continuing with the above example, BMAU analyzes the MSB and LSB of the input data to perform the appropriate multiplication function: when both the MSB and LSB are zero, the calculation process can be skipped and the output is directly set to zero, thereby improving the calculation efficiency; when the LSB is non-zero, the specified bit shift operation is performed to ensure the accurate alignment of the input data, which ensures the accuracy and consistency during the data conversion process; when the MSB is non-zero, the input is processed to ensure that it effectively participates in the multiplication operation, while avoiding unnecessary calculation overhead; when both the MSB and LSB are non-zero, the multiplier based on the lookup table LUT is calculated, and multiple partial products are summed to generate the final product to ensure the accuracy of the multiplication.

[0027] The preferred bit operation analysis unit BMAU of the embodiment of the present invention can also be used to optimize the process of computing overhead when processing sparse and irregular matrices. Among them, through parallel computing and pipeline operation, the bit slice calculation unit processes non-zero elements and merges partial products through reduction operations. The addition operation is started in each clock cycle and allocated to the digital signal processing unit DSP to improve the computing speed and resource utilization.

[0028] The LUT-based multiplier provided by the embodiment of the present invention includes an integrated bit-level BMAU and a bit-level multiplier unit to support operations of different bit widths. At the same time, a shared bit slice calculation unit (PE) optimizes the critical path of the calculation to ensure consistency under different bit width configurations.

[0029] Please refer to Figure 3 For example, when processing sparse matrices, the embodiment of the present invention uses a bit slice computing unit to optimize the computing overhead. Through parallel computing and pipeline operations, the bit slice computing unit can efficiently process non-zero elements and merge partial products through reduction operations. Among them, the addition operation is started at each clock cycle and allocated to the DSP to improve the computing speed and resource utilization. In addition, stream optimization and inlining techniques help reduce latency and improve performance, which is particularly suitable for real-time digital signal processing applications.

[0030] In a preferred embodiment of the present invention, the bit slice calculation unit processes non-zero elements through parallel calculation and pipeline operation, and merges partial products through reduction operation, which may include: determining the size of the convolution kernel, identifying non-zero elements, and grouping them into multiple data blocks according to the positions of the non-zero elements; marking multiple data blocks, and loading each data block into their respective designated positions; after integrating the multiple data blocks into a complete convolution kernel, preparing for calculation processing. Each bit slice calculation unit performs multiplication calculation on the loaded data according to a predetermined operation, and in this process, the generated partial products are subsequently merged through reduction operation.

[0031] Please refer to Figure 3, continuing with the above example, the kernel size is determined and non-zero (NNZ) elements are identified, which are grouped into multiple data blocks according to their positions. The kernel data blocks, for example, labeled K1, K2, and K3, are loaded into their respective designated locations, ensuring that each bit slice calculation unit receives the data required for subsequent operations. After integrating the data blocks into a complete convolution kernel, the bit slice calculation unit is ready for efficient computational processing. Each bit slice calculation unit performs multiplication calculations on the loaded data according to the predetermined operation. In this process, the generated partial products are then merged through reduction operations. The bit slice calculation unit sums the intermediate results and finally generates the output result in the bit slice calculation stage.

[0032] In a preferred embodiment of the present invention, the addition operation is started in each clock cycle and allocated to the digital signal processing unit DSP, which may include: the core addition unit adds two operands and sets a completion flag; in a parallel manner, the addition operation can be started in each clock cycle, and the addition operation in the parallel stage is allocated to the DSP.

[0033] Please refer to Figure 3 ,Continuing the above example, the core addition unit adds two operands and sets the completion flag to ensure pipeline parallelism and efficient use of hardware resources. Through parallelism, the addition operation can be started in every clock cycle, thus achieving high throughput. The addition operation in the parallel stage is assigned to the DSP to improve the calculation speed and resource utilization.

[0034] In a preferred embodiment of the present invention, the bit slice computation unit may further be configured to: utilize a column template function to process a column addition operation by initializing input, managing intermediate results, and ensuring full pipelining; and utilize a top-level function to coordinate the addition process, which may include, for example, managing input data flow, calling a column addition pipeline, and storing the final result.

[0035] Please refer to Figure 3 ,Continuing with the above example, the bit slice compute unit column template ,function handles a column addition operation by initializing the input, managing the intermediate results, and ensuring ,full pipeline operation, thus maximizing the throughput.,Based on this, the top-level function coordinates the entire addition ,process, which may include managing the input data flow, invoking the column addition ,pipeline, and storing the final result.

[0036] Further preferably, the bit slice calculation unit may be configured to: adopt data multiplexing and streamlined configuration to ensure that the matrix data fits into the bit slice calculation unit array, and realize data flow and parallel processing through two calculation stages.

[0037] Please refer to Figure 4, to illustrate by way of example, in order to further improve computing efficiency, an embodiment of the present invention adopts data reuse and streamlined configuration to ensure that matrix data can be efficiently adapted to the bit slice computing unit array (i.e., PE array), and realizes data flow and parallel processing through two computing stages (e.g., cycle 1 and cycle 2). This ensures optimal data reuse while improving memory bandwidth utilization and computing efficiency to meet high throughput and large-scale data processing requirements. In order to further improve performance, an embodiment of the present invention adopts a stream optimization method and an inline method to realize data reuse to reduce latency. For example, a stream optimization instruction can indicate a first-in-first-out queue (FIFO) to promote efficient data flow. These optimizations significantly improve performance, making the implementation particularly suitable for high-throughput, real-time digital signal processing applications.

[0038] In a preferred embodiment of the present invention, the data multiplexing and streamlined configuration are adopted to ensure that the matrix data fits into the bit slice calculation unit array, and realizes data flow and parallel processing through two calculation stages, which may include: dividing the matrix data into preset slice units to utilize the locality of data and optimize memory bandwidth; fitting the matrix data into the bit slice calculation unit array, so that the data is transmitted through the bit slice calculation unit in two calculation stages. Each bit slice calculation unit calculates a partial sum and passes the data to the next bit slice calculation unit.

[0039] Please refer to Figure 4 , following the above example, an embodiment of the present invention utilizes data multiplexing to implement a sparse matrix design for pipeline parallelism. The data matrix can be divided into preset slice (T) units to exploit data locality and optimize memory bandwidth. This approach can efficiently fit matrix blocks into the bit slice compute unit array. In the streamline array configuration, data is transmitted through the bit slice compute unit in two stages (e.g., cycle 1 and cycle 2), representing different stages of calculation; each bit slice compute unit calculates a partial sum and passes the data to the next unit, thereby achieving efficient data flow and parallelism.

[0040] In a preferred embodiment of the present invention, the bit slice calculation unit can also be configured to: use a loop operation to handle the iterative process of traversing the slice unit in multiple cycles, eliminating the stride factor, wherein a new operation is started in each cycle; control logic to ensure that the calculation results are stored in the output matrix; adaptively divide the irregular matrix by analyzing the dimension of the matrix data; the bit slice calculation unit array selects an appropriate slice size to support parallel computing.

[0041] Please refer to Figure 4, continuing with the above example, the embodiment of the present invention manages the iterative process of traversing the slice unit in multiple cycles by using loop operations, eliminating the stride factor. For example, the first loop retrieves elements A-O, while the subsequent loops access subsets such as D, J, E, G, H, O, and F. This strategy ensures optimal data reuse and significantly improves computational efficiency and performance when processing large-scale data sets. New operations can be initiated at each clock cycle, thereby improving throughput. In addition, the control logic of the embodiment of the present invention ensures that the calculation results are stored in the output matrix at the appropriate time. The adaptive partitioning of irregular matrices in the embodiment of the present invention involves analyzing the matrix dimensions to determine the optimal structure and processing requirements. Based on these dimensions, the PE array (e.g., PE0,0, PE1,0, ..., PEN,0) receives input from a dedicated memory buffer (memory buffer, MK) and interacts through interconnected data paths. Data flows vertically into the bit slice computing unit, allowing efficient data allocation, and communication between the bit slice computing units facilitates accumulation operations. The bit slice computing unit array selects an appropriate slice size to support parallel computing, thereby optimizing the entire matrix processing process.

[0042] In an embodiment of the present invention, the multiplication (MUL) operation is performed by feeding the input variables (I0, I1, IG, IN) into the corresponding bit slice calculation units, and sharing the intermediate results through communication between the bit slice calculation units. The clock signal regulates the different stages of the operation to ensure that the data processing of all bit slice calculation units is carried out synchronously. The addition (ADD) stage is represented by NNZ[Num], which indicates the accumulation of non-zero data elements across all bit slice calculation units. The data flow between the bit slice calculation units forms a chain sequence, in which each bit slice calculation unit contributes its part of the result and finally summarizes the intermediate results. Once the calculation is completed, the output result will be written back to the memory to ensure the correct storage of the processed data.

[0043] Accordingly, an embodiment of the present invention provides a bit-slicing accelerator for efficient convolutional neural network reasoning, and the bit-slicing accelerator includes a multiplier based on a lookup table LUT. The multiplier of the lookup table LUT includes an integrated bit-level bit operation analysis unit BMAU and a bit-level bit-level multiplier unit to support operations of different bit widths. The embodiment of the present invention can effectively utilize data sparsity in deep neural network reasoning, and significantly improve the reasoning efficiency and energy efficiency in resource and energy-constrained environments. At the same time, adaptive sparse operations and flexible bit width support enable the embodiment of the present invention to flexibly adapt to the computing needs of different row layers, greatly improving the adaptability and overall performance of the hardware accelerator.

[0044] Furthermore, for the bit slice computing unit, the embodiment of the present invention utilizes core partitioning, effective bit slice computing, and a fully pipelined addition mechanism to optimize computing efficiency. By combining the digital signal processing unit DSP, significant performance improvement is achieved when processing sparse and irregular matrices. For dynamic computing arrays, the embodiment of the present invention utilizes preset bit slice computing units and a two-stage systolic array data flow to maximize data reuse. In addition, communication and adaptive partitioning between bit slice computing units further improve the efficiency of DNN reasoning computing.

[0045] Accordingly, the embodiments of the present invention can claim the following significant advantages: 1) The efficiency of sparsity utilization is significantly improved. Existing technologies often fail to fully utilize the data sparsity in DNNs, resulting in a waste of computing and storage resources. The embodiments of the present invention use an adaptive sparse lookup table and a bit slice computing unit to refine data sparsity at the bit level, greatly improving the utilization efficiency of sparse data, thereby significantly reducing computing energy consumption.

[0046] 2) Flexible bit width support. Most traditional hardware accelerators use fixed-precision bit slice computing units, which lack flexible support for different bit widths. The embodiment of the present invention integrates a multiplier based on a lookup table (LUT) and a shared bit slice computing unit, which can support operations with different bit widths and flexibly adapt to the requirements of different precisions at each layer in the neural network, thereby achieving the best balance between precision and energy efficiency at different levels.

[0047] 3) Comprehensively improved computing performance. By combining the bit-slicing computing unit with the full-pipeline addition mechanism and the digital signal processing unit, the embodiment of the present invention achieves significant performance improvement when processing sparse and irregular matrices. It not only improves the data processing speed, but also reduces latency and improves the overall system throughput.

[0048] 4) Dynamically configured computing array. The dynamic computing array design of the embodiment of the present invention can adaptively divide the bit slice computing units according to the actual computing needs, and use the data flow of the two-stage systolic array to maximize data reuse. Different from the existing fixed-structure computing array, the dynamic configuration of the embodiment of the present invention can better adapt to the computing density requirements of different network layers, thereby improving the reasoning efficiency, especially in scenarios with limited resources and energy consumption.

[0049] 5) Static power consumption is significantly reduced. Most of the bit slice calculation units in the prior art are always kept in an activated state, resulting in high static power consumption. The embodiments of the present invention reduce the power consumption of unnecessary bit slice calculation units by performing clock enable control based on the bit slice calculation unit, thereby significantly reducing the static power consumption of the overall system.

[0050] In summary, the present invention has significant advantages in sparsity utilization, bit width flexibility, computing performance, dynamic adaptability and energy efficiency, and is particularly suitable for application in DNN reasoning tasks with limited resources and energy consumption.

[0051] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0052] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0053] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0054] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0055] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0056] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0057] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0058] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0059] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A bit-slicing accelerator for efficient convolutional neural network inference, characterized in that: The bit-slicing accelerator includes a multiplier based on a lookup table LUT, and the multiplier based on the lookup table LUT includes a bit operation analysis unit BMAU and a bit-level multiplier unit, wherein the bit operation analysis unit BMAU uses the LUT mechanism to manage the bit-level multiplication operation and divide the input data into bit segments, and the bit-level multiplier unit processes the grouping of continuous multiplier bits after the bit segment division to realize bit-level data processing.

2. The bit-slicing accelerator according to claim 1, characterized in that: The multiplier based on the lookup table LUT configures the operation data as follows: Counting non-zero bits in the most significant bit MSB and the least significant bit LSB of the input data, and storing the counting result in a flip-flop; Interface with the LUT address generator, generate partial products, and calculate through the LUT, and store the calculation results in the flip-flop; Based on the non-zero bits in the MSB and the LSB, a bit path is determined.

3. The bit slice accelerator according to claim 2, characterized in that: The step of determining a bit path according to non-zero bits in the most significant bit MSB and the least significant bit LSB comprises: When both the MSB and the LSB are zero, the calculation process is skipped and the output is set to zero; When the LSB is non-zero, performing a specified shift operation to ensure alignment of the input data; When the MSB is non-zero, processing the input data to ensure effective participation in a multiplication operation; When both the MSB and the LSB are non-zero, multiple partial products are calculated and summed to generate a final product.

4. The bit slice accelerator according to claim 2, characterized in that: The bit operation analysis unit BMAU is also used to optimize the process of computing overhead when processing sparse and irregular matrices. Through parallel computing and pipeline operations, the bit slice calculation unit processes non-zero elements and merges partial products through reduction operations. The addition operation is started in each clock cycle and allocated to the digital signal processing unit DSP to improve the computing speed and resource utilization.

5. The bit slice accelerator according to claim 4, characterized in that: The bit slice calculation unit processes non-zero elements through parallel calculation and pipeline operation, and merges partial products through reduction operation, including: Determine the convolution kernel size, identify the non-zero elements, and group them into multiple data blocks according to their positions; Marking the plurality of data blocks and loading each data block into a respective designated location; After the multiple data blocks are integrated into a complete convolution kernel, calculation processing is prepared, wherein each bit slice calculation unit performs multiplication calculation on the loaded data according to a predetermined operation, during which the partial products generated are subsequently merged through a reduction operation.

6. The bit slice accelerator according to claim 4, characterized in that: The addition operation is started in each clock cycle and is assigned to the digital signal processing unit DSP, including: The core addition unit adds the two operands and sets the completion flag; In a parallel manner, the addition operation can be started in each clock cycle, and the addition operation in the parallel stage is distributed to the DSP.

7. The bit slice accelerator according to claim 4, characterized in that: The bit slice calculation unit is further configured to: Use column template functions to process a column addition operation by initializing inputs, managing intermediate results, and ensuring full pipelining; And using the top-level function, coordinate the addition process, including managing the input data flow, calling the column addition pipeline, and storing the final result.

8. The bit slice accelerator according to claim 4, characterized in that: The bit slice calculation unit is further configured to: Data multiplexing and streamlined configuration are adopted to ensure that matrix data fits into the bit slice computing unit array, and data flow and parallel processing are achieved through two computing stages.

9. The bit slice accelerator according to claim 8, characterized in that: The data reuse and streamlined configuration are adopted to ensure that the matrix data fits into the bit slice computing unit array, and data flow and parallel processing are realized through two computing stages, including: Dividing the matrix data into preset slice units to utilize data locality and optimize memory bandwidth; The matrix data is adapted into the array of bit slice processing units so that the data is transmitted through the bit slice computation units in two computation stages, wherein each bit slice computation unit computes a partial sum and passes the data to the next bit slice computation unit.

10. The bit slice accelerator according to claim 9, characterized in that: The bit slice calculation unit is further configured to: Using a loop operation to handle the iterative process of traversing the slice unit in multiple cycles, excluding the stride factor, wherein in each cycle, a new operation is started; Control logic to ensure that the calculation results are stored in the output matrix; By analyzing the dimension of the matrix data, the irregular matrix is ​​adaptively divided; The bit slice calculation unit array selects an appropriate slice size to support parallel calculation.

Images