Neural network calculation acceleration method and system

Abstract

The invention provides a neural network calculation acceleration method and system, and the method comprises the steps: extracting an amplitude bit of input vector data and an amplitude bit of weight vector data, and generating a zero tag through logic operation; the zero label represents whether the product of the input vector data and the weight vector data is zero or not; dynamically aggregating the non-zero products according to the zero label to form an aggregated product; and dynamically adjusting the calculation scale of the adder tree according to the number of non-zero products in the aggregated products, and only performing accumulation operation on the non-zero products. According to the method, invalid calculation can be effectively reduced, the calculation efficiency and the utilization efficiency of hardware resources are improved, and the method is suitable for accelerated calculation of various neural network models.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for accelerating neural network computation. Background Technology

[0002] With the rapid development of artificial intelligence technology, neural networks play an important role in fields such as image classification, object detection, and natural language processing. In neural network computation, multiplication-accumulation operations occupy a core position. However, existing computation methods typically do not consider zero values ​​in the input and weights, incorporating all data into the computation process. This results in a large number of invalid computations with products of zero. These invalid computations not only consume significant resources but also fail to have any real impact on the output feature map, severely affecting computational efficiency and the utilization efficiency of hardware resources. Summary of the Invention

[0003] This invention provides a method and system for accelerating neural network computation, addressing the shortcomings of existing neural network computation methods that typically fail to consider zero values ​​in the input and weight data, leading to a large amount of invalid computation and severely impacting computational efficiency and hardware resource utilization. This invention effectively reduces invalid computation, improves computational efficiency and hardware resource utilization, and is applicable to accelerating the computation of various neural network models.

[0004] This invention provides a method for accelerating neural network computation, comprising: extracting the amplitude bits of input vector data and the amplitude bits of weight vector data, and generating zero labels through logical operations; the zero labels are labels that characterize whether the product of input vector data and weight vector data is zero; dynamically aggregating non-zero products according to the zero labels to form an aggregated product; dynamically adjusting the computation scale of the addition tree according to the number of non-zero products in the aggregated product, and performing accumulation operations only on non-zero products.

[0005] According to a neural network computation acceleration method provided by the present invention, the step of extracting the amplitude bits of input vector data and weight vector data, and generating a zero label through logical operations, includes: using original code to represent input vector data and weight vector data to extract the amplitude bits of the input vector data and the amplitude bits of the weight vector data; the original code includes a sign bit and an amplitude bit; inputting the amplitude bits of the input vector data into a first OR gate to obtain the input label output by the first OR gate; inputting the amplitude bits of the weight vector data into a second OR gate to obtain the weight label output by the second OR gate; and inputting the input label and the weight label into an AND gate to obtain the zero label output by the AND gate.

[0006] According to a neural network computation acceleration method provided by the present invention, the step of dynamically aggregating non-zero products to form an aggregated product based on the zero label includes: using a hierarchical aggregation method, dividing every 16 products into multiple levels for processing; in each level, non-zero products are aggregated into a first region and zero products are aggregated into a second region based on the zero label to obtain an aggregated product; the first region and the second region have different directions.

[0007] According to a neural network computation acceleration method provided by the present invention, the step of dynamically adjusting the computation scale of the addition tree based on the number of non-zero products in the aggregated product includes: selectively enabling or disabling circuits at each level of the addition tree based on the number of non-zero products in the aggregated product to adapt to the sparsity of the input vector data.

[0008] According to the neural network computation acceleration method provided by the present invention, the method further includes: performing an XOR operation on the sign bit of the input vector data and the sign bit of the weight vector data to determine the sign of the product; and allocating the product to different computing circuits for processing according to the sign of the product.

[0009] According to a neural network computation acceleration method provided by the present invention, the step of allocating the product to different computing circuits for processing according to the sign of the product includes: when the sign of the product is 1, allocating the product to a negative value computing circuit for processing; and when the sign of the product is 0, allocating the product to a positive value computing circuit for processing.

[0010] This invention also provides a neural network computation acceleration system, comprising: a generation module for extracting the amplitude bits of input vector data and weight vector data, and generating zero labels through logical operations; the zero labels are labels characterizing whether the product of the input vector data and the weight vector data is zero; a dynamic aggregation module for dynamically aggregating non-zero products according to the zero labels to form an aggregated product; and a scale adjustment module for dynamically adjusting the computation scale of the addition tree according to the number of non-zero products in the aggregated product, performing accumulation operations only on non-zero products.

[0011] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement any of the neural network computation acceleration methods described above.

[0012] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the neural network computation acceleration method described above.

[0013] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the neural network computation acceleration methods described above.

[0014] The present invention provides a method and system for accelerating neural network computation, which can effectively reduce invalid computation, improve computational efficiency and hardware resource utilization efficiency, and is applicable to the accelerated computation of various neural network models. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating a neural network computation acceleration method provided by the present invention.

[0017] Figure 2 This is a schematic diagram of the specific process of a neural network computing acceleration method provided by the present invention.

[0018] Figure 3 This is a schematic diagram of the non-zero tag generation circuit provided by the present invention.

[0019] Figure 4 This is a schematic diagram of the effective product aggregation circuit provided by the present invention.

[0020] Figure 5 This is a schematic diagram illustrating the principle of the hierarchical addition tree adapted for sparse data provided by the present invention.

[0021] Figure 6 This is a schematic diagram of the structure of a neural network computing acceleration system provided by the present invention.

[0022] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] With the rapid development of artificial intelligence, related applications such as image classification, object detection, and natural language processing are playing an increasingly important role in people's lives and work. As a crucial research area in artificial intelligence, neural networks, due to their high efficiency, high accuracy, and strong robustness, have achieved remarkable results in various environments and tasks, becoming a cutting-edge direction in AI research. Various neural network models have emerged, playing different roles in different fields. Traditional artificial neural networks, such as Convolutional Neural Networks (CNNs), have achieved great success, primarily through deep local feature extraction and spatial translation invariance to achieve high accuracy and performance on different tasks. Neuromorphic computing, such as third-generation neural networks—Spiking Neural Networks (SNNs)—attempts to mimic the operation of the human brain's nervous system, imitating its biosensing and information processing architecture. It utilizes the spatiotemporal sparsity of pulses to achieve higher speed and energy efficiency, as well as higher robustness and biological interpretability. It is considered capable of unleashing the greater potential of neuromorphic computing, facilitating further exploration of neurobehavior and brain science. In recent years, the development of generative pre-trained Transformer (GPT) models has taken artificial intelligence a big step forward. Its most basic structure is the well-established Transformer model, which has demonstrated its powerful capabilities in fields such as language and writing.

[0025] The computation of neural network models requires hardware support. Because the training and inference processes of neural networks involve significant computational and storage demands, traditional central processing units (CPUs) often cannot meet their high-performance and low-latency requirements, while large GPUs cannot meet the power consumption and latency constraints of edge devices. Therefore, to overcome this deficiency and deploy neural network models to mobile or edge devices for tasks, dedicated hardware accelerators for neural network computation need to be designed. In recent years, researchers have proposed and extensively studied dedicated neural network accelerators, such as TrueNorth, C-DNN, and PAICORE. As dedicated hardware, it is required to improve the computational efficiency and throughput of neural networks. Compared to general-purpose processors, neural network accelerators can accelerate core operations such as convolution, pooling, and nonlinear activation through model-hardware co-optimization, thereby providing faster computation speeds and lower power consumption.

[0026] Regardless of the specific neural network model, it typically consists of multiple structurally similar functional components, also known as "layers." Data is passed between layers, and each layer processes the received input and weights using different mechanisms. This usually involves multiplying the corresponding values ​​and summing the products to obtain a partial sum—a common "multiply-accumulate" operation in neural network computation. Convolutional layers in CNNs and SNNs typically require sliding convolutional kernels across the input feature map, performing multiply-accumulate operations on all input channels at each location. When there are many input channels, this results in a huge computational burden. Fully connected layers and Transformer models directly use matrix or vector multiplication, which also involves large-scale multiply-accumulate operations when there are many dimensions.

[0027] In accelerator design, the multiply-accumulate unit is the most crucial component, and also the main contributor to area and power consumption. Existing neural network computing accelerators typically use a combination of multipliers and adders to perform multiply-accumulate operations. Statistical data shows that the input and weight data in neural network models often contain a large number of zeros, exhibiting high sparsity. Since the product is zero when one of the two operands in a multiplication operation is zero, many calculated products during multiply-accumulate operations are zero and contribute nothing to the final result. Therefore, effectively utilizing this sparsity characteristic can significantly improve the accelerator's computational efficiency. It is necessary to develop a dynamic adaptation processing method and hardware circuitry for unstructured sparse data in neural networks, enabling more flexible and efficient neural network accelerator designs.

[0028] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for accelerating neural network computation provided by the present invention.

[0029] This invention provides a method for accelerating neural network computation, comprising: 101: Extract the amplitude bits of the input vector data and the amplitude bits of the weight vector data, and generate zero labels through logical operations; the zero label is a label that indicates whether the product of the input vector data and the weight vector data is zero; 102: Based on the zero label, dynamically aggregate non-zero products to form an aggregated product; 103: Based on the number of non-zero products in the aggregated product, dynamically adjust the calculation scale of the addition tree, and only perform accumulation operations on non-zero products.

[0030] Sparsity is an important property in the computation of neural network models, and this property can be used to speed up computation. This invention considers the sparsity of both input and weight data, and proposes an effective product aggregation circuit. Using original code encoding rules, it is applicable to the computation process of various neural network models and reduces the frequency of node voltage switching during computation. It includes a non-zero label generation circuit. For each pair of input-weight data (input vector data and weight vector data), it extracts all amplitude bits and generates zero labels through logical operations. Based on the zero labels, the effective product aggregation circuit can aggregate non-zero products together. For example, every 16 products are processed within one effective product aggregation circuit, operating in four levels. The first level (L1) consists of 8 groups of 2 products, and each subsequent group is formed by merging the two groups from the previous level. Within each group, non-zero products are moved to the right based on the zero label or the calculated non-zero product count, while zero products are discarded. The non-zero product count is obtained by calculating the sum of the two corresponding zero labels contained in the previous level, representing the number of non-zero products in the group. In the final output vector, all non-zero products are clustered on the right, while zero products on the left are ignored in future calculations. The total count of non-zero products from the 16 products is obtained for subsequent calculations. This invention improves the adaptation to unstructured sparsity of data while maintaining computational accuracy.

[0031] This invention employs a hierarchical adder tree adapted to sparse data. It can sense the sparsity of the input data and dynamically adjust the size of the activated computational circuits. It accumulates the effective non-zero products filtered by the effective product aggregation circuit to obtain the result. This invention works well with the effective product aggregation circuit to achieve significant power savings. The higher the data sparsity and the more irregular the data distribution, the smaller the size of the activated computational circuits, further improving computational efficiency. For example, to improve adaptability to real-world data, each hierarchical adder can receive up to 16 products. Based on the final non-zero product count calculated by the corresponding effective product aggregation circuit, the incoming product vector is truncated, and effective products are dynamically allocated to appropriate levels in the adder tree. Simultaneously, unnecessary circuit components are disabled, thus saving power. If the adder tree receives more than 16 input data items, multiple hierarchical adder trees can be combined to achieve dynamic control of the input.

[0032] The data processing algorithm and hardware processor design method proposed in this invention are applicable to both ASICs and FPGAs. This invention has been implemented in ASICs using tools such as Synopsys' Design Compiler and Prime Time, and has passed comprehensive testing, ensuring the feasibility and high performance of the algorithm and hardware design. The proposed invention can achieve a neuron scale exceeding 10,000, with a peak energy efficiency of 40 TOPS / W at 500 MHz. Typically, the Gaussian distribution of data in a model results in sparsity exceeding 50%. Measurements show that the proposed technique can achieve up to 54% energy savings, providing efficient support for most artificial intelligence workloads. Simultaneously, the proposed technique reduces the bitmap storage and computational operations required in structured sparse design, further simplifying the model training and deployment process.

[0033] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the specific process of a neural network computation acceleration method provided by the present invention.

[0034] As a preferred embodiment, extracting the amplitude bits of the input vector data and the amplitude bits of the weight vector data, and generating a zero label through logical operations, includes: representing the input vector data and the weight vector data using sign-magnitude representation to extract the amplitude bits of the input vector data and the amplitude bits of the weight vector data; the sign-magnitude representation includes a sign bit and an amplitude bit; inputting the amplitude bits of the input vector data into a first OR gate to obtain the input label output by the first OR gate; inputting the amplitude bits of the weight vector data into a second OR gate to obtain the weight label output by the second OR gate; and inputting the input label and the weight label into an AND gate to obtain the zero label output by the AND gate.

[0035] As a preferred embodiment, the method further includes: performing an XOR operation on the sign bits of the input vector data and the sign bits of the weight vector data to determine the sign of the product; and allocating the product to different computing circuits for processing according to the sign of the product.

[0036] As a preferred embodiment, the product is assigned to different computing circuits for processing according to the sign of the product, including: when the sign of the product is 1, the product is assigned to a negative value computing circuit for processing; when the sign of the product is 0, the product is assigned to a positive value computing circuit for processing.

[0037] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the non-zero tag generation circuit provided by the present invention.

[0038] In this embodiment, the non-zero tag generation circuit in the effective product aggregation circuit uses original code as the data representation method. The data distribution and encoding characteristics of original code naturally reduce the voltage flipping frequency during calculation. Simultaneously, the sign-amplitude representation method of original code also simplifies the amplitude calculation process. For each pair of input-weight data, all amplitude bits are extracted. For example, both the input vector and weight vector are 8-bit vectors, with the highest bit being the sign bit and the remaining 7 bits being amplitude bits. Through logical operations (such as OR), the first OR gate performs a bitwise OR operation on the amplitude bits of the input vector, outputting the input tag; the second OR gate performs a bitwise OR operation on the amplitude bits of the weight vector, outputting the weight tag. For the OR operation, the output is "1" if any input is "1", and "0" only when all inputs are "0". Therefore, multiplying any number by the amplitude bit corresponding to an OR gate output of 0 results in 0, and the associated multiplication and accumulation can be skipped. Then, the input tag and weight tag are input into the AND gate. For AND operations, the output is "0" if any input is "0", and "1" only if all inputs are "1". In other words, if either the input label or the weight label is 0, the product will be 0, and the related accumulation can be skipped. An AND gate output of 1 indicates that the product of the input and weights is definitely not 0 and needs to be fed into the multiplication and accumulation process.

[0039] The sign bits of the input vector data and the weight vector data can be fed into an XOR gate to determine where they will be processed (positive or negative value). In the XOR operation, if the two input levels are different, the output is a high level (1); if the two input levels are the same, the output is a low level (0). That is, if the two inputs are different, the XOR gate outputs a high level. Specifically, if the XOR gate output is 1, it means the input and weight have opposite signs and need to be processed in the negative value calculation circuit; conversely, if the output is 0, it means the input and weight have the same sign and need to be processed in the positive value calculation circuit.

[0040] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the effective product aggregation circuit provided by the present invention.

[0041] As a preferred embodiment, non-zero products are dynamically aggregated based on zero labels to form an aggregated product, including: using a hierarchical aggregation method, processing each set of 16 products into multiple levels; in each level, non-zero products are aggregated into a first region and zero products are aggregated into a second region based on zero labels to obtain an aggregated product; the first region and the second region have different directions.

[0042] In this embodiment, based on the zero tags generated by the non-zero tag generation circuit, the effective product aggregation circuit aggregates non-zero products together. To minimize the input size of the critical path and the multiplexer (MUX), every 16 products are processed within one effective product aggregation circuit, and the process operates in four levels. The first level (L1) consists of eight groups of two products. In each subsequent level, each group is formed by merging the two groups from the previous level. Within each group, non-zero products are moved to a first region (e.g., the right side) based on the zero tag or the calculated non-zero product count, while zero products are moved to a second region (e.g., the left side), and zero products are discarded. For example, in L1, whether data is moved and the direction of movement are determined by the right-side zero tag of each pair of data. If the right-side zero tag is 0, it means the right-side data is zero and must be discarded; therefore, it cannot be on the right, and the two products need to be swapped. Conversely, if the right-side data is non-zero, the original order can be maintained regardless of whether the left-side data is zero. Each non-zero product count is the sum of the two corresponding zero labels contained in the previous level, representing the number of non-zero products in the group.

[0043] In L2, if the non-zero product count on the right is 0, it means that both products in the right group are 0 and need to be replaced by the left group. The number of non-zero products in the left group doesn't matter, because the non-zero products are always on the right and the zero products on the left; shifting the entire group to the right maintains the original order of non-zero products on the right and zero products on the left. If the non-zero product count is 1, the left group only shifts one position to the right, replacing the second (leftmost) product in the right group. Finally, if the non-zero product count is 2, the order remains unchanged. Similar operations are performed in subsequent levels. Therefore, in the final output vector, all non-zero products are clustered on the right, while the zero products on the left are ignored in future calculations.

[0044] The method of this invention also simultaneously obtains the total non-zero product count value among the 16 products. This count is essential for subsequent calculations to determine the size of the activated circuit. The effective product aggregation circuit proposed in this invention utilizes this essential calculation characteristic, as well as the characteristic of unified right-gathering of effective non-zero products within a group, eliminating complex shift judgment logic. This invention simultaneously considers the unstructured sparsity of inputs and weights, providing greater versatility without imposing specific sparsity constraints, and can adapt well to data in real-world environments. By reducing the potential shift target for each data point, the effective product aggregation circuit eliminates the need for large-input multiplexers, achieving area savings of up to 29%.

[0045] Please refer to Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of the hierarchical addition tree adapted for sparse data provided by the present invention.

[0046] As a preferred embodiment, the computational scale of the addition tree is dynamically adjusted according to the number of non-zero products in the aggregated product, including: selectively enabling or disabling circuits at each level of the addition tree according to the number of non-zero products in the aggregated product to adapt to the sparsity of the input vector data.

[0047] Considering that if the adder tree design has a fixed input, many input ports will lack valid data due to the presence of numerous zeros. Therefore, the adder tree design must be able to perceive the sparsity of the input data and dynamically adjust the scale of the activated computational circuitry. In this embodiment, valid non-zero products filtered by the valid product aggregation circuit are accumulated through a dynamically hierarchical adder tree adapted to sparse data to obtain the multiply-accumulate result. Specifically, each hierarchical adder can receive up to 16 products. Based on the final non-zero product count calculated by the corresponding valid product aggregation circuit, the incoming product vector is truncated, and valid products are dynamically allocated to the appropriate level of the adder tree. For example, if there are 7 non-zero products among the 16 product vectors, products numbered #0~7 (7 non-zero products and 1 zero) will be stored in the second-level register, while products numbered #8~15 will be discarded. Then, the hierarchical addition tree starts calculation from level 2, while all adders and circuits in level 1 are disabled, thus saving some power consumption. If the number of non-zero products is less than 4, the third level circuit is activated, and the first two levels of circuits are all disabled.

[0048] Typically, addition trees receive far more than 16 input data points. In such cases, multiple hierarchical addition trees can be combined to achieve dynamic control of the input. For example, if there are 144 products to be input, they can be divided into two parts: nine 4-level dynamic hierarchical adders and one 9-1 adder. After the hierarchical addition tree, the 9-1 adder directly and immediately accumulates all the partial sums to obtain the final result.

[0049] By integrating an effective product aggregation circuit and a hierarchical addition tree, this invention can dynamically determine the activated computational circuit based on the unstructured sparsity of the input and weights, thus significantly improving the perception and processing capabilities for unstructured sparse data.

[0050] The neural network computing acceleration system provided by the present invention is described below. The neural network computing acceleration system described below can be referred to in correspondence with the neural network computing acceleration method described above.

[0051] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of a neural network computing acceleration system provided by the present invention.

[0052] The present invention also provides a neural network computing acceleration system, comprising: a generation module 601, used to extract the amplitude bits of the input vector data and the amplitude bits of the weight vector data, and generate zero labels through logical operations; the zero label is a label characterizing whether the product of the input vector data and the weight vector data is zero; a dynamic aggregation module 602, used to dynamically aggregate non-zero products according to the zero labels to form an aggregated product; and a scale adjustment module 603, used to dynamically adjust the computing scale of the addition tree according to the number of non-zero products in the aggregated product, and only perform accumulation operations on non-zero products.

[0053] Figure 7 An example is a schematic diagram of the structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 701, a communication interface 702, a memory 703, and a communication bus 704. The processor 701, communication interface 702, and memory 703 communicate with each other via the communication bus 704. The processor 701 can call logical instructions in the memory 703 to execute a neural network computation acceleration method. This method includes: extracting the amplitude bits of the input vector data and the amplitude bits of the weight vector data, and generating zero labels through logical operations; the zero label is a label characterizing whether the product of the input vector data and the weight vector data is zero; dynamically aggregating non-zero products based on the zero label to form an aggregated product; and dynamically adjusting the computational scale of the addition tree based on the number of non-zero products in the aggregated product, performing accumulation operations only on non-zero products.

[0054] Furthermore, the logical instructions in the aforementioned memory 703 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the neural network computation acceleration method provided by the above methods. The method includes: extracting the amplitude bits of the input vector data and the amplitude bits of the weight vector data, and generating zero labels through logical operations; the zero label is a label that characterizes whether the product of the input vector data and the weight vector data is zero; dynamically aggregating non-zero products according to the zero label to form an aggregated product; dynamically adjusting the computation scale of the addition tree according to the number of non-zero products in the aggregated product, and only performing accumulation operations on non-zero products.

[0056] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the neural network computation acceleration method provided by the above methods. The method includes: extracting the amplitude bits of the input vector data and the amplitude bits of the weight vector data, and generating zero labels through logical operations; the zero label is a label characterizing whether the product of the input vector data and the weight vector data is zero; dynamically aggregating non-zero products according to the zero label to form an aggregated product; dynamically adjusting the computational scale of the addition tree according to the number of non-zero products in the aggregated product, and performing accumulation operations only on non-zero products.

[0057] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0058] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for accelerating neural network computation, characterized in that, include: Extract the magnitude bits of the input vector data and the magnitude bits of the weight vector data, and generate zero labels through logical operations; The zero label is a label that indicates whether the product of the input vector data and the weight vector data is zero; Based on the zero label, non-zero products are dynamically aggregated to form an aggregated product; The computational scale of the addition tree is dynamically adjusted based on the number of non-zero products in the aggregated product, and the accumulation operation is performed only on non-zero products.

2. The neural network computation acceleration method according to claim 1, characterized in that, The process of extracting the magnitude bits of the input vector data and the magnitude bits of the weight vector data, and generating zero labels through logical operations, includes: The input vector data and weight vector data are represented using original code to extract the magnitude bits of the input vector data and the magnitude bits of the weight vector data; the original code includes a sign bit and a magnitude bit. The magnitude bits of the input vector data are input into the first OR gate to obtain the input label output by the first OR gate; The magnitude bits of the weight vector data are input into the second OR gate to obtain the weight labels output by the second OR gate; The input label and the weight label are input into an AND gate to obtain the zero label output by the AND gate.

3. The neural network computation acceleration method according to claim 1, characterized in that, The step of dynamically aggregating non-zero products based on the zero label to form an aggregated product includes: A hierarchical aggregation approach is adopted, dividing each set of 16 multiplicands into multiple levels for processing; Each level, based on the zero label, aggregates non-zero products into the first region and zero products into the second region to obtain aggregated products; the directions of the first and second regions are different.

4. The neural network computation acceleration method according to claim 1, characterized in that, The step of dynamically adjusting the computational scale of the addition tree based on the number of non-zero products in the aggregated product includes: Based on the number of non-zero products in the aggregated product, selectively enable or disable the circuitry at each level of the adder tree to accommodate the sparsity of the input vector data.

5. The neural network computation acceleration method according to any one of claims 1 to 4, characterized in that, Also includes: The sign bit of the input vector data and the sign bit of the weight vector data are XORed to determine the sign of the product; The product is distributed to different computing circuits for processing based on its sign.

6. The neural network computation acceleration method according to claim 5, characterized in that, The process of allocating the product to different computing circuits for processing based on the sign of the product includes: If the sign of the product is 1, the product is assigned to the negative value calculation circuit for processing; If the sign of the product is 0, the product is assigned to the positive value calculation circuit for processing.

7. A neural network computing acceleration system, characterized in that, include: The generation module is used to extract the magnitude bits of the input vector data and the magnitude bits of the weight vector data, and generate zero labels through logical operations. The zero label is a label that indicates whether the product of the input vector data and the weight vector data is zero; The dynamic aggregation module is used to dynamically aggregate non-zero products based on the zero label to form an aggregated product; The scaling module is used to dynamically adjust the calculation scale of the addition tree based on the number of non-zero products in the aggregated product, and only perform accumulation operations on non-zero products.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the neural network computation acceleration method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the neural network computation acceleration method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the neural network computation acceleration method as described in any one of claims 1 to 6.

Images