Neural network acceleration method and device based on reconfigurable approximate multiplier

Abstract

The invention discloses a neural network acceleration method and device based on a reconfigurable approximate multiplier, and the method comprises the steps: dividing the quantized weight data of each network layer of a neural network model into a plurality of weight intervals, and carrying out the independent replacement of the approximate multiplier; sorting all weight intervals into a weight interval queue in combination with the network precision after approximate replacement and the multiply-accumulate calculation amount; approximate multiplier replacement of AX2 and AX1 approximate modes is carried out on the weight interval queues in sequence until the network precision reasoned by the neural network model after replacement reaches a precision reduction threshold value; and finally, generating a reconfigurable approximate multiplier replacement configuration of each weight interval under each network layer of the neural network model. The invention aims to comprehensively consider the weight importance difference and the calculation load difference of the neural network, and dynamically distribute different approximate modes of the approximate multiplier for different weights so as to realize energy saving.

Description

Technical Field

[0001] This invention relates to the field of neural network accelerator technology, and specifically to a neural network acceleration method and apparatus based on a reconfigurable approximate multiplier. Background Technology

[0002] With the rapid development of artificial intelligence, edge computing, and big data processing technologies, the demand for energy-efficient, high-performance general-purpose computing engines and application-specific integrated circuits (ASICs) is increasing daily. These technological advancements place higher demands on the energy efficiency of integrated circuits and computing systems. Although advancements in semiconductor technology and energy-efficient design technology have significantly improved the performance of computing systems, the rapid increase in total energy consumption has also brought numerous challenges.

[0003] In deep learning, especially in convolutional neural networks, multiplication and addition are the primary computational operations. However, the implementation overhead and latency of multipliers are far greater than those of adders. Therefore, approximating multipliers can more effectively reduce the hardware overhead of the system. By using approximate multipliers, power consumption and hardware costs can be significantly reduced while maintaining sufficient accuracy, which is crucial for designing high-performance, low-power computing systems. Since the data in neural networks is inherently redundant, previous work has employed methods such as pruning and quantization to reduce unnecessary data volume. Some special layers (such as nonlinear layers) filter feature data, ignoring some features, indicating that neural networks themselves also perform a certain degree of approximation and are fault-tolerant to input data.

[0004] However, different applications have different requirements for multipliers. The specific choice of approximate multiplier needs to be based on the characteristics of the specific application. Among them, neural networks are an important application scenario for approximate multipliers. Neural networks are usually multi-layered structures, and the sensitivity of different layers is different. Therefore, it is necessary to consider the mapping method of the approximate multiplier in different layers.

[0005] Currently, the design of neural network accelerators incorporating approximate multipliers can be categorized into the following main approaches based on the granularity of replacement: 1. Using approximate multipliers for the entire computation array; 2. Using different approximate multipliers for different layers; 3. Using different approximate multipliers for weights; 4. Using different approximate multipliers alternately for the MAC array. From the perspective of approximate multipliers, they can be divided into two types: one is based on fixed-error multipliers, mainly using the first, second, and fourth approaches mentioned above; the other is based on configurable-precision approximate multipliers, i.e., the third approach mentioned above.

[0006] Design methods for approximate multipliers based on fixed errors lack the ability to dynamically adjust precision at runtime, limiting their applicability in dynamic environments such as neural network inference, as some applications or specific content within those applications may have extremely low tolerance for approximations. Therefore, current research primarily focuses on using approximate multipliers with configurable precision. Approximate multipliers based on configurable precision mainly rely on their variable precision characteristics to replace approximate multipliers for neural network weights, using approximate multipliers of different precisions for each weight. Current methods for assigning approximate multipliers to weights mainly face two problems: first, how to perform the mapping. Currently, there are two main mapping methods: one is to explore symmetric approximate multipliers based on the symmetry of the weights, and the other is to use as many approximate multipliers of various precisions as possible, based on the minimum usage ratio of multipliers. However, neither approach takes into account the impact of the importance of neural network weights on the neural network, nor does it consider the differences in computational cost between different layers in addition to differences in fault tolerance. The second issue is how to adjust the precision of the reconfigurable multiplier based on the weights. Currently, there are two main solutions: one is to store the control signal in the weight information for very fine-grained mapping, but this increases the model size to some extent; the other is to use an additional component for real-time weight judgment, but if the weight division is very fine-grained, this will result in a large area and delay of the control component, reducing the overall benefit. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a neural network acceleration method and apparatus based on a reconfigurable approximation multiplier, which addresses the above-mentioned problems in the prior art. The present invention aims to use a reconfigurable approximation multiplier to dynamically allocate different approximation modes of the approximation multiplier to different weights by comprehensively considering the differences in the importance of neural network weights and the differences in computational load, so as to achieve energy saving.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A neural network acceleration method based on a reconfigurable approximate multiplier includes the following steps: S1, obtain the quantized weight data and multiply-accumulate computation amount of each network layer of the neural network model after training and quantization; S2 divides the quantized weight data of each network layer into multiple weight intervals; S3. For each network layer, replace the default multiplier with a more approximate multiplier for each weight interval, and infer the network accuracy after approximating each weight interval from the obtained neural network model. S4. Weight intervals whose accuracy drops beyond a preset threshold after approximation or whose ranking is a specified proportion behind all network layers are designated as non-fault-tolerant weight intervals, while the remaining weight intervals are designated as fault-tolerant weight intervals. The fault-tolerant weight intervals are divided into multiple groups based on the computational cost of multiplication and accumulation, and then sorted within each group according to the accuracy of the approximation, thus obtaining multiple weight interval queues for the fault-tolerant weight intervals. The non-fault-tolerant weight intervals are sorted according to the accuracy of the approximation to generate a single weight interval queue. S5 merges multiple weight interval queues of fault-tolerant weight intervals and one weight interval queue of non-fault-tolerant weight intervals into a complete weight interval queue. S6, perform AX2 approximation multiplier replacement on the complete weight interval queue until the network accuracy of the replaced neural network model inference reaches the first accuracy decrease threshold. S7, determine whether the number of weight intervals replaced by the approximation multiplier in the AX2 approximation mode is less than the specified proportion of the total number of weight intervals in the complete weight interval queue. If it is true, retain the approximation multiplier replacement in the AX2 approximation mode; otherwise, cancel the approximation multiplier replacement in the AX2 approximation mode. S8, perform approximate multiplier replacement for the complete weight interval queue using the AX1 approximation mode until the network accuracy of the replaced neural network model inference reaches the second accuracy reduction threshold. S9, Generate a reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer of the neural network model. The approximate multiplier replacement configuration is used to control the approximate mode adopted by the reconfigurable approximate multiplier in the neural network acceleration device when performing multiplication operations in each weight interval of each network layer during the inference of the neural network model in the neural network acceleration device.

[0009] Optionally, when dividing the quantized weight data of each network layer into multiple weight intervals in step S2, the quantized weight data of each network layer is sorted by size and then divided into multiple weight intervals with the same or equivalent number of weights.

[0010] Optionally, in step S3, replacing the default multiplier with a more approximate multiplier for each weight interval means replacing the default multiplier with a more approximate multiplier than the AX2 approximation mode for each weight interval.

[0011] Optionally, in step S5, when merging multiple weight interval queues of fault-tolerant weight intervals and one weight interval queue of non-fault-tolerant weight intervals into a complete weight interval queue, the multiple weight interval queues of fault-tolerant weight intervals are sorted in descending order of multiplication and accumulation computation, and the one weight interval queue of non-fault-tolerant weight intervals is located after the multiple weight interval queues of fault-tolerant weight intervals.

[0012] Optionally, the reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer of the neural network model generated in step S9 includes the weight boundaries of N-1 adjacent weight intervals and the approximate multiplier approximation layer configuration Conf, where N is the total number of weight intervals. The weight boundaries are used to determine the weight interval to which each weight data belongs. The approximate multiplier approximation layer configuration Conf records the approximate multiplier control signals Ctrl corresponding to N weight intervals. Each approximate multiplier control signal Ctrl is used to control the approximation mode adopted by the reconfigurable approximate multiplier when performing multiplication operations.

[0013] Optionally, after generating the reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer in step S9, the method further includes saving the reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer in the corresponding network layer parameters.

[0014] The present invention also provides a neural network acceleration device for applying the neural network acceleration method based on reconfigurable approximation multipliers, comprising multiple control units and an approximation operation array composed of reconfigurable approximation multipliers. The multiple control units are respectively connected to the input terminals of the approximation operation array. Each control unit includes N-1 comparators and a selector. The N-1 comparators are used to compare N-1 pairs of input data to obtain N-1 comparison signals, such that the comparison signal output by the comparator corresponding to the weight interval to which the input weight data W belongs is 1, and the comparison signals output by the other comparators are 0. The N-1 pairs of input data consist of the input weight data W and the weight boundaries of N-1 adjacent weight intervals. The selector is used to find the approximation multiplier control signal Ctrl corresponding to the weight interval with a comparison signal of 1 in the approximation layer configuration Conf of the current network layer and send it to the approximation operation array to control the approximation mode adopted by the reconfigurable approximation multiplier when performing multiplication operation on the input weight data W.

[0015] The present invention also provides a neural network acceleration device based on a reconfigurable approximate multiplier, comprising a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the neural network acceleration method based on the reconfigurable approximate multiplier.

[0016] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the neural network acceleration method based on a reconfigurable approximate multiplier by a processor.

[0017] The present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the neural network acceleration method based on a reconfigurable approximate multiplier via a processor.

[0018] Compared with existing technologies, the present invention mainly achieves the following beneficial effects: The neural network acceleration method based on reconfigurable approximate multipliers of the present invention defines non-fault-tolerant weight intervals as weight intervals whose accuracy drops beyond a preset threshold after approximation or whose weight intervals are ranked at a specified proportion after all network layers, and the remaining weight intervals as fault-tolerant weight intervals. The fault-tolerant weight intervals are divided into multiple groups according to the computational cost of multiplication and accumulation, and sorted within each group according to the accuracy of the approximate replacement network, thereby obtaining multiple weight interval queues of fault-tolerant weight intervals. The non-fault-tolerant weight intervals are sorted according to the accuracy of the approximate replacement network to generate a weight interval queue. By comprehensively considering the importance of neural network weight intervals and the differences in computational load between different layers, different approximate multipliers are assigned to different weight data to achieve energy savings. It can utilize approximate multipliers with different approximation levels for different weight intervals of different layers to improve energy efficiency and meet the accuracy requirements of different scenarios, and has the advantages of good energy saving effect and high energy efficiency. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the weighted interval division method in an embodiment of the present invention.

[0021] Figure 3 This diagram illustrates the experimental results of approximate replacement of weight intervals in the five-layer neural network model in this embodiment. (a) represents the computational cost of each convolutional layer, (b) represents the network accuracy after approximate replacement of weight intervals in layer L1, (c) represents the network accuracy after approximate replacement of weight intervals in layer L2, (d) represents the network accuracy after approximate replacement of weight intervals in layer L3, (e) represents the network accuracy after approximate replacement of weight intervals in layer L4, and (f) represents the network accuracy after approximate replacement of weight intervals in layer L5.

[0022] Figure 4 This is a schematic diagram illustrating an example of obtaining the computational load of each layer and the inference after interval substitution in an embodiment of the present invention.

[0023] Figure 5 The following is an example diagram illustrating the use of AX2 exceeding 10% in this embodiment of the invention, where (a) is the complete weight interval queue after replacing the approximate multiplier in the AX2 approximation mode, and (b) is the complete weight interval queue after replacing the approximate multiplier in the AX1 approximation mode.

[0024] Figure 6 The following is an example diagram illustrating the cancellation of AX2 replacement when the usage ratio of AX2 does not exceed 10% in an embodiment of the present invention. (a) is the complete weight interval queue after replacing the approximate multiplier in the AX2 approximation mode, and (b) is the complete weight interval queue after replacing the approximate multiplier in the AX1 approximation mode.

[0025] Figure 7 This is a schematic diagram of the key structure of the neural network acceleration device in an embodiment of the present invention.

[0026] Figure 8 This is a schematic diagram of the control unit in the neural network acceleration device in an embodiment of the present invention. Detailed Implementation

[0027] Assuming that the neural network weights will remain in the computation array for a long time, the key to the neural network acceleration method based on reconfigurable approximate multipliers in this embodiment lies in the configurable approximate multiplier with dynamically switching precision at runtime and the design of a dynamically switching systolic array. It also incorporates a weight interval importance ranking method that comprehensively considers the differences in computational load and the importance of network weights between different layers of the neural network, and a design space exploration method based on a two-step AX2 / AX1 approximation mode for approximate multiplier replacement to accelerate the approximate multiplier allocation process. To enable those skilled in the art to better understand the technical solution of this invention, the technical solution of this invention will be further described in detail below with reference to the accompanying drawings in the embodiments of this invention.

[0028] like Figure 1 As shown, the neural network acceleration method based on a reconfigurable approximate multiplier in this embodiment includes the following steps: S1, obtain the quantized weight data and multiply-accumulate computation amount of each network layer of the neural network model after training and quantization; S2 divides the quantized weight data of each network layer into multiple weight intervals; S3. For each network layer, replace the default multiplier with a more approximate multiplier for each weight interval, and infer the network accuracy after approximating each weight interval from the obtained neural network model. S4. Weight intervals whose accuracy drops beyond a preset threshold after approximation or whose ranking is a specified proportion behind all network layers are designated as non-fault-tolerant weight intervals, while the remaining weight intervals are designated as fault-tolerant weight intervals. The fault-tolerant weight intervals are divided into multiple groups based on the computational cost of multiplication and accumulation, and then sorted within each group according to the accuracy of the approximation, thus obtaining multiple weight interval queues for the fault-tolerant weight intervals. The non-fault-tolerant weight intervals are sorted according to the accuracy of the approximation to generate a single weight interval queue. S5 merges multiple weight interval queues of fault-tolerant weight intervals and one weight interval queue of non-fault-tolerant weight intervals into a complete weight interval queue. S6, perform AX2 approximation multiplier replacement on the complete weight interval queue until the network accuracy of the replaced neural network model inference reaches the first accuracy decrease threshold. S7, determine whether the number of weight intervals replaced by the approximation multiplier in the AX2 approximation mode is less than the specified proportion of the total number of weight intervals in the complete weight interval queue. If it is true, retain the approximation multiplier replacement in the AX2 approximation mode; otherwise, cancel the approximation multiplier replacement in the AX2 approximation mode. S8, perform approximate multiplier replacement for the complete weight interval queue using the AX1 approximation mode until the network accuracy of the replaced neural network model inference reaches the second accuracy reduction threshold. S9, Generate a reconfigurable approximate multiplier replacement configuration for each weight interval of each network layer in the neural network model. This approximate multiplier replacement configuration is used to control the approximation mode adopted by the reconfigurable approximate multiplier in the neural network acceleration device when performing multiplication operations in each weight interval of each network layer during inference of the neural network model. In this embodiment, the reconfigurable approximate multiplier has two approximation modes (AX1 and AX2) and one accurate mode (AX0). AX2 is the mode with the lowest power consumption and the largest error, AX1 is the mode with the smaller error, and AX0 is the accurate mode.

[0029] Before step S1 in this embodiment, the neural network model needs to be trained and quantized. Since the training and quantization of neural network models are well-known methods, their implementation details will not be described here. After training and quantization are completed, step S1 can be used to analyze the trained and quantized neural network structure, and to statistically analyze the quantized weight data and multiply-accumulate (MAC) computation of each network layer. The neural network acceleration method based on reconfigurable approximate multipliers in this embodiment is applicable to neural network models with various structures. In this embodiment, a five-layer neural network model (including network layers L1 to L5, which can be convolutional layers and fully connected layers, etc.) is used as an example to further explain the neural network acceleration method based on reconfigurable approximate multipliers in this embodiment.

[0030] In step S2 of this embodiment, when dividing the quantized weight data of each network layer into multiple weight intervals, this includes sorting the quantized weight data of each network layer according to size, and then dividing it into multiple weight intervals with the same or equivalent number of weights. For example, as an optional implementation, such as... Figure 2 As shown, the quantized weight data of each network layer can be sorted by size and then divided into N=4 weight intervals Q1 to Q4 with the same or equivalent number of weights (the difference in number does not exceed a preset threshold). In this embodiment, the quantized weight data of the five convolutional layers and the computational cost of each layer are saved, and then processed as follows: Figure 2 The partitioning method shown divides the weights of each convolutional layer into four weight intervals with a similar amount of data. For example, if a convolutional layer has 10,000 weights, then each weight interval is allocated approximately 2,500 weights.

[0031] Since approximate replacement of a single weight interval in a single layer has no significant impact, in step S3 of this embodiment, replacing the default multiplier with a more approximate multiplier for each weight interval means replacing the default multiplier with an approximate multiplier that has a higher approximation level than the AX2 approximation mode. After replacing the divided weight intervals individually with an approximate multiplier that has a slightly higher approximation level than AX2, inference is performed to obtain the network accuracy after approximate replacement, which serves as the basis for distinguishing the importance of each weight interval. The accuracy of the approximate multiplier used is lower than that of the actual approximate multiplier used, which is beneficial for distinguishing the sensitivity of different weight intervals to approximate replacement. In this embodiment, replacing these weight intervals with an approximate multiplier with a higher approximation level yields the following result: Figure 3 The network inference accuracy after the approximate replacement is shown. Figure 3This diagram illustrates the experimental results of approximate weight interval replacement for a five-layer neural network model (including layers L1 to L5) in this embodiment. (a) represents the computational cost of each convolutional layer; (b) represents the network accuracy after approximate weight interval replacement for layer L1; (c) represents the network accuracy after approximate weight interval replacement for layer L2; (d) represents the network accuracy after approximate weight interval replacement for layer L3; (e) represents the network accuracy after approximate weight interval replacement for layer L4; and (f) represents the network accuracy after approximate weight interval replacement for layer L5. Weight intervals with a significant decrease in accuracy after replacement are selected and considered to be fault-intolerant; the remaining intervals are considered fault-tolerant.

[0032] In step S4 of this embodiment, the weight intervals in which the accuracy of the approximate replacement network decreases by more than a preset threshold or is ranked at the bottom of all network layers by a specified percentage are considered as unforgivable weight intervals, while the remaining weight intervals are considered as forgivable weight intervals. As an optional implementation, in step S4 of this embodiment, when the weight intervals in which the accuracy of the approximate replacement network decreases by more than a preset threshold or is ranked at the bottom of all network layers by a specified percentage are considered as unforgivable weight intervals, the unforgivable intervals are selected based on two indicators: first, the weight intervals in which the accuracy of the approximate replacement network decreases by more than 20%; second, the weight intervals in which the accuracy is in the bottom 10% of the whole are considered as unforgivable intervals. The final unforgivable part is determined by the union of the two indicators.

[0033] like Figure 4 As shown, in step S4 of this embodiment, when dividing the fault-tolerant weight intervals into multiple groups based on the multiply-accumulate computational cost, specifically, the network layers are divided into three groups according to the difference in computational cost: a maximum computational cost group, a normal computational cost group, and a minimum computational cost group. Within each group, the weight intervals are sorted according to the network precision after approximation, thus obtaining multiple queues of fault-tolerant weight intervals. For non-fault-tolerant weight intervals, a single queue of weight intervals is generated by sorting them according to the network precision after approximation. As an optional implementation, in this embodiment, the division into three groups is based on the layer's average multiply-accumulate computational cost. Weight intervals exceeding twice the layer's average multiply-accumulate computational cost are considered to belong to the maximum computational cost group, those below half the layer's average multiply-accumulate computational cost are considered to belong to the minimum computational cost group, and the remaining weight intervals belong to the normal computational cost group. Figure 4 The fault-tolerant intervals are divided into three queues based on computational cost: the maximum computational cost group, the moderate computational cost group, and the minimum computational cost group. Within each group, the intervals are sorted from highest to lowest inference precision. Finally, all weighted intervals are reorganized in the order of the maximum computational cost group, the moderate computational cost group, the minimum computational cost group, and the non-fault-tolerant interval group.

[0034] In step S5 of this embodiment, when merging multiple weight interval queues of fault-tolerant weight intervals and one weight interval queue of non-fault-tolerant weight intervals into a complete weight interval queue, the multiple weight interval queues of fault-tolerant weight intervals are sorted in descending order of multiplication and accumulation computation, and the one weight interval queue of non-fault-tolerant weight intervals is located after the multiple weight interval queues of fault-tolerant weight intervals.

[0035] In step S6 of this embodiment, the approximate multipliers of the AX2 approximation mode are replaced for the complete weight interval queue until the network accuracy of the replaced neural network model inference reaches the first accuracy reduction threshold. The replacement of the approximate multipliers of the AX2 approximation mode for the complete weight interval queue can be performed using a binary search method to find the weight intervals for AX2 approximation mode replacement, thereby determining the largest weight interval that can use the AX2 approximation mode and meets the accuracy reduction threshold. The first accuracy reduction threshold is constrained by the maximum accuracy reduction threshold, and its value is less than the maximum accuracy reduction threshold. It can be the maximum accuracy reduction threshold multiplied by a coefficient less than 1. As an optional implementation, in this embodiment, the first accuracy reduction threshold is 50% of the maximum accuracy reduction threshold.

[0036] In step S7 of this embodiment, it is determined whether the number of weight intervals replaced by the approximate multiplier in the AX2 approximation mode is less than a specified proportion of the total number of weight intervals in the complete weight interval queue. If it is true, the approximate multiplier replacement in the AX2 approximation mode is retained; otherwise, the approximate multiplier replacement in the AX2 approximation mode is cancelled. The specified proportion can be determined according to actual needs. For example, as an optional implementation, in this embodiment, it is determined whether the number of weight intervals replaced by the approximate multiplier in the AX2 approximation mode is less than 10% of the total number of weight intervals in the complete weight interval queue. If it is true, the approximate multiplier replacement in the AX2 approximation mode is retained; otherwise, the approximate multiplier replacement in the AX2 approximation mode is cancelled. However, after entering step S8, the approximate multiplier replacement in the AX1 approximation mode is still applied to all weight intervals, targeting the complete weight interval queue. Figure 5 The diagram shows the case where the number of weight intervals replaced by the approximate multiplier in the AX2 approximation mode exceeds 10%. In this case, the approximate multiplier replacement in the AX2 approximation mode is retained, and after proceeding to step S8, the approximate multiplier replacement in the AX1 approximation mode is performed on the unreplaced weight intervals. Figure 6 The example shown illustrates the case where the number of weight intervals replaced by the approximate multiplier in the AX2 approximation mode does not exceed 10%. In this case, the approximate multiplier replacement in the AX2 approximation mode is canceled, and after proceeding to step S8, the approximate multiplier replacement in the AX1 approximation mode is still applied to all weight intervals, targeting the complete weight interval queue. For example... Figure 5As shown, AX2, which has a larger approximation error and lower power consumption, is first used for replacement. Since the inference accuracy after approximation replacement was sorted, a bisection method is used to explore the weight range that can replace the AX2 multiplier in order to accelerate the exploration of the design space for approximation replacement, until the accuracy degradation threshold is reached. Then, for the remaining weight range that has not been replaced by the approximation multiplier, the bisection method is also used to replace the AX1 multiplier until the accuracy degradation threshold is reached, finally obtaining the configuration strategy of the approximation multiplier that meets the accuracy degradation threshold. The following supplementary explanations and discussions are made regarding the neural network weight and computationally driven neural network accelerator architecture based on approximation calculation in this embodiment: The non-fault-tolerant weight range mainly refers to two indicators: one is the weight range with an accuracy degradation of more than 20%, and the other is the weight range with an accuracy in the bottom 10% of the total computation. The non-fault-tolerant weight range is the union of the two. The division of the maximal group and the computationally intensive minima is mainly measured by the average layer computation, which can be selected as twice the average layer computation. When using AX2 for approximation replacement, the accuracy degradation threshold used is 50% of the overall system degradation threshold, leaving some accuracy margin for AX1 replacement. When using AX1 for approximate replacement, if the weight interval replaced by AX2 is too small (e.g., 10%), then AX2 is considered unsuitable for the instance, and the previous approximate replacement by AX2 is canceled. The approximate replacement by AX1 is applied to the entire weight interval instead of the remaining weight interval. Figure 6 As shown. After the above steps, the approximate multiplier configuration for the four weight intervals of each neural layer is obtained, and the approximate multiplier configuration information is stored in the corresponding neural network layer.

[0037] In step S8 of this embodiment, the approximate multipliers of the AX1 approximation mode are replaced for the complete weight interval queue until the network accuracy of the replaced neural network model inference reaches the second accuracy reduction threshold. The replacement of the approximate multipliers of the AX1 approximation mode for the complete weight interval queue can be performed using a binary search method to find the weight intervals for AX1 approximation mode replacement, thereby determining the largest weight interval that can use the AX1 approximation mode and meets the accuracy reduction threshold. The second accuracy reduction threshold is constrained by the maximum accuracy reduction threshold, and its value is not greater than the maximum accuracy reduction threshold. As an optional implementation, in this embodiment, the second accuracy reduction threshold is 100% of the maximum accuracy reduction threshold.

[0038] In step S9 of this embodiment, the reconfigurable approximate multiplier replacement configuration for each weight interval in each network layer of the generated neural network model includes weight boundaries of N-1 adjacent weight intervals and an approximate multiplier approximation layer configuration Conf, where N is the total number of weight intervals. The weight boundaries are used to determine the weight interval to which each weight data belongs. The approximate multiplier approximation layer configuration Conf records the approximate multiplier control signals Ctrl corresponding to the N weight intervals. Each approximate multiplier control signal Ctrl is used to control the approximation mode used by the reconfigurable approximate multiplier when performing multiplication operations. In this embodiment, one reconfigurable approximate multiplier configuration requires 2 bits of data, therefore, one network layer requires 8 bits of data to store the approximate multiplier configurations for four weight intervals, and the width of the approximate multiplier approximation layer configuration Conf is 8 bits.

[0039] As an optional implementation, in step S9 of this embodiment, after generating the reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer of the neural network model, the reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer is further stored in the corresponding network layer parameters, thereby realizing the integration of the reconfigurable approximate multiplier replacement configuration and the network layer parameters of the neural network model.

[0040] Those skilled in the art will understand that the neural network acceleration device driven by approximate calculation of neural network weights and computational load proposed in this embodiment can be implemented by adding logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. As an optional implementation, such as... Figure 7 and Figure 8 As shown, this embodiment also provides a neural network acceleration device for applying the neural network acceleration method based on reconfigurable approximation multipliers, including multiple control units (control unit #1 to control unit #M) and an approximation operation array composed of reconfigurable approximation multipliers. The multiple control units are respectively connected to the input end of the approximation operation array. Each control unit includes N-1 comparators and a selector. The N-1 comparators are used to compare N-1 pairs of input data to obtain N-1 comparison signals, such that the comparison signal output by the comparator corresponding to the weight interval to which the input weight data W belongs is 1, and the comparison signals output by the other comparators are 0. The N-1 pairs of input data consist of the input weight data W and the weight boundaries of N-1 adjacent weight intervals. The selector is used to find the approximation multiplier control signal Ctrl corresponding to the weight interval with a comparison signal of 1 in the approximation layer configuration Conf of the current network layer and send it to the approximation operation array to control the approximation mode adopted by the reconfigurable approximation multiplier when performing multiplication operation on the input weight data W. Figure 7 In the middle, W1~WM There are M parallel weighted data points W, with weighted data points W1 to W2. M Before entering the approximate operation array, the data is processed by control units #1 to #M to generate approximate multiplier control signals Ctrl1 to Ctrl1 that flow with the weights. M Approximate multiplier control signals Ctrl1 to Ctrl2 M and weighted data W1~W M Input the approximate operation array together. Weight data W1~W M As metadata for the operation, the approximate multiplier control signals Ctrl1 to Ctrl2 are... M Used to configure the approximation modes (AX0, AX1, and AX2) of the reconfigurable approximation multiplier. Control unit i receives weight data W. i The approximation layer configuration Conf consists of three boundaries L1, L2, and L3, divided by four weight intervals, and an 8-bit approximation multiplier. Three comparators are used to receive L1 and W respectively. i L2 and W i L3 and W i The input selector determines the weight W. i To determine which weight range it belongs to, the selector chooses the corresponding 2-bit approximation multiplier control signal Ctrl from the 8-bit approximation multiplier approximation layer configuration Conf. i Ctrl i [1] and Ctrl i [0] represents the 2-bit approximate multiplier control signal Ctrl. i The high and low bits allow for the application of approximate multipliers with different approximation levels to different weight ranges for different layers, thereby improving energy efficiency and meeting the accuracy requirements of different scenarios. When loading weights, the weights are input into... Figure 7 Figure 8 The control unit shown generates the runtime configuration signal for the approximate multiplier after passing through comparators, selectors, and other units. This process can be completed during weight preloading and can overlap with the computation time of the computation array.

[0041] In summary, the neural network acceleration method and apparatus described in this embodiment employ a reconfigurable precision approximation multiplier, using different approximation modes for different weight data, thereby reducing the overall system energy consumption. This embodiment focuses on the importance of neural network weights and the computational overhead of each layer's operations, thus reducing the overall system energy consumption. The neural network acceleration device described in this embodiment is an independent design of the computation array, independent of the overall system. Each network layer can configure the computation array at runtime according to its own configuration, without restrictions on the neural network structure. The neural network acceleration device described in this embodiment focuses on the importance of neural network weights and the computational overhead of each layer's operations, thus reducing the overall system energy consumption and improving the energy efficiency of the network acceleration device. Furthermore, this embodiment also provides a neural network acceleration device based on a reconfigurable approximate multiplier, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the neural network acceleration method based on the reconfigurable approximate multiplier.

[0042] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the neural network acceleration method based on a reconfigurable approximate multiplier by a processor.

[0043] Furthermore, this embodiment also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the neural network acceleration method based on a reconfigurable approximate multiplier via a processor.

[0044] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0045] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A neural network acceleration method based on a reconfigurable approximate multiplier, characterized in that, Includes the following steps: S1, obtain the quantized weight data and multiply-accumulate computation amount of each network layer of the neural network model after training and quantization; S2 divides the quantized weight data of each network layer into multiple weight intervals; S3. For each network layer, replace the default multiplier with a more approximate multiplier for each weight interval, and infer the network accuracy after approximating each weight interval from the obtained neural network model. S4. Weight intervals whose accuracy drops beyond a preset threshold after approximation or whose ranking is a specified proportion behind all network layers are designated as non-fault-tolerant weight intervals, while the remaining weight intervals are designated as fault-tolerant weight intervals. The fault-tolerant weight intervals are divided into multiple groups based on the computational cost of multiplication and accumulation, and then sorted within each group according to the accuracy of the approximation, thus obtaining multiple weight interval queues for the fault-tolerant weight intervals. The non-fault-tolerant weight intervals are sorted according to the accuracy of the approximation to generate a single weight interval queue. S5 merges multiple weight interval queues of fault-tolerant weight intervals and one weight interval queue of non-fault-tolerant weight intervals into a complete weight interval queue. S6, perform AX2 approximation multiplier replacement on the complete weight interval queue until the network accuracy of the replaced neural network model inference reaches the first accuracy decrease threshold. S7, determine whether the number of weight intervals replaced by the approximation multiplier in the AX2 approximation mode is less than the specified proportion of the total number of weight intervals in the complete weight interval queue. If it is true, retain the approximation multiplier replacement in the AX2 approximation mode; otherwise, cancel the approximation multiplier replacement in the AX2 approximation mode. S8, perform approximate multiplier replacement for the complete weight interval queue using the AX1 approximation mode until the network accuracy of the replaced neural network model inference reaches the second accuracy reduction threshold. S9, Generate a reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer of the neural network model. The approximate multiplier replacement configuration is used to control the approximate mode adopted by the reconfigurable approximate multiplier in the neural network acceleration device when performing multiplication operations in each weight interval of each network layer during the inference of the neural network model in the neural network acceleration device.

2. The neural network acceleration method based on a reconfigurable approximate multiplier according to claim 1, characterized in that, In step S2, when dividing the quantized weight data of each network layer into multiple weight intervals, the quantized weight data of each network layer is sorted by size and then divided into multiple weight intervals with the same or equivalent number of weights.

3. The neural network acceleration method based on a reconfigurable approximate multiplier according to claim 1, characterized in that, In step S3, replacing the default multiplier with a more approximate multiplier for each weight interval means replacing the default multiplier with a more approximate multiplier than the AX2 approximation mode for each weight interval.

4. The neural network acceleration method based on a reconfigurable approximate multiplier according to claim 1, characterized in that, In step S5, when merging multiple weight interval queues of fault-tolerant weight intervals and one weight interval queue of non-fault-tolerant weight intervals into a complete weight interval queue, the multiple weight interval queues of fault-tolerant weight intervals are sorted in descending order of multiplication and accumulation computation, and the one weight interval queue of non-fault-tolerant weight intervals is located after the multiple weight interval queues of fault-tolerant weight intervals.

5. The neural network acceleration method based on a reconfigurable approximate multiplier according to claim 1, characterized in that, The reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer of the neural network model generated in step S9 includes the weight boundaries of N-1 adjacent weight intervals and the approximate multiplier approximation layer configuration Conf, where N is the total number of weight intervals. The weight boundaries are used to determine the weight interval to which each weight data belongs. The approximate multiplier approximation layer configuration Conf records the approximate multiplier control signals Ctrl corresponding to N weight intervals. Each approximate multiplier control signal Ctrl is used to control the approximation mode adopted by the reconfigurable approximate multiplier when performing multiplication operations.

6. The neural network acceleration method based on a reconfigurable approximate multiplier according to claim 5, characterized in that, After generating the reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer in step S9, the method further includes saving the reconfigurable approximate multiplier replacement configuration for each weight interval under each network layer in the corresponding network layer parameters.

7. A neural network acceleration device for applying the neural network acceleration method based on a reconfigurable approximate multiplier as described in claim 5 or 6, characterized in that, The system includes multiple control units and an approximation operation array composed of reconfigurable approximation multipliers. The multiple control units are connected to the input terminals of the approximation operation array. Each control unit includes N-1 comparators and a selector. The N-1 comparators compare N-1 pairs of input data to obtain N-1 comparison signals, such that the comparator outputting the comparison signal corresponding to the weight interval to which the input weight data W belongs is 1, and the comparison signals output by the other comparators are 0. The N-1 pairs of input data consist of the input weight data W and the weight boundaries of N-1 adjacent weight intervals. The selector finds the approximation multiplier control signal Ctrl corresponding to the weight interval with a comparison signal of 1 in the approximation layer configuration Conf of the current network layer and sends it to the approximation operation array to control the approximation mode used by the reconfigurable approximation multipliers when performing multiplication operations on the input weight data W.

8. A neural network acceleration device based on a reconfigurable approximate multiplier, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the neural network acceleration method based on a reconfigurable approximate multiplier as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the neural network acceleration method based on a reconfigurable approximate multiplier as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the neural network acceleration method based on a reconfigurable approximate multiplier as described in any one of claims 1 to 6.

Images