Anomaly-aware low-precision training hardware system based on learnable threshold

Abstract

The application discloses a kind of based on abnormal value perception low-precision training hardware system of learnable threshold, belongs to neural network training field, it aims at solving when carrying out deep neural network training on edge computing device, how to effectively handle abnormal value while guaranteeing quantization precision Problem.It includes by updating the threshold of abnormal value to improve model quantization performance, specifically includes the following steps: configure abnormal value threshold real-time monitoring and identify abnormal value in input calculation data, and according to the identification result, the calculation data is perceived quantization;The calculation data after perception quantization is carried out data conversion, and data coding is carried out, and the calculation scene is determined according to data coding result;Based on calculation scene, assign calculation task to processing unit cluster;Abnormal value gradient is calculated by gradient descent algorithm, and learnable abnormal value threshold update is carried out;After network training is completed, performance evaluation is carried out, and the result is arranged and output.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of neural network training, more particularly to an abnormal value perception low-precision training hardware system based on a learnable threshold. BACKGROUND

[0002] Deep neural networks have achieved remarkable success in various task domains, and their accuracy and capabilities continue to improve. However, as models become deeper and larger, higher demands are placed on the storage capacity and computing performance of computing platforms. In order to perform neural network training on edge devices with limited performance, model compression methods are often used to reduce performance requirements. Among them, quantization is a commonly used technique that uses lower precision data for network training, thereby greatly reducing the demand for storage and computing performance. Traditional low-precision quantization methods often saturate large-amplitude abnormal values to the quantization threshold in order to ensure quantization accuracy. However, these abnormal values have a high contribution to the accuracy of deep neural networks, especially current widely attention large language models. If these abnormal values are directly excluded, it will seriously damage the network performance. Therefore, how to effectively handle abnormal values while ensuring quantization accuracy has become a challenge. Traditional abnormal value-aware quantization methods will statistically analyze the data distribution online, which will result in a huge hardware overhead and lead to a decrease in computing performance and an extension of the training period.

[0003] The invention patent with publication number CN116561534B discloses a method and system for improving the precision of roadside sensors based on self-supervised learning, wherein the method comprises: installing multiple edge computing devices and multiple standard components on the roads in the target area, wherein each standard component includes multiple standard roadside sensors; for each data collection component, adjusting the working parameters of the data collection component; for each edge computing device, establishing a precision improvement model, generating multiple training samples based on the historical data collected by the multiple data collection components corresponding to the edge computing device and the historical data collected by at least one standard component corresponding to the edge computing device, and training the precision improvement model based on the multiple training samples; for each edge computing device, based on the trained precision improvement model, calibrate the real-time data collected by the multiple data collection components corresponding to the edge computing device. Although the prior art has the advantages of improving the precision of existing roadside sensors and reducing the cost of improvement and reconstruction. It still cannot solve the problem of how to effectively handle abnormal values while ensuring quantization accuracy when performing deep neural network training on edge computing devices. Therefore, in order to overcome these limitations, the present application proposes an abnormal value perception low-precision training hardware system based on a learnable threshold. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application aims to provide a low-precision training hardware system based on a learnable threshold value, which solves the problem of how to effectively process abnormal values while ensuring quantization accuracy when training a deep neural network on an edge computing device. During network training, abnormal values in input calculation data are monitored and identified in real time, and the calculation data is perceptually quantized according to the identification results, which can quickly respond to changes in the data and avoid negative effects on model performance caused by abnormal values, thereby improving the generalization ability of the system.

[0005] To achieve the above object, the present application provides the following technical scheme:

[0006] A low-precision training hardware system based on a learnable threshold value, comprising the following steps of updating the threshold value of abnormal values online:

[0007] Configure an abnormal value threshold to monitor and identify abnormal values in input calculation data in real time, and perceptually quantize the calculation data according to the identification results;

[0008] Convert the perceptually quantized calculation data and encode the data, and determine the calculation scenario according to the data encoding result;

[0009] Assign the calculation task to the processing unit cluster based on the calculation scenario, wherein the processing unit cluster is used for mixed precision calculation;

[0010] Calculate the abnormal value gradient by the gradient descent algorithm and update the learnable abnormal value threshold;

[0011] After the network training is completed, the performance is evaluated, and the trained model, its parameters and the performance evaluation results are sorted and output.

[0012] Specifically, the specific steps of perceptual quantization include:

[0013] Apply a clipping function to limit the calculation data within a fixed range;

[0014] Quantize the clipped calculation data by a level mapping function, which converts the clipped data into low-precision representation;

[0015] Configure a learnable abnormal value threshold for identifying and dividing the calculation data;

[0016] Mark the calculation data less than the abnormal value threshold as normal value, and use linear quantization method to clip and map to obtain the quantization result;

[0017] Mark the calculation data greater than or equal to the abnormal value threshold as abnormal value, first perform data translation on the abnormal value, and then perform clipping and mapping to obtain the quantization result.

[0018] Specifically, the calculation formula of the data-aware quantization includes:

[0019] The calculation formula of the data-aware quantization for the normal value marked data is:

[0020]

[0021] wherein, x N is the calculation data determined as the normal value, a is a learnable abnormal value threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q b by the mapping function Π(·), b is a data bit width, is a low-precision representation of x N ;

[0022] Specifically, the calculation formula of the data-aware quantization includes:

[0023] The calculation formula of the data-aware quantization for the abnormal value marked data is:

[0024]

[0025] wherein, x o is the calculation data determined as the abnormal value, x max is the maximum value of the received calculation data, a is a learnable abnormal value threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q b by the mapping function Π(·), b is a data bit width, is a low-precision representation of x o .

[0026] Specifically, the calculation formula of the data conversion is:

[0027] The normal value is converted into a low-bit-width tensor data representation in the form of OAQ, and the abnormal value is converted into two parts of low-bit-width tensor data and high-bit-width tensor data in the form of OAQ participating in the calculation, that is:

[0028] N Tensor = OAQ_Tensor(low_precision)

[0029] O Tensor = OAQ_Tensor(low_precision) + threshold(high_precision)

[0030] wherein, N Tensor is the tensor form of the calculation data marked as the normal value converted, O Tensoris the tensor form of the calculation data marked as an outlier, OAQ_Tensor(low_precision) is the low-bit-width tensor data representation in the OAQ form, and threshold(high_precision) is the high-bit-width tensor data.

[0031] Specifically, the specific steps of data encoding include:

[0032] Traverse the entire calculation data sequence, and mark each element, marking outliers as 1 and normal values as 0.

[0033] Initialize an empty encoding list for storing the encoded data.

[0034] Start from the first element of the calculation data sequence, and traverse the entire sequence one by one.

[0035] If the current element is a normal value, check whether the next element is also a normal value, if it is a normal value, continue to check the subsequent elements until an outlier or the end of the data sequence is encountered, record the number of consecutive normal values and the value of the last normal value, and add them to the encoding list.

[0036] After processing the current element, continue to check the next element until the entire data sequence is processed.

[0037] Specifically, the steps of determining the calculation scenario include:

[0038] The processing unit array fuses adjacent processing units into a processing unit cluster according to the decoding information.

[0039] According to the category of the calculation data, adjust the usage of the processing unit cluster, and allocate the calculation scenario, that is, for the case where both operands are normal values, use one processing unit to complete the calculation, and send multiple groups of calculation data into different processing units for parallel calculation.

[0040] Specifically, the steps of determining the calculation scenario further include:

[0041] For the mixed case where the operands are normal values and outliers respectively, the low-bit-width tensor data representation in the OAQ form and the high-bit-width tensor data of the outliers are allocated to different processing units in the processing unit cluster for calculation with the normal values respectively.

[0042] For the extreme case where both operands are outliers, reuse two processing units in the processing unit cluster in time to combine and calculate the low-bit-width tensor data representation in the OAQ form and the high-bit-width tensor data of the input data.

[0043] Specifically, the specific steps of mixed-precision calculation include:

[0044] According to the data transformation of the calculation data, a high-bit-width part and a low-bit-width part of the calculation data are obtained, the low-bit-width part is taken as a first input of the mixed precision calculation unit, the high-bit-width part is taken as a second input of the mixed precision calculation unit, the first input and the second input are calculated by using a product-sum array of the mixed precision calculation unit, and a low-bit-width calculation result in a low-bit-width format representation of the calculation data is obtained.

[0045] Specifically, the abnormal value gradient calculation includes:

[0046] For normal values, the calculation formula of the learnable abnormal value threshold gradient is:

[0047]

[0048] wherein, is the partial derivative, x N is the calculation data determined as normal values, alpha is the learnable abnormal value threshold, the clipping function [x,1] clips the data to [-1,1], and then the data is mapped into the quantization level Q b by the mapping function Π(·), b is the data bit width, is the low-precision representation of x N ;

[0049] For abnormal values, the calculation formula of the learnable abnormal value threshold gradient is:

[0050]

[0051] wherein, is the partial derivative, x N is the calculation data determined as normal values, alpha is the learnable abnormal value threshold, the clipping function [x,1] clips the data to [-1,1], and then the data is mapped into the quantization level Q b by the mapping function Π(·), b is the data bit width, is the low-precision representation of x N .

[0052] The beneficial effects of the present application are:

[0053] 1. By monitoring and identifying outliers in input data in real-time, and handling normal values and outliers with different quantization strategies, the system can effectively balance computational resource consumption and model performance. For normal values, a linear quantization method is used for clipping mapping to maintain the relative relationship of the data unchanged; for outliers, data translation is performed before clipping mapping to reduce errors. This targeted processing method helps to improve the generalization ability of the model, while avoiding the potential negative impact of outliers on model performance. In addition, by dynamically adjusting the outlier threshold to adapt to the characteristics of different data sets, the allocation and use of computational resources are further optimized.

[0054] 2. The introduction of a learnable outlier threshold allows the model to automatically adjust the threshold size based on the received data. This means that the model can better adapt to new data environments when faced with changing data distributions, thereby improving its generalization ability and stability. By updating the outlier threshold through gradient descent algorithm, the model continuously optimizes during training, further improving the prediction accuracy and reliability of the model.

[0055] 3. After the network training is completed, the performance evaluation is carried out, and the trained model and its parameters and performance evaluation results are sorted and output. Not only helps users understand the training effect of the model, but also provides important reference information for subsequent model deployment and application. By analyzing the accuracy, recall rate, F1 score and other indicators on the test set, the performance of the model can be evaluated comprehensively, and potential problems can be found and solved in a timely manner. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A flowchart of a hardware system for outlier-aware low-precision training based on a learnable threshold;

[0057] Figure 2 A flowchart of computing data identification, quantization and encoding;

[0058] Figure 3 A schematic diagram of the steps for determining the computing scenario;

[0059] Figure 4 A schematic diagram of mixed-precision computing. DETAILED DESCRIPTION

[0060] For reference Figure 1 , the embodiment introduces a hardware system for outlier-aware low-precision training based on a learnable threshold, which includes updating the threshold of outliers online, specifically including the following steps:

[0061] Configure the outlier threshold to monitor and identify outliers in the input computing data in real time, and perform perceptual quantization on the computing data according to the identification result;

[0062] The data of the perceptual quantized calculation data is converted and encoded, and the calculation scene is determined according to the data encoding result;

[0063] The calculation task is allocated to the processing unit cluster based on the calculation scene, and the processing unit cluster is used for performing mixed precision calculation;

[0064] The abnormal value gradient is calculated by the gradient descent algorithm, and the learnable abnormal value threshold is updated;

[0065] After the network training is completed, the performance is evaluated, and the trained model and its parameters and performance evaluation results are sorted and output.

[0066] Please refer to Figure 2 In this embodiment, before the training starts, an initial abnormal value threshold is set, and during the forward propagation of the network, each calculation data point of the input layer is monitored, and for each data point, it is judged whether it exceeds the preset abnormal value threshold. If the calculation data is less than the abnormal value threshold, it is marked as a normal value, otherwise it is marked as an abnormal value, and according to the identification result of the calculation data, the normal value and the abnormal value of the calculation data are respectively perceptual quantized. For the perceptual quantized calculation data, according to whether it is a normal value or an abnormal value, bit width conversion is performed respectively. The normal value is converted to a narrower bit width to save storage space, while the abnormal value needs a wider bit width to maintain accuracy. The input calculation data is encoded to form a sequence, which can show which data is normal and which is abnormal. To help the subsequent processing unit identify and appropriately process these data. Based on the result of data encoding, the current calculation scene is determined, different calculation modes are allocated according to different normal value and abnormal value distribution, and the processing strategy is adjusted to achieve the best running effect. Then, receive information about the current calculation scene, including the type of input calculation data, i.e. whether it contains abnormal values, and the processing unit cluster configuration of these calculation data, according to the selected processing unit cluster configuration, the calculation task is allocated to the corresponding processing unit cluster. Ensure that each processing unit obtains appropriate tasks and can efficiently execute tasks. During each forward propagation process, record the gradient information of all input calculation data points. Calculate the gradient of each parameter with respect to the loss function, including weights and abnormal value threshold. For the abnormal value threshold, it is updated by the gradient descent algorithm, and the learning rate controls the update speed of the abnormal value threshold. After the network training is completed, the performance is evaluated, the test data set is used to test the trained model, and the accuracy, recall rate, F1 score and other indicators of the model are recorded to help understand the performance of the model on different categories. All weight parameters in the model are exported and saved, the learnable abnormal value threshold and other related parameters are saved, the parameters used in the quantization process are recorded and saved, so as to reproduce the same quantization effect in actual application.

[0067] Preferably, the specific steps of perceptual quantization include;

[0068] The clipping function is applied to limit the calculation data within a fixed range, ensuring data consistency and stability, and avoiding quantization errors caused by excessive data range. At the same time, it also provides a unified benchmark for subsequent quantization operations.

[0069] The calculation data after clipping is quantized by the level mapping function, which converts the clipped data into low-precision representation, i.e., converts floating-point numbers or other high-precision format data into integers or low-precision floating-point numbers, etc. To reduce storage space and computing resource consumption, and improve the running efficiency of the model on resource-constrained devices.

[0070] A learnable outlier threshold is configured to identify and divide the calculation data, to determine which calculation data is considered abnormal in the current data distribution, and to provide special processing strategies for these outliers.

[0071] The calculation data less than the outlier threshold is marked as normal value, and the linear quantization method is used for clipping mapping to obtain its quantization result; for data within the normal range, the linear quantization method can maintain the relative relationship and distribution characteristics of the data unchanged, while reducing the precision loss of the data. This method is suitable for most regular data and can effectively reduce the storage and computing cost of data.

[0072] The calculation data greater than or equal to the outlier threshold is marked as an outlier, and the data is first translated, and then clipped and mapped to obtain its quantization result. Since outliers often have large values or deviate from the regular distribution, directly quantizing them with linear quantization may cause large errors or distortions. Therefore, these data need to be translated to around 0 first, and then quantized. This can better preserve the information of outliers while reducing quantization errors. In addition, the adjusted scaling factor can be flexibly adjusted according to the characteristics of outliers to achieve more refined quantization effects.

[0073] Preferably, the calculation formula of perceptual quantization of calculation data includes:

[0074] The calculation formula of perceptual quantization of calculation data marked as normal value is:

[0075]

[0076] where x N is the calculation data determined as normal value, α is the learnable outlier threshold, the clipping function [x,1] clips the data to [-1,1], and then the mapping function Π(·) maps the data to the quantization level Q b , b is the data bit width, is x N a low-precision representation of x

[0077] The calculation data marked as an outlier, the perception quantization formula is:

[0078]

[0079] where x o is the calculation data determined as an outlier, x max is the maximum value of the received calculation data, and a is a learnable outlier threshold value. The clipping function [x,1] clips the data to [-1,1], and then the data is mapped to the quantization level Q b by the mapping function Π(·). b is the data bit width, is a low-precision representation of x o .

[0080] According to the relationship between the input data and the outlier threshold a, different quantization strategies are used for processing. For normal values, a is directly used as a scaling factor for quantization; for outliers, they are first shifted to the vicinity of 0, and then quantized using the adjusted scaling factor (x max - a). This ensures that different types of data can be properly quantized, thereby improving the overall performance and generalization ability of the model.

[0081] Preferably, the specific steps of data conversion include:

[0082] Obtain the classification of the calculation data, distinguishing between normal values and outliers;

[0083] For calculation data marked as normal values, convert them to low-bit-width data representation; for calculation data marked as outliers, convert them to high-bit-width data representation; for example, for data marked as normal values, a low-bit-width representation strategy is adopted. That is, these data are converted from the original bit format to a lower bit format, for example, from 32-bit floating-point numbers to 16-bit integers or lower bit representations. This conversion helps to reduce storage requirements and improve computational efficiency, while maintaining sufficient precision to support routine computing tasks. For data marked as outliers, a high-bit-width representation strategy is performed. These data need to be kept at a higher precision due to their importance and sensitivity. Therefore, they are converted to a higher bit format, such as from 32-bit floating-point numbers to 64-bit, to ensure that important information is not lost when processing these critical data.

[0084] Preferably, the calculation formula for data conversion is:

[0085] The normal values are converted into low-bit-width tensor data representation in OAQ form, and the abnormal values are converted into two parts of low-bit-width tensor data and high-bit-width tensor data in OAQ form, which participate in the calculation, i.e.:

[0086] N Tensor = OAQ_Tensor(low_precision)

[0087] O Tensor = OAQ_Tensor(low_precision) + threshold(high_precision)

[0088] Wherein, N Tensor is the tensor form of the calculation data marked as normal value conversion, O Tensor is the tensor form of the calculation data marked as abnormal value, OAQ_Tensor(low_precision) is the low-bit-width tensor data representation in OAQ form, and threshold(high_precision) is the high-bit-width tensor data.

[0089] Preferably, the data conversion can also include:

[0090] Determining the value range of normal values and abnormal values, including the maximum value and the minimum value;

[0091] For each normal value x, it is converted into low-bit-width representation using the following formula:

[0092]

[0093] Wherein, Low bitwidth (x) is the low-bit-width representation of the normal value, which maps the value of x to a low-bit-width integer range, x min is the minimum value of the normal value, x max is the maximum value of the normal value, [·] is the floor function, and n low is the number of bits of the low-bit-width.

[0094] For each abnormal value y, it is converted into high-bit-width representation using the following formula:

[0095]

[0096] Wherein, High bitwidth (y) is the high-bit-width representation of the abnormal value, which maps the value of y to a high-bit-width integer range, y min is the minimum value of the abnormal value, y max is the maximum value of the abnormal value, [·] is the floor function, and n high is the number of bits of the high-bit-width.

[0097] Preferably, the specific steps of data encoding include:

[0098] Traverse the entire calculation data sequence and mark each element, marking the abnormal value as 1 and the normal value as 0. This is to distinguish which are abnormal values ​​and which are normal values, preparing for subsequent encoding.

[0099] Initialize an empty code list to store the encoded data. The code list is used to record the data after run-length encoding for subsequent storage and transmission.

[0100] Starting from the first element of the calculated data sequence, we traverse the entire sequence one by one. For each element, we determine whether it is a normal value or an abnormal value and deal with it according to different situations.

[0101] If the current element is a normal value (i.e., the value is 0), it is necessary to check whether the next element is also a normal value. If so, continue to check the subsequent elements until an abnormal value is encountered or the data sequence ends. In this process, the number of consecutive normal values ​​and the value of the last normal value are recorded. Then, this information is added to the encoding list. This can compress the consecutive normal values ​​and only retain their number and the value of the last normal value, thereby reducing the storage space occupied. If the current element is an abnormal value (i.e., the value is 1), there is no need to perform compression processing and add it directly to the encoding list. Because the appearance of abnormal values ​​may mean a special pattern or change in the data sequence, this information needs to be retained for subsequent analysis.

[0102] After processing the current element, you need to continue checking the next element and repeat the above steps until the entire data sequence has been processed, ensuring that all elements are correctly encoded into the encoding list.

[0103] The encoded results are output for subsequent storage, transmission, or further analysis and processing. Encoding effectively reduces the storage space occupied by the data sequence and improves the efficiency and speed of data processing. Furthermore, because outlier information is retained, more in-depth analysis and research can be conducted on anomalies within the data sequence.

[0104] Preferably, the step of determining the calculation scenario includes:

[0105] The processing unit array selectively merges adjacent processing units into a processing unit cluster based on the decoded information;

[0106] Adjust the usage of processing unit clusters based on the type of computational data and allocate computational scenarios. For example, if all operands are normal values, use one processing unit to complete the computation, and send multiple sets of computational data to different processing units for parallel computation.

[0107] For the mixed case where the operands are normal and abnormal values respectively, the low-bit-width and high-bit-width tensor data in OAQ form of abnormal values are allocated to different processing units in the processing unit cluster, and are calculated with normal values respectively to reduce the calculation delay.

[0108] For the extreme case where the operands are abnormal values, two processing units in the processing unit cluster are time-multiplexed to combine and calculate the low-bit-width and high-bit-width tensor data in OAQ form of input data respectively to avoid complex data scheduling logic.

[0109] In the embodiment, to efficiently process different types of data calculation, the processing unit array selectively fuses adjacent processing units into a processing unit cluster according to the decoding information to assist in completing the calculation under different task scenarios. Please refer to Figure 3 , wherein ① and ③ represent the case where the input operands are normal values, ② represents the case where the input operands are normal and abnormal values, and ④ represents the case where the input operands are abnormal values, and the details are as follows Figure 3 In (a), since the input operands are normal values, one processing unit can complete the calculation, so in order to improve the hardware utilization, the calculation data ① and ③ are sent to two processing units in the processing unit cluster for calculation. The details are as follows Figure 3 In (b), the input operands have abnormal values, in order to reduce the calculation delay, the OAQ-Tensor of abnormal values and the abnormal value threshold are allocated to different processing units in the processing unit cluster, and are calculated with normal values respectively. The details are as follows Figure 3In (c), the input operands are all outliers. To handle this extreme case and avoid complex data scheduling logic, the system time-multiplexes two processing units in the processing unit cluster to combine and compute the OAQ-Tensor of the input data and the outlier threshold data, respectively. By fusing adjacent processing units into a processing unit cluster according to the decoding information and dynamically adjusting the usage of the processing unit cluster according to different data types, resources can be flexibly allocated to adapt to different computing needs. This dynamic resource management capability is the key to improving hardware utilization and computing efficiency. For the case where the operands are all normal values, one processing unit is used to complete the computation, and multiple groups of computing data are sent to different processing units for parallel computation. This can significantly improve the computing throughput and reduce the waiting time, thereby accelerating the overall computing process. For the mixed case containing normal values and outliers, the low-width tensor data in the OAQ form of the outlier and the high-width tensor data are allocated to different processing units in the processing unit cluster for separate computation with the normal values. This helps to reduce the computing delay caused by outliers and ensures that the computation of normal values is not affected by outliers. For the extreme case where the operands are all outliers, time-multiplexing two processing units in the processing unit cluster is used to combine and compute, which can avoid complex data scheduling logic, simplify the processing flow, reduce management costs, and maintain the continuity and stability of the computation.

[0110] Preferably, the specific steps of the mixed-precision computation include:

[0111] According to the data transformation of the computing data, the high-width part and the low-width part of the computing data are obtained, wherein the low-width part is the first input of the mixed-precision computation unit, the high-width part is the second input of the mixed-precision computation unit, and the product accumulation array of the mixed-precision computation unit is used to compute the first input and the second input to obtain the low-width computation result in the low-width format representation of the computing data.

[0112] In this embodiment, please refer to Figure 4 The processing unit includes four 4-bit multiply-accumulate units, which can support 4-bit and 8-bit mixed-precision computation according to the actual precision of the current input computing data. When the input is 8-bit high-precision data, the data is split into high-4-bit data and low-4-bit data, which are sent to the MAC unit for computation, and then the partial sum result is used to complete the complete 8-bit computation using a shifter and an addition tree. When the input data is 4-bit, the four MAC units can perform parallel computation and output. Specifically, an example of mixed-precision computation is as follows:

[0113] A x B = (A H x 2 4 + AL )+(B H ×2 4 +B L )

[0114] =(A H ×B H ×2 8 )+(A H ×B L ×2 4 )+(A L ×B H ×2 4 )+(A L ×B L )

[0115] =(A H ×B H <<8)+(A H ×B L <<4)+(A L ×B H <<4)+(A L ×B L )

[0116] where A and B are two 8-bit high-precision data, which are split into high 4-bit data A H , B H and low 4-bit data A L , B L . These data are combined by the formula to perform multiplication operation, and finally an accurate result is obtained. In the case of not sacrificing too much precision, multiplication operation is performed using lower computing resources. Multiplication operation can be more effectively performed on some hardware platforms.

[0117] Preferably, for normal values, the calculation formula of the learnable abnormal value threshold gradient is:

[0118]

[0119] where x N is the calculation data determined as normal value, a is the learnable abnormal value threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q b by the mapping function Π(·), b is the data bit width, is the low-precision representation of x N .

[0120] Preferably, for abnormal values, the calculation formula of the learnable abnormal value threshold gradient is:

[0121]

[0122] where x N is the calculated data determined as normal value, a is the learnable outlier threshold, the clipping function [x,1] clips the data to [-1,1], and then the data is mapped to the quantization levels Q b by the mapping function Π(·), b is the data bit width, is the low-precision representation of x N .

[0123] Preferably, the specific steps of performance evaluation include:

[0124] Test the trained model using the test dataset, record the accuracy, recall rate, F1 score and other indicators of the model to help understand the performance of the model on different categories. Calculate the loss function value of the model on the validation set or test set to evaluate the generalization ability of the model, generate a confusion matrix, and analyze the classification performance of the model on each category in detail to identify possible misclassification situations.

[0125] Export and save all weight parameters in the model, save the learnable outlier threshold and other related parameters, record and save the parameters used in the quantization process, such as quantization bit number, quantization range, etc. to reproduce the same quantization effect in actual application.

[0126] According to the results of performance evaluation, generate a detailed report. The report includes the performance indicators of the model, key parameters, important findings during training, and recommended improvement measures. Save the trained model and its parameters in a standard format, including ONNX, TensorFlow SavedModel, for easy deployment and use on other systems or platforms. Use visualization tools, including TensorBoard, Matplotlib, to display the training process, performance changes and final results of the model to more intuitively understand the performance of the model.

[0127] Working principle and effect:

[0128] An outlier-aware low-precision training hardware system based on learnable threshold is applied to the field of neural network training, aiming to solve the problem of how to effectively handle outliers while ensuring quantization accuracy when training deep neural networks on edge computing devices. It includes updating the threshold of outliers online, specifically including the following steps:

[0129] During the network training process, the input computational data is monitored in real time, and outliers are identified based on a pre-set outlier threshold. According to the detection results, different quantization strategies are applied to normal values and outliers to balance computational resource consumption and model performance. The perceptually quantized computational data is subjected to data conversion, including the application of a clipping function and the use of a level mapping function, to convert the data into a low-precision representation. The computational scenario is determined based on the results of the data conversion, in preparation for subsequent task allocation. The computational tasks are allocated to processing unit clusters based on the determined computational scenario. The processing unit clusters are used to perform mixed-precision computation, in which normal values are represented using low-bit-width tensor data, while outliers are represented using both high-bit-width and low-bit-width tensor data. The outlier gradient is calculated using a gradient descent algorithm, and the outlier threshold is updated to optimize model performance. After the network training is completed, performance evaluation is performed, and the trained model, its parameters, and the performance evaluation results are output. Through perceptual quantization and mixed-precision computation, unnecessary high-precision computation is reduced, and the utilization efficiency of computational resources is improved. The learnable outlier threshold can be dynamically adjusted according to the characteristics of the data set, making the model more flexible and versatile. Although low-precision computation is used, through reasonable quantization strategies and outlier processing, good model performance can still be maintained. Through mixed-precision computation, the model size can be reduced without sacrificing too much performance, reducing storage and transmission costs.

[0130] In summary, a hardware system for outlier-aware low-precision training based on a learnable threshold significantly improves computational efficiency while ensuring model performance, making it suitable for resource-constrained environments.

[0131] The above only describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solutions falling within the scope of the present application should be considered within the protection scope of the present application. It should be noted that for ordinary technical personnel in the technical field, some improvements and refinements without departing from the principles of the present application should also be considered within the protection scope of the present application.

Claims

1. A learnable threshold based outlier aware low-precision training hardware system, comprising: The method comprises updating a threshold value of an abnormal value through online updating, and specifically comprises the following steps: An abnormal value threshold value is configured to monitor and identify abnormal values in input calculation data in real time, and the calculation data is perceptually quantized according to the identification result; The specific steps of the perceptual quantization include: A clipping function is applied to limit the calculation data in a fixed interval; The calculation data after clipping is quantized through a level mapping function, and the mapping function converts the clipped data into a low-precision representation; A learnable abnormal value threshold value is configured to identify and divide the calculation data; The calculation data less than the abnormal value threshold value is marked as a normal value, and a linear quantization method is used for clipping mapping to obtain the quantization result; The calculation data greater than or equal to the abnormal value threshold value is marked as an abnormal value, and the abnormal value is first translated in data, and then clipped and mapped to obtain the quantization result; The calculation data after perceptual quantization is converted and encoded, and the calculation scenario is determined according to the data encoding result; The specific steps of determining the calculation scenario include: The processing unit array fuses adjacent processing units into a processing unit cluster according to the decoding information; According to the category of the calculation data, the use mode of the processing unit cluster is adjusted, and the calculation scenario is allocated, that is, for the case where both operands are normal values, one processing unit is used to complete the calculation, and multiple groups of calculation data are sent into different processing units for parallel calculation; For the mixed case where the operands are normal values and abnormal values, the low-bit-width tensor data representation and the high-bit-width tensor data of the abnormal value in the form of OAQ are allocated to different processing units in the processing unit cluster for calculation with the normal value respectively; For the extreme case where both operands are abnormal values, two processing units in the processing unit cluster are time-multiplexed to combine and calculate the low-bit-width tensor data representation and the high-bit-width tensor data of the input data in the form of OAQ respectively; Based on the calculation scenario, a calculation task is allocated to the processing unit cluster, and the processing unit cluster is used to perform mixed-precision calculation; The abnormal value gradient is calculated through a gradient descent algorithm, and the learnable abnormal value threshold value is updated; The abnormal value gradient calculation includes: For normal values, the calculation formula of the learnable abnormal value threshold gradient is: ； wherein, is a partial derivative, is a calculated data determined as a normal value, is a learnable abnormal value threshold, a clipping function clips the data to and then maps the data to quantization levels by a mapping function , is a data bit width, is a low precision representation of ;​ For abnormal values, the calculation formula of the learnable abnormal value threshold gradient is: ； wherein, is a partial derivative, is a calculated data determined as a normal value, is a learnable abnormal value threshold, a clipping function clips data to and then maps data to quantization levels by a mapping function , is a data bit width, is a low precision representation of ;​ After the network training is completed, performance evaluation is performed, and the trained model, its parameters and the performance evaluation result are arranged and output.

2. The learnable threshold based outlier-aware low-precision training hardware system of claim 1, wherein, The calculation formula of the perceptual quantization of the calculation data includes: The calculation formula of the perceptual quantization of the calculation data marked as a normal value is: ； wherein, is the calculated data that is determined to be a normal value, is a learnable threshold for abnormal values, a clipping function clips the data to and then maps the data to quantization levels by a mapping function , , is the data bit width, is a low precision representation of .

3. The learnable threshold based outlier-aware low-precision training hardware system of claim 2, wherein, The calculation formula of the perceptual quantization of the calculation data includes: The calculation formula of the perceptual quantization of the calculation data marked as an abnormal value is: ； wherein, is the calculated data determined as an outlier, is the maximum value of the received calculated data, is the learnable outlier threshold, the clipping function clips the data to and then maps the data into quantization levels by the mapping function , is the data bit width, is the low precision representation of .​ 4. The learnable threshold based outlier-aware low-precision training hardware system of claim 1, wherein, The calculation formula of the data conversion is: The normal value is converted into a low-bit-width tensor data representation in the form of OAQ, and the abnormal value is converted into a low-bit-width tensor data and a high-bit-width tensor data in the form of OAQ to participate in calculation, that is: ； ； wherein, is a tensor form of the computed data conversion labeled as normal value, is a tensor form of the computed data labeled as abnormal value, is a low bit-width tensor data representation in the form of OAQ, is a high bit-width tensor data.

5. The learnable threshold based outlier aware low-precision training hardware system as claimed in claim 1, wherein, The specific steps of the data encoding include: The entire calculation data sequence is traversed, and each element is labeled, with abnormal values labeled as 1 and normal values labeled as 0; An empty encoding list is initialized to store the encoded data; Starting from the first element of the data sequence, traverse the entire sequence one by one; If the current element is a normal value, check whether the next element is also a normal value, if it is a normal value, continue to check the subsequent elements until an abnormal value is encountered or the data sequence ends, record the number of consecutive normal values and the value of the last normal value, and add them to the encoding list; After processing the current element, continue to check the next element until the entire data sequence is processed.

6. The learnable threshold based outlier aware low-precision training hardware system of claim 1, wherein, The specific steps of the mixed precision calculation include: According to the data transformation of the calculation data, obtain the high-bit-width part and the low-bit-width part of the calculation data, wherein the low-bit-width part is used as the first input of the mixed precision calculation unit, the high-bit-width part is used as the second input of the mixed precision calculation unit, the product accumulation array of the mixed precision calculation unit is used to calculate the first input and the second input, and the low-bit-width calculation result in the low-bit-width format representation of the calculation data is obtained.

Images