Abnormal value perception low-precision training hardware system based on learnable threshold

Abstract

The invention discloses an abnormal value perception low-precision training hardware system based on a learnable threshold value, belongs to the field of neural network training, and aims to solve the problem of how to effectively process an abnormal value while ensuring quantization precision when deep neural network training is performed on edge computing equipment. The method comprises the step of improving the quantitative performance of a model by updating a threshold value of an abnormal value on line, and specifically comprises the following steps: configuring the threshold value of the abnormal value, monitoring and identifying the abnormal value in input calculation data in real time, and performing perceptual quantification on the calculation data according to an identification result; performing data conversion on the calculation data after perceptual quantization, performing data coding, and determining a calculation scene according to a data coding result; allocating calculation tasks to a processing unit cluster based on the calculation scene; calculating an abnormal value gradient through a gradient descent algorithm, and updating a learnable abnormal value threshold value; after network training is completed, performance evaluation is carried out, and results are sorted and output.

Description

Technical Field

[0001] The present invention relates to the field of neural network training, and more specifically to an outlier-aware low-precision training hardware system based on a learnable threshold. Background Art

[0002] Deep neural networks have achieved remarkable success in various task areas, and their accuracy and capabilities are constantly improving. However, as models become deeper and larger, higher requirements are placed on the storage capacity and computing performance of the computing platform. In order to train neural networks on edge devices with limited performance, model compression methods are usually used to reduce performance requirements. Among them, quantization is a commonly used technique that greatly reduces the storage and computing performance requirements by using lower-precision data for network training. In order to ensure quantization accuracy, traditional low-precision quantization methods usually saturate large-amplitude outliers to the quantization threshold. However, these outliers have a high contribution to the accuracy of deep neural networks, especially the large language models that are currently receiving much attention. If these outliers are directly removed, the network performance will be seriously damaged. Therefore, how to effectively handle outliers while ensuring quantization accuracy has become a challenge. Traditional outlier-aware quantization methods collect data distribution online, but this will incur great hardware overhead, resulting in reduced computing performance and longer training cycles.

[0003] The invention patent with announcement number CN116561534B discloses a method and system for improving the accuracy of roadside sensors based on self-supervised learning, wherein the method includes: installing multiple edge computing devices and multiple groups of standard components on the road in the target area, wherein each group of standard components includes multiple standard roadside sensors; for each group of data acquisition components, adjusting the working parameters of the data acquisition components; for each edge computing device, establishing an accuracy improvement model, generating multiple training samples based on historical data collected by multiple groups of data acquisition components corresponding to the edge computing device and historical data collected by at least one group of standard components corresponding to the edge computing device, and training the accuracy improvement model based on multiple training samples; for each edge computing device, calibrating the real-time data collected by multiple groups of data acquisition components corresponding to the edge computing device based on the trained accuracy improvement model. Although the prior art has the advantages of improving the accuracy of existing roadside sensors and reducing the cost of upgrading and transformation. It still fails to solve the problem of how to effectively handle outliers while ensuring quantization accuracy when training deep neural networks on edge computing devices. Therefore, in order to overcome these limitations, the present invention proposes an outlier-aware low-precision training hardware system based on a learnable threshold. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an outlier-aware low-precision training hardware system based on a learnable threshold, which solves the problem of how to effectively handle outliers while ensuring quantization accuracy when training deep neural networks on edge computing devices. During the network training process, outliers in the input calculation data are monitored and identified in real time, and the calculation data is perceived and quantized based on the identification results, which can quickly respond to changes in the data, avoid the negative impact of model performance caused by outliers, and improve the generalization ability of the system.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A low-precision training hardware system for outlier perception based on a learnable threshold includes updating the threshold of outliers online, specifically including the following steps:

[0007] Configure outlier thresholds to monitor and identify outliers in input computing data in real time, and perform perceptual quantification of computing data based on the identification results;

[0008] Perform data conversion and data encoding on the computational data after perceptual quantization, and determine the computational scenario based on the data encoding result;

[0009] Allocate computing tasks to processing unit clusters based on computing scenarios, where the processing unit clusters are used to perform mixed-precision computing;

[0010] Calculate the outlier gradient through the gradient descent algorithm and perform a learnable outlier threshold update;

[0011] After the network training is completed, a performance evaluation is performed, and the trained model, its parameters, and performance evaluation results are organized and output.

[0012] Specifically, the specific steps of perceptual quantization include:

[0013] Apply the clipping function to limit the calculated data to a fixed interval;

[0014] quantizing the cropped computational data through a level mapping function, which converts the cropped data into a low-precision representation;

[0015] Configure a learnable outlier threshold to identify and segment the computational data;

[0016] The calculated data that is smaller than the outlier threshold is marked as normal value, and the linear quantization method is used to perform clipping mapping to obtain the quantization result;

[0017] The calculated data that is greater than or equal to the outlier threshold is marked as an outlier. The outliers are firstly shifted and then clipped and mapped to obtain their quantified results.

[0018] Specifically, the calculation formula for calculating data-aware quantization includes:

[0019] The calculation data marked as normal values, the calculation formula of perceptual quantization is:

[0020]

[0021] Among them, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of ;

[0022] Specifically, the calculation formula for calculating data perception quantization also includes:

[0023] The calculation formula for the data marked as outliers is:

[0024]

[0025] Among them, x o is the calculated data judged as outliers, x max is the maximum value of the received computational data, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x o A low-precision representation of .

[0026] Specifically, the calculation formula for data conversion is:

[0027] The normal value is converted into a low-bit width tensor data representation in the form of OAQ, and the abnormal value is converted into two parts of low-bit width tensor data and high-bit width tensor data in the form of OAQ for calculation, namely:

[0028] N Tensor =OAQ_Tensor(low_precision)

[0029] O Tensor =OAQ_Tensor(low_precision)+threshold(high_precision)

[0030] Among them, N Tensor is a tensor form of the computational data transformation marked as normal value, O TensorIt is the tensor form of the computational data marked as outliers, OAQ_Tensor(low_precision) is the low-bitwidth tensor data representation in OAQ form, and threshold(high_precision) is the high-bitwidth tensor data.

[0031] Specifically, the specific steps of data encoding include:

[0032] Traverse the entire calculation data sequence, mark each element, mark the abnormal value as 1, and mark the normal value as 0;

[0033] Initialize an empty encoding list to store the encoded data;

[0034] Start by calculating the first element of the data sequence and traverse the entire sequence one by one;

[0035] If the current element is a normal value, it is necessary to check whether the next element is also a normal value. If it is a normal value, continue to check the following elements until an abnormal value is encountered or the data sequence ends. The number of consecutive normal values ​​and the value of the last normal value are recorded and added to the encoding list;

[0036] After processing the current element, continue to check the next element until the entire data sequence has been processed and output the result of the encoded sequence.

[0037] Specifically, the steps of determining the calculation scene include:

[0038] The processing unit array fuses adjacent processing units into a processing unit cluster according to the decoded information;

[0039] According to the category of calculation data, the usage of the processing unit cluster is adjusted and the calculation scenarios are allocated. That is, when the operands are all normal values, one processing unit is used to complete the calculation, and multiple groups of calculation data are sent to different processing units for parallel calculation.

[0040] Specifically, the step of determining the calculation scene also includes:

[0041] For the mixed case of normal values ​​and abnormal values ​​in operands, the low-bitwidth tensor data representation and high-bitwidth tensor data of the abnormal values ​​in OAQ form are distributed to different processing units in the processing unit cluster and calculated separately from the normal values;

[0042] For the extreme case where all operands are abnormal values, two processing units in the processing unit cluster are time-multiplexed to combine and calculate the low-bit-width tensor data representation and high-bit-width tensor data in the OAQ form of the input data, respectively.

[0043] Specifically, the specific steps of mixed precision calculation include:

[0044] According to the data transformation of the calculation data, the high-bitwidth part and the low-bitwidth part of the calculation data are obtained, wherein the low-bitwidth part is used as the first input of the mixed precision calculation unit, and the high-bitwidth part is used as the second input of the mixed precision calculation unit. The first input and the second input are calculated using the product-addition array of the mixed precision calculation unit to obtain a low-bitwidth calculation result represented in a low-bitwidth format of the calculation data.

[0045] Specifically, the outlier gradient calculation includes:

[0046] For normal values, the learnable outlier threshold gradient is calculated as:

[0047]

[0048] in, is the partial derivative, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of ;

[0049] For outliers, the learnable outlier threshold gradient is calculated as:

[0050]

[0051] in, is the partial derivative, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of .

[0052] Beneficial effects of the present invention:

[0053] 1. By real-time monitoring and identifying outliers in input data, and using different quantization strategies to handle normal values ​​and outliers, the system can effectively balance computing resource consumption and model performance. For normal values, a linear quantization method is used for cropping and mapping to keep the relative relationship of the data unchanged; for outliers, data translation is performed first and then cropping and mapping is performed to reduce errors. This targeted processing method helps to improve the generalization ability of the model while avoiding the potential negative impact of outliers on model performance. In addition, by dynamically adjusting the outlier threshold to adapt to the characteristics of different data sets, the allocation and use of computing resources are further optimized.

[0054] 2. A learnable outlier threshold is introduced, allowing the model to automatically adjust the threshold size based on the data received. This means that the model can better adapt to new data environments when faced with changing data distributions, thereby improving its generalization ability and stability. Updating the outlier threshold through the gradient descent algorithm ensures that the model is continuously optimized during the training process, further improving the model's prediction accuracy and reliability.

[0055] 3. After the network training is completed, the performance evaluation is carried out, and the trained model, its parameters and performance evaluation results are sorted and output. This not only helps users understand the training effect of the model, but also provides important reference information for subsequent model deployment and application. By conducting a detailed analysis of indicators such as accuracy, recall, and F1 score on the test set, the performance of the model can be comprehensively evaluated, and potential problems can be discovered and resolved in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of a hardware system for outlier-aware low-precision training based on learnable thresholds;

[0057] Figure 2 Flowchart for computational data identification, quantification and coding;

[0058] Figure 3 Schematic diagram of the steps identified for the computational scenario;

[0059] Figure 4 Schematic diagram of mixed precision computing. DETAILED DESCRIPTION

[0060] See also Figure 1 This embodiment introduces a low-precision training hardware system for outlier perception based on a learnable threshold, including updating the threshold of outliers online, specifically including the following steps:

[0061] Configure outlier thresholds to monitor and identify outliers in input computing data in real time, and perform perceptual quantification of computing data based on the identification results;

[0062] Perform data conversion and data encoding on the computational data after perceptual quantization, and determine the computational scenario based on the data encoding result;

[0063] Allocate computing tasks to processing unit clusters based on computing scenarios, where the processing unit clusters are used to perform mixed-precision computing;

[0064] Calculate the outlier gradient through the gradient descent algorithm and perform a learnable outlier threshold update;

[0065] After the network training is completed, a performance evaluation is performed, and the trained model, its parameters, and performance evaluation results are organized and output.

[0066] See also Figure 2 In this embodiment, before the training starts, an initial outlier threshold is set. During the forward propagation of the network, each calculated data point of the input layer is monitored, and for each data point, it is determined whether it exceeds the preset outlier threshold. If the calculated data is less than the outlier threshold, it is marked as a normal value, otherwise it is marked as an outlier. According to the recognition result of the calculated data, the normal value and the outlier of the calculated data are perceptually quantized respectively. For the perceptually quantized calculated data, bit width conversion is performed respectively according to whether it is a normal value or an outlier. Normal values ​​are converted to narrower bit widths to save storage space, while outliers require wider bit widths to maintain accuracy. The input calculated data is encoded to form a sequence that can show which data are normal values ​​and which are outliers. This helps the subsequent processing units to identify and properly process these data. Based on the result of data encoding, the current calculation scenario is determined, and different calculation modes are allocated according to different normal value and outlier distributions, and the processing strategy of adjusting itself is implemented to achieve the best operation effect. Then, receive information about the current computing scenario, including the type of input computing data, that is, whether it contains outliers, and the processing unit cluster configuration of these computing data, and assign the computing tasks to the corresponding processing unit cluster according to the selected processing unit cluster configuration. Ensure that each processing unit has obtained the appropriate task and can perform the task efficiently. During each forward propagation process, record the gradient information of all input computing data points. Calculate the gradient of each parameter with respect to the loss function, including weights and outlier thresholds. For the outlier threshold, update it through the gradient descent algorithm, and control the update speed of the outlier threshold through the learning rate. After the network training is completed, perform performance evaluation, test the trained model with the test data set, and record the model's accuracy, recall rate, F1 score and other indicators to help understand the performance of the model on different categories. Export and save all weight parameters in the model, save the learnable outlier threshold and other related parameters, and record and save the parameters used in the quantization process so that the same quantization effect can be reproduced in actual applications.

[0067] Preferably, the specific steps of perceptual quantization include:

[0068] Applying the clipping function limits the calculated data to a fixed interval to ensure the consistency and stability of the data and avoid quantization errors caused by a large data range. At the same time, this also provides a unified benchmark for subsequent quantization operations.

[0069] The cropped calculation data is quantized through the level mapping function, which converts the cropped data into low-precision representation, that is, converting floating point numbers or other high-precision format data into integers or low-precision floating point formats, etc., to reduce the consumption of storage space and computing resources and improve the running efficiency of the model on resource-constrained devices.

[0070] Configure a learnable outlier threshold to identify and divide the calculated data to determine which calculated data are considered abnormal in the current data distribution, thereby providing special processing strategies for these outliers.

[0071] The calculated data that is less than the outlier threshold is marked as normal value, and the linear quantization method is used to crop and map to obtain its quantization result; for data within the normal range, the linear quantization method can keep the relative relationship and distribution characteristics of the data unchanged, while reducing the loss of data accuracy. This method is suitable for most conventional data and can effectively reduce the storage and computing costs of data.

[0072] The calculated data that is greater than or equal to the outlier threshold is marked as an outlier. The outliers are first shifted and then cropped and mapped to obtain their quantization results. Since outliers often have large values ​​or characteristics that deviate from the normal distribution, directly linearly quantizing them may cause large errors or distortions. Therefore, it is necessary to first shift these data to near 0 and then quantize them. This can better retain the information of the outliers while reducing the quantization error. In addition, the adjusted scaling factor can be flexibly adjusted according to the characteristics of the outliers to achieve a more refined quantization effect.

[0073] Preferably, the calculation formula for calculating data perception quantization includes:

[0074] The calculation data marked as normal values, the calculation formula of perceptual quantization is:

[0075]

[0076] Among them, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of ;

[0077] The calculation formula for the data marked as outliers is:

[0078]

[0079] Among them, x o is the calculated data judged as outliers, x max is the maximum value of the received computational data, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x o A low-precision representation of .

[0080] Different quantization strategies are used according to the relationship between the input data and the outlier threshold α. For normal values, α is directly used as the scaling factor for quantization; for outliers, they are first translated to near 0, and then the adjusted scaling factor (x max -α) to ensure that different types of data can be properly quantized, thereby improving the overall performance and generalization ability of the model.

[0081] Preferably, the specific steps of data conversion include:

[0082] Obtain the classification of calculated data and distinguish between normal values ​​and abnormal values;

[0083] For the calculation data marked as normal values, it is converted into a low-bit-width data representation; for the calculation data marked as abnormal values, it is converted into a high-bit-width data representation; illustratively, for the data marked as normal values, a low-bit-width representation strategy is adopted. That is, these data are converted from the original bit format to a lower bit format, for example, from a 32-bit floating point number to a 16-bit integer or a lower bit representation. This conversion helps to reduce storage requirements and improve computing efficiency while maintaining sufficient accuracy to support conventional computing tasks. For data marked as abnormal values, a high-bit-width representation strategy is implemented. These data need to retain higher precision due to their importance and sensitivity. Therefore, they are converted into a higher-bit data format, for example, from 32-bit floating point numbers to 64 bits, to ensure that important information is not lost when processing these critical data.

[0084] Preferably, the calculation formula for data conversion is:

[0085] The normal value is converted into a low-bit width tensor data representation in the form of OAQ, and the abnormal value is converted into two parts of low-bit width tensor data and high-bit width tensor data in the form of OAQ for calculation, namely:

[0086] N Tensor =OAQ_Tensor(low_precision)

[0087] O Tensor =OAQ_Tensor(low_precision)+threshold(high_precision)

[0088] Among them, N Tensor is a tensor form of the computational data transformation marked as normal value, O Tensor is the tensor form of the computational data marked as outliers, OAQ_Tensor(low_precision) is the low-bitwidth tensor data representation in OAQ form, and threshold(high_precision) is the high-bitwidth tensor data;

[0089] Preferably, data conversion may further include:

[0090] Determine the range of normal and abnormal values, including maximum and minimum values;

[0091] For each normal value x, convert it to a low-bitwidth representation using the following formula:

[0092]

[0093] Among them, Low bitwidth (x) is the normal value represented by the low bit width, and the value of x is mapped to a low bit width integer range, x min is the minimum value of the normal value, x max is the maximum value of the normal value, [·] is the floor rounding function, n low is the number of bits in the low width.

[0094] For each outlier value y, convert it to high-bitwidth representation using the following formula:

[0095]

[0096] Among them, High bitwidth (y) is the abnormal value represented by the high bit width. The value of y is mapped to a high bit width integer range. min is the minimum value of the outliers, y max is the maximum value of the outlier, [·] is the floor function, n high Is the number of bits in the high bit width.

[0097] Preferably, the specific steps of data encoding include:

[0098] Traverse the entire calculation data sequence, mark each element, mark the abnormal value as 1, and mark the normal value as 0. This is to distinguish which are abnormal values ​​and which are normal values, in preparation for subsequent encoding.

[0099] Initialize an empty encoding list to store the encoded data. The encoding list is used to record the data after run-length encoding for subsequent storage and transmission.

[0100] Starting from the first element of the calculated data sequence, we traverse the entire sequence one by one. For each element, we determine whether it is a normal value or an abnormal value, and handle it according to different situations.

[0101] If the current element is a normal value (i.e., the value is 0), you need to check whether the next element is also a normal value. If so, continue to check the following elements until you encounter an abnormal value or the data sequence ends. In this process, record the number of consecutive normal values ​​and the value of the last normal value. Then, add this information to the encoding list. In order to compress the consecutive normal values, only retain their number and the value of the last normal value, thereby reducing the storage space occupied. If the current element is an abnormal value (i.e., the value is 1), there is no need to compress it and add it directly to the encoding list. Because the appearance of abnormal values ​​may mean some special pattern or change in the data sequence, this information needs to be retained for subsequent analysis.

[0102] After processing the current element, you need to continue checking the next element and repeat the above steps until the entire data sequence has been processed to ensure that all elements are correctly encoded into the encoding list.

[0103] The result of the encoding is output for subsequent storage, transmission or further analysis and processing. Through encoding, the storage space occupied by the data sequence is effectively reduced, and the efficiency and speed of data processing are improved. At the same time, since the information of the outliers is retained, the abnormal conditions in the data sequence can be analyzed and studied more deeply.

[0104] Preferably, the step of determining the calculation scene includes:

[0105] The processing unit array selectively merges adjacent processing units into a processing unit cluster according to the decoded information;

[0106] According to the type of calculation data, the use of the processing unit cluster is adjusted to allocate the calculation scenarios. That is, when the operands are all normal values, one processing unit is used to complete the calculation, and multiple groups of calculation data are sent to different processing units for parallel calculation.

[0107] For the mixed case of normal values ​​and abnormal values, the low-bitwidth tensor data representation of the abnormal values ​​in OAQ form and the high-bitwidth tensor data are assigned to different processing units in the processing unit cluster and calculated separately from the normal values ​​to reduce the calculation delay;

[0108] For the extreme case where all operands are abnormal values, two processing units in the processing unit cluster are time-multiplexed to combine and calculate the low-bitwidth tensor data representation and high-bitwidth tensor data in the OAQ form of the input data, respectively, to avoid complex data scheduling logic.

[0109] In this embodiment, in order to efficiently process different types of data calculations, the processing unit array selectively merges adjacent processing units into processing unit clusters based on the decoded information to assist in completing calculations in different task scenarios. Figure 3 , where ① and ③ represent the case where the input operands are all normal values, ② represents the case where the input operands are a mixture of normal values ​​and abnormal values, and ④ represents the case where the input operands are all abnormal values. Figure 3 In (a), since the input operands are all normal values, the calculation can be completed using one processing unit. Therefore, in order to improve hardware utilization, the calculation data ①③ are sent to two processing units in the processing unit cluster for calculation. Figure 3 In (b), there are outliers in the input operands. In order to reduce the computational delay, the outlier OAQ-Tensor and outlier threshold are assigned to different processing units in the processing unit cluster and are calculated separately with the normal values. Figure 3In (c), the input operands are all abnormal values. In order to handle the extreme case where all operands are abnormal values ​​and to avoid complex data scheduling logic, the system reuses two processing units in the processing unit cluster in time to combine and calculate the OAQ-Tensor and abnormal value threshold data of the input data ④ respectively. By fusing adjacent processing units into a processing unit cluster according to the decoding information and dynamically adjusting the use of the processing unit cluster according to different data types, resources can be flexibly allocated to adapt to different computing needs. This dynamic resource management capability is the key to improving hardware utilization and computing efficiency. For the case where all operands are normal values, one processing unit is used to complete the calculation, and multiple groups of calculation data are sent to different processing units for parallel calculation. The calculation throughput can be significantly improved, the waiting time can be reduced, and the overall calculation process can be accelerated. For the mixed case containing normal values ​​and abnormal values, the low-bit width tensor data representation of the abnormal value in the 0AQ form and the high-bit width tensor data are allocated to different processing units in the processing unit cluster and calculated separately from the normal value. It helps to reduce the calculation delay caused by abnormal values ​​and ensure that the calculation of normal values ​​is not affected by abnormal values. For the extreme case where all operands are outliers, by time-reusing two processing units in the processing unit cluster for combination and calculation, complex data scheduling logic can be avoided, the processing flow is simplified, and management costs are reduced, while also maintaining the continuity and stability of the calculation.

[0110] Preferably, the specific steps of mixed precision calculation include:

[0111] According to the data transformation of the calculation data, the high-bitwidth part and the low-bitwidth part of the calculation data are obtained, wherein the low-bitwidth part is used as the first input of the mixed precision calculation unit, and the high-bitwidth part is used as the second input of the mixed precision calculation unit. The first input and the second input are calculated using the product-addition array of the mixed precision calculation unit to obtain a low-bitwidth calculation result represented in a low-bitwidth format of the calculation data.

[0112] In this example, see Figure 4 The processing unit contains four 4-bit multiplication and accumulation units, which perform parallel calculations based on the actual precision of the current input calculation data, and can support 4-bit and 8-bit mixed precision calculations. When the input is 8-bit high-precision data, the data is split into high 4-bit data and low 4-bit data, and sent to the MAC unit to complete the calculation. The shifter and addition tree are used to complete the complete 8-bit calculation for the partial sum result. When the input data is 4 bits, the 4 MAC units can complete the corresponding calculations in parallel and output them. Specifically, an example of mixed precision calculation is as follows:

[0113] A×B=(A H ×2 4 +AL )+(B H ×2 4 +B L )

[0114] =(A H ×B H ×2 8 )+(A H ×B L ×2 4 )+(A L ×B H ×2 4 )+(A L ×B L )

[0115] =(A H ×B H <<8)+(A H ×B L <<4)+(A L ×B H <<4)+(A L ×B L )

[0116] Among them, A and B are two 8-bit high-precision data, which are split into high 4-bit data A H , B H and lower 4 bits of data A L , B L The data is combined and multiplied by this formula to get an accurate result. The multiplication is performed using lower computing resources without sacrificing too much accuracy. The multiplication can be performed more efficiently on some hardware platforms.

[0117] Preferably, for normal values, the calculation formula of the learnable outlier threshold gradient is:

[0118]

[0119] Among them, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of .

[0120] Preferably, for outliers, the calculation formula of the learnable outlier threshold gradient is:

[0121]

[0122] Among them, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of .

[0123] Preferably, the specific steps of performance evaluation include:

[0124] Use the test data set to test the trained model and record the model's accuracy, recall, F1 score and other indicators to help understand the model's performance on different categories. Calculate the model's loss function value on the validation set or test set to evaluate the model's generalization ability, generate a confusion matrix, and analyze the model's classification performance in each category in detail to identify possible misclassifications.

[0125] Export and save all weight parameters in the model, save the learnable outlier threshold and other related parameters, record and save the parameters used in the quantization process, such as the number of quantization bits, quantization range, etc., so as to reproduce the same quantization effect in practical applications.

[0126] Generate a detailed report based on the results of the performance evaluation. The report includes the model's performance indicators, key parameters, important findings during the training process, and recommended improvement measures. Save the trained model and its parameters in standard formats, including ONNX and TensorFlow SavedModel, to facilitate deployment and use on other systems or platforms. Use visualization tools, including TensorBoard and Matplotlib, to display the model's training process, performance changes, and final results to more intuitively understand the model's performance.

[0127] Working principle and effect:

[0128] A low-precision training hardware system based on learnable thresholds for outlier perception is applied in the field of neural network training. It aims to solve the problem of how to effectively handle outliers while ensuring quantization accuracy when training deep neural networks on edge computing devices. It includes updating the threshold of outliers online, specifically including the following steps:

[0129] During the network training process, the input computing data is monitored in real time, and outliers are identified based on the preset outlier threshold. According to the detection results, different quantization strategies are applied to normal values ​​and outliers to balance computing resource consumption and model performance. Data conversion is performed on the computing data after perceptual quantization, including the application of clipping functions and the use of level mapping functions to convert the data into low-precision representation. The computing scenario is determined based on the results of the data conversion to prepare for subsequent task allocation. According to the determined computing scenario, the computing tasks are assigned to the processing unit cluster. The processing unit cluster is used to perform mixed precision calculations, in which normal values ​​are represented by low-bit width tensor data, and outliers are represented by both high-bit width and low-bit width tensor data. The outlier gradient is calculated by the gradient descent algorithm, and the outlier threshold is updated to optimize the model performance. After the network training is completed, performance evaluation is performed, and the trained model, its parameters and performance evaluation results are sorted and output. Through perceptual quantization and mixed precision calculation, unnecessary high-precision calculations are reduced and the utilization efficiency of computing resources is improved. The learnable outlier threshold can be dynamically adjusted according to the characteristics of the data set, making the model more flexible and universal. Although low-precision calculations are used, good model performance can still be maintained through reasonable quantization strategies and outlier processing. Mixed-precision calculations can reduce model size and storage and transmission costs without sacrificing too much performance.

[0130] In summary, a hardware system for low-precision training with outlier awareness based on learnable thresholds achieves significant improvement in computational efficiency while ensuring model performance, and is suitable for resource-constrained environments.

[0131] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A low-precision training hardware system for outlier perception based on learnable thresholds, characterized in that: Including updating the threshold of outliers online, specifically including the following steps: Configure outlier thresholds to monitor and identify outliers in input computing data in real time, and perform perceptual quantification of computing data based on the identification results; Perform data conversion and data encoding on the computational data after perceptual quantization, and determine the computational scenario based on the data encoding result; Allocate computing tasks to processing unit clusters based on computing scenarios, where the processing unit clusters are used to perform mixed-precision computing; Calculate the outlier gradient through the gradient descent algorithm and perform a learnable outlier threshold update; After the network training is completed, a performance evaluation is performed, and the trained model, its parameters, and performance evaluation results are organized and output.

2. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 1, characterized in that: The specific steps of the perceptual quantization include: Apply the clipping function to limit the calculated data to a fixed interval; quantizing the cropped computational data through a level mapping function, which converts the cropped data into a low-precision representation; Configure a learnable outlier threshold to identify and segment the computational data; The calculated data that is smaller than the outlier threshold is marked as normal value, and the linear quantization method is used to perform clipping mapping to obtain the quantization result; The calculated data that is greater than or equal to the outlier threshold is marked as an outlier. The outliers are firstly shifted and then clipped and mapped to obtain their quantified results.

3. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 2, characterized in that: The calculation formula for calculating data perception quantization includes: The calculation data marked as normal values, the calculation formula of perceptual quantization is: Among them, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of .

4. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 3, characterized in that: The calculation formula for calculating data perception quantization also includes: The calculation formula for the data marked as outliers is: Among them, x o is the calculated data judged as outliers, x max is the maximum value of the received computational data, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x o A low-precision representation of .

5. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 1, characterized in that: The calculation formula for the data conversion is: The normal value is converted into a low-bit width tensor data representation in the form of OAQ, and the abnormal value is converted into two parts of low-bit width tensor data and high-bit width tensor data in the form of OAQ for calculation, namely: N Tensor =OAQ_Tensor(low_precision) O Tensor =OAQ_Tensor(low_precision)+threshold(high_precision) Among them, N Tensor is a tensor form of the computational data transformation marked as normal value, O Tensor It is the tensor form of the computational data marked as outliers, OAQ_Tensor(low_precision) is the low-bitwidth tensor data representation in OAQ form, and threshold(high_precision) is the high-bitwidth tensor data.

6. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 1, characterized in that: The specific steps of data encoding include: Traverse the entire calculation data sequence, mark each element, mark the abnormal value as 1, and mark the normal value as 0; Initialize an empty encoding list to store the encoded data; Start by calculating the first element of the data sequence and traverse the entire sequence one by one; If the current element is a normal value, it is necessary to check whether the next element is also a normal value. If it is a normal value, continue to check the following elements until an abnormal value is encountered or the data sequence ends. The number of consecutive normal values ​​and the value of the last normal value are recorded and added to the encoding list; After processing the current element, continue to check the next element until the entire data sequence has been processed and output the result of the encoded sequence.

7. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 1, characterized in that: The step of determining the computing scenario includes: The processing unit array fuses adjacent processing units into a processing unit cluster according to the decoded information; According to the category of calculation data, the usage of the processing unit cluster is adjusted and the calculation scenarios are allocated. That is, when the operands are all normal values, one processing unit is used to complete the calculation, and multiple groups of calculation data are sent to different processing units for parallel calculation.

8. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 1, characterized in that: The step of determining the computing scenario also includes: For the mixed case of normal values ​​and abnormal values ​​in operands, the low-bitwidth tensor data representation and high-bitwidth tensor data of the abnormal values ​​in OAQ form are distributed to different processing units in the processing unit cluster and calculated separately from the normal values; For the extreme case where all operands are abnormal values, two processing units in the processing unit cluster are time-multiplexed to combine and calculate the low-bit-width tensor data representation and high-bit-width tensor data in the OAQ form of the input data, respectively.

9. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 1, characterized in that: The specific steps of the mixed precision calculation include: According to the data transformation of the calculation data, the high-bitwidth part and the low-bitwidth part of the calculation data are obtained, wherein the low-bitwidth part is used as the first input of the mixed precision calculation unit, and the high-bitwidth part is used as the second input of the mixed precision calculation unit. The first input and the second input are calculated using the product-addition array of the mixed precision calculation unit to obtain a low-bitwidth calculation result represented in a low-bitwidth format of the calculation data.

10. The outlier-aware low-precision training hardware system based on a learnable threshold as claimed in claim 1, characterized in that: The outlier gradient calculation includes: For normal values, the learnable outlier threshold gradient is calculated as: in, is the partial derivative, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of ; For outliers, the learnable outlier threshold gradient is calculated as: in, is the partial derivative, x N is the calculated data judged as normal value, α is the learnable outlier threshold, the clipping function [x, 1] clips the data to [-1, 1], and then the data is mapped to the quantization level Q through the mapping function Π(·) b , b is the data bit width, is x N A low-precision representation of .

Images