High-precision floating point in-memory computing system and method based on outlier separation

Abstract

The invention belongs to the technical field of in-memory computing, and discloses a high-precision floating-point in-memory computing system and method based on outlier separation, and the method comprises the steps: analyzing the index distribution of an input data group, and judging that an outlier extreme value exists and starting an outlier mode when a difference value between a maximum index and an in-group average or second maximum index is detected to exceed a preset threshold value; in the in-memory calculation main body path, mask processing is carried out on the detected extreme maximum value at the input end, pre-alignment and displacement are carried out on the remaining normal data again according to the local maximum index of the normal data, and parallel mantissa multiply-accumulate calculation is completed in the array; meanwhile, in the digital bypass path, an external digital multiplier is used for independently calculating the product of the outlier; the calculation results of the two paths are aligned and added in a digital domain, and a final high-precision calculation result is output. Through the design of dynamic outlier detection and a hybrid calculation path, the precision of floating point in-memory calculation is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of in-memory computing technology, and in particular relates to a high-precision floating-point in-memory computing system and method based on outlier separation. Background Technology

[0002] With the development of artificial intelligence algorithms, in order to maintain inference accuracy, Digital CIM (Digital In-Memory Computation) has gradually evolved from fixed-point computation to support floating-point (FP) computation. Existing floating-point in-memory computation IPs typically adopt a pre-aligned block floating-point (MAC) architecture. That is, before performing multiply-accumulate (MAC) operations, a set of input data and weights are grouped (e.g., 32 data per group), and the maximum exponent in the group is found as the shared exponent. The remaining data in the group are then right-shifted and aligned according to the difference between their own exponents and the shared maximum exponent. Subsequently, the mantissas are multiplied and accumulated within the CIM array. However, the input data in neural networks typically exhibits high variance and a long-tailed distribution. When a set of input data contains "outliers" where the exponents are significantly larger than the others, the mantissas of the remaining normal values ​​are shifted to the right (e.g., more than 16 bits) to align with this largest shared exponent, causing the mantissas to underflow to zero. This "large number eats small number" phenomenon means that the in-memory computing array actually only calculates the outlier, losing the calculation results of other valid information. This leads to serious errors in the final summation result, greatly reducing the model's inference accuracy. Summary of the Invention

[0003] The purpose of this invention is to provide a high-precision floating-point in-memory computing system and method based on outlier separation, so as to solve the problems existing in the prior art.

[0004] To achieve the above objectives, this invention provides a high-precision floating-point in-memory computing system based on outlier separation, comprising:

[0005] The Computational Memory Array (CIM Array) is used to store weight data and perform parallel multiply-accumulate operations.

[0006] The preprocessing and control module is used to analyze the exponential distribution of the input data and is configured with outlier detection logic. When an outlier is detected in the input data group whose maximum exponent exceeds a preset threshold, an outlier control signal is generated.

[0007] The input driving module, in response to the outlier control signal, replaces outlier data in the input data group with zero values ​​and inputs the processed remaining data to the storage computing array;

[0008] An auxiliary computing module is used to read the weight data corresponding to the outlier data and perform independent multiplication operations while the storage computing array is performing calculations.

[0009] The accumulator fusion module is used to add the output of the storage computing array to the output of the auxiliary computing module.

[0010] Optionally, the auxiliary computing module utilizes the read / write ports of the storage computing array to read outlier weight data in parallel during the array's MAC operation cycle, without occupying additional computing cycles.

[0011] Optionally, in outlier mode, the preprocessing and control module uses the largest exponent among the remaining non-outlier data as a benchmark to perform mantissa shift pre-alignment on the remaining data, thereby preventing the loss of mantissa precision of the non-outlier data.

[0012] On the other hand, to achieve the above objectives, this invention provides a high-precision floating-point in-memory calculation method based on outlier separation, comprising:

[0013] Dynamic outlier detection: During the pre-alignment stage, a threshold determination logic is added. The difference between the largest index and the average index (or the second largest index) is calculated. If it exceeds a preset threshold, the group is determined to contain an "outlier extreme value".

[0014] Hybrid Computing Path:

[0015] Path A (CIM Main Flow): The detected extreme large value channels are masked at the input (i.e., replaced with 0), the remaining normal data (such as the remaining 31) are re-pre-aligned and shifted, and the mantissa MAC calculation is completed in parallel within the CIM array.

[0016] Path B (Digital Bypass Flow): Utilizing the CIM architecture's support for "parallel computation and read / write", while Path A is being computed, the weight data corresponding to the extreme large values ​​is read from the array, and the product of the pair of data is computed separately using an external digital multiplier.

[0017] Result fusion: The CIM accumulation result of path A and the digital multiplication result of path B are finally added in the digital domain to obtain a high-precision MAC output.

[0018] The technical effects of this invention are as follows:

[0019] This invention significantly improves the accuracy of floating-point in-memory computation through dynamic outlier detection and a hybrid computation path design. It avoids the phenomenon of entire small signal groups being "zeroed out" due to a single extreme value, preserving the effective computational accuracy of the vast majority of data within the group. The advancement of this technology lies in its proposed hybrid architecture of "detection-separation-parallel computation-fusion." Utilizing the in-memory computing array's ability to support "parallel computation and read / write," it bypasses digital logic to process a very small number of outliers, allowing the in-memory computing array to process the main data after outlier removal. This design achieves high-precision floating-point matrix-vector multiplication without blocking the main array's computational throughput or significantly increasing hardware overhead and latency. This technology is primarily applied in high-performance AI chip design, digital signal processing, and high-precision floating-point in-memory computing circuit design, and is particularly suitable for deep learning accelerators with high inference accuracy requirements. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 Extreme activation values ​​lead to a loss of precision.

[0023] Figure 2 This is a schematic diagram of the statistical results of the input distribution of a typical MLP network in an embodiment of the present invention;

[0024] Figure 3 This is a flowchart of the extreme value separation-parallel computation-fusion operation in an embodiment of the present invention. Detailed Implementation

[0025] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0026] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.

[0027] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.

[0028] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0029] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] Example 1

[0031] like Figure 1 - Figure 3 As shown, this embodiment provides a high-precision floating-point in-memory computing system based on outlier separation. It belongs to the hardware acceleration design scheme of Generalized Matrix-Vector Multiplication (GEMV) and can be applied to all neural network models involving matrix-vector multiplication. The system specifically includes:

[0032] Computational Memory Array (CIM Array): Used to store weight data and perform parallel multiply-accumulate operations;

[0033] Preprocessing and control module: used to analyze the exponential distribution of input data, configured with outlier detection logic, when outlier data with a maximum exponent exceeding a preset threshold is detected in the input data group, an outlier control signal is generated;

[0034] Input driving module: In response to the outlier control signal, it replaces outlier data in the input data group with zero values ​​and inputs the processed remaining data to the storage computing array;

[0035] Auxiliary calculation module: used to read the weight data corresponding to the outlier data and perform independent multiplication operations while the storage computing array is performing calculations;

[0036] Accumulation and fusion module: Used to add the output results of the storage computing array to the output results of the auxiliary computing module.

[0037] In practice, the auxiliary computing module utilizes the read / write ports of the storage computing array to read outlier weight data in parallel during the array's MAC operation cycle, without occupying additional computing cycles.

[0038] In practice, the preprocessing and control module, in outlier mode, uses the largest exponent among the remaining non-outlier data as a benchmark to perform mantissa shift pre-alignment on the remaining data, thereby preventing the loss of mantissa precision of the non-outlier data.

[0039] It is feasible to execute the "in-memory MAC calculation of the remaining data" and the "digital field multiplication of the maximum value" in parallel, and finally add the two results together to obtain the final calculation result.

[0040] This embodiment is primarily applied to AI inference scenarios requiring the processing of high dynamic range data, particularly for deep neural network models such as Multilayer Perceptrons (MLPs) and Transformers. In these models, activation values ​​often exhibit a long-tailed distribution; while a very small percentage of extreme large values ​​represent a tiny fraction, they can severely degrade the accuracy of traditional block-to-floating-point architectures. The solution provided in this embodiment is well-suited for integration into high-energy-efficiency AI acceleration chips at the edge or in the cloud. It maintains the high energy efficiency and high density advantages of in-memory computing while addressing the accuracy bottleneck of traditional architectures when dealing with complex data distributions. As large models increasingly demand higher computational accuracy, this technology ensures that in-memory computing chips maintain high fidelity when performing complex inference tasks, demonstrating broad commercial application prospects. It can drive the application of in-memory computing technology in high-end scenarios with stringent requirements for both accuracy and energy efficiency, such as autonomous driving, intelligent security, and data centers.

[0041] The specific implementation process of this embodiment includes:

[0042] Step 1: Preprocessing and Detection

[0043] The input buffer reads in a set of floating-point data (e.g., 32).

[0044] The exponent comparator finds the maximum exponent E. MAX And calculate the difference ∆E between it and other index characteristics (such as the mean or second largest value) within the group.

[0045] Decision: If ∆E > Threshold (e.g., the shift limit corresponding to the mantissa bit width), enter outlier mode; otherwise, execute regular block floating-point CIM mode.

[0046] Step 2: Data Separation (Outlier Pattern)

[0047] Lock the index position (Index k) of extremely large values.

[0048] CIM Input Reconstruction: Set the input data at index k to 0 (Input masking).

[0049] Realignment: The remaining 31 data points are aligned based on their local maximum exponents. Since extreme values ​​have been removed, the shared exponent is smaller, and the tails of the remaining data points are preserved instead of being zeroed out.

[0050] Step 3: Parallel computation execution

[0051] In-memory matrix multiplication is performed: the reconstructed 31 data points are fed into the CIM array and subjected to parallel MAC operations with the stored weight mantissas to obtain the result Partial_SUM. CIM .

[0052] Digital logic assistance: Utilizing the read port of the CIM array, the corresponding weight W is directly read based on the index k. k Input is performed in the digital logic unit outside the array. k ×W k Precise floating-point / fixed-point multiplication yields Product. Outlier .

[0053] Leveraging the ability of CIM IP to perform computation and read / write operations in parallel, the two steps described above are executed concurrently in time, without overriding each other and without incurring additional time overhead.

[0054] Step 4: Result Fusion

[0055] Partial_SUM CIM With Product Outlier Align the results, add them together, and output the final result.

[0056] This embodiment statistically analyzes the input distribution of a typical MLP network. The data is mainly concentrated between 0.001 and 1, while the extreme large values ​​of 10 to 10000 account for a very low percentage (<2%).

[0057] Probabilistic inference: At a granularity of 32 channels per group, the probability of a single extreme value appearing in each group of data is the highest, and multiple extreme values ​​are extremely rare. Therefore, optimizing only for a single outlier can yield extremely high returns.

[0058] Statistics show that the probability of extreme values ​​is extremely low (<2%). The digital bypass logic not only has a small area, but is also activated only under specific conditions.

[0059] In summary, this embodiment addresses the issue of decreased overall computational accuracy caused by a few extreme values ​​when processing high dynamic range input data in existing block floating-point CIMs. By dynamically monitoring and using hybrid computation paths, it significantly improves the computational accuracy of floating-point matrix-vector multiplication without substantially increasing hardware overhead. This embodiment avoids the phenomenon of "zeroing out" of an entire group of small signals due to a single extreme value, preserving the effective computational accuracy of the vast majority of data within the group. This embodiment fully utilizes the parallel read / write and computation capabilities of the CIM array, and through pipelined operations, it ensures that processing outliers does not block the computational throughput of the main array.

[0060] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A high-precision floating-point in-memory computing system based on outlier separation, characterized in that, include: The storage computing array is configured to store weighted data and perform parallel multiply-accumulate operations; The preprocessing and control module is configured to detect the maximum exponent in the input floating-point data. When the difference between the feature values ​​of the maximum exponent and the remaining exponents is greater than a preset threshold, it is determined to be outlier data. Generate an outlier control signal, replace outlier data in the input floating-point data with zero values ​​according to the outlier control signal, and input the remaining floating-point data into the storage computing array for multiplication and accumulation operations; The auxiliary calculation module is configured to read the weight data corresponding to the outlier data during the multiplication-accumulation operation of the storage computing array, and perform independent multiplication operations based on the read weight data; The accumulator fusion module is configured to add the output of the storage computing array to the output of the auxiliary computing module to obtain the final multiply-accumulate result.

2. The high-precision floating-point in-memory computing system based on outlier separation according to claim 1, characterized in that, The preprocessing and control module specifically includes: The preprocessing unit is configured to analyze the exponential distribution of the input floating-point data, determine the maximum exponent based on the exponential distribution analysis results, and determine the maximum exponent as outlier data when the difference between the feature values ​​of the maximum exponent and the remaining exponents is greater than a preset threshold, and generate an outlier control signal. The data separation unit is configured to lock the index position of the outlier data according to the outlier control signal, replace the floating-point data corresponding to the index position with zero value, perform mantissa shift pre-alignment on the remaining floating-point data based on the maximum exponent in the remaining floating-point data, and input the aligned floating-point data into the storage computing array for multiplication and accumulation operations.

3. The high-precision floating-point in-memory computing system based on outlier separation according to claim 1, characterized in that, During the multiplication and accumulation operation of the storage computing array, the auxiliary computing module uses the read and write ports of the storage computing array to read the weight data corresponding to the outlier data in parallel.

4. A high-precision floating-point in-memory computation method based on outlier separation, applied to a high-precision floating-point in-memory computation system based on outlier separation as described in any one of claims 1-3, characterized in that, include: S1. Obtain the floating-point data to be processed; S2. Input the floating-point data into the exponent comparator to analyze the exponent distribution. Determine the maximum exponent based on the analysis results. Calculate the difference between the characteristic values ​​of the maximum exponent and the remaining exponents. If the difference is less than a preset threshold, execute the conventional block floating-point digital domain in-memory calculation mode. If the difference is greater than the preset threshold, determine that the maximum exponent is outlier data, generate an outlier control signal, and execute step S3. S3. Replace outlier data in the input floating-point data with zero values ​​according to the outlier control signal, and input the remaining floating-point data into the storage computing array for multiplication and accumulation operations to obtain the first calculation result; During the multiplication-accumulation operation in the storage computing array, the weight data corresponding to the outlier data is read, and an independent multiplication operation is performed based on the read weight data to obtain the second calculation result; S4. Align the first calculation result and the second calculation result, add them together, and output the final multiplication and accumulation operation result.

5. The high-precision floating-point in-memory calculation method based on outlier separation according to claim 4, characterized in that, Step S3 involves replacing outlier data in the input floating-point data with zero values ​​according to the outlier control signal, specifically including: Based on the outlier control signal, the index position of the outlier data is locked, and the floating-point data corresponding to the index position is replaced with zero value.

6. The high-precision floating-point in-memory calculation method based on outlier separation according to claim 5, characterized in that, Step S3 involves performing independent multiplication operations based on the read weighted data, specifically including: Based on the index position, the weight corresponding to the outlier data is read using the read / write port of the storage computing array. Based on the weight data corresponding to the outlier data, precise floating-point / fixed-point multiplication is performed in the digital logic unit outside the storage computing array to obtain the second calculation result.

Images