A data processing method, system, chip and storage medium
By identifying and processing micro-scaled floating-point format data in the computing chip and using multiplication-free conversion processing, the problem of computing chips being incompatible with the MXFP format is solved, achieving efficient format conversion and cost savings.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XINXIN HANGTU (SUZHOU) TECHNOLOGY CO LTD
- Filing Date
- 2026-03-10
- Publication Date
- 2026-06-16
Smart Images
Figure CN121809355B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of chip technology, and in particular to a data processing method, system, chip, and storage medium. Background Technology
[0002] With the explosive growth in the number of parameters in deep learning models, computational efficiency and storage bandwidth have become major bottlenecks. To address this issue, organizations such as the Open Compute Project have proposed the Microscaling Floating-Point (MXFP) format for quantizing model parameters.
[0003] The core feature of the MXFP format is block scaling, where a group of consecutive data shares a scaling factor, and each element retains only a low-precision mantissa. However, current mainstream computing chips are mainly designed based on the standard floating-point format, leading to hardware incompatibility issues with the MXFP format. Redesigning native hardware to support the MXFP format would require overhauling the existing data path, resulting in long design cycles, high costs, and the inability to protect existing intellectual property cores. Summary of the Invention
[0004] This invention provides a data processing method, system, chip, and storage medium that enables existing computing chips to support the MXFP format, avoiding problems such as excessive costs caused by redesigning hardware.
[0005] According to one aspect of the present invention, a data processing method is provided, comprising:
[0006] Obtain the raw data in micro-scaling floating-point format, and identify the target format parameters and target granularity parameters corresponding to the raw data;
[0007] Floating-point data and the corresponding scaling factor are extracted from the original data according to the target format parameters;
[0008] Based on the target granularity parameter, the floating-point data and the corresponding scaling factor are queued and reassembled respectively to obtain the floating-point data sequence and the scaling factor sequence;
[0009] Perform multiplication-free conversion processing based on the floating-point data sequence, the scaling factor sequence, and the target format parameters to obtain target data in standard floating-point format.
[0010] In this embodiment, by setting format parameters and granularity parameters, and extracting floating-point data and scaling factors based on format parameters, and caching and reorganizing floating-point data and scaling factors based on granularity parameters, format conversion from micro-scaling floating-point format to standard floating-point format can be achieved. This enables existing computing chips to support the MXFP format, avoiding problems such as excessive costs caused by redesigning hardware.
[0011] According to another aspect of the present invention, a data processing system is provided, including a controller and various multiplication-free conversion units;
[0012] The controller is configured to acquire raw data in micro-scaling floating-point format and identify the target format parameter and target granularity parameter corresponding to the raw data; extract floating-point data and the corresponding scaling factor from the raw data according to the target format parameter; cache and reassemble the floating-point data and the corresponding scaling factor in a queue according to the target granularity parameter to obtain a floating-point data sequence and a scaling factor sequence; and send the floating-point data sequence, the scaling factor sequence, and the target format parameter to each multiplication-free conversion unit.
[0013] Each multiplication-free conversion unit is used to perform multiplication-free conversion processing based on the floating-point data sequence, the scaling factor sequence, and the target format parameters to obtain target data in standard floating-point format.
[0014] According to another aspect of the present invention, a chip is provided, comprising the data processing system described in any embodiment of the present invention.
[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program configured to cause a processor to execute and implement the data processing method described in any embodiment of the present invention.
[0016] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the data processing method described in any embodiment of the present invention.
[0017] The technical solution of this invention involves acquiring raw data in micro-scaling floating-point format and identifying the target format parameters and target granularity parameters corresponding to the raw data; extracting floating-point data and corresponding scaling factors from the raw data according to the target format parameters; queuing and reassembling the floating-point data and corresponding scaling factors according to the target granularity parameters to obtain a floating-point data sequence and a scaling factor sequence; performing multiplication-free conversion processing based on the floating-point data sequence, scaling factor sequence, and target format parameters to obtain target data in standard floating-point format; by adding format conversion logic from micro-scaling floating-point format to standard floating-point format, existing computing chips can support the MXFP format, avoiding problems such as excessive costs caused by redesigning hardware.
[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart of a data processing method provided according to Embodiment 1 of the present invention;
[0021] Figure 2 This is a flowchart of a data processing method provided according to Embodiment 2 of the present invention;
[0022] Figure 3 This is a flowchart of a method for obtaining a target scaling factor pointer according to Embodiment 2 of the present invention;
[0023] Figure 4 This is a schematic diagram of the structure of a data processing system according to Embodiment 3 of the present invention;
[0024] Figure 5 This is a schematic diagram of the structure of a chip according to Embodiment 4 of the present invention. Detailed Implementation
[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0026] It should be noted that the terms "first," "second," "modification," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0027] Example 1
[0028] Figure 1 This is a flowchart of a data processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where micro-scaling floating-point format data is converted into standard floating-point format to achieve model inverse quantization. This method can be executed by a data processing system, which can be configured in a chip. Figure 1 As shown, the method includes:
[0029] S110. Obtain the raw data in micro-scaling floating-point format, and identify the target format parameters and target granularity parameters corresponding to the raw data.
[0030] The micro-scaling floating-point format can be MXFP8, MXFP4, or similar formats. In this embodiment, to support different data formats and scaling granularities, format parameters and granularity parameters can be set. For example, format descriptors and granularity descriptors can be set to represent format parameters and granularity parameters, respectively. The format parameters define the bit field structure and transformation of the data stream, specifically including the exponent bit width, mantissa bit width, and offset parameters. The granularity parameters define the scaling factor (Scale) update logic, specifically including the step size threshold, broadcast range value, and event enable parameters.
[0031] In this embodiment, raw data in MXFP format can be received from the input bus, and the target format parameters and target granularity parameters of the current configuration can be read from the programmable configuration register group.
[0032] The identification of the target format parameters and target granularity parameters corresponding to the original data may include:
[0033] Identify the target format identifier and target granularity identifier corresponding to the raw data;
[0034] The target format parameters are obtained based on the target format identifier and the preset correspondence between format identifiers and format parameters. The target granularity parameters are obtained based on the target granularity identifier and the preset correspondence between granularity identifiers and granularity parameters.
[0035] In this embodiment, the correspondence between format identifiers and format parameters, as well as the correspondence between granularity identifiers and granularity parameters, can be preset and stored in the programmable configuration register group. Therefore, after acquiring the raw data, the target format identifier and target granularity identifier can be identified based on the format definition of the raw data. Then, based on the identification results, the corresponding target format parameters and target granularity parameters can be determined by searching the preset correspondence in the programmable configuration register group.
[0036] The advantage of the above settings is that they can improve the versatility of format conversion and support different data formats.
[0037] S120. Extract floating-point data and corresponding scaling factors from the original data according to the target format parameters.
[0038] Specifically, firstly, the original microdata and scaling factor are separated from the original data; then, the original microdata is decoded based on the target format parameters to extract floating-point data. The floating-point data consists of a sign component, an exponent component, and a mantissa component.
[0039] Extracting floating-point data and the corresponding scaling factor from the original data according to the target format parameters may include:
[0040] The original data is split into original microdata and corresponding scaling factors;
[0041] Based on the target format parameters, the exponent bit width and mantissa bit width are obtained, and based on the exponent bit width and mantissa bit width, the sign component, exponent component and mantissa component are extracted from the original micro data.
[0042] The floating-point data is generated based on the sign component, the exponent component, and the mantissa component.
[0043] In this embodiment, hardware circuitry can be used to extract floating-point data. Specifically, firstly, according to the format definition of the original data, it is split into a set of original microdata and a scaling factor, and a synchronization tag is added to each. Next, the exponent bit width and mantissa bit width are extracted from the target format parameters, and a dynamic barrel shifter is used to automatically right-shift each original microdata to align the exponent bits according to the exponent bit width, thereby extracting the corresponding exponent component. Then, a programmable bitmask generator is used to generate the current AND mask according to the mantissa bit width, and the AND mask is bitwise ANDed with the original microdata to extract the corresponding mantissa component. Finally, the sign component at the beginning of the original microdata can be concatenated with the extracted corresponding exponent and mantissa components to generate the corresponding floating-point data.
[0044] It is worth noting that regardless of the specific MXFP format of the raw data, the physical transistor path through which the data flows is exactly the same, only the control signals are different. This avoids the waste of on-chip area caused by designing separate circuits for each format.
[0045] The advantage of the above settings is that they enable adaptive parsing of raw data in multiple formats, avoiding the problem of format rigidity.
[0046] S130. Based on the target granularity parameter, the floating-point data and the corresponding scaling factor are queued and reassembled respectively to obtain the floating-point data sequence and the scaling factor sequence.
[0047] It's important to note that the logical definition granularity of the MXFP format often doesn't match the physical throughput granularity of the computing core. The computing core can easily match an incorrect scaling factor, leading to computational errors or efficiency collapse. Here, logical definition granularity represents the block size of the front-end input data stream, i.e., the number of data items sharing a scaling factor. Physical throughput granularity represents the amount of data that a computing core can process in parallel per cycle.
[0048] To address the aforementioned issues, this embodiment employs two parallel and independent data queues. The first data queue stores floating-point data and buffers high-throughput mantissa streams. The second data queue stores scaling factors and maintains the logical ownership of data blocks. Typically, the data queues can be First-In-First-Out (FIFO) queues.
[0049] Specifically, after acquiring the floating-point data and scaling factor, they can be stored in their respective data queues. Then, based on the physical throughput granularity M of the computing core, the first data queue can be controlled to output M floating-point data in each clock cycle to form a floating-point data sequence. At the same time, the second data queue can be controlled to output the scaling factor corresponding to the M floating-point data to form a scaling factor sequence.
[0050] S140. Perform multiplication-free conversion processing based on the floating-point data sequence, the scaling factor sequence, and the target format parameters to obtain target data in standard floating-point format.
[0051] Specifically, first, the current floating-point data and the corresponding current scaling factor are extracted from the floating-point data sequence and scaling factor sequence, respectively. Next, the bias parameter and mantissa width are parsed from the target format parameters. Then, exponential addition and subtraction operations are performed on the exponent component, current scaling factor, and bias parameter in the current floating-point data to obtain the result. Finally, based on the mantissa width, the result is concatenated with the sign component and mantissa component in the current floating-point data to obtain intermediate data. Then, the mantissa of the intermediate data is padded with zeros according to the standard floating-point format to generate the final target data in the standard floating-point format. The standard floating-point format can be a 16-bit floating-point format (FP16), a 32-bit floating-point format (FP32), etc.
[0052] The process of performing multiplication-free conversion based on the floating-point data sequence, the scaling factor sequence, and the target format parameters to obtain target data in standard floating-point format may include:
[0053] Based on the floating-point data sequence and the correspondence between floating-point data and scaling factors, obtain the boundary pointers corresponding to each scaling factor in the scaling factor sequence, and generate a boundary pointer sequence based on each boundary pointer;
[0054] Data allocation is performed in each multiplication-free conversion unit according to the floating-point data sequence, the scaling factor sequence, and the boundary pointer sequence to obtain the floating-point data and scaling factor corresponding to each multiplication-free conversion unit;
[0055] The bias parameter and mantissa width are obtained according to the target format parameter, and the multiplication-free conversion unit performs multiplication-free conversion processing according to the corresponding floating-point data and scaling factor, as well as the bias parameter and the mantissa width, to obtain target data in standard floating-point format.
[0056] In this embodiment, a dynamic routing mechanism based on boundary pointers is introduced to ensure that each multiplication-free conversion unit can correctly match the scaling factor. Specifically, while controlling the data queue to output the floating-point data sequence and the scaling factor sequence, a set of boundary pointers is output to form a boundary pointer sequence. The boundary pointer Pi indicates the valid cutoff index of the i-th scaling factor Si in the floating-point data sequence, corresponding one-to-one with the scaling factor Si. Then, through the digital comparator set in each multiplication-free conversion unit Lanej, the unit identifier j is compared with the value of each boundary pointer Pi, and the boundary pointer corresponding to each multiplication-free conversion unit is determined according to the numerical comparison result, and then the corresponding scaling factor is determined based on the boundary pointer. For example, if j is less than or equal to P0, then Lane j is determined to correspond to P0 and S0; if j is greater than P0 and less than or equal to P1, then Lane j is determined to correspond to P1 and S1, and so on.
[0057] While determining the scaling factor corresponding to the multiplication-free conversion unit, the floating-point data at the corresponding position in the floating-point data sequence can be read according to the unit identifier j, and used as the floating-point data corresponding to Lane j. For example, for Lane 0, the 0th floating-point data in the floating-point data sequence is read; for Lane 1, the 1st floating-point data in the floating-point data sequence is read.
[0058] The multiplication-free conversion unit can be a parameterized calculation unit, and its internal circuit structure does not change with the data format. The multiplication-free conversion unit can consist of independent 9-bit adders / subtractors and dynamic shifters / multiplexers. The multiplication-free conversion process can include three steps: exponent path alignment, mantissa path alignment, and sign path pass-through. In the exponent path alignment step, the inputs are the exponent component Exp_{Data}, the scaling factor Exp_{Scale}, and the bias parameter Bias_Config of the floating-point data. Bias_Config is input to the subtraction port of the adder. The adder executes the formula New_Exp = Exp_{Scale} + Exp_{Data} - Bias_Config to calculate the exponent component New_Exp in standard floating-point format. Specifically, when the subformat of MXFP is E4M3 (4-bit Exponent, 3-bit Mantissa), Bias_Config=15; when the subformat is E5M2 (5-bit Exponent, 2-bit Mantissa), Bias_Config=31. Bias_Config=2^N-1, where N is the total number of bits in the exponent component. The multiplication-free conversion unit does not need to "know" whether the current format is E4M3 or E5M2; it is only responsible for subtracting the input value, thus achieving complete decoupling of the logic.
[0059] In the mantissa path alignment step, the input consists of the mantissa component Mant_Data and the mantissa width Mant_Width of the floating-point data. Taking the standard floating-point format FP16 as an example, if Mant_Width = 3 (E4M3), the high 3 bits of Mant_Data are mapped to the high 3 bits (Bits 9-7) of the FP16 mantissa through a dynamic shifter / multiplexer group, and the low 7 bits are forced low (zero padding). If Mant_Width = 2 (E5M2), the high 2 bits of Mant_Data are mapped to the high 2 bits (Bits 9-8) of the FP16 mantissa, and the low 8 bits are forced low. In other words, the extracted Mant_Data is directly fed into the dynamic shifter / multiplexer group, which determines where to append these bits to the mantissa of the standard floating-point format based on Mant_Width.
[0060] In the symbol path pass-through step, the input is the sign component, Sign_Data, of floating-point data. Since the scaling factor for MXFP format is typically defined as a non-negative real number, Sign_Data can be directly mapped to the sign bit (Bit 15) of standard floating-point format. If a signed scaling factor is supported, an XOR logic is added, based on the formula Sign_Out = Sign_DataS. Sign_Scale calculates the sign component Sign_Out in standard floating-point format. Among them, This represents the XOR operation, and Sign_Scale represents the sign component of the scaling factor. Therefore, after the above steps, the target data in standard floating-point format can be finally obtained.
[0061] In this embodiment, by converting multiplication into addition in the exponent field, the use of a floating-point multiplier can be avoided, reducing computational complexity. Secondly, the format conversion process is completely decoupled from the core calculation process, eliminating the need to modify the complex control logic within the core. Furthermore, the independent adder allows exponent calculation to be completed in parallel during the preprocessing stage before the data enters the computation core, ensuring that the data enters the core at full speed and avoiding resource contention. Finally, the adder has very few logic gates, avoiding the wiring complexity and timing risks associated with reusing core resources.
[0062] The technical solution of this invention involves acquiring raw data in micro-scaling floating-point format and identifying the target format parameters and target granularity parameters corresponding to the raw data; extracting floating-point data and corresponding scaling factors from the raw data according to the target format parameters; queuing and reassembling the floating-point data and corresponding scaling factors according to the target granularity parameters to obtain a floating-point data sequence and a scaling factor sequence; performing multiplication-free conversion processing based on the floating-point data sequence, scaling factor sequence, and target format parameters to obtain target data in standard floating-point format; by adding format conversion logic from micro-scaling floating-point format to standard floating-point format, existing computing chips can support the MXFP format, avoiding problems such as excessive costs caused by redesigning hardware.
[0063] Example 2
[0064] Figure 2 This is a flowchart of a data processing method provided in Embodiment 2 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with one or more of the above implementation methods. For example... Figure 2 As shown, the method includes:
[0065] S210. Obtain the raw data in micro-scaling floating-point format, and identify the target format parameters and target granularity parameters corresponding to the raw data.
[0066] S220. Extract floating-point data and corresponding scaling factors from the original data according to the target format parameters.
[0067] S230. Cache the floating-point data to the first data queue and cache the scaling factor to the second data queue.
[0068] The first data queue and the second data queue can be parallel and independent FIFO queues.
[0069] S240. Based on the logical definition granularity of the micro-scaling floating-point format and the physical throughput granularity of the computing core, determine the current granularity remapping mode, and based on the current granularity remapping mode, extract the floating-point data sequence of the current clock cycle from the first data queue.
[0070] It should be noted that the logical definition granularity N of MXFP and the physical throughput granularity M of the computing core are often non-integer multiples or misaligned. To avoid computational errors or efficiency collapse caused by this, this embodiment decouples the "logical definition" of data from the "physical transmission" of hardware. It is important to note that when aggregating or splitting data, the original logical order of the data flow must be strictly maintained, adjusting only the timing and bit width of the physical transmission. This ensures that the correct scaling factor sequence can be derived subsequently based solely on the issue count (Issue_Count), without requiring complex out-of-order rearrangement logic.
[0071] Among them, the granularity remapping mode can include multiple aggregation mode, split adaptation mode and non-aligned recombination mode. The definitions of various modes are shown in Table 1.
[0072] Table 1. Granularity Remapping Mode Definition
[0073]
[0074] The multiple aggregation mode is suitable for scenarios where the front-end data blocks are small, the back-end computing core throughput is large, and the multiple relationship (i.e., M equals K multiplied by N, where K is an integer greater than 1) is satisfied. The buffer is configured with a storage depth of at least M data bits. During operation, the buffer continuously receives and temporarily stores K raw MXFP data blocks (containing K independent scaling factors and K multiplied by N raw micro-data). When the Kth data block is written, the buffer sends the accumulated M (i.e., K multiplied by N) raw micro-data along with the K corresponding scaling factors in parallel to the subsequent M-way multiplication-free conversion unit within one clock cycle. By adopting the "K-in, 1-out" strategy, the problem that small-granularity data streams cannot feed large-throughput computing cores is solved, significantly improving the utilization of back-end computing resources.
[0075] Secondly, the split-adaptation mode is suitable for scenarios where the front-end data block is large, the back-end computing core throughput is small, and the multiple relationship is satisfied (i.e., N equals K multiplied by M, where K is an integer greater than 1). The buffer receives a complete raw data block (containing one scaling factor and N raw micro-data). The buffer splits the N raw micro-data according to the bit width M of the computing core, and delivers them in K consecutive clock cycles. In the first clock cycle, the 0th to M-1th data are output, along with the shared scaling factor; in the i-th clock cycle, the (i-1) multiplied by M to i multiplied by M-1 data are output, along with the shared scaling factor again; until the K-th clock cycle is completed. During this process, the buffer only performs time-series slicing on the data, ensuring that each batch of sliced data can obtain the correct scaling factor support in the multiplication-free conversion unit. Through the "1-in-K-out" strategy, the adaptation of the large-granularity compression format to the fine-grained tensor computing core is achieved.
[0076] Furthermore, the unaligned reassembly mode is suitable for complex scenarios where there is no integer multiple relationship between the input granularity N and the output granularity M (i.e., N is not divisible by M, and M is not divisible by N). The buffer is configured as a FIFO-based streaming structure. Let L be the least common multiple of N and M. The system takes L data points as a complete large cycle. At the input end, the system continuously receives L divided by N original front-end data blocks. Inside the buffer, data blocks from different sources are tightly packed across the boundaries of the original scaling factor. At the output end, once the amount of valid data in the buffer reaches or exceeds M, the system immediately truncates the first M data points for transmission, and the remaining data is retained at the beginning of the buffer for subsequent concatenation.
[0077] S250. Obtain the transmit count value corresponding to the first data queue in the current clock cycle, and obtain the target scaling factor pointer according to the transmit count value and the target granularity parameter, and extract the scaling factor sequence of the current clock cycle from the second data queue based on the target scaling factor pointer.
[0078] The issue count (Issue_Count) indicates how many floating-point data items were output from the first data queue during the current clock cycle. The scaling factor pointer (Ptr) points to the address of the scaling factor in the second data queue.
[0079] Obtaining the target scaling factor pointer based on the emission count value and the target granularity parameter may include:
[0080] Based on the target granularity parameters, obtain the step size threshold, broadcast range value, and event enable parameters;
[0081] Based on the transmit count value, the step size threshold, and the event enable parameter, if a preset update trigger condition is detected, the current scaling factor pointer is obtained, and the current scaling factor pointer is updated according to the broadcast range value to obtain the target scaling factor pointer.
[0082] Among them, Step_Limit defines the toggle threshold of the internal counter, Broadcast_Range defines the modulo operation range of the scaling factor pointer, and Event_Enable defines whether to disable automatic counting and enable external signal triggering.
[0083] The preset update trigger condition refers to the pre-defined conditions required to trigger an update of the scaling factor pointer, which may include a step size threshold condition and an external event condition. In this embodiment, if Issue_Count, Step_Limit, and Event_Enable are found to meet the step size threshold condition or the external event condition, it can be determined that the preset update trigger condition is met. Then, Ptr = (Ptr + 1) % Broadcast_Range can be executed to update the scaling factor pointer, and the updated scaling factor pointer is used as the target scaling factor pointer. That is, the current scaling factor pointer is incremented by 1, and a modulo operation is performed with the broadcast range to achieve cyclic or single-step operation. If it is determined that the preset update trigger condition is not met, Ptr remains unchanged, and the current scaling factor is reused, i.e., the current scaling factor pointer is used as the target scaling factor pointer. The target scaling factor pointer is the scaling factor pointer required for the current clock cycle, corresponding one-to-one with the floating-point data in the floating-point data sequence.
[0084] The detection of a preset update trigger condition based on the emission count value, the step size threshold, and the event enable parameter may include:
[0085] Obtain the current loop index value and determine whether the current loop index value is less than the emission count value. If so, increment the internal count value to obtain the current internal count value.
[0086] If the current internal count value is detected to be greater than or equal to the step size threshold, or the event enable parameter is a first preset value and the external event signal is high, then it is determined that the preset update trigger condition is met.
[0087] In a specific example, the process of obtaining the target scaling factor pointer can be as follows: Figure 3 As shown. An internal counter is set to record how many floating-point data points have reused the current scaling factor. The external event signal (External_Signal) is a bypass input signal used to indicate special boundaries in the data stream, such as token end, row end, etc.
[0088] Specifically, within each clock cycle, the transmit count value Issue_Count is received, and the loop index value i = 0 is initialized. Next, it is determined whether i is less than Issue_Count. If so, the internal counter is incremented to obtain the current internal count value Counter; otherwise, the process waits for the next clock cycle. Then, it is determined whether Counter is greater than or equal to Step_Limit, or whether Event_Enable = 1 and External_Signal is high. If either condition is met, the scaling factor pointer is updated based on Ptr = (Ptr + 1) % Broadcast_Range, and the internal counter is reset, clearing Counter to zero to prepare for the next round of counting. If neither condition is met, the current scaling factor pointer is maintained. Finally, i = i + 1, and the next round of judgment is performed until all target scaling factor pointers for the current clock cycle are obtained.
[0089] In this embodiment, by setting the target granularity parameter, the same set of control logic can be reused across multiple granularity modes. Specifically, for the Per-Block Scaling (Per-Block) granularity mode, Step_Limit=N (e.g., 16 or 32), Broadcast_Range= (Maximum value), Event_Enable=0, the core circuit behaves as standard step logic; every N data points consumed, the pointer automatically increments by 1 to read the next scaling factor. For per-Tensor scaling (Per-Tensor) granularity mode, Step_Limit= (Maximum value), Broadcast_Range=1, Event_Enable=0. Since the step threshold is infinitely large, the internal counter will never overflow and the range is limited to 1. After reading the first scaling factor, the scaling factor pointer is "deadlocked" at address 0.
[0090] For the Per-Channel Scaling granularity mode, Step_Limit=1, Broadcast_Range=Channel_Num (e.g., 64), Event_Enable=0, the internal counter is updated every step (i.e., for each data point), and the scaling factor pointer cycles continuously between 0 and (C-1). For the Per-Token Scaling granularity mode, which scales text based on the basic unit of a large language model, Step_Limit= Broadcast_Range= Event_Enable=1 disables automatic count updates, and the circuit switches to listening to the "Row End" in the bypass input signal, forcibly incrementing the scaling factor pointer only when the signal changes.
[0091] S260. Perform multiplication-free conversion processing based on the floating-point data sequence, the scaling factor sequence, and the target format parameters to obtain target data in standard floating-point format.
[0092] The technical solution of this invention caches the floating-point data in a first data queue and the scaling factor in a second data queue. Based on the logical definition granularity of the micro-scaling floating-point format and the physical throughput granularity of the computing core, a current granularity remapping mode is determined. Based on the current granularity remapping mode, the floating-point data sequence for the current clock cycle is extracted from the first data queue. The emit count value corresponding to the first data queue within the current clock cycle is obtained. Based on the emit count value and the target granularity parameter, a target scaling factor pointer is obtained. Based on the target scaling factor pointer, the scaling factor sequence for the current clock cycle is extracted from the second data queue. By caching the floating-point data and scaling factor separately, selecting different granularity remapping modes for floating-point data extraction based on different logical definition granularities and physical throughput granularities, and extracting the scaling factor based on the scaling factor pointer, the peak throughput of the computing core can be maintained, pipeline bubbles can be eliminated, fast format / granularity switching can be achieved, and the inference efficiency of heterogeneous hierarchical models can be greatly improved.
[0093] In one specific implementation of this embodiment, it is assumed that the original data adopts the E4M3 format of MXFP8, the scaling granularity is Per Block (Block Size N=16), and the physical throughput granularity of the backend computing core is M=32 data points / cycle. At this time, there exists a situation where M=2 The relationship is a multiple of N. The specific processing flow is as follows: Step 1: Initialize configuration; the controller configures the system to MXFP_Mode, with the following format parameters: Exp_Width=4, Mant_Width=3, Bias_Config=15; granularity parameters: Step_Limit=16, Broadcast_Range= , Event_Enable=0. Step 2: Separation and extraction of data and scaling factor; Transmit continuous MXFP8 data blocks on the data bus, send the extracted Scale to the Scale FIFO, and send the original micro data (Data) to the Data FIFO. Step 3: Multiple aggregation of the buffer (2 in, 1 out). Among them, (1) Status monitoring: The Data FIFO accumulates 32 data (2 Blocks); (2) Transmission action: The buffer triggers the "transmission" signal and sends Issue_Count=32 to the Scale adapter module. At the same time, the Data FIFO maps the first 16 data to Lane 0-15 and the last 16 data to Lane 16-31. The Scale FIFO pops Scale A and Scale B and generates the boundary pointer Boundary_Ptr=15. Step 4: Parallel format conversion; 32 non-multiplication conversion units (Converter Lane) work in parallel. Among them, (1) Route selection: Lane 0-15 selects Scale A according to the boundary pointer, and Lane 16-31 selects Scale B; (2) Exponent calculation: Read the global configuration Bias_Config=15, and execute New_Exp=Exp_{Scale}+Exp_{Data}-15; (3) Mantissa mapping: The hardware automatically extracts the lower 3 bits of the input data and connects them to the higher 3 bits of the FP16 mantissa. Step 5: Core calculation; The back-end FP16 calculation unit receives 32 uniform standard FP16 data at time T2 and performs calculations.
[0094] To address the mismatch between the Block Size defined in the MXFP standard and the width of the computing core, this embodiment does not employ a complex full-scale crossover switch. Instead, it utilizes simple FIFO threshold control to achieve multiplicative aggregation and issuance. This technical solution eliminates pipeline bubbles, preventing idle lanes in the computing core due to excessively small data blocks, thus ensuring 100% utilization of backend computing resources. Secondly, it reduces control complexity by using a boundary pointer mechanism to resolve the correspondence between 32 data points and 2 scales in one go, eliminating the need to store scale indexes separately for each data point and significantly saving on-chip control logic area.
[0095] In another specific implementation of this embodiment, the dynamic adaptability of the data processing method when handling complex large models is demonstrated. For example, the large model is a modern network that combines a Convolutional Neural Network (CNN) and a Transformer. Assume the application scenario is: first, a convolutional layer (ConvLayer) is executed, followed immediately by a Transformer attention layer. This solution achieves seamless dynamic switching between different data formats and different quantization granularities.
[0096] Among them, scenario A: convolutional layer processing (configuration: E5M2+Per Channel). In view of the high dynamic range requirement of convolutional layers, the E5M2 format is adopted and Per Channel quantization is used to maintain channel features. Specifically, step one: configuration stage; set parameters Exp_Width=5, Mant_Width=2, Bias_Config=31, Step_Limit=1, Broadcast_Range=64, Event_Enable=0. Step two: parsing stage; according to the register value, automatically adjust the bitmask logic and extract 5-bit exponent component and 2-bit mantissa component from the input data stream. Step three: adaptation stage; configure Scale FIFO to a 64-depth circular addressing mode. Assuming that the input feature map is transmitted in NHWC order, the Scale pointer is automatically incremented by 1 for each data transmission; when the pointer reaches 63, it automatically wraps back to 0. Scale[0]...Scale
[63] in H Repeated broadcasting in W loops. Step 4: Conversion stage; The multiplication-free conversion unit reads the global Bias_Config=31 and performs exponential addition; simultaneously, the mantissa adapter maps the 2-bit input mantissa to the high 2 bits of the FP16. Step 5: Result; The high dynamic range E5M2 data stream is converted to FP16 in real time and seamlessly enters the computing core.
[0097] Scenario B: Attention Layer Processing (Configuration: E4M3 + Per Token). After the convolutional layer computation is complete, the hardware immediately switches to the Transformer layer. To address the high precision requirements of the attention mechanism, the E4M3 format is used, coupled with Per Token quantization. Specifically, Step 1: Dynamic Switching; Upon receiving the layer switching command, update the parameters Exp_Width=4, Mant_Width=3, Bias_Config=15, Step_Limit= Broadcast_Range= Event_Enable=1. Step 2: Parsing Phase; The hardware logic responds immediately without reset, switching the mask to extract the 4-bit exponent and 3-bit mantissa. Step 3: Adaptation Phase; Switching to row-synchronous update mode, the Scale FIFO enters a "hold" state, locking after reading a Scale; The Scale pointer is only updated when a Token end marker (Row End) is detected in the data stream or the internal counter reaches Len=128. This ensures that all dimensions within the same Token share the same Scale. Step 4: Transformation Phase; The multiplication-free transformation unit automatically adjusts to use Bias_Config=15 for exponentiation, and the mantissa adapter is changed to map the 3-bit mantissa. Step 5: Result; The high-precision E4M3 data stream seamlessly enters the computation core.
[0098] This embodiment breaks through the traditional hardware limitation of "one format corresponding to one circuit". By introducing a bit-field configurable resolution and multi-modal addressing controller, the hardware is upgraded from an "application-specific integrated circuit" to a "general-purpose decoding engine". This embodiment achieves extremely fast context switching. Compared to the latency required for reconfiguration of traditional field-programmable gate arrays or dedicated core switching, this solution only requires updating registers and does not require draining the pipeline, greatly improving the inference efficiency of heterogeneous models. Secondly, it achieves perfect granular decoupling. For per-channel circular addressing and per-token row synchronous updates, the data preprocessing overhead of the central processing unit is completely eliminated, significantly reducing end-to-end system latency. Finally, it maximizes hardware resource utilization. The same set of adder and shifter logic is reused for two completely different formats, E5M2 and E4M3, without increasing additional computing power or area, yet achieving computing power universality.
[0099] This invention proposes a universal dequantization architecture and data processing method. Without modifying the internal logic of the existing floating-point operation core, an intelligent adaptation layer is introduced at the front end of the pipeline to achieve full functionality reuse through the following four mechanisms: (1) Multi-format adaptive parsing: Based on programmable bitmask templates, it supports dynamic parsing of any sub-format such as E4M3 and E5M2. (2) Universal scaling granularity adaptation: Through broadcast and step control mechanisms, it uniformly supports four mainstream granularities: Per-Tensor / Channel / Block / Token. (3) Low-precision elastic reassembly: It solves the mismatch between the front-end logic definition granularity (N) and the back-end physical throughput granularity (M). (4) Parallel multiplication-free conversion: It uses exponential addition to replace floating-point multiplication to achieve single-cycle mapping.
[0100] Example 3
[0101] Figure 4 This is a schematic diagram of the structure of a data processing system provided in Embodiment 3 of the present invention. Figure 4 As shown, the data processing system 30 includes a controller 31 and various multiplication-free conversion units 32;
[0102] The controller 31 is configured to acquire raw data in micro-scaling floating-point format and identify the target format parameter and target granularity parameter corresponding to the raw data; extract floating-point data and the corresponding scaling factor from the raw data according to the target format parameter; cache and reassemble the floating-point data and the corresponding scaling factor in a queue according to the target granularity parameter to obtain a floating-point data sequence and a scaling factor sequence; and send the floating-point data sequence, the scaling factor sequence and the target format parameter to each multiplication-free conversion unit 32.
[0103] Each multiplication-free conversion unit 32 is used to perform multiplication-free conversion processing based on the floating-point data sequence, the scaling factor sequence, and the target format parameters to obtain target data in standard floating-point format.
[0104] The controller 31 may further include a multi-format adaptive parsing module, which consists of a dynamic barrel shifter and a programmable bitmask generator, used to parse and obtain the exponent and mantissa components of the floating-point data respectively. Secondly, the controller 31 may also include a series of programmable configuration registers for storing configuration information such as format parameters and granularity parameters, as well as data such as floating-point data and scaling factors. The multiplication-free conversion unit 32 may consist of a digital comparator, an adder, and a shifter.
[0105] The technical solution of this invention involves: acquiring raw data in micro-scaled floating-point format through controller 31 and identifying the target format parameters and target granularity parameters corresponding to the raw data; extracting floating-point data and corresponding scaling factors from the raw data according to the target format parameters; queuing and reassembling the floating-point data and corresponding scaling factors according to the target granularity parameters to obtain a floating-point data sequence and a scaling factor sequence; performing multiplication-free conversion processing by each multiplication-free conversion unit 32 based on the floating-point data sequence, scaling factor sequence, and target format parameters to obtain target data in standard floating-point format; and executing the format conversion logic from micro-scaled floating-point format to standard floating-point format by adding controller 31 and multiplication-free conversion units 32, thereby enabling existing computing chips to support the MXFP format and avoiding problems such as excessive costs caused by redesigning hardware.
[0106] Example 4
[0107] Figure 5This is a schematic diagram of a chip structure provided in Embodiment 4 of the present invention. The chip 40 includes the data processing system 30 described in any embodiment of the present invention. It is understood that the chip 40 may also include conventional modules such as a computing core and a bus; this embodiment does not specifically limit its inclusion in these modules.
[0108] In some embodiments, the data processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. When the computer program is executed by a processor, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the data processing method by any other suitable means (e.g., by means of firmware).
[0109] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), system-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0110] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0111] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0112] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0113] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact via a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server.
[0114] This embodiment may also include a computer program product, which includes a computer program that, when executed by a processor, implements the data processing method provided in any embodiment of the present invention.
[0115] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0116] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A data processing method, characterized in that, include: Obtain the raw data in micro-scaling floating-point format, and identify the target format parameters and target granularity parameters corresponding to the raw data; The original data is split into original micro data and corresponding scaling factors, and the original micro data is decoded based on the target format parameters to extract floating-point data, which consists of a sign component, an exponent component and a mantissa component. The floating-point data is cached in the first data queue, and the scaling factor is cached in the second data queue; Based on the physical throughput granularity of the computing core, extract the floating-point data sequence of the current clock cycle from the first data queue; Obtain the transmit count value corresponding to the first data queue in the current clock cycle, and obtain the target scaling factor pointer based on the transmit count value and the target granularity parameter, and extract the scaling factor sequence of the current clock cycle from the second data queue based on the target scaling factor pointer; Extract the current floating-point data and the corresponding current scaling factor from the floating-point data sequence and the scaling factor sequence, respectively; Obtain the bias parameter and mantissa width based on the target format parameter, and perform multiplication-free conversion processing based on the current floating-point data, the current scaling factor, the bias parameter, and the mantissa width to obtain target data in standard floating-point format.
2. The method according to claim 1, characterized in that, Identifying the target format parameters and target granularity parameters corresponding to the original data includes: Identify the target format identifier and target granularity identifier corresponding to the raw data; The target format parameters are obtained based on the target format identifier and the preset correspondence between format identifiers and format parameters. The target granularity parameters are obtained based on the target granularity identifier and the preset correspondence between granularity identifiers and granularity parameters.
3. The method according to claim 1, characterized in that, Decoding the original microdata based on the target format parameters to extract floating-point data includes: Based on the target format parameters, the exponent bit width and mantissa bit width are obtained, and based on the exponent bit width and mantissa bit width, the sign component, exponent component and mantissa component are extracted from the original micro data. The floating-point data is generated based on the sign component, the exponent component, and the mantissa component.
4. The method according to claim 1, characterized in that, Based on the physical throughput granularity of the computing core, the floating-point data sequence of the current clock cycle is extracted from the first data queue, including: Based on the logical definition granularity of the micro-scaling floating-point format and the physical throughput granularity of the computing core, the current granularity remapping mode is determined, and based on the current granularity remapping mode, the floating-point data sequence of the current clock cycle is extracted from the first data queue.
5. The method according to claim 1, characterized in that, Based on the emission count value and the target granularity parameter, obtain the target scaling factor pointer, including: Based on the target granularity parameters, obtain the step size threshold, broadcast range value, and event enable parameters; Based on the transmit count value, the step size threshold, and the event enable parameter, if a preset update trigger condition is detected, the current scaling factor pointer is obtained, and the current scaling factor pointer is updated according to the broadcast range value to obtain the target scaling factor pointer.
6. The method according to claim 5, characterized in that, Based on the emission count value, the step size threshold, and the event enable parameter, a preset update trigger condition is detected, including: Obtain the current loop index value and determine whether the current loop index value is less than the emission count value. If so, increment the internal count value to obtain the current internal count value. If the current internal count value is detected to be greater than or equal to the step size threshold, or the event enable parameter is a first preset value and the external event signal is high, then it is determined that the preset update trigger condition is met.
7. The method according to claim 1, characterized in that, Extracting the current floating-point data and the corresponding current scaling factor from the floating-point data sequence and the scaling factor sequence respectively includes: Based on the floating-point data sequence and the correspondence between floating-point data and scaling factors, obtain the boundary pointers corresponding to each scaling factor in the scaling factor sequence, and generate a boundary pointer sequence based on each boundary pointer; Data allocation is performed in each multiplication-free conversion unit according to the floating-point data sequence, the scaling factor sequence, and the boundary pointer sequence to obtain the floating-point data and scaling factor corresponding to each multiplication-free conversion unit; Based on the current floating-point data, the current scaling factor, the bias parameter, and the mantissa bit width, a multiplication-free conversion process is performed to obtain target data in standard floating-point format, including: Each of the aforementioned multiplication-free conversion units performs multiplication-free conversion processing based on the corresponding floating-point data and scaling factor, as well as the bias parameter and the mantissa bit width, to obtain target data in standard floating-point format.
8. A data processing system, characterized in that, Includes the controller and each multiplication-free conversion unit; The controller is used to acquire raw data in micro-scaling floating-point format and identify the target format parameters and target granularity parameters corresponding to the raw data; The original data is split into original micro data and corresponding scaling factors, and the original micro data is decoded based on the target format parameters to extract floating-point data, which consists of a sign component, an exponent component and a mantissa component. The floating-point data is cached in the first data queue, and the scaling factor is cached in the second data queue; Based on the physical throughput granularity of the computing core, extract the floating-point data sequence of the current clock cycle from the first data queue; Obtain the transmit count value corresponding to the first data queue in the current clock cycle, and obtain the target scaling factor pointer based on the transmit count value and the target granularity parameter, and extract the scaling factor sequence of the current clock cycle from the second data queue based on the target scaling factor pointer; And send the floating-point data sequence, the scaling factor sequence, and the target format parameters to each of the multiplication-free conversion units; Each multiplication-free conversion unit is used to extract the current floating-point data and the corresponding current scaling factor from the floating-point data sequence and the scaling factor sequence, respectively. The bias parameter and mantissa width are obtained according to the target format parameter, and multiplication-free conversion is performed according to the current floating-point data, the current scaling factor, the bias parameter and the mantissa width to obtain target data in standard floating-point format.
9. A chip, characterized in that, Including the data processing system as described in claim 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the data processing method according to any one of claims 1-7.