Data processing method, system, apparatus, device, and storage medium
By performing precision adaptive processing on the mantissa of floating-point numbers and designing miniaturized multipliers, and expanding the parallel physical channels, the problem of excessive hardware resources and power consumption in floating-point operations is solved, achieving a balance between efficient computational parallelism and precision.
Patent Information
- Application Number
- CN202610416457.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-31
- Publication Date
- 2026-06-26
Smart Images
Figure CN122284948A_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of floating-point arithmetic technology, and in particular to a data processing method, system, apparatus, device, and storage medium. Background Technology
[0002] In computing and data processing, floating-point multiplication and addition operations are widely used, and the performance of floating-point operations directly affects the efficiency of algorithm execution. However, compared with fixed-point operations, floating-point operations are more complex and require more hardware resources and power consumption.
[0003] In existing technologies, when performing multiplication and addition operations on floating-point data, the floating-point mantissa occupies a large number of bits. During the multiplication of the mantissa bits, a multiplier with a larger area is actually required, resulting in higher power consumption. In order to balance factors such as area and performance, the number of channels is reduced, which leads to computing power delay. Summary of the Invention
[0004] This application provides a data processing method, system, apparatus, device, and storage medium that can achieve a balance between accuracy and hardware area, power consumption, and computing power. The technical solution of this application is as follows:
[0005] On the one hand, a data processing method is provided, applied to a processor, the method comprising:
[0006] Obtain a calculation instruction, which is used to perform floating-point arithmetic operations on a first operand and a second operand, wherein the first operand and the second operand are represented in a floating-point format, and the floating-point format has a first mantissa bit width;
[0007] Perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively to obtain the first processed mantissa corresponding to the first operand and the second processed mantissa corresponding to the second operand; the effective bit width corresponding to the first processed mantissa and the second processed mantissa is smaller than the bit width of the first mantissa;
[0008] The first arithmetic unit is configured according to the effective bit width to perform a multiplication operation on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result; the operation bit width of the first arithmetic unit is smaller than the operation bit width of the second arithmetic unit, and the second arithmetic unit is used to perform a multiplication operation on the mantissa portions corresponding to the first operand and the second operand respectively.
[0009] Based on the exponent portions corresponding to the first operand and the second operand, and the intermediate mantissa result, an arithmetic operation result is generated;
[0010] The method is executed synchronously by multiple parallel physical channels, the number of which is configured to be no less than the number of threads in a thread bundle, so that parallel processing of at least one complete thread bundle can be completed in a single operation cycle.
[0011] Optionally, the floating-point format includes single-precision floating-point format, half-precision floating-point format, and 16-bit floating-point format.
[0012] Optionally, the precision adaptive processing operation includes a deterministic precision reduction operation, which truncates the high-order bits of the mantissa portions of the first operand and the second operand, so that the bit width of the first processing mantissa and the second processing mantissa is a predetermined value.
[0013] Optionally, the predetermined value is determined based on at least one of the following: the precision loss allowed by the target computing task, the mantissa bit width of the floating-point format, power consumption constraints, area constraints, or performance constraints.
[0014] Optionally, the precision adaptive processing operation includes a dynamic precision adjustment operation, which dynamically determines the effective bit width corresponding to the first processing mantissa and the second processing mantissa based on the numerical range, distribution characteristics, or statistical information corresponding to the first operand and / or the second operand, respectively.
[0015] Optionally, configuring the first arithmetic unit corresponding to the effective bit width includes:
[0016] Among multiple arithmetic units with different fixed operation bit widths, the arithmetic unit whose operation bit width matches the effective bit width is determined as the first arithmetic unit;
[0017] or,
[0018] The configurable array of computing units is logically partitioned or resource-mapped to form the first computing unit; the bit width of the first computing unit is matched with the effective bit width.
[0019] Optionally, the number of physical channels is an integer multiple of the number of threads in the thread bundle, wherein the integer is greater than or equal to 1, so that the arithmetic operations corresponding to each thread in the thread bundle can be executed in parallel within a single operation cycle.
[0020] Optionally, generating the arithmetic result based on the exponent portions of the first operand and the second operand, and the intermediate mantissa result, includes:
[0021] Perform exponential operations corresponding to the arithmetic operations on the exponent portions of the first operand and the second operand respectively to obtain the exponential processing result;
[0022] The exponent processing result and the intermediate mantissa result are aligned and normalized to generate the arithmetic operation result.
[0023] On the other hand, a data processing system is provided, including:
[0024] The instruction decoding module is used to decode the received calculation instruction, which is used to instruct the execution of floating-point arithmetic operations on a first operand and a second operand. The first operand and the second operand are represented in floating-point format, and the floating-point format has a first mantissa bit width.
[0025] An operand acquisition module, connected to the instruction decoding module, is used to acquire the first operand and the second operand based on the decoding result;
[0026] The operand operation module, connected to the operand acquisition module, includes multiple parallel first physical channels, the number of which is not less than the number of threads in a thread bundle, for synchronously executing the floating-point arithmetic operations.
[0027] Each of the first physical channels includes:
[0028] The mantissa processing unit is used to perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively to obtain a first processed mantissa corresponding to the first operand and a second processed mantissa corresponding to the second operand; the effective bit width corresponding to the first processed mantissa and the second processed mantissa is smaller than the first mantissa bit width.
[0029] The multiplier unit, whose operational bit width matches the effective bit width, is used to perform multiplication on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result;
[0030] An exponent processing unit is used to perform an exponent operation corresponding to the arithmetic operation on the exponent portions of the first operand and the second operand respectively, to obtain an exponent processing result.
[0031] The result synthesis unit is used to generate an arithmetic operation result based on the exponent processing result and the intermediate mantissa result;
[0032] The write-back module, connected to the operand operation module, is used to write the arithmetic operation result back to the register.
[0033] Optionally, the operand operation module further includes multiple second physical channels configured in parallel. The second physical channels are used to perform arithmetic operations in half-precision floating-point format or 16-bit floating-point format. The number of the second physical channels is an integer multiple of the number of threads in the thread bundle.
[0034] Optionally, the number of the second physical channels is twice the number of the first physical channels.
[0035] On the other hand, a data processing apparatus is provided, the apparatus comprising:
[0036] The acquisition module is used to acquire calculation instructions, which are used to perform floating-point arithmetic operations on a first operand and a second operand. The first operand and the second operand are represented in floating-point format, and the floating-point format has a first mantissa bit width.
[0037] The precision processing module is used to perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively, to obtain a first processed mantissa corresponding to the first operand and a second processed mantissa corresponding to the second operand; the effective bit width corresponding to the first processed mantissa and the second processed mantissa is smaller than the bit width of the first mantissa;
[0038] The arithmetic unit configuration module is used to configure a corresponding first arithmetic unit according to the effective bit width, so as to perform a multiplication operation on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result; the arithmetic bit width of the first arithmetic unit is smaller than the arithmetic bit width of the second arithmetic unit, and the second arithmetic unit is used to perform a multiplication operation on the mantissa portions corresponding to the first operand and the second operand respectively;
[0039] The arithmetic operation result generation module is used to generate an arithmetic operation result based on the exponent parts corresponding to the first operand and the second operand, as well as the intermediate mantissa result.
[0040] The floating-point arithmetic operations are executed synchronously by multiple parallel physical channels, the number of which is configured to be no less than the number of threads in a thread bundle, so that parallel processing of at least one complete thread bundle can be completed in a single operation cycle.
[0041] On the other hand, a data processing device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the data processing method as described above.
[0042] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the data processing method as described above.
[0043] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing method described above.
[0044] The data processing method, system, apparatus, device, and storage medium provided in this application have the following technical advantages:
[0045] The technical solution provided in this application is applied to a processor to obtain calculation instructions. These instructions are used to perform floating-point arithmetic operations on a first operand and a second operand. The first and second operands are represented in floating-point format, which has a first mantissa bit width. Precision adaptive processing is performed on the mantissa portions corresponding to the first and second operands to obtain a first processed mantissa corresponding to the first operand and a second processed mantissa corresponding to the second operand. The effective bit width corresponding to the first and second processed mantissas is less than the first mantissa bit width. A corresponding first arithmetic unit is configured according to the effective bit width to perform multiplication operations on the first and second processed mantissas, generating an intermediate mantissa result. The arithmetic bit width of the first arithmetic unit is less than the arithmetic bit width of the second arithmetic unit, which performs multiplication operations on the mantissa portions corresponding to the first and second operands. An arithmetic operation result is generated based on the exponent portions corresponding to the first and second operands and the intermediate mantissa result. The method is executed synchronously by multiple parallel physical channels, the number of which is configured to be no less than the number of threads in a thread bundle, so that parallel processing of at least one complete thread bundle is completed within a single computation cycle. Therefore, while ensuring a certain level of computational accuracy, it saves space and improves computational parallelism, thereby achieving a balance between accuracy and hardware area, power consumption, and computing power. Attached Figure Description
[0046] To more clearly illustrate the technical solutions of the present invention, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.
[0047] Figure 1 This is a schematic diagram of the application environment of a data processing method provided in an embodiment of this application;
[0048] Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of this application;
[0049] Figure 3 This is a schematic diagram of a process for generating arithmetic operation results provided in an embodiment of this application;
[0050] Figure 4 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of this application;
[0051] Figure 5 This is a block diagram of a data processing apparatus provided in an embodiment of this application;
[0052] Figure 6 This is a block diagram of a terminal electronic device for data processing provided in an embodiment of this application;
[0053] Figure 7 This is a block diagram of a server electronic device for data processing provided in an embodiment of this application. Detailed Implementation
[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0055] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.
[0056] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:
[0057] Floating-point format: A computer data type used to represent numerical values with a decimal part. It consists of a sign part, an exponent part, and a mantissa part, and is capable of representing numerical values with a large range and high precision.
[0058] The exponent part: In floating-point data, this part represents the order of magnitude of the value. In floating-point multiplication, the exponent parts of two floating-point numbers are added together to obtain the exponent part of the product.
[0059] The mantissa portion: In floating-point data, the part used to represent the significant digits of the value. In floating-point multiplication, the mantissa portions of two floating-point numbers are multiplied to obtain the mantissa portion of the product.
[0060] Calculation instructions: These are instructions received and executed by the processor or computing unit to perform arithmetic operations on floating-point data.
[0061] Physical path: In a parallel computing architecture, a physical path refers to a hardware unit or logical path used to independently execute computational tasks. Multiple physical paths can process different data pairs simultaneously, thereby significantly improving the overall computational throughput.
[0062] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment for a data processing method provided in an embodiment of this application. The application environment may include at least a server 100 and a terminal 200.
[0063] In an optional embodiment, terminal 200 can be used to provide object-oriented data processing services. Specifically, terminal 200 can be, but is not limited to, electronic devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices, or software running on the aforementioned electronic devices, such as applications. Optionally, the operating system running on the electronic device can be, but is not limited to, Android, iOS, Linux, Windows, etc.
[0064] Server 100 can provide backend services for terminal 200. Specifically, server 100 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.
[0065] In addition, it should be noted that, Figure 1 The examples shown are merely one application environment for a data processing method, and the embodiments in this specification are not limited to the above.
[0066] In the embodiments described in this specification, the server 100 and the terminal 200 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.
[0067] Current mainstream floating-point computing is based on fixed-precision formats (such as FP64 double precision, FP32 single precision, FP16 half precision, BF16 brain floating-point, and TF32 tensor floating-point). Their design logic focuses on general precision coverage but lacks adaptability to differentiated scenario requirements. On one hand, high-precision formats (such as FP64 and FP32) require a large amount of hardware area for their arithmetic units (such as multipliers) (e.g., the 24*24 mantissa multiplier in FP32 and the 53*53 mantissa multiplier in FP64), resulting in high hardware costs and high power consumption, making them difficult to adapt to resource-constrained scenarios such as edge terminals and microcontrollers (MCUs). On the other hand, while low-precision formats (such as FP16 and BF16) simplify hardware design, their fixed precision and lack of flexibility cannot meet the mixed requirements of coarse-grained simulation in scientific computing and high-precision computation in core areas, or the need to balance high throughput in AI inference intermediate layers with high precision in the output layer. Existing floating-point computing technologies have significant bottlenecks in terms of precision flexibility, hardware resource adaptability, cross-architecture compatibility, and meeting the needs of different scenarios.
[0068] Based on this, this application proposes a data processing method that saves hardware area by truncating the mantissa and miniaturizing the multiplier during floating-point multiplication, while expanding the number of parallel physical channels. This saves area while improving computational parallelism, thus achieving a balance between accuracy and hardware area, power consumption, and computing power.
[0069] The following describes a data processing method according to this application. Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of this application. This specification provides the operational steps of the method described in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only execution order. In actual system or server products, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or drawings. Specifically, as... Figure 2 As shown, the above method may include:
[0070] S201: Obtain calculation instructions.
[0071] In one specific embodiment, the above-described calculation instructions can be used to perform floating-point arithmetic operations on the first operand and the second operand. The first operand and the second operand can be represented in floating-point format. Floating-point format data is a computer data format used to represent numerical values with a decimal part. Floating-point format data consists of a sign bit, an exponent bit, and a mantissa bit, and can represent numerical values with a large range and high precision. The floating-point format has a first mantissa bit width. Specifically, the floating-point format can include single-precision floating-point format (FP32), half-precision floating-point format (FP16), and brain floating-point format (BF16). The corresponding floating-point format's mantissa bit width (i.e., the first mantissa bit width) can be 23 bits, 10 bits, or 7 bits.
[0072] In practical applications, after receiving a calculation instruction, the instruction can be parsed to obtain the operand information to be calculated, which may include the operands and their storage addresses, as well as the computation pipeline information.
[0073] S203: Perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively to obtain the first processed mantissa corresponding to the first operand and the second processed mantissa corresponding to the second operand.
[0074] In one specific embodiment, the effective bit widths corresponding to the first and second processed mantissas are each smaller than the bit width of the first mantissa, while the effective bit widths corresponding to the first and second processed mantissas are the same. Based on the precision requirements of floating-point operations, the mantissa portions corresponding to the first and second operands can be partially truncated to obtain first and second processed mantissas that are smaller than the original floating-point format mantissa bit width.
[0075] Optionally, the required precision can be automatically adjusted according to actual application needs to determine the required mantissa bit width to be truncated / retained, and partial mantissa truncation processing can be performed based on this. Optionally, the precision adaptive processing operation can include a deterministic precision reduction operation, which truncates the high-order bits of the mantissa portions corresponding to the first and second operands to make the bit widths of the first and second processed mantissas reach a predetermined value. Specifically, the predetermined value can be determined through application layer constraints, data layer characteristics, and hardware layer limitations. Specifically, the predetermined value can be determined based on at least one of the following: the allowable precision loss of the target computing task, the mantissa bit width of the floating-point format, power consumption constraints, area constraints, or performance constraints. For example, based on the tolerance of the target computing task for the error of the computing result, the maximum acceptable precision loss for different computing stages can be determined, thereby specifying the optimal mantissa bit width (i.e., the predetermined value) for the computing of that stage.
[0076] Optionally, the effective mantissa bit width required for the current calculation can be dynamically determined based on the numerical characteristics of the first operand and / or the second operand. Specifically, the precision adaptive processing operation may include a dynamic precision adjustment operation, which dynamically determines the effective bit width corresponding to the first and second processed mantissas based on the numerical range, distribution characteristics, or statistical information corresponding to the first operand and / or the second operand.
[0077] S205: Configure the corresponding first arithmetic unit according to the effective bit width to perform a multiplication operation on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result.
[0078] In one specific embodiment, the bit width of the first arithmetic unit is smaller than that of the second arithmetic unit. The second arithmetic unit can be used to perform multiplication operations on the mantissa portions of the first and second operands respectively. In practical applications, taking the first and second operands as examples where both are FP32 floating-point format, the mantissa bit width of the first and second operands is 23 bits. A 24*24 multiplier (i.e., the second arithmetic unit) is actually needed for mantissa multiplication. In this embodiment, the lower 8 bits of the mantissa are discarded, retaining the higher 15 bits and the implicit 1 bit, requiring only a 16*16 multiplier (i.e., the first arithmetic unit) for mantissa multiplication. Furthermore, based on the saved multiplier area, an additional 16 parallel processing channels can be added, resulting in 32 parallel processing channels. This allows for the parallel completion of element-wise multiplication operations by 32 threads within a single processing cycle. Theoretically, the area ratio between a 24*24 multiplier and a 16*16 multiplier is (24*24) / (16*16), which is 2.25. The estimated actual multiplier area ratio is between 1.5 and 1.8. By truncating the mantissa, a smaller multiplier can be configured to perform mantissa multiplication, thus saving area and doubling the number of operation channels.
[0079] In the above embodiments, a portion of the mantissa portion corresponding to the first operand and the second operand is truncated from the high bits and a portion of the low bits is discarded to obtain the first processed mantissa and the second processed mantissa. Thus, a first arithmetic unit with matching bit width can be configured according to the effective bit width of the first processed mantissa and the second processed mantissa to perform multiplication operations on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result. By truncating the mantissa and miniaturizing the multiplier, hardware area is saved, while the number of parallel computing channels is expanded. While ensuring a certain level of computational accuracy, the occupied area is saved, and the computational parallelism is improved, thereby achieving a balance between accuracy and hardware area, power consumption, and computing power.
[0080] In an optional embodiment, the first arithmetic unit configured according to the effective bit width may include:
[0081] Among multiple arithmetic units with different fixed operation bit widths, the arithmetic unit whose operation bit width matches the effective bit width is determined as the first arithmetic unit;
[0082] or,
[0083] The configurable array of computing units is logically partitioned or its resources are mapped to form the first computing unit.
[0084] In one specific embodiment, the operation bit width of the first arithmetic unit matches the effective bit width. Specifically, the operation bit width of the first arithmetic unit is typically the effective bit width + 1. Specifically, resource partitioning for an array of arithmetic units with non-fixed operation bit widths can involve dividing the entire multiplier array into a large number of small-bit-width multiplication units. For example, the hardware resources can be divided into 32 independent 16*16 multipliers, with one multiplier per physical channel. Alternatively, multiple small-bit-width multipliers can be logically merged to form a high-bit-width multiplier. Specifically, resource mapping for the configurable arithmetic unit array can involve mapping the elements to be computed to the corresponding arithmetic units to adapt to different precision requirements. In practical applications, when the scheduler dispatches instructions, the instructions contain information about the precision requirements. The controller dynamically routes the elements to be computed to the corresponding arithmetic units based on this information.
[0085] S207: Generate the arithmetic operation result based on the exponents of the first and second operands and the intermediate mantissas.
[0086] In one specific embodiment, the above method can be executed synchronously by multiple parallel physical channels. The number of physical channels can be configured to be no less than the number of threads in a thread bundle, so that parallel processing of at least one complete thread bundle can be completed within a single computation cycle. Specifically, the number of physical channels is an integer multiple of the number of threads in the thread bundle, where the integer is greater than or equal to 1, so that the arithmetic operations corresponding to each thread in the thread bundle can be executed in parallel within a single computation cycle.
[0087] Optional, such as Figure 3 As shown, the arithmetic operation result generated based on the exponents of the first and second operands and the intermediate mantissas can include:
[0088] S301: Perform exponential operations corresponding to arithmetic operations on the exponent parts of the first operand and the second operand respectively to obtain the exponential processing result;
[0089] S303: Align and normalize the exponent processing result with the intermediate mantissa result to generate the arithmetic operation result.
[0090] In one specific embodiment, the exponent portions corresponding to the first operand and the second operand are added together to obtain the exponent processing result. Optionally, after adding the exponent portions corresponding to the first operand and the second operand, the addition result can be shifted (i.e., biased) to obtain a corrected exponent processing result. For example, if the exponent portion of the first operand is Ea and the exponent portion of the second operand is Eb, adding the exponent portions corresponding to the first operand and the second operand (Ea + Eb) and subtracting a preset bias value from the result yields the corrected exponent processing result.
[0091] In one specific embodiment, the arithmetic operation result can be data conforming to the floating-point number specification. Optionally, the exponent processing result can be aligned, the intermediate mantissa result can be rounded, and then normalized to obtain the arithmetic operation result. Specifically, the exponent processing result is aligned because the result of mantissa multiplication may not be within the normalized floating-point representation range. Based on the numerical range of the mantissa multiplication result, the exponent processing result is aligned accordingly to ensure that subsequent mantissas can be correctly interpreted. Specifically, the intermediate mantissa result is rounded because the result of mantissa multiplication may be a very long value, requiring rounding of the excess lower digits to bring it within the normalized range of the floating-point type. Specifically, the exponent processing result and the intermediate mantissa result are normalized to ensure that the mantissa portion of the final result is in a standard format. Normalization includes checking and adjusting the exponent and mantissa to conform to the corresponding floating-point standard. For example, if mantissa overflow occurs due to rounding, the exponent needs to be adjusted again.
[0092] Optionally, the arithmetic operation result also includes a sign result. This is achieved by XORing the sign parts of the two operands. For example, if both operands have positive signs, the sign result is positive; if one operand has a positive sign and the other has a negative sign, the sign result is negative. Specifically, the final determined sign result, exponent result, and mantissa result are combined to generate the arithmetic operation result.
[0093] In one specific implementation, the first and second operands can be any first element of the first vector and any second element of the second vector, respectively. The first and second elements can correspond one-to-one, and both can be floating-point data. Specifically, the first and second vectors are vectors to be used for inner product operations. These vectors can be vectors corresponding to feature data, which can be input data from a large model, such as semantic data, image data, or video data. Specifically, the first and second vectors can be feature vectors input to a preset neural network model. The structure of the preset neural network model can be set according to actual application requirements; for example, it can be a self-attention network. Correspondingly, the first vector can be a query vector, and the second vector can be a key vector. Specifically, the query vector can be obtained by multiplying the feature data with the corresponding query weights, and the key vector can be obtained by multiplying the feature data with the corresponding key weights.
[0094] Specifically, a first preset number of operation channels can be used to perform parallel multiplication of a first preset number of first elements and their corresponding second elements. Optionally, the first preset number can be greater than or equal to a second preset number, which can be the number of first elements included in the first vector or the number of second elements included in the second vector. That is, the number of expanded operation channels is greater than or equal to the number of elements in the vector, and the operation of the elements in the vector can be completed in parallel within one operation cycle, i.e., one vector operation task. Specifically, when the first preset number is equal to the second preset number, for example, both can be 32, that is, based on 32 operation channels, 32 pairs of first and second element multiplications can be completed in parallel within one operation cycle, i.e., the current vector operation task is completed.
[0095] In practical applications, the second preset number can be the number of threads in a thread bundle, and the first preset number can be greater than or equal to the second preset number, that is, not less than the number of threads in a thread bundle. This allows for parallel processing of at least one complete thread bundle within a single computation cycle, based on the first preset number of computation channels. Specifically, the first preset number can be an integer multiple of the number of threads in the thread bundle, and this integer can be greater than or equal to 1, enabling the parallel execution of element-wise operations corresponding to each thread in the thread bundle within a single computation cycle.
[0096] In one specific implementation, taking SIMT supporting 32 threads as an example, corresponding to the same number of 32-bit single-instruction multi-threaded channels, 32 threads can be processed in one cycle. However, such a design consumes a huge amount of chip area. Existing technologies reduce the number of channels, for example, to 16 channels, but this reduces throughput and causes computational latency. In this application, vector operation instructions are dispatched to 32 operation channels of the Vector Unit. Each operation channel is allocated a pair of elements, each element being a 32-bit floating-point number. Each pair of elements consists of elements from the first vector and the second vector, respectively. The 32 channels operate in parallel, and each channel independently processes a pair of element multiplications. During the operation of each channel, the sign bits of the two elements are multiplied first, i.e., XORed, to obtain the sign operation result. For example, if the signs of both elements are positive, the sign operation result is positive; if the sign of one element is positive and the sign of the other element is negative, the sign operation result is negative. Next, the exponent bits of the two elements are added and shifted to obtain the exponent operation result. Then, the mantissa bits of the two elements are partially truncated, discarding the lower 8 bits and retaining the higher 15 bits. This is then input into a 16*16 multiplier for multiplication to obtain the mantissa operation result. Afterwards, the exponent operation result is adjusted based on the mantissa operation result (e.g., the exponent is increased if the mantissa overflows), the mantissa is rounded to reduce errors, and normalized to ensure it is within the normalization range of the floating-point type. The final determined sign bit, exponent bit, and mantissa bits are combined to generate the operation result corresponding to each channel. The operation results corresponding to each of the 32 operation channels are then accumulated to obtain the vector operation result of the two vectors.
[0097] As can be seen from the technical solutions provided in the embodiments of this specification above, the calculation instruction obtained in this specification is used to perform floating-point arithmetic operations on the first operand and the second operand. The first operand and the second operand are represented in floating-point format, and the floating-point format has a first mantissa bit width. Precision adaptive processing is performed on the mantissa portions corresponding to the first operand and the second operand to obtain the first processed mantissa corresponding to the first operand and the second processed mantissa corresponding to the second operand. The effective bit width corresponding to the first processed mantissa and the second processed mantissa is less than the first mantissa bit width. The corresponding first arithmetic unit is configured according to the effective bit width to perform calculations. The first and second mantissas are multiplied to generate an intermediate mantissa result. The bit width of the first arithmetic unit is smaller than that of the second arithmetic unit. The second arithmetic unit performs multiplication on the mantissa portions of the first and second operands respectively. Based on the exponent portions of the first and second operands and the intermediate mantissa result, an arithmetic result is generated. The method is executed synchronously by multiple parallel physical channels, the number of which is configured to be no less than the number of threads in a thread bundle, enabling parallel processing of at least one complete thread bundle within a single computation cycle. Thus, while maintaining a certain level of computational accuracy, it saves space and improves computational parallelism, achieving a balance between accuracy and hardware area, power consumption, and computing power.
[0098] This invention also provides a data processing system, comprising:
[0099] The instruction decoding module is used to decode the received calculation instructions. The calculation instructions are used to instruct the first operand and the second operand to perform floating-point arithmetic operations. The first operand and the second operand are represented in floating-point format, and the floating-point format has a first mantissa bit width.
[0100] The operand acquisition module is connected to the instruction decoding module and is used to acquire the first operand and the second operand based on the decoding result.
[0101] The operand operation module, connected to the operand acquisition module, includes multiple parallel first physical channels. The number of first physical channels is not less than the number of threads in a thread bundle, and is used to synchronously execute floating-point arithmetic operations.
[0102] Each first physical channel includes:
[0103] The mantissa processing unit is used to perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively to obtain the first processed mantissa corresponding to the first operand and the second processed mantissa corresponding to the second operand; the effective bit width corresponding to the first processed mantissa and the second processed mantissa is smaller than the first mantissa bit width.
[0104] The multiplier unit, whose operational bit width matches the effective bit width, is used to perform multiplication on the first and second processed mantissas to generate an intermediate mantissa result.
[0105] The exponent processing unit is used to perform exponent operations corresponding to arithmetic operations on the exponent parts of the first operand and the second operand respectively, and obtain the exponent processing result.
[0106] The result synthesis unit is used to generate arithmetic operation results based on the exponent processing results and the intermediate mantissa results;
[0107] The write-back module, connected to the operand operation module, is used to write the results of arithmetic operations back to the register.
[0108] Specifically, the system may also include a thread beam scheduling module for sending computation instructions to the instruction decoding module.
[0109] Specifically, the operand operation module also includes multiple second physical channels configured in parallel. The second physical channels are used to perform arithmetic operations in half-precision floating-point format or 16-bit floating-point format. The number of second physical channels is an integer multiple of the number of threads in the thread bundle.
[0110] Specifically, the number of second physical channels is twice the number of first physical channels.
[0111] In practical applications, the system architecture is as follows: Figure 4 As shown, taking vector operations as an example, the WarpScheduler coordinates instruction dispatch to ensure that all operation lanes execute the same instructions synchronously, but the internal processing of each operation lane is independent. Within each cycle, the WarpScheduler sends vector operation instructions to the Execution Unit. The Instruction Decoder parses the instructions sent by the WarpScheduler. This Instruction Decoder is the first stage of the Execution Unit, responsible for extracting various information from the instructions and then passing them to the next module. The Instruction Decoder parses the instructions to obtain operand information, pipeline control information, and the addresses of the Scalar General Purpose Register (SGPR) / Vector General Purpose Register (VGPR), which are then passed to the OperandFetch module. The OperandFetch module is the second stage of the Execution Unit pipeline. The relevant information of the vector to be operated on is sent to the Vector Unit (i.e., the aforementioned Operand Unit) or the Scalar Unit for further decoding.
[0112] Vector or scalar operation modules can employ pipelines of different bit widths to achieve this. Figure 4 Taking the example shown, the vector operation module or scalar operation module can adopt a 32-bit pipeline. The scalar operation module processes scalar calculations and only needs to allocate one physical channel. In the vector operation module, the 32-bit pipeline is expanded from 16 physical channels to 32 physical channels (i.e., the first physical channel), thus allocating 32 physical channels to asynchronously execute the aforementioned floating-point operations. The difference from single-precision floating-point calculation is that when performing mantissa multiplication, only the high 15 bits of the mantissa are taken, and then a 16*16 multiplier is used to perform mantissa multiplication, replacing the 24*24 multiplier of single-precision floating-point. During the floating-point calculation process, each channel obtains a 32-bit operand, and the arithmetic and logic unit performs the aforementioned sign bit multiplication, exponent bit addition and shifting, mantissa bit multiplication, exponent bit alignment, rounding, and normalization post-processing of the operands using the methods described above. Additionally, the vector operation module can include 64 physical channels configured in parallel (i.e., a second physical channel), which can be used to perform arithmetic operations in half-precision floating-point or 16-bit floating-point format. The write-back unit is the final stage of the execution module, writing the calculation results back to the scalar general-purpose register / vector general-purpose register.
[0113] This invention also provides a data processing apparatus, such as... Figure 5 As shown, the device includes:
[0114] The acquisition module 510 is used to acquire a calculation instruction, which is used to perform floating-point arithmetic operations on a first operand and a second operand. The first operand and the second operand are represented in a floating-point format, and the floating-point format has a first mantissa bit width.
[0115] The precision processing module 520 is used to perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively to obtain a first processed mantissa corresponding to the first operand and a second processed mantissa corresponding to the second operand; the effective bit width of the first processed mantissa and the second processed mantissa is smaller than the bit width of the first mantissa.
[0116] The arithmetic unit configuration module 530 is configured to configure a corresponding first arithmetic unit according to the effective bit width, so as to perform a multiplication operation on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result; the arithmetic bit width of the first arithmetic unit is smaller than the arithmetic bit width of the second arithmetic unit, and the second arithmetic unit is used to perform a multiplication operation on the mantissa portions corresponding to the first operand and the second operand respectively.
[0117] The arithmetic operation result generation module 540 is used to generate an arithmetic operation result based on the exponent parts corresponding to the first operand and the second operand, as well as the intermediate mantissa result.
[0118] The method is executed synchronously by multiple parallel physical channels, the number of which is configured to be no less than the number of threads in a thread bundle, so that parallel processing of at least one complete thread bundle can be completed in a single operation cycle.
[0119] Optionally, the floating-point format includes single-precision floating-point format, half-precision floating-point format, and 16-bit floating-point format.
[0120] Optionally, the precision adaptive processing operation includes a deterministic precision reduction operation, which truncates the high-order bits of the mantissa portions of the first operand and the second operand, so that the bit width of the first processing mantissa and the second processing mantissa is a predetermined value.
[0121] Optionally, the predetermined value is determined based on at least one of the following: the precision loss allowed by the target computing task, the mantissa bit width of the floating-point format, power consumption constraints, area constraints, or performance constraints.
[0122] Optionally, the precision adaptive processing operation includes a dynamic precision adjustment operation, which dynamically determines the effective bit width corresponding to the first processing mantissa and the second processing mantissa based on the numerical range, distribution characteristics, or statistical information corresponding to the first operand and / or the second operand, respectively.
[0123] Optionally, the computing unit configuration module 530 includes:
[0124] An arithmetic unit configuration unit is used to determine the arithmetic unit whose arithmetic bit width matches the effective bit width among a plurality of arithmetic units with different fixed arithmetic bit widths as the first arithmetic unit;
[0125] or,
[0126] The configurable array of computing units is logically partitioned or resource-mapped to form the first computing unit; the bit width of the first computing unit is matched with the effective bit width.
[0127] Optionally, the number of physical channels is an integer multiple of the number of threads in the thread bundle, wherein the integer is greater than or equal to 1, so that the arithmetic operations corresponding to each thread in the thread bundle can be executed in parallel within a single operation cycle.
[0128] Optionally, the arithmetic operation result generation module 540 includes:
[0129] An exponentiation unit is used to perform exponentiation operations on the exponent portions of the first operand and the second operand, corresponding to the arithmetic operations, to obtain an exponentiation processing result.
[0130] The arithmetic operation result generation unit is used to align and normalize the exponent processing result and the intermediate mantissa result to generate the arithmetic operation result.
[0131] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0132] Figure 6 This is a block diagram of a terminal electronic device for data processing provided in an embodiment of this application. The electronic device can be a terminal, and its internal structure diagram can be as follows. Figure 6 As shown, the electronic device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.
[0133] Figure 7 This is a block diagram of a server electronic device for data processing provided in an embodiment of this application. The electronic device can be a server, and its internal structure diagram can be as follows. Figure 7 As shown, this electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.
[0134] Those skilled in the art will understand that Figure 6 or Figure 7The structures shown are merely block diagrams of some structures related to the disclosed solutions of this invention, and do not constitute a limitation on the electronic devices to which the disclosed solutions of this invention are applied. Specific electronic devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.
[0135] The present invention also provides a data processing device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the data processing method disclosed in the present invention.
[0136] The present invention also provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the data processing method disclosed in the present invention.
[0137] The present invention also provides a computer program product that, when run on an electronic device, causes the electronic device to execute the data processing method disclosed in the present invention.
[0138] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.
[0139] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0140] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.
[0141] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.
[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0143] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0144] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.
[0145] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.
Claims
1. A data processing method, characterized in that, Applied to a processor, the method includes: Obtain a calculation instruction, which is used to perform floating-point arithmetic operations on a first operand and a second operand, wherein the first operand and the second operand are represented in a floating-point format, and the floating-point format has a first mantissa bit width; Perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively to obtain the first processed mantissa corresponding to the first operand and the second processed mantissa corresponding to the second operand; the effective bit width corresponding to the first processed mantissa and the second processed mantissa is smaller than the bit width of the first mantissa; The first arithmetic unit is configured according to the effective bit width to perform a multiplication operation on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result; the operation bit width of the first arithmetic unit is smaller than the operation bit width of the second arithmetic unit, and the second arithmetic unit is used to perform a multiplication operation on the mantissa portions corresponding to the first operand and the second operand respectively. Based on the exponent portions corresponding to the first operand and the second operand, and the intermediate mantissa result, an arithmetic operation result is generated; The method is executed synchronously by multiple parallel physical channels, the number of which is configured to be no less than the number of threads in a thread bundle, so that parallel processing of at least one complete thread bundle can be completed in a single operation cycle.
2. The method according to claim 1, characterized in that, The floating-point formats include single-precision floating-point format, half-precision floating-point format, and 16-bit floating-point format.
3. The method according to claim 1, characterized in that, The precision adaptive processing operation includes a deterministic precision reduction operation, which truncates the high-order bits of the mantissa portions of the first operand and the second operand, so that the bit width of the first processing mantissa and the second processing mantissa is a predetermined value.
4. The method according to claim 3, characterized in that, The predetermined value is determined based on at least one of the following: the allowable precision loss for the target computation task, the mantissa bit width of the floating-point format, power consumption constraints, area constraints, or performance constraints.
5. The method according to claim 1, characterized in that, The precision adaptive processing operation includes a dynamic precision adjustment operation, which dynamically determines the effective bit width corresponding to the first processing mantissa and the second processing mantissa based on the numerical range, distribution characteristics or statistical information corresponding to the first operand and / or the second operand.
6. The method according to claim 1, characterized in that, The first arithmetic unit configured according to the effective bit width includes: Among multiple arithmetic units with different fixed operation bit widths, the arithmetic unit whose operation bit width matches the effective bit width is determined as the first arithmetic unit; or, The configurable array of computing units is logically partitioned or resource-mapped to form the first computing unit; the bit width of the first computing unit is matched with the effective bit width.
7. The method according to claim 1, characterized in that, The number of physical channels is an integer multiple of the number of threads in the thread bundle, and the integer is greater than or equal to 1, so that the arithmetic operations corresponding to each thread in the thread bundle can be executed in parallel within a single operation cycle.
8. The method according to claim 1, characterized in that, The step of generating an arithmetic operation result based on the exponent portions of the first operand and the second operand, and the intermediate mantissa result, includes: Perform exponential operations corresponding to the arithmetic operations on the exponent portions of the first operand and the second operand respectively to obtain the exponential processing result; The exponent processing result and the intermediate mantissa result are aligned and normalized to generate the arithmetic operation result.
9. A data processing system, characterized in that, include: The instruction decoding module is used to decode the received calculation instruction, which is used to instruct the execution of floating-point arithmetic operations on a first operand and a second operand. The first operand and the second operand are represented in floating-point format, and the floating-point format has a first mantissa bit width. An operand acquisition module, connected to the instruction decoding module, is used to acquire the first operand and the second operand based on the decoding result; The operand operation module, connected to the operand acquisition module, includes multiple parallel first physical channels, the number of which is not less than the number of threads in a thread bundle, for synchronously executing the floating-point arithmetic operations. Each of the first physical channels includes: The mantissa processing unit is used to perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively to obtain a first processed mantissa corresponding to the first operand and a second processed mantissa corresponding to the second operand; the effective bit width corresponding to the first processed mantissa and the second processed mantissa is smaller than the first mantissa bit width. The multiplier unit, whose operational bit width matches the effective bit width, is used to perform multiplication on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result; An exponent processing unit is used to perform an exponent operation corresponding to the arithmetic operation on the exponent portions of the first operand and the second operand respectively, to obtain an exponent processing result. The result synthesis unit is used to generate an arithmetic operation result based on the exponent processing result and the intermediate mantissa result; The write-back module, connected to the operand operation module, is used to write the arithmetic operation result back to the register.
10. The data processing system according to claim 9, characterized in that, The operand operation module also includes multiple second physical channels configured in parallel. The second physical channels are used to perform arithmetic operations in half-precision floating-point format or 16-bit floating-point format. The number of the second physical channels is an integer multiple of the number of threads in the thread bundle.
11. The data processing system according to claim 10, characterized in that, The number of the second physical channels is twice the number of the first physical channels.
12. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire calculation instructions, which are used to perform floating-point arithmetic operations on a first operand and a second operand. The first operand and the second operand are represented in floating-point format, and the floating-point format has a first mantissa bit width. The precision processing module is used to perform precision adaptive processing on the mantissa portions corresponding to the first operand and the second operand respectively, to obtain a first processed mantissa corresponding to the first operand and a second processed mantissa corresponding to the second operand; the effective bit width corresponding to the first processed mantissa and the second processed mantissa is smaller than the bit width of the first mantissa; The arithmetic unit configuration module is used to configure a corresponding first arithmetic unit according to the effective bit width, so as to perform a multiplication operation on the first processed mantissa and the second processed mantissa to generate an intermediate mantissa result; the arithmetic bit width of the first arithmetic unit is smaller than the arithmetic bit width of the second arithmetic unit, and the second arithmetic unit is used to perform a multiplication operation on the mantissa portions corresponding to the first operand and the second operand respectively; The arithmetic operation result generation module is used to generate an arithmetic operation result based on the exponent parts corresponding to the first operand and the second operand, as well as the intermediate mantissa result. The floating-point arithmetic operations are executed synchronously by multiple parallel physical channels, the number of which is configured to be no less than the number of threads in a thread bundle, so that parallel processing of at least one complete thread bundle can be completed in a single operation cycle.
13. A data processing device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the data processing method as described in any one of claims 1 to 8.
14. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the data processing method as described in any one of claims 1 to 8.
15. A computer program product, characterized in that, The computer program product includes at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the data processing method as described in any one of claims 1 to 8.