Floating point calculation method, device, electronic device and storage medium
By performing exponential operation and mantissa pre-alignment of floating-point vectors to avoid order shifts, the problem of low efficiency of floating-point vector multiplication and addition operations in AI processors is solved, and more efficient calculations and lower costs are achieved.
Patent Information
- Application Number
- CN202410268795.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-08
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-03-08
AI Technical Summary
Existing AI processors need to perform order shift operations when performing floating-point vector multiplication and addition operations, which increases chip complexity and cost and reduces computing efficiency.
By performing exponential operation and pre-alignment of floating-point vectors, avoiding order shift operations during multiplication and addition operations, and performing multiplication and addition calculations directly.
It improves computing efficiency, reduces algorithm complexity and chip cost, and reduces the loss of effective values in the mantissa and improves calculation accuracy.
Smart Images

Figure CN118034642B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a floating-point operation method, device, electronic device and storage medium. Background Art
[0002] At present, inner products are widely used in some mainstream AI (Artificial Intelligence) processors in the industry. For example, in the reasoning and training of neural networks, all network layers involving convolution require a large number of multiplication and addition operations. In the reasoning and training of neural networks, the commonly used data type is floating point. Compared with fixed-point data, floating-point data has a larger dynamic range and can represent a much larger range of values than fixed-point numbers.
[0003] As the size and computational complexity of neural network (NN) models continue to increase, traditional AI processors usually perform a large amount of data movement between independent computing and memory. In-memory computing (CIM) reduces data movement and improves computing efficiency by integrating multiplication and addition operations into memory, such as static random access memory (SRAM) and resistive RAM (ReRAM).
[0004] In the prior art, when performing multiplication and addition operations on floating-point vectors, the floating-point elements at the same position in the vector are first multiplied by a floating-point multiplier, and then the multiplication results are hierarchically added through a floating-point addition tree to finally obtain the calculation result. Since the exponents of the multiplication results at different positions in the vector are different, it is necessary to perform an aligning shift operation on the multiplication results before the addition operation, that is, the exponents of the multiplication results are aligned and the mantissas are shifted accordingly. Performing aligning shift operations in the memory computing unit will increase the complexity and cost of the chip, reduce computing efficiency and increase storage requirements. Summary of the invention
[0005] In view of the above problems, the purpose of the present invention is to provide a floating-point operation method, device, electronic device and storage medium, which eliminates the need for order shift operations during multiplication and addition operations, improves computing efficiency, and reduces algorithm complexity and chip cost.
[0006] According to a first aspect of the present invention, there is provided a floating-point operation method, comprising: obtaining a first vector and a second vector according to a model operation instruction, wherein the first vector includes a plurality of first elements, the second vector includes a plurality of second elements, the first elements and the second elements are floating-point data, and the first elements and the second elements correspond one to one; performing an exponential operation on all elements in the first vector and all elements in the second vector to obtain an exponential calculation result; performing mantissa pre-alignment on all elements in the first vector and all elements in the second vector according to the exponential calculation result to obtain a first mantissa vector and a second mantissa vector; performing multiplication and addition operation on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication and addition result; and performing normalization processing according to the exponential calculation result and the mantissa multiplication and addition result to obtain a model operation result.
[0007] Preferably, performing exponential operation on all elements in the first vector and all elements in the second vector to obtain an exponential calculation result includes: taking out exponential parts of multiple first elements and multiple second elements respectively to obtain multiple first exponents and multiple second exponents; adding the multiple first exponents and the multiple second exponents correspondingly to generate multiple exponential sums; determining the largest exponential sum from the multiple exponential sums as the maximum exponential sum; and using the multiple exponential sums and the maximum exponential sum as the exponential calculation result.
[0008] Preferably, performing mantissa pre-alignment on all elements in the first vector and all elements in the second vector according to the exponent calculation result to obtain a first mantissa vector and a second mantissa vector includes: preprocessing the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector to obtain multiple first intermediate mantissas and multiple second intermediate mantissas, wherein the first intermediate mantissas and the second intermediate mantissas both include multiple redundant bits; subtracting the maximum exponent sum from the multiple exponent sums to obtain multiple exponent differences; calculating a first shift value of the multiple first intermediate mantissas and a second shift value of the multiple second intermediate mantissas according to the multiple exponent differences, the sum of the first shift value and the second shift value at the corresponding position being the corresponding exponent difference, and the first shift value and the second shift value being non-negative integers; performing mantissa pre-alignment on the multiple first intermediate mantissas and the multiple second intermediate mantissas according to the first shift value and the second shift value to obtain the first mantissa vector and the second mantissa vector.
[0009] Preferably, calculating the first shift value of all first elements in the first vector and the second shift value of all second elements in the second vector according to the multiple exponent difference values includes: calculating the number of redundant bits for the multiple first intermediate mantissas and the multiple second intermediate mantissas respectively to obtain multiple first redundant numbers and multiple second redundant numbers; adding the multiple first redundant numbers and the multiple second redundant numbers correspondingly to generate multiple redundant sums; comparing the multiple exponent difference values with the corresponding redundant sums; when the exponent difference value is less than or equal to the redundant sum, calculating the first shift value and the second shift value according to the first redundant number and the exponent difference value; when the exponent difference value is greater than the redundant sum, calculating the first shift value and the second shift value according to the average of the exponent difference values.
[0010] Preferably, calculating the first shift value and the second shift value based on the first redundant number and the exponent difference includes: comparing the exponent difference with the first redundant number; when the exponent difference is less than or equal to the first redundant number, using the exponent difference as the first shift value and the second shift value is 0; when the exponent difference is greater than the first redundant number, using the first redundant number as the first shift value, and using the difference between the exponent difference and the first redundant number as the second shift value.
[0011] Preferably, calculating the first shift value and the second shift value based on the mean of the exponent difference includes: when the exponent difference is a positive even number, taking the mean value of the exponent difference as the first shift value and taking the difference between the exponent difference and the first shift value as the second shift value; when the exponent difference is a positive odd number, taking the mean value of the difference between the exponent difference and 1 as the first shift value and taking the difference between the exponent difference and the first shift value as the second shift value.
[0012] Preferably, calculating the first shift value and the second shift value according to the mean of the exponential difference includes: performing a subtraction operation on the exponential difference and the redundant sum to obtain the difference to be allocated; when the difference to be allocated is a positive even number, taking the sum of the average value of the difference to be allocated and the first redundant number as the first shift value, and taking the difference between the exponential difference and the first shift value as the second shift value; when the difference to be allocated is a positive odd number, taking the sum of the average value of the difference between the difference to be allocated and 1 and the first redundant number as the first shift value, and taking the difference between the exponential difference and the first shift value as the second shift value.
[0013] Preferably, the floating-point operation method further includes: grouping the multiple first elements in the first vector and the multiple second elements in the second vector to obtain L first sub-vectors and L second sub-vectors, respectively, the first sub-vector includes M first elements, the second sub-vector includes M second elements, and L and M are both positive integers; extracting the exponent part of all the first elements in the L first sub-vectors and all the second elements in the L second sub-vectors to obtain L first exponent calculation results; performing mantissa pre-alignment on the L first sub-vectors and the L second sub-vectors according to the L first exponent calculation results to obtain L first mantissa sub-vectors and L second mantissa sub-vectors; performing multiplication and addition operations on the L first mantissa sub-vectors and the L second mantissa sub-vectors to obtain L mantissa multiplication and addition results; obtaining a first sum vector according to the L first exponent calculation results and the L mantissa multiplication and addition results, the first sum vector including the L first sum results.
[0014] Preferably, the floating-point operation method also includes: taking out the exponent part of all the first sum results in the first sum vector respectively to obtain the second exponent calculation result; shifting all the first sum results according to the second exponent result to obtain multiple first and intermediate mantissas; summing the multiple first and intermediate mantissas to obtain the second sum result; and obtaining the model operation result according to the second exponent calculation result and the second sum result.
[0015] Preferably, the floating-point operation method also includes: grouping the first sum vector to obtain multiple first sub-sum vectors, each first sub-sum vector includes P first sum results, P is a positive integer; respectively taking out the exponential part of all the first sum results in each sub-sum vector to obtain a second exponential calculation result; shifting all the first sum results in each sub-sum vector according to the second exponential calculation result to obtain a corresponding intermediate sum vector; summing all the elements in the intermediate sum vector to obtain a corresponding second sum result; normalizing the second sum result according to the second exponential calculation result to obtain a third sum result, and the third sum result is floating-point data; and performing floating-point summing on multiple third sum results to obtain a model operation result.
[0016] According to a second aspect of the present invention, there is provided a floating-point operation device, comprising: an acquisition module, for acquiring a first vector and a second vector according to a model operation instruction, wherein the first vector includes a plurality of first elements, the second vector includes a plurality of second elements, the first elements and the second elements are floating-point data, and the first elements and the second elements correspond to each other one by one; an exponential operation module, for performing an exponential operation on all elements in the first vector and all elements in the second vector to obtain an exponential calculation result; a mantissa pre-alignment module, for performing mantissa pre-alignment on all elements in the first vector and all elements in the second vector according to the exponential calculation result to obtain a first mantissa vector and a second mantissa vector; a multiplication-addition operation module, for performing a multiplication-addition operation on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication-addition result; and a shift module, for performing normalization processing on the exponential calculation result and the mantissa multiplication-addition result to obtain a model operation result.
[0017] Preferably, the exponential operation module is also used to extract the exponential parts of multiple first elements and multiple second elements respectively to obtain multiple first exponents and multiple second exponents; add the multiple first exponents and the multiple second exponents correspondingly to generate multiple exponential sums; determine the largest exponential sum from the multiple exponential sums as the maximum exponential sum; and use the multiple exponential sums and the maximum exponential sum as the exponential calculation results.
[0018] Preferably, the mantissa pre-alignment module includes: a preprocessing unit, used to preprocess the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector to obtain multiple first intermediate mantissas and multiple second intermediate mantissas, wherein the first intermediate mantissas and the second intermediate mantissas both include multiple redundant bits; a difference calculation unit, used to subtract the maximum exponent sum from the multiple exponent sums to obtain multiple exponent differences; a shift value calculation unit, used to calculate the first shift value of all first elements in the first vector and the second shift value of all second elements in the second vector according to the multiple exponent differences, the sum of the first shift value and the second shift value at the corresponding position is the corresponding exponent difference, and the first shift value and the second shift value are non-negative integers; a mantissa pre-alignment unit, used to perform mantissa pre-alignment on the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector according to the first shift value and the second shift value to obtain a first mantissa vector and a second mantissa vector.
[0019] Preferably, the floating-point operation device also includes: a vector grouping module, used to group multiple first elements in the first vector and multiple second elements in the second vector to obtain L first sub-vectors and L second sub-vectors, respectively, the first sub-vector includes M first elements, and the second sub-vector includes M second elements.
[0020] Preferably, the exponential operation module is also used to extract the exponent part of all the first elements in the L first sub-vectors and all the second elements in the L second sub-vectors respectively to obtain L first exponent calculation results; the mantissa pre-alignment module is also used to perform mantissa pre-alignment on the L first sub-vectors and the L second sub-vectors according to the L first exponent calculation results to obtain L first mantissa sub-vectors and L second mantissa sub-vectors; the multiplication and addition operation module is also used to perform multiplication and addition operations on the L first mantissa sub-vectors and the L second mantissa sub-vectors to obtain L mantissa multiplication and addition results; the floating-point operation device also includes: a sum module, which is used to obtain a first sum vector according to the L first exponent calculation results and the L mantissa multiplication and addition results, and the first sum vector includes L first sum results.
[0021] Preferably, the exponential operation module is also used to extract the exponential part of all the first sum results in the first sum vector to obtain the second exponential calculation result; the shift module is also used to shift all the first sum results by the right order according to the second exponential result to obtain multiple first and intermediate mantissas; the sum module is also used to sum the multiple first and intermediate mantissas to obtain the second sum result; and obtain the model operation result according to the second exponential calculation result and the second sum result.
[0022] Preferably, the vector grouping module is also used to group the first sum vector to obtain multiple first sub-sum vectors, each first sub-sum vector includes P first sum results, P is a positive integer; the exponential operation module is also used to take out the exponential part of all the first sum results in each sub-sum vector to obtain a second exponential calculation result; the shift module is also used to shift all the first sum results in each sub-sum vector according to the second exponential calculation result to obtain a corresponding intermediate sum vector; the sum module is also used to sum all the elements in the intermediate sum vector to obtain a corresponding second sum result; the shift module is also used to normalize the second sum result according to the second exponential calculation result to obtain a third sum result, and the third sum result is floating-point data; the sum module is also used to sum multiple third sum results as floating-point numbers to obtain a model operation result.
[0023] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned floating-point operation method when executing the computer program.
[0024] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the floating-point operation method described above is implemented. According to a fifth aspect of the present invention, a computer program product or a computer program is provided, and the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the methods provided by the above aspects.
[0025] The floating-point operation method, device, electronic device and storage medium provided by the present invention perform mantissa pre-alignment on all elements in a first vector and a second vector to obtain a first mantissa vector and a second mantissa vector, perform multiplication and addition operations on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication and addition result, and directly perform the multiplication and addition operations in a calculation unit without performing an order shift operation during the multiplication and addition operation, thereby improving calculation efficiency and reducing algorithm complexity and chip cost.
[0026] Furthermore, preprocessing all elements in the first vector and the second vector to increase redundant bits, and then pre-aligning the mantissas of all elements in the first vector and the second vector can reduce the loss of valid values in the mantissa, thereby reducing precision loss and improving calculation accuracy.
[0027] Furthermore, the first vector and the second vector are grouped, and the mantissas of all elements of each group of sub-vectors are pre-aligned before multiplication and addition operations are performed directly. The grouping method and the number of elements of each group of sub-vectors can be determined according to the bit width of the storage and computing unit, thereby improving the configurability of the storage and computing unit, making it suitable for storage and computing units with different bit widths and having high versatility. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0029] Figure 1 A flow chart showing a floating-point operation method provided by a first embodiment of the present invention;
[0030] Figure 2 A flowchart of step S130 in the floating-point operation method provided by the first embodiment of the present invention is shown;
[0031] Figure 3 A flow chart showing a floating-point operation method provided by a second embodiment of the present invention;
[0032] Figure 4 A calculation schematic diagram of a floating-point operation method provided by a second embodiment of the present invention is shown;
[0033] Figure 5 A flow chart showing a floating-point operation method provided by a third embodiment of the present invention;
[0034] Figure 6 A calculation schematic diagram of a floating-point operation method provided by a third embodiment of the present invention is shown;
[0035] Figure 7 A schematic diagram showing the structure of a floating-point operation device provided by an embodiment of the present invention is shown;
[0036] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0037] Various embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, the same elements are represented by the same or similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not drawn to scale.
[0038] The specific implementation of the present invention is further described in detail below in conjunction with the drawings and examples.
[0039] Figure 1 FIG. 1 is a flow chart showing a floating point operation method provided by an embodiment of the present invention. Figure 1 As shown, the floating-point operation method provided in the embodiment of the present invention can be applied to the inner product operation scenario, and the floating-point operation method includes the following steps.
[0040] In step S110, a first vector and a second vector are obtained according to a model operation instruction.
[0041] In this embodiment, the first vector and the second vector are input data of the model. The first vector includes a plurality of first elements, the second vector includes a plurality of second elements, the first elements and the second elements are floating point data, and the first elements and the second elements correspond one to one. The floating point data have the same floating point number type, such as fp32, bf16 or fp16, fp8, etc.
[0042] It can be understood that the model includes a neural network model, and the model operation instruction refers to an instruction for performing an inner product operation on two vectors. The first vector and the second vector are vectors for performing an inner product operation, and are vectors input to the model. First, the first vector and the second vector are obtained according to the model operation instruction, wherein the first vector includes N first elements, and the second vector includes N second elements.
[0043] The floating point format of floating point numbers generally follows the IEEE754 standard, which includes three parts: S / E / M. S is the sign bit (Sign), E is the exponent bit (exponent, exp), and M is the mantissa bit (mantissa, man), as shown in the following table. Common floating point numbers include FP16, FP32, and FP64. The sign bit is 1 bit, and the data bit width occupied by the exponent bit and mantissa bit of different floating point types is different.
[0044] Sign bit (sign) Exponent (exp) Mantissa (man) FP16 1bit 5bit 10bit FP32 1bit 8bit 23bit FP64 1bit 11bit 52bit
[0045] For two floating point numbers A and B, A = (-1) Sa ×Ma×2 Ea , B=(-1) Sb ×Mb×2 Eb , where Sa is the sign bit of A, Ma is the mantissa of A, and Ea is the exponent of A. The result of A×B is: A×B=(-1) Sa+Sb ×Ma×Mb×2 (Ea+Eb) ; It can be seen that the fixed-point multiplication result of Ma×Mb and the fixed-point addition result of Ea+Eb need to be calculated. For the floating-point inner product operation, that is, the multiplication and addition operation of two vectors is completed, the vector a=[a0,a1,……,a N-1 ] and b=[b0,b1,……,b N-1 ] is defined as: a·b=a0×b0+a1×b1+……+a N-1 ×b N-1 .
[0046] In step S120, exponential operation is performed on all elements in the first vector and all elements in the second vector to obtain an exponential calculation result.
[0047] In this embodiment, the index parts of the first elements and the second elements are respectively taken out to obtain a plurality of first indexes (Ea0, Ea1, ... Ea N-1 ) and a plurality of second indices (Eb0, Eb1, ... Eb N-1 ); add the first indexes and the second indexes correspondingly to generate the sum of the indexes (Ea0+Eb0, Ea1+Eb1, ... Ea N-1 +Eb N-1 ); determine the largest index sum from the multiple index sums as the maximum index sum Emax; and convert the multiple index sums (Ea0+Eb0, Ea1+Eb1, ... Ea N-1 +Eb N-1 ) and the maximum index and Emax as the index calculation results.
[0048] In step S130, mantissa pre-alignment is performed on all elements in the first vector and all elements in the second vector according to the exponent calculation result to obtain a first mantissa vector and a second mantissa vector.
[0049] In this embodiment, the maximum index sum Emax is added to the multiple index sums (Ea0+Eb0, Ea1+Eb1, ... Ea N-1 +Eb N-1 ) are subtracted to obtain multiple index differences (ΔE0, ΔE1, ..., ΔE N-1 ), where ΔE0=Emax-(Ea0+Eb0), ΔE1=Emax-(Ea1+Eb1), ΔE N-1 =Emax-(Ea N-1 +Eb N-1 ). The first shift value Va of the mantissa of all the first elements in the first vector and the second shift value Vb of the mantissa of all the second elements in the second vector are calculated according to the multiple exponent differences, and the sum of the first shift value Va of the mantissa of the first element and the second shift value Vb of the mantissa of the second element is equal to the corresponding exponent difference ΔE, for example, Va0+Vb0=ΔE0, Va1+Vb1=ΔE1, ..., Va N-1 +Vb N-1 =ΔE N-1 The mantissa Ma of the first element and the mantissa Mb of the second element are respectively pre-aligned according to the first shift value Va and the second shift value Vb (i.e., the mantissa Ma of the first element is shifted right by Va digits and the mantissa Mb of the second element is shifted by Vb digits), to obtain the first mantissa vector Mas and the second mantissa vector Mbs.
[0050] In a preferred embodiment, see Figure 2 , step S130 includes step S131-step S134.
[0051] In step S131, the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector are preprocessed to obtain a plurality of first intermediate mantissas and a plurality of second intermediate mantissas, wherein the first intermediate mantissas and the second intermediate mantissas each include a plurality of redundant bits.
[0052] In this embodiment, the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector are read and preprocessed to obtain multiple first intermediate mantissas Mar and multiple second intermediate mantissas Mbr. Specifically, the mantissas of the multiple first elements and the mantissas of the multiple second elements are respectively saved in a higher precision fixed-point number format, the first intermediate mantissa has multiple redundant bits added to the mantissa of the first element, and the second intermediate mantissa has multiple redundant bits added to the mantissa of the second element, for example, saved in the form of complement code or original code.
[0053] Taking FP16 as an example, the 10-bit mantissa Ma of the first element is read, and a 16-bit fixed-point number (INT16) is used to save the mantissa Ma of the first element to obtain the first intermediate mantissa Mar. Similarly, the 10-bit mantissa Mb of the second element is read, and a 16-bit fixed-point number (INT16) is used to save the mantissa Mb of the first element to obtain the first intermediate mantissa Mbr. For example, for the normalized elements, an implicit 1 is added to the left of the highest bit of the 10-bit mantissa M. At this time, the mantissa M requires 11 bits. The mantissa M is converted into a complement code according to the sign bit. At this time, the complement code requires at least 12 bits. 16 bits are used to save the mantissa M. In order to improve the accuracy, the complement code is shifted left by 4 bits. At this time, the complement code format is Q2.14, that is, the integer part is 2 bits and the decimal part is 14 bits.
[0054] In step S132, a subtraction operation is performed on the maximum exponential sum and the multiple exponential sums to obtain multiple exponential differences.
[0055] In this embodiment, the maximum index sum Emax is added to the multiple index sums (Ea0+Eb0, Ea1+Eb1, ... Ea N-1 +Eb N-1 ) are subtracted to obtain multiple index differences (ΔE0, ΔE1, ΔE N-1 ), where ΔE0=Emax-(Ea0+Eb0), ΔE1=Emax-(Ea1+Eb1), ΔE N-1 =Emax-(Ea N-1 +Eb N-1 ). The first shift value Va of the mantissa of all the first elements in the first vector and the second shift value Vb of the mantissa of all the second elements in the second vector are calculated according to the multiple exponent differences, and the sum of the first shift value Va of the mantissa of the first element and the second shift value Vb of the mantissa of the second element is equal to the corresponding exponent difference ΔE, for example, Va0+Vb0=ΔE0, Va1+Vb1=ΔE1, ..., Va N-1 +Vb N-1 =ΔE N-1 .
[0056] In step S133, the first shift value of all the first elements in the first vector and the second shift value of all the second elements in the second vector are calculated according to the multiple exponent differences, the sum of the first shift value and the second shift value at the corresponding positions is the corresponding exponent difference, and the first shift value and the second shift value are non-negative integers.
[0057] In this embodiment, the number of redundant bits is calculated for multiple first intermediate mantissas Mar and multiple second intermediate mantissas Mbr respectively to obtain multiple first redundant numbers Ra and multiple second redundant numbers Rb; the multiple first redundant numbers Ra and the multiple second redundant numbers Rb are added correspondingly to generate multiple redundant sums Rsum; the multiple exponent difference values ΔE are compared with the corresponding redundant sums Rsum; when the exponent difference value ΔE is less than or equal to the redundant sum Rsum, the first shift value Va and the second shift value Vb are calculated according to the first redundant number Ra and the exponent difference value ΔE; when the exponent difference value ΔE is greater than the redundant sum Rsum, the first shift value Va and the second shift value Vb are calculated according to the average of the exponent difference values ΔE.
[0058] Specifically, when the exponential difference ΔE is less than or equal to the redundancy sum Rsum, the exponential difference ΔE is compared with the first redundancy number Ra; when the exponential difference ΔE is less than or equal to the first redundancy number Ra, the exponential difference ΔE is used as the first shift value Va, and the second shift value Vb is 0, that is, Va=ΔE, Vb=0; when the exponential difference ΔE is greater than the first redundancy number Ra, the first redundancy number Ra is used as the first shift value Va, and the difference between the exponential difference ΔE and the first redundancy number Ra is used as the second shift value Vb, that is, Va=Ra, Vb=ΔE-Ra.
[0059] When the exponential difference ΔE is greater than the redundancy sum Rsum, when the exponential difference ΔE is a positive even number, the average value of the exponential difference ΔE is used as the first shift value Va, and the difference between the exponential difference ΔE and the first shift value Va is used as the second shift value Vb, that is, Va=ΔE / 2, Vb=ΔE-Va=ΔE / 2; when the exponential difference ΔE is a positive odd number, the average value of the difference between the exponential difference ΔE and 1 is used as the first shift value Va, and the difference between the exponential difference ΔE and the first shift value Va is used as the second shift value Vb, that is, Va=(ΔE-1) / 2, Vb=ΔE-Va=(ΔE+1) / 2.
[0060] In a preferred embodiment, when the exponential difference ΔE is greater than the redundancy sum Rsum, a subtraction operation is performed on the exponential difference ΔE and the redundancy sum Rsum to obtain a difference to be allocated ΔEk=ΔE-Ra-Rb; when the difference to be allocated ΔEk is a positive even number, the sum of the average value of the difference to be allocated ΔEk and the first redundancy number Ra is used as the first shift value Va, and the difference between the exponential difference ΔE and the first shift value Va is used as the second shift value Vb, that is, Va=Ra+(ΔE-Rsum) / 2=( When the difference value ΔEk to be allocated is a positive odd number, the sum of the average value of the difference between the difference value ΔEk to be allocated and 1 and the first redundant number Ra is used as the first shift value Va, and the difference between the exponential difference ΔE and the first shift value Va is used as the second shift value Vb, that is, Va=Ra+(ΔE-Rsum-1) / 2=(ΔE+Ra-Rb-1) / 2, Vb=ΔE-Va=(ΔE-Ra+Rb+1) / 2.
[0061] In step S134, mantissa pre-alignment is performed on a plurality of first intermediate mantissas and a plurality of second intermediate mantissas according to the first shift value and the second shift value to obtain a first mantissa vector and a second mantissa vector.
[0062] In this embodiment, a plurality of first intermediate mantissas Mar are right-shifted by a first shift value Va and a plurality of second intermediate mantissas Mbr are right-shifted by a second shift value Vb to perform mantissa pre-alignment to obtain a first mantissa vector Mas and a second mantissa vector Mbs, wherein the first mantissa vector Mas=[Mas0, Mas1, ..., Mas N-1 ], the second mantissa vector Mbs = [Mbs0, Mbs1, ..., Mbs N-1 ]. In the process of right shifting the mantissa, RNE (round to nearest even) can be used to make a rounding to improve the accuracy.
[0063] In step S140, a multiplication and addition operation is performed on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication and addition result.
[0064] In this embodiment, the first mantissa vector Mas and the second mantissa vector Mbs are multiplied and added to obtain a mantissa multiplication and addition result Macc, Macc=Mas0×Mbs0+Mas1×Mbs1+…+Mas N-1 ×Mbs N-1 .
[0065] In step S150, a normalization process is performed on the exponent calculation result and the mantissa multiplication and addition result to obtain a model operation result.
[0066] In this embodiment, the exponent calculation result and the mantissa multiplication and addition result are used to construct a normalized floating point number. The mantissa multiplication and addition result is shifted, and the exponent calculation result is processed according to the shift to be within the normalized range of the floating point type.
[0067] The floating-point operation method provided by the present invention performs mantissa pre-alignment on all elements in a first vector and a second vector to obtain a first mantissa vector and a second mantissa vector, performs multiplication and addition operations on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication and addition result, and directly performs the multiplication and addition operations in a calculation unit without performing an order shift operation in the accumulation process, thereby improving calculation efficiency and reducing algorithm complexity and chip cost.
[0068] Furthermore, preprocessing all elements in the first vector and the second vector to increase redundant bits, and then pre-aligning the mantissas of all elements in the first vector and the second vector can reduce the loss of valid values in the mantissa, thereby reducing precision loss and improving calculation accuracy.
[0069] Figure 3 FIG. 2 is a flow chart showing a floating point operation method provided by the second embodiment of the present invention. Figure 3 As shown, the floating-point operation method includes the following steps.
[0070] In step S201, a first vector and a second vector are obtained according to a model operation instruction.
[0071] In this embodiment, the first vector and the second vector are input data of the model. The first vector includes a plurality of first elements, the second vector includes a plurality of second elements, the first elements and the second elements are floating point data, and the first elements and the second elements correspond one to one. The floating point data have the same floating point number type, such as fp32, bf16 or fp16, fp8, etc.
[0072] It can be understood that the model includes a neural network model, and the model operation instruction refers to an instruction for performing an inner product operation on two vectors. The first vector and the second vector are vectors for performing an inner product operation, and are vectors input to the model. First, the first vector and the second vector are obtained according to the model operation instruction, wherein the first vector includes N first elements, and the second vector includes N second elements.
[0073] In step S202, the first vector and the second vector are grouped to obtain L first sub-vectors and L second sub-vectors, respectively. The first sub-vector includes M first elements, and the second sub-vector includes M second elements.
[0074] In this embodiment, the number of elements contained in each sub-vector is calculated according to the bit width of the storage and computing unit and the data type of the element. For example, the bit width of the storage and computing unit is 128 bits, and the floating-point type of the element is FP16, that is, the number of bits required to store each element is 16 bits, and each sub-vector contains 8 elements.
[0075] Specifically, see Figure 4 , if N is an integer multiple of M, then L = N / M, and the first subvectors are a1 = [a0, a1, ..., a M-1 ],a2=[a M , a M+2 , ..., a 2M-1 ],……,aL=[a N-M , a N-M , ..., a N-1 ]; the second sub-vectors are b1=[b0, b1, ..., b M-1 ],b2=[b M , b M+2 , ..., b 2M-1 ],……,bL=[b N-M , b N-M , ..., b N-1 ].
[0076] If N is not an integer multiple of M, the first vector and the second vector need to be expanded, and then the expanded vectors are grouped. For example, Mc elements (c is the remainder of N divided by M) can be added after the last elements of the first vector and the second vector, and the added elements are all 0, so L = k + 1 (k is the quotient of N divided by M); then the first sub-vectors are a1 = [a0, a1, ..., a M-1 ],a2=[a M , a M+2 , ..., a 2M-1 ],……,aL=[a kM , a kM+1 , ..., a kM+M-1 ]; the second sub-vectors are b1=[b0, b1, ..., b M-1 ],b2=[b M , b M+2 , ..., b 2M-1 ],……,bL=[b kM , b kM+1 , ..., b kM+M-1 ].
[0077] In step S203, exponential parts are taken out from all first elements in the L first sub-vectors and all second elements in the L second sub-vectors to obtain L first exponential calculation results.
[0078] In this embodiment, for each first sub-vector in the L first sub-vectors and each second sub-vector in the L second sub-vectors, the exponent part is taken out from the first element and the second element with the same position corresponding to each other, and the corresponding first exponent calculation result is calculated. For example, the exponent part is taken out from all the first elements in the i-th first sub-vector and all the second elements in the i-th second sub-vector to obtain a plurality of first exponents (Ea (i-1)M , Ea (i-1)M+1 , ... Ea iM-1 ) and multiple second indices (Eb (i-1)M , Eb (i-1)M+1 , ... Eb 3M-1 ); add the multiple first indices and the multiple second indices correspondingly to generate the sum of multiple indices of the third sub-vector (Ea (i-1)M +Eb (i-1)M , Ea (i-1)M+1 +Eb (i-1)M+1 , ... Ea iM-1 +Eb iM-1 ); determine the maximum exponential sum from the multiple exponential sums of the i-th sub-vector as the maximum exponential sum E1max_i of the i-th sub-vector; and convert the multiple exponential sums (Ea (i-1)M +Eb (i-1)M , Ea (i-1)M+1 +Eb (i-1)M+1 , ... Ea iM-1 +Eb iM-1 ) and the maximum index and E1max_i are used as the calculation results of the first index of the i-th sub-vector, where i is a positive integer and i≤L.
[0079] In step S204, mantissa pre-alignment is performed on the L first sub-vectors and the L second sub-vectors according to the calculation results of the L first exponents to obtain L first mantissa sub-vectors and L second mantissa sub-vectors.
[0080] In this embodiment, for each first subvector in the L first subvectors and each second subvector in the L second subvectors, the first elements and the second elements with the same corresponding positions are pre-aligned to obtain the L first mantissa subvectors and the L second mantissa subvectors. For example, according to the calculation result of the first exponent of the i-th subvector, all the first elements in the i-th first subvector and all the second elements in the i-th second subvector are pre-aligned to obtain the i-th first mantissa subvector and the i-th second mantissa subvector.
[0081] Specifically, the maximum index sum E1max_i of the ith sub-vector and the multiple index sums (Ea (i-1)M +Eb (i-1)M , Ea (i-1)M+1 +Eb(i-1)M+1 , ... Ea iM-1 +Eb iM-1 ) is subtracted to obtain multiple index differences (ΔE (i-1)M , ΔE (i-1)M+1 , ..., ΔE iM-1 ), where ΔE (i-1)M =E1max_i-(Ea (i-1)M +Eb (i-1)M ),ΔE (i-1)M+1 =E1max_i-(Ea (i-1)M+1 +Eb (i-1)M+1 ),ΔE iM-1 =E1max_i-(Ea iM-1 +Eb iM-1 ) Calculate the first shift value Va of the mantissa of all the first elements in the i-th sub-vector according to the multiple exponential differences of the i-th sub-vector i and the second shift value Vb of the mantissa of all the second elements in the i-th subvector i , the sum of the first shift value Va of the mantissa of the first element at the corresponding position and the second shift value Vb of the mantissa of the second element is equal to the corresponding exponent difference ΔE, for example, Va (i-1)M +Vb (i-1)M =ΔE (i-1)M , Va (i-1)M+1 +Vb (i-1)M+1 =ΔE (i-1)M+1 , ..., Va iM-1 +Vb iM-1 =ΔE iM-1 According to the first shift value Va and the second shift value Va, the mantissa Ma of the first element and the mantissa Mb of the second element are respectively pre-aligned (i.e., the mantissa Ma of the first element is shifted right by Va digits and the mantissa Mb of the second element is shifted by Vb digits), and the first mantissa subvector Mas of the i-th subvector is obtained. i and the second mantissa subvector Mbs i .
[0082] The mantissa pre-alignment operation of the first sub-vector and the second sub-vector at the corresponding position is the same as that in the first embodiment, and will not be described again.
[0083] In step S205, multiplication and addition operations are performed on the L first mantissa sub-vectors and the L second mantissa sub-vectors respectively to obtain L mantissa multiplication and addition results.
[0084] In this embodiment, for each first mantissa vector in the L first mantissa vectors and each second mantissa vector in the L first mantissa vectors, the first mantissa and the second mantissa corresponding to the same position are multiplied and added to obtain the mantissa multiplication and addition results at the corresponding positions. Therefore, the L first mantissa sub-vectors and the L second mantissa sub-vectors are multiplied and added to obtain L mantissa multiplication and addition results Msum.
[0085] In step S206, a first sum vector is obtained according to the L first exponent calculation results and the L mantissa multiplication and addition results, where the first sum vector includes the L first sum results.
[0086] In this embodiment, for the maximum exponent sum and the L mantissa multiplication and addition results among the L first exponent calculation results, the maximum exponent sum and the mantissa multiplication and addition results with the same position correspondence constitute the first sum result. That is, the i-th maximum exponent sum is used as the exponent of the i-th first sum result, and the i-th mantissa multiplication and addition result is used as the mantissa of the i-th first sum result. The L first sum results constitute the first sum vector Sum. Sum=[Sum1, Sum2, ..., Sum L ],Sum1=a1×b1=Msum1×2 E1max_1 , Sum2=a2×b2=Msum2×2 E1max_2 ; ..., Sum L =aL×bL=Msum L ×2 E1max_L .
[0087] In step S207, the exponential parts of all the first sum results in the first sum vector are respectively taken out to obtain the second exponential calculation results.
[0088] In this embodiment, the exponential parts of the L first sum results are taken out to obtain L first sum indices [E1max_1, E1max_2, ..., E1max_L], and the maximum exponent E2max and the L first sum indices [E1max_1, E1max_2, ..., E1max_L] among the L first sum indices are taken as the second exponent calculation results. Among them, the L first sum indices are the maximum exponent sum among the L first exponent calculation results, that is, the i-th first sum index is the maximum exponent sum among the i-th exponent calculation results.
[0089] In step S208, all first sum results are shifted and aligned according to the second exponent calculation result to obtain a plurality of first and intermediate mantissas.
[0090] In this embodiment, the maximum exponent E2max in the second exponent calculation result is subtracted from the L first sum exponents [E1max_1, E1max_2, ..., E1max_L] to obtain L exponent differences, and then the mantissas of the L first sum results are shifted according to the L exponent differences to obtain L first sum intermediate mantissas.
[0091] In step S209, a plurality of first and intermediate mantissas are summed to obtain a second sum result.
[0092] In this embodiment, L first and intermediate mantissas are added to obtain a second sum result Macc. The number of bits required to store the second sum result Macc changes dynamically according to the size of L.
[0093] In step S210, a model operation result is obtained according to the second index calculation result and the second sum result.
[0094] In this embodiment, the maximum exponent E2max in the second exponential calculation result and the second sum result Macc are used to construct a normalized floating point number; the model operation result Acc is obtained by shifting the second sum result Macc and processing the second exponential calculation result according to the shift so that it is within the normalized range of the floating point type.
[0095] The floating-point operation method provided by the present invention groups the first vector and the second vector, and directly performs multiplication and addition operations after pre-aligning the mantissas of all elements of each group of sub-vectors. The grouping method and the number of elements of each group of sub-vectors can be determined according to the bit width of the storage and calculation unit, thereby improving the configurability of the storage and calculation unit, making it suitable for storage and calculation units with different bit widths, and having high versatility.
[0096] Figure 5 FIG. 4 is a flow chart showing a floating point operation method provided by the third embodiment of the present invention. Figure 5 As shown, the floating-point operation method includes the following steps.
[0097] In step S301, a first vector and a second vector are obtained according to a model operation instruction.
[0098] In step S302, the first vector and the second vector are grouped to obtain L first sub-vectors and L second sub-vectors, respectively. The first sub-vector includes M first elements, and the second sub-vector includes M second elements.
[0099] In step S303, exponential parts are taken out from all first elements in the L first sub-vectors and all second elements in the L second sub-vectors to obtain L first exponential calculation results.
[0100] In step S304, mantissa pre-alignment is performed on the L first sub-vectors and the L second sub-vectors according to the calculation results of the L first exponents to obtain L first mantissa sub-vectors and L second mantissa sub-vectors.
[0101] In step S305, multiplication and addition operations are performed on the L first mantissa sub-vectors and the L second mantissa sub-vectors respectively to obtain L mantissa multiplication and addition results.
[0102] In step S306, a first sum vector is obtained according to the L first exponent calculation results and the L mantissa multiplication and addition results, where the first sum vector includes the L first sum results.
[0103] Steps S301 to S306 in this embodiment are the same as steps S201 to S206 in the first embodiment, and are not described again herein.
[0104] In step S307, the first sum vectors are grouped to obtain a plurality of first sub-sum vectors, each of which includes P first sum results.
[0105] In this embodiment, see Figure 6 , group the L first sum results in the first sum vector to obtain H first sub-sum vectors, each of which includes P first sum results. If L is an integer multiple of P, then H = L / P; if L is not an integer multiple of P, the first sum vector is expanded and then grouped again, and the number of zero elements that need to be added to the expansion of the first sum vector is Pd, d is the remainder of L divided by P, H = e+1, and e is the quotient of L divided by P.
[0106] In step S308, the exponential parts of all the first sum results in each sub-sum vector are taken out to obtain the second exponential calculation results.
[0107] In this embodiment, the exponents of all first sum results in each sub-sum vector and the maximum exponent are used as the second exponent calculation result.
[0108] In step S309, all first sum results in each sub-sum vector are shifted and aligned according to the second index calculation result to obtain multiple intermediate sum vectors.
[0109] In this embodiment, each sub-sum vector performs a subtraction operation according to its corresponding maximum exponent and the exponents of all first sum results in each sub-sum vector to obtain multiple exponent differences, and the mantissas of all first sum results are shifted according to the multiple exponent differences to obtain corresponding multiple first and intermediate mantissas, and the multiple first and intermediate mantissas constitute an intermediate sum vector. Multiple sub-vectors are pre-aligned by mantissas to obtain multiple intermediate sum vectors.
[0110] In step S310, all elements in each intermediate sum vector are summed to obtain a plurality of second sum results.
[0111] In this embodiment, all elements in each intermediate sum vector are summed to obtain a plurality of second sum results.
[0112] In step S311, the second sum result is normalized according to the second exponent calculation result to obtain a third sum result, and the third sum result is floating point data.
[0113] In this embodiment, the exponents in the second exponent calculation results with the same positions and the second sum result are used to construct a normalized floating point number. By shifting the third sum result Macc and processing the second exponent calculation result according to the shift so that it is within the normalized range of the floating point type, the third sum results Acc1, Acc2, ..., Acc are obtained. P Since the grouping is performed during the summation process so that the number of elements in each group is preset, the number of bits required to store the sum result is fixed.
[0114] In step S312, a plurality of third sum results are summed up as floating-point numbers to obtain a model operation result.
[0115] In this embodiment, multiple third sum results are directly added to obtain the model calculation result Acc.
[0116] The floating-point operation method provided by the present invention groups the first vector and the second vector, and directly performs multiplication and addition operations after pre-aligning the mantissas of all elements of each group of sub-vectors. The grouping method and the number of elements of each group of sub-vectors can be determined according to the bit width of the storage and calculation unit, thereby improving the configurability of the storage and calculation unit, making it suitable for storage and calculation units with different bit widths, and having high versatility.
[0117] Figure 7 FIG. 2 is a schematic diagram showing the structure of a floating-point operation device provided by an embodiment of the present invention. Figure 7 As shown, the floating point operation device includes an acquisition module 710 , an exponent operation module 720 , a mantissa pre-alignment module 730 , a multiplication and addition operation module 740 and a shift module 750 .
[0118] The acquisition module 710 is used to acquire a first vector and a second vector according to a model operation instruction, wherein the first vector includes multiple first elements, the second vector includes multiple second elements, the first elements and the second elements are floating-point data, and the first elements and the second elements correspond one to one.
[0119] In this embodiment, the first vector and the second vector are input data of the model. The first vector includes a plurality of first elements, the second vector includes a plurality of second elements, the first elements and the second elements are floating point data, and the first elements and the second elements correspond one to one. The floating point data have the same floating point number type, such as fp32, bf16 or fp16, fp8, etc.
[0120] For two floating point numbers A and B, A = (-1) Sa ×Ma×2 Ea , B=(-1) Sb ×Mb×2 Eb , where Sa is the sign bit of A, Ma is the mantissa of A, and Ea is the exponent of A. The result of A×B is: A×B=(-1) Sa+Sb ×Ma×Mb×2 (Ea+Eb) ; It can be seen that the fixed-point multiplication result of Ma×Mb and the fixed-point addition result of Ea+Eb need to be calculated. For the floating-point inner product operation, that is, the multiplication and addition operation of two vectors is completed, the vector a=[a0,a1,……,a N-1 ] and b=[b0,b1,……,b N-1 ] is defined as: a·b=a0×b0+a1×b1+……+a N-1 ×b N-1 .
[0121] The exponential operation module 720 is used to perform exponential operation on all elements in the first vector and all elements in the second vector to obtain an exponential calculation result.
[0122] In this embodiment, the exponential operation module 720 extracts the exponent part of each of the first elements and the second elements to obtain a plurality of first exponents (Ea0, Ea1, ..., Ea2). N-1 ) and a plurality of second indices (Eb0, Eb1, ... Eb N-1 ); add the first indexes and the second indexes correspondingly to generate the sum of the indexes (Ea0+Eb0, Ea1+Eb1, ... Ea N-1 +Eb N-1 ); determine the largest index sum from the multiple index sums as the maximum index sum Emax; and convert the multiple index sums (Ea0+Eb0, Ea1+Eb1, ... Ea N-1 +Eb N-1 ) and the maximum index and Emax as the index calculation results.
[0123] The mantissa pre-alignment module 730 is used to perform mantissa pre-alignment on all elements in the first vector and all elements in the second vector according to the exponent calculation result to obtain a first mantissa vector and a second mantissa vector.
[0124] In this embodiment, the mantissa pre-alignment module 730 combines the maximum exponent sum Emax with the multiple exponent sums (Ea0+Eb0, Ea1+Eb1, ... Ea N-1 +Eb N-1 ) are subtracted to obtain multiple index differences (ΔE0, ΔE1, ..., ΔE N-1 ), where ΔE0=Emax-(Ea0+Eb0), ΔE1=Emax-(Ea1+Eb1), ΔE N-1 =Emax-(Ea N-1 +Eb N-1 ). The first shift value Va of the mantissa of all the first elements in the first vector and the second shift value Vb of the mantissa of all the second elements in the second vector are calculated according to the multiple exponent differences, and the sum of the first shift value Va of the mantissa of the first element and the second shift value Vb of the mantissa of the second element is equal to the corresponding exponent difference ΔE, for example, Va0+Vb0=ΔE0, Va1+Vb1=ΔE1, ..., Va N-1 +Vb N-1 =ΔE N-1 The mantissa Ma of the first element and the mantissa Mb of the second element are respectively pre-aligned according to the first shift value Va and the second shift value Vb (i.e., the mantissa Ma of the first element is shifted right by Va digits and the mantissa Mb of the second element is shifted by Vb digits), to obtain the first mantissa vector Mas and the second mantissa vector Mbs.
[0125] In a preferred embodiment, the mantissa pre-alignment module 730 includes a pre-processing unit 731 , a difference calculation unit 732 , a shift value calculation unit 733 and a mantissa pre-alignment unit 734 . Among them, the preprocessing unit 731 is used to preprocess the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector to obtain multiple first intermediate mantissas and multiple second intermediate mantissas, wherein the first intermediate mantissas and the second intermediate mantissas both include multiple redundant bits; the difference calculation unit 732 is used to subtract the maximum exponent sum from the multiple exponent sums to obtain multiple exponent differences; the shift value calculation unit 733 is used to calculate the first shift value of all first elements in the first vector and the second shift value of all second elements in the second vector according to the multiple exponent differences, the sum of the first shift value and the second shift value at the corresponding position is the corresponding exponent difference, and the first shift value and the second shift value are non-negative integers; the mantissa pre-alignment unit 734 is used to perform mantissa pre-alignment on the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector according to the first shift value and the second shift value to obtain the first mantissa vector and the second mantissa vector. The mantissa pre-alignment module provided in this embodiment corresponds to the mantissa pre-alignment step provided in the first embodiment, and the specific description refers to the first embodiment, which is not repeated here.
[0126] The multiplication-addition operation module 740 is used to perform a multiplication-addition operation on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication-addition result.
[0127] In this embodiment, the first mantissa vector Mas and the second mantissa vector Mbs are multiplied and added to obtain a mantissa multiplication and addition result Macc, Macc=Mas0×Mbs0+Mas1×Mbs1+…+Mas N-1 ×Mbs N-1 .
[0128] The shift module 750 is used to perform normalization processing on the exponent calculation result and the mantissa multiplication and addition result to obtain a model operation result.
[0129] In this embodiment, the exponent calculation result and the mantissa multiplication and addition result are used to construct a normalized floating point number. The mantissa multiplication and addition result is shifted, and the exponent calculation result is processed according to the shift to be within the normalized range of the floating point type.
[0130] In a preferred embodiment, the floating-point operation device also includes a vector grouping module 760 and a module 770, wherein the vector grouping module 760 is used to group the first vector and the second vector respectively to obtain L first sub-vectors and L second sub-vectors, the first sub-vector includes M first elements, and the second sub-vector includes M second elements.
[0131] The exponential operation module 720 is further configured to extract exponential parts from all first elements in the L first sub-vectors and all second elements in the L second sub-vectors to obtain L first exponential calculation results.
[0132] The mantissa pre-alignment module 730 is further configured to perform mantissa pre-alignment on the L first sub-vectors and the L second sub-vectors according to the L first exponent calculation results to obtain L first mantissa sub-vectors and L second mantissa sub-vectors.
[0133] The multiplication-addition operation module 740 is further configured to perform multiplication-addition operations on the L first mantissa sub-vectors and the L second mantissa sub-vectors to obtain L mantissa multiplication-addition results.
[0134] The sum module 770 is used to obtain a first sum vector according to the L first exponent calculation results and the L mantissa multiplication and addition results, wherein the first sum vector includes the L first sum results.
[0135] The exponential operation module 720 is further used to extract the exponential parts of all the first sum results in the first sum vector to obtain the second exponential calculation results.
[0136] The shift module 750 is further configured to shift all first sum results by an order according to the second exponent result to obtain a plurality of first and intermediate mantissas.
[0137] The sum module 770 is further used for summing a plurality of first and intermediate mantissas to obtain a second sum result; and obtaining a model operation result according to the second exponent calculation result and the second sum result.
[0138] The floating-point operation device provided in this embodiment corresponds to the floating-point operation method provided in the second embodiment. The specific description is referred to the second embodiment and will not be repeated here.
[0139] In a preferred embodiment, the vector grouping module 760 is further used to group the first sum vector into multiple first sub-sum vectors, each of which includes P first sum results, where P is a positive integer.
[0140] The exponential operation module 720 is further used to extract the exponential part of all the first sum results in each sub-sum vector to obtain the second exponential calculation result.
[0141] The shift module 750 is further configured to shift all first sum results in each sub-sum vector according to the second exponent calculation result to obtain a corresponding intermediate sum vector.
[0142] The sum module 770 is further used to sum all elements in the intermediate sum vector to obtain a corresponding second sum result.
[0143] The shift module 750 is further used to perform normalization processing on the second sum result according to the second exponent calculation result to obtain a third sum result, and the third sum result is floating point data.
[0144] The sum module 770 is also used to perform floating point summation on multiple third sum results to obtain a model operation result.
[0145] The floating-point operation device provided in this embodiment corresponds to the floating-point operation method provided in the third embodiment. The specific description is referred to the third embodiment and will not be repeated here.
[0146] The floating-point operation device provided by the present invention performs mantissa pre-alignment on all elements in a first vector and a second vector to obtain a first mantissa vector and a second mantissa vector, performs multiplication and addition operations on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication and addition result, and directly performs multiplication and addition operations in a calculation unit without performing an order shift operation in the accumulation process, thereby improving calculation efficiency and reducing algorithm complexity and chip cost.
[0147] Furthermore, preprocessing all elements in the first vector and the second vector to increase redundant bits, and then pre-aligning the mantissas of all elements in the first vector and the second vector can reduce the loss of valid values in the mantissa, thereby reducing precision loss and improving calculation accuracy.
[0148] Furthermore, the first vector and the second vector are grouped, and the mantissas of all elements of each group of sub-vectors are pre-aligned before multiplication and addition operations are performed directly. The grouping method and the number of elements of each group of sub-vectors can be determined according to the bit width of the storage and computing unit, thereby improving the configurability of the storage and computing unit, making it suitable for storage and computing units with different bit widths and having high versatility.
[0149] Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 8 As shown, the electronic device 800 of this embodiment includes: at least one processor 801 ( Figure 8 Only one is shown in the figure) a processor, a memory 802, and a computer program 803 stored in the memory 802 and executable on the at least one processor 801, and when the processor 801 executes the computer program 803, the steps of the above-mentioned computational graph processing method are implemented.
[0150] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 8This is merely an example of the electronic device 800 and does not constitute a limitation on the electronic device 800 . The electronic device 800 may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.
[0151] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0152] In some embodiments, the memory 802 may be an internal storage unit of the electronic device 800, such as a hard disk or memory of the electronic device 800. In other embodiments, the memory 802 may also be an external storage device of the electronic device 800, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 800. Further, the memory 802 may also include both an internal storage unit of the electronic device 800 and an external storage device. The memory 802 is used to store an operating system, an application program, a boot loader (Boot Loader), data and other programs, such as the program code of the computer program, etc. The memory 802 may also be used to temporarily store data that has been output or is to be output.
[0153] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0154] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0156] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0157] According to the embodiments of the present invention as described above, these embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the above description. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.
Claims
1. A floating point operation method, characterized in that: include: According to the model operation instruction, a first vector and a second vector are obtained, wherein the first vector includes a plurality of first elements, the second vector includes a plurality of second elements, the first elements and the second elements are floating-point data, and the first elements and the second elements correspond to each other one by one; Performing an exponential operation on all elements in the first vector and all elements in the second vector to obtain an exponential calculation result; Perform mantissa pre-alignment on all elements in the first vector and all elements in the second vector according to the exponent calculation result to obtain a first mantissa vector and a second mantissa vector; Performing a multiplication and addition operation on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication and addition result; Performing normalization processing on the exponent calculation result and the mantissa multiplication and addition result to obtain a model operation result; The first vector and the second vector are grouped to obtain a plurality of first sub-vectors and a plurality of second sub-vectors, respectively, and the number of elements contained in each first sub-vector and each second sub-vector is determined according to the bit width of the storage unit and the data type of the element; Performing exponential operations on all elements in the first vector and all elements in the second vector to obtain exponential calculation results includes: extracting exponential parts of a plurality of first elements and a plurality of second elements respectively to obtain a plurality of first exponents and a plurality of second exponents; Adding a plurality of first indices and a plurality of second indices correspondingly to each other to generate a plurality of index sums; Determine the largest exponential sum from the multiple exponential sums as the maximum exponential sum; Taking the plurality of index sums and the maximum index sum as the index calculation results; Pre-aligning the mantissas of all elements in the first vector and all elements in the second vector according to the exponent calculation result to obtain a first mantissa vector and a second mantissa vector includes: Preprocessing the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector to obtain a plurality of first intermediate mantissas and a plurality of second intermediate mantissas, wherein the first intermediate mantissas and the second intermediate mantissas each include a plurality of redundant bits; Subtracting the maximum exponential sum from the multiple exponential sums to obtain multiple exponential difference values; Calculate first shift values of multiple first intermediate mantissas and second shift values of multiple second intermediate mantissas according to the multiple exponent difference values, the sum of the first shift value and the second shift value at corresponding positions is the corresponding exponent difference value, and the first shift value and the second shift value are non-negative integers; Mantissa pre-alignment is performed on a plurality of first intermediate mantissas and a plurality of second intermediate mantissas according to the first shift value and the second shift value to obtain a first mantissa vector and a second mantissa vector.
2. The floating-point operation method according to claim 1, wherein: Calculating first shift values for all first elements in the first vector and second shift values for all second elements in the second vector according to the plurality of exponent differences comprises: Calculating the number of redundant bits for each of the plurality of first intermediate mantissas and the plurality of second intermediate mantissas to obtain a plurality of first redundant numbers and a plurality of second redundant numbers; Adding a plurality of first redundant numbers and a plurality of second redundant numbers correspondingly to generate a plurality of redundant sums; comparing a plurality of said index differences with corresponding said redundant sums; When the exponent difference is less than or equal to the redundancy sum, calculating a first shift value and a second shift value according to the first redundancy number and the exponent difference; When the index difference is greater than the redundant sum, a first shift value and a second shift value are calculated according to a mean value of the index difference.
3. The floating-point operation method according to claim 2, wherein: Calculating the first shift value and the second shift value according to the first redundant number and the exponent difference includes: comparing the index difference with a first redundant number; When the exponent difference is less than or equal to the first redundant number, the exponent difference is used as the first shift value, and the second shift value is 0; When the index difference is greater than the first redundant number, the first redundant number is used as the first shift value, and the difference between the index difference and the first redundant number is used as the second shift value.
4. The floating-point operation method according to claim 2, wherein: Calculating the first shift value and the second shift value according to the mean of the exponential difference values includes: When the exponent difference is a positive even number, taking an average value of the exponent difference as a first shift value and taking a difference between the exponent difference and the first shift value as a second shift value; When the exponent difference is a positive odd number, an average value of the difference between the exponent difference and 1 is used as a first shift value, and a difference between the exponent difference and the first shift value is used as a second shift value.
5. The floating-point operation method according to claim 2, wherein: Calculating the first shift value and the second shift value according to the mean of the exponential difference values includes: performing a subtraction operation on the index difference and the redundant sum to obtain a difference to be allocated; When the difference to be allocated is a positive even number, taking the sum of the average value of the difference to be allocated and the first redundant number as the first shift value, and taking the difference between the exponential difference and the first shift value as the second shift value; When the difference to be allocated is a positive odd number, the sum of an average value of the difference between the difference to be allocated and 1 and a first redundant number is used as a first shift value, and the difference between the exponential difference and the first shift value is used as a second shift value.
6. The floating-point operation method according to claim 1, wherein: Also includes: Grouping the multiple first elements in the first vector and the multiple second elements in the second vector respectively to obtain L first sub-vectors and L second sub-vectors, where the first sub-vector includes M first elements, and the second sub-vector includes M second elements, and L and M are both positive integers; Extracting exponential parts from all first elements in the L first sub-vectors and all second elements in the L second sub-vectors to obtain L first exponential calculation results; Pre-aligning the mantissas of the L first sub-vectors and the L second sub-vectors according to the calculation results of the L first exponents to obtain L first mantissa sub-vectors and L second mantissa sub-vectors; Performing multiplication and addition operations on the L first mantissa subvectors and the L second mantissa subvectors respectively to obtain L mantissa multiplication and addition results; A first sum vector is obtained according to the L first exponent calculation results and the L mantissa multiplication and addition results, wherein the first sum vector includes the L first sum results.
7. The floating-point operation method according to claim 6, wherein: Also includes: Take out the exponential part of all the first sum results in the first sum vector respectively to obtain the second exponential calculation result; Shifting all first sum results according to the second exponent result to obtain a plurality of first and intermediate mantissas; The plurality of first and middle mantissas are summed to obtain a second sum result; A model operation result is obtained according to the second index calculation result and the second sum result.
8. The floating-point operation method according to claim 6, wherein: Also includes: The first sum vector is grouped to obtain a plurality of first sub-sum vectors, each of which includes P first sum results, where P is a positive integer; Take out the exponential part of all the first sum results in each sub-sum vector to obtain the second exponential calculation result; Shift all first sum results in each sub-sum vector according to the calculation result of the second index to obtain a corresponding intermediate sum vector; Summing all elements of the intermediate sum vector to obtain the corresponding second sum result; Normalize the second sum result according to the second exponent calculation result to obtain a third sum result, where the third sum result is floating point data; The model operation result is obtained by performing floating point summation on multiple third sum results.
9. A floating point operation device, characterized in that: include: an acquisition module, configured to acquire a first vector and a second vector according to a model operation instruction, wherein the first vector includes a plurality of first elements, the second vector includes a plurality of second elements, the first elements and the second elements are floating-point data, and the first elements and the second elements correspond to each other one by one; An exponential operation module, used for performing exponential operation on all elements in the first vector and all elements in the second vector to obtain an exponential calculation result; a mantissa pre-alignment module, configured to perform mantissa pre-alignment on all elements in the first vector and all elements in the second vector according to the exponent calculation result to obtain a first mantissa vector and a second mantissa vector; A multiplication-addition operation module, used for performing a multiplication-addition operation on the first mantissa vector and the second mantissa vector to obtain a mantissa multiplication-addition result; A shift module, used for performing normalization processing on the exponent calculation result and the mantissa multiplication and addition result to obtain a model operation result; A vector grouping module, used for grouping the first vector and the second vector respectively to obtain a plurality of first sub-vectors and a plurality of second sub-vectors, wherein the number of elements contained in each first sub-vector and each second sub-vector is determined according to the bit width of the storage and calculation unit and the data type of the element; The exponential operation module is further used to extract the exponential parts of the multiple first elements and the multiple second elements respectively to obtain multiple first exponents and multiple second exponents; add the multiple first exponents and the multiple second exponents correspondingly to generate multiple exponential sums; determine the largest exponential sum from the multiple exponential sums as the maximum exponential sum; and use the multiple exponential sums and the maximum exponential sum as the exponential calculation results; The mantissa pre-alignment module is also used to pre-process the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector to obtain multiple first intermediate mantissas and multiple second intermediate mantissas, wherein the first intermediate mantissas and the second intermediate mantissas both include multiple redundant bits; subtract the maximum exponent sum from the multiple exponent sums to obtain multiple exponent differences; calculate the first shift value of the multiple first intermediate mantissas and the second shift value of the multiple second intermediate mantissas according to the multiple exponent differences, the sum of the first shift value and the second shift value at the corresponding position is the corresponding exponent difference, and the first shift value and the second shift value are non-negative integers; perform mantissa pre-alignment on the multiple first intermediate mantissas and the multiple second intermediate mantissas according to the first shift value and the second shift value to obtain the first mantissa vector and the second mantissa vector.
10. The floating-point operation device according to claim 9, characterized in that: The mantissa pre-alignment module comprises: a preprocessing unit, configured to preprocess the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector to obtain a plurality of first intermediate mantissas and a plurality of second intermediate mantissas, wherein the first intermediate mantissas and the second intermediate mantissas each include a plurality of redundant bits; a difference calculation unit, configured to perform a subtraction operation on the maximum exponential sum and the multiple exponential sums to obtain multiple exponential differences; a shift value calculation unit, configured to calculate first shift values of all first elements in the first vector and second shift values of all second elements in the second vector according to the multiple exponent difference values, the sum of the first shift value and the second shift value at corresponding positions being the corresponding exponent difference value, and the first shift value and the second shift value being non-negative integers; A mantissa pre-alignment unit is used to perform mantissa pre-alignment on the mantissas of all first elements in the first vector and the mantissas of all second elements in the second vector according to the first shift value and the second shift value to obtain a first mantissa vector and a second mantissa vector.
11. The floating-point operation device according to claim 9, characterized in that: The vector grouping module is used to group multiple first elements in the first vector and multiple second elements in the second vector to obtain L first sub-vectors and L second sub-vectors, respectively. The first sub-vector includes M first elements, and the second sub-vector includes M second elements.
12. The floating-point operation device according to claim 11, characterized in that: The exponential operation module is further used to extract the exponential parts of all the first elements in the L first sub-vectors and all the second elements in the L second sub-vectors to obtain L first exponential calculation results; the mantissa pre-alignment module is further used to perform mantissa pre-alignment on the L first sub-vectors and the L second sub-vectors according to the L first exponential calculation results to obtain L first mantissa sub-vectors and L second mantissa sub-vectors; The multiplication and addition operation module is further used to perform multiplication and addition operations on the L first mantissa sub-vectors and the L second mantissa sub-vectors to obtain L mantissa multiplication and addition results; The floating-point operation device further comprises: The sum module is used to obtain a first sum vector according to L first exponent calculation results and L mantissa multiplication and addition results, wherein the first sum vector includes L first sum results.
13. The floating-point operation device according to claim 12, characterized in that: The exponential operation module is also used to extract the exponential part of all the first sum results in the first sum vector to obtain the second exponential calculation result; The shift module is further used to shift all first sum results according to the second exponent result to obtain a plurality of first and intermediate mantissas; The sum module is also used for summing a plurality of first and intermediate mantissas to obtain a second sum result; and obtaining a model operation result according to the second exponent calculation result and the second sum result.
14. The floating-point operation device according to claim 12, characterized in that: The vector grouping module is further used for grouping the first sum vector to obtain a plurality of first sub-sum vectors, each of which includes P first sum results, where P is a positive integer; The exponential operation module is also used to extract the exponential part of all the first sum results in each sub-sum vector to obtain the second exponential calculation result; The shift module is further used to shift all the first sum results in each sub-sum vector according to the second exponent calculation result to obtain a corresponding intermediate sum vector; The sum module is also used to sum all elements in the intermediate sum vector to obtain a corresponding second sum result; The shift module is further used to perform normalization processing on the second sum result according to the second exponent calculation result to obtain a third sum result, wherein the third sum result is floating point data; The sum module is also used to perform floating point summation on multiple third sum results to obtain a model operation result.
15. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the floating-point operation method according to any one of claims 1 to 8 when executing the computer program.
16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the floating-point operation method according to any one of claims 1 to 8 is implemented.
