Data processing device and method, processor and electronic equipment

Abstract

The embodiment of the invention provides a data processing device and method, a processor and electronic equipment, and relates to the technical field of computers, and the device comprises a floating-point number coding part composed of a first extraction unit, a search unit and a coding unit. The first extraction unit is used for extracting sign bits and data bits of the floating-point number to be coded in the vector to be coded. The searching unit is used for respectively searching indexes corresponding to the maximum floating-point number and the minimum floating-point number in the vector to be coded based on the data bits of the floating-point number to be coded, so as to obtain a maximum index and a minimum index; and the coding unit is used for coding the to-be-coded floating-point number into n-bit logarithm floating-point format representation based on the target digit n, the sign bit of the to-be-coded floating-point number, the data bit of the to-be-coded floating-point number, the maximum index and the minimum index to obtain a coded vector. In this way, the floating-point number in the logarithm floating-point format can be expressed efficiently.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing device, method, processor and electronic equipment. Background Art

[0002] A floating-point format refers to the specific structure used to store and represent floating-point numbers in a computer. Floating-point formats include logarithmic floating-point formats (LogFMT-nBit) and non-logarithmic floating-point formats. Non-logarithmic floating-point formats are defined by the IEEE 754 standard, such as TF32, BF16, and FP8. Logarithmic floating-point formats are designed to reduce communication bandwidth while maintaining high model accuracy. Logarithmic floating-point formats map floating-point numbers to a logarithmic space, significantly reducing data transmission bit width while maintaining high model accuracy.

[0003] In order to improve data transmission efficiency, how to efficiently represent floating-point numbers in logarithmic floating-point format during data processing has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] Embodiments of the present invention provide a data processing device, method, processor, and electronic device, which can solve the problem of how to efficiently represent floating-point numbers in a logarithmic floating-point format.

[0005] In order to solve the above problems, an embodiment of the present invention discloses a data processing device, comprising: a floating point encoding component composed of a first extraction unit, a search unit and an encoding unit; The first extraction unit is used to extract the sign bit and data bit of the floating point number to be encoded in the vector to be encoded; The search unit is configured to search for the indexes corresponding to the maximum floating point number and the minimum floating point number in the vector to be encoded based on the data bits of the floating point number to be encoded, respectively, to obtain the maximum index and the minimum index; The encoding unit is used to encode the floating-point number to be encoded into an n-bit logarithmic floating-point format representation based on a target number of bits n, a sign bit of the floating-point number to be encoded, a data bit of the floating-point number to be encoded, the maximum index, and the minimum index to obtain an encoded vector.

[0006] On the other hand, an embodiment of the present invention discloses a data processing method, which is applied to the above-mentioned data processing device, and the method includes: Extracting the sign bit and data bit of the floating point number to be encoded in the vector to be encoded; Based on the data bits of the floating-point number to be encoded, searching for the indexes corresponding to the maximum floating-point number and the minimum floating-point number in the vector to be encoded, respectively, to obtain the maximum index and the minimum index; Based on a target number of bits n, a sign bit of the floating-point number to be encoded, data bits of the floating-point number to be encoded, the maximum index, and the minimum index, the floating-point number to be encoded is encoded into an n-bit logarithmic floating-point format representation to obtain an encoded vector.

[0007] An embodiment of the present invention further discloses a processor, comprising the above-mentioned data processing device.

[0008] On the other hand, an embodiment of the present invention discloses an electronic device, comprising: the above-mentioned processor, memory, communication interface and communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the aforementioned method.

[0009] An embodiment of the present invention further discloses a machine-readable medium having instructions stored thereon. When executed by one or more processors, the processors are enabled to execute the method described above.

[0010] The embodiment of the present invention includes the following advantages: the data processing device for a processor provided by the embodiment of the present invention includes a floating-point encoding component composed of a first extraction unit, a search unit, and an encoding unit. The first extraction unit is used to extract the sign bit and data bit of the floating-point number to be encoded in the vector to be encoded. The search unit is used to search for the indexes corresponding to the maximum floating-point number and the minimum floating-point number in the vector to be encoded based on the data bits of the floating-point number to be encoded, and obtain the maximum index and the minimum index. The encoding unit is used to encode the floating-point number to be encoded into an n-bit logarithmic floating-point format representation based on the target number of bits n, the sign bit of the floating-point number to be encoded, the data bits of the floating-point number to be encoded, the maximum index, and the minimum index, to obtain an encoded vector. In this way, the provided data processing device, as hardware, supports encoding operations for the logarithmic floating-point format, thereby improving the efficiency of encoding to a certain extent and realizing efficient representation of floating-point numbers in the logarithmic floating-point format.

[0011] In an embodiment of the present invention, communication between different data processing devices uses a logarithmic floating-point format. When performing data calculations, the data processing devices first use a floating-point decoding component to convert the logarithmic floating-point format to a non-logarithmic floating-point format, ensuring data accuracy during the calculations. Furthermore, the provision of the floating-point decoding component enables the processor to support decoding operations specific to the logarithmic floating-point format, thereby improving decoding efficiency to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments of the present invention.

[0013] Figure 11 is a schematic diagram of a data processing device according to an embodiment of the present invention; Figure 2 1 is a schematic structural diagram of a floating-point number encoding component provided by an embodiment of the present invention; Figure 3 is a structural diagram of a first searcher provided by an embodiment of the present invention; Figure 4 It is an algorithm implementation circuit schematic diagram in the prior art; Figure 5 1 is a circuit diagram of an encoding subunit provided by an embodiment of the present invention; Figure 6 is a schematic diagram of an encoding process provided by an embodiment of the present invention; Figure 7 is a schematic diagram of a framework of another data processing device provided by an embodiment of the present invention; Figure 8 1 is a circuit diagram of a reduction unit provided by an embodiment of the present invention; Figure 9 This is a schematic diagram of a decoding process provided by an embodiment of the present invention; Figure 10 is a schematic diagram of a multiplication-addition operation provided by an embodiment of the present invention; Figure 11 This is a flowchart of a data processing method provided by an embodiment of the present invention; Figure 12 It is a structural diagram of an electronic device provided by an embodiment of the present invention.

[0014] Figure markings: 11-floating point encoding component, 111-first extraction unit, 112-search unit, 113-encoding unit, 1131-logarithmic subunit, 1132-first splicing subunit, 1133-encoding subunit, 1134-second splicing subunit, 121-second extraction unit, 122-restoration unit, 123-decoding unit, 12-floating point decoding component, 13-data input component, 14-data operation component. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0016] The data processing device provided in embodiments of the present invention can be applied to any computing node in a distributed system. The computing node can be an electronic device, such as a computer or server. Deep learning models, such as the Large Language Model (LLM), can be deployed in the distributed system. During operation, each computing node, after processing data, needs to transmit the resulting data to other computing nodes.

[0017] Exemplarily, the model can be a large language model BLOOM-176B based on the Transformer architecture, which contains 176 billion parameters. The inference process requires 48 computing nodes to run, and data needs to be transferred between the computing nodes. The data transferred can be the intermediate calculation results of the large language model, and the transfer method can be: the hidden layer data of the model is divided into blocks and calculated on different computing nodes, and then collected to a certain node and continued to be transferred backward to continue the calculation. It can also be that the hidden layer of the model is divided into multiple groups, each group uses a computing node for calculation, and a pipeline is formed between different computing nodes to transfer the intermediate calculation results from front to back.

[0018] The data transmitted may be a vector composed of floating-point numbers, and there are a variety of floating-point formats available for use. For example, TF32, BF16, FP8, and, in order to reduce the communication bandwidth while maintaining a high model accuracy, a logarithmic floating-point format (LogFMT-nBit) is proposed for the model. Among them, LogFMT-nBit can map floating-point numbers to a logarithmic space. Since the data transmission bit width of the logarithmic floating-point format is relatively small, only n bits, it significantly reduces the data transmission bit width while maintaining a high model accuracy. However, the current processor does not provide a functional component for encoding and decoding the logarithmic floating-point format, so as to efficiently represent the floating-point numbers in the logarithmic floating-point format, that is, to efficiently convert the floating-point number representation in the logarithmic floating-point format. To this end, an embodiment of the present invention provides a data processing device.

[0019] Figure 1 Schematic diagram of a data processing device according to an embodiment of the present invention. The data processing device can be applied to a processor such as Figure 1 As shown, the device includes a floating point encoding component 11 composed of a first extraction unit 111, a search unit 112 and an encoding unit 113, the first extraction unit 111 is connected to the search unit 112 and the encoding unit 113, and the search unit 112 is connected to the encoding unit 113; The first extraction unit 111 is configured to extract the sign bit and data bits of the floating-point number to be encoded from the vector to be encoded.

[0020] The search unit 112 is configured to search for the indexes corresponding to the maximum floating point number and the minimum floating point number in the vector to be encoded based on the data bits of the floating point number to be encoded, and obtain the maximum index and the minimum index.

[0021] The encoding unit 113 is configured to encode the floating-point number to be encoded into an n-bit logarithmic floating-point format representation based on a target number of bits n, a sign bit of the floating-point number to be encoded, data bits of the floating-point number to be encoded, the maximum index, and the minimum index to obtain an encoded vector.

[0022] The floating-point encoding component 11 provided in the embodiment of the present invention provides a specific hardware implementation solution. The floating-point numbers included in the vector to be encoded are the floating-point numbers to be encoded. The vector to be encoded may include q floating-point numbers, i.e., the vector to be encoded includes q floating-point numbers to be encoded. The vector to be encoded may be externally input or generated by other components in the data processing device, which is not limited in the embodiment of the present invention. The floating-point numbers to be encoded may be floating-point numbers according to the IEEE standard.

[0023] The first extraction unit 111 extracts the sign bit of each floating-point number to be encoded, and uses the part other than the sign bit as an independent data bit, outputs the extracted data bits of each floating-point number to be encoded to the search unit 112, and outputs the extracted sign bit and data bits of each floating-point number to be encoded to the encoding unit 113.

[0024] Furthermore, the maximum floating-point number is used to represent the maximum value in the vector to be encoded. That is, the maximum floating-point number is the non-zero floating-point number with the largest absolute value in the vector to be encoded. The minimum floating-point number is used to represent the minimum value in the vector to be encoded. That is, the minimum floating-point number is the non-zero floating-point number with the smallest absolute value in the vector to be encoded. The maximum index and minimum index are used to indicate the relative positions of the maximum and minimum floating-point numbers in the vector to be encoded, that is, to indicate the order of the maximum and minimum floating-point numbers in the vector to be encoded.

[0025] For a floating-point number to be encoded, the data bits of the floating-point number to be encoded represent the size of the floating-point number to be encoded. The search unit 112 receives the data bits of the floating-point number to be encoded input by the first extraction unit 111. Based on the data bits of each encoded floating-point number in the vector to be encoded, the maximum floating-point number and the minimum floating-point number are searched, and the indexes corresponding to the maximum floating-point number and the minimum floating-point number are obtained respectively.

[0026] For example, the vector to be encoded can be represented as {x1, x2, ..., x q The encoding specification of the logarithmic floating point format may include: among the q floating point numbers to be encoded in the vector to be encoded, the non-zero floating point number with the smallest absolute value is x i , the non-zero floating-point number with the largest absolute value is x j . Correspondingly, the minimum index and the maximum index can be i and j respectively.

[0027] After obtaining the maximum index j and the minimum index i, the search unit 112 may output the maximum index and the minimum index to the encoding unit 113. Accordingly, the encoding unit 113 receives the sign bit and the data bit of the floating point number to be encoded output by the first extraction unit 111, and receives the maximum index and the minimum index output by the search unit 112. The encoding unit 113 may determine the maximum logarithmic representation and the minimum logarithmic representation of the vector to be encoded required for encoding based on the maximum index and the minimum index. The maximum logarithmic representation max of the vector to be encoded may be denoted as a j =ln|x j |, the minimum logarithm representation min of the vector to be encoded can be recorded as a i =ln|x i |.

[0028] The specific value of the target number of bits n can be preset or externally input. The encoding unit 113 can pre-store first constants corresponding to different values ​​of n, and the first constant corresponding to the specific value of the target number of bits n is the first coefficient. The specific value of the target number of bits is predetermined according to the encoding specification of the logarithmic encoding format. The specific value of n represents the total number of bits included in each floating-point number in the final encoded vector. For example, the specific value of n can be pre-defined in the floating-point encoding component 11 of the data processing device of each computing node of the distributed system to facilitate use during encoding.

[0029] The logarithmic floating-point format representation obtained by encoding a single floating-point number to be encoded according to the logarithmic floating-point format is called the encoded floating-point number. That is, the encoded floating-point number is the floating-point representation obtained by converting the floating-point number to be encoded. The encoded vectors each include the encoded floating-point numbers obtained by encoding each of the q floating-point numbers to be encoded. A single encoded floating-point number contains n bits. For example, when the specific value of n is 8, the single encoded floating-point number contains 8 bits, and when the specific value of n is 10, the single encoded floating-point number contains 10 bits. The logarithmic floating-point format of a single encoded floating-point number includes: 1 sign bit (denoted as S) and n-1 scale bits (i.e., data bits) that reflect the ratio between linear space and logarithmic space.

[0030] In summary, the data processing device for a processor provided in an embodiment of the present invention includes a floating-point encoding component 11 consisting of a first extraction unit 111, a search unit 112, and an encoding unit 113. The first extraction unit 111 is used to extract the sign bit and data bits of the floating-point number to be encoded in the vector to be encoded. The search unit 112 is used to search for the indexes corresponding to the maximum floating-point number and the minimum floating-point number in the vector to be encoded based on the data bits of the floating-point number to be encoded, respectively, to obtain the maximum index and the minimum index. The encoding unit 113 is used to encode the floating-point number to be encoded into an n-bit logarithmic floating-point format representation based on the target number of bits n, the sign bit of the floating-point number to be encoded, the data bits of the floating-point number to be encoded, the maximum index, and the minimum index, to obtain an encoded vector. In this way, the data processing device, as hardware, supports encoding operations for the logarithmic floating-point format, thereby improving the encoding efficiency to a certain extent and achieving efficient representation of floating-point numbers in the logarithmic floating-point format.

[0031] Each vector to be encoded is equivalent to a data block. When encoding in the LogFMT-nBit format, the maximum and minimum logarithmic representations are locally calculated for the vector to be encoded. Encoding is performed based on these maximum and minimum logarithmic representations, and the data block size can be flexibly adjusted based on actual needs. This allows for flexible adjustment of the representation range for data blocks of varying sizes, enabling encoding of data blocks of varying sizes in a distributed environment to accommodate diverse data distributions.

[0032] Optionally, Figure 2 1 is a schematic diagram of the structure of a floating-point number encoding component 11 provided in an embodiment of the present invention. The search unit 112 includes a first searcher and a second searcher. The input ends of the first searcher and the second searcher are respectively connected to the output end of the first extraction unit 111, and the output ends of the first searcher and the second searcher are respectively connected to the input end of the encoding unit 113.

[0033] The first finder is used to find the index corresponding to the maximum floating point number in the vector to be encoded based on the data bits of the floating point number to be encoded, and obtain the maximum index.

[0034] The second finder is used to find the index corresponding to the minimum floating point number in the vector to be encoded based on the data bits of the floating point number to be encoded, and obtain the minimum index.

[0035] The first extraction unit 111 may include a first output terminal and a second output terminal. The first output terminal is connected to the input terminal of the first finder, and the extracted data bits of each floating-point number to be encoded are input to the first finder for the first finder to search for a maximum index. The second output terminal is connected to the input terminal of the second finder, and the extracted data bits of each floating-point number to be encoded are input to the second finder for the second finder to search for a minimum index.

[0036] In an embodiment of the present invention, two searchers are set respectively: a first searcher and a second searcher, so that the first searcher and the second searcher can search for the maximum index and the minimum index respectively based on the data bits of all floating-point numbers to be encoded output by the first extraction unit 111 in parallel, thereby improving the encoding efficiency.

[0037] Optionally, both the first searcher and the second searcher include multiple comparator layers. The first comparator layer includes q / 2 comparators, where q is the number of floating-point numbers included in the vector to be encoded. The two input ends of each comparator in the first comparator layer receive the data bits of the two floating-point numbers to be encoded, respectively. The two input ends of each comparator in the next comparator layer are connected to the output ends of the two comparators in the current comparator layer, respectively. The next comparator layer is any comparator layer among the multiple comparator layers except the first comparator layer, and the current comparator layer is the previous comparator layer of the next comparator layer. Accordingly, the number of comparators in the current comparator layer is twice the number of comparators in the next comparator layer.

[0038] The comparator in the first finder is used to output the maximum value and the index of the maximum value among the data bits of the two input floating-point numbers to be encoded; the comparator in the second finder is used to output the minimum value and the index of the minimum value among the data bits of the two input floating-point numbers to be encoded. The first finder includes a comparator for selecting the maximum value from the data bits of the two input floating-point numbers to be encoded and outputting the maximum value and the index of the maximum value; the second finder includes a comparator for selecting the minimum value from the data bits of the two input floating-point numbers to be encoded and outputting the minimum value and the index of the minimum value.

[0039] The first searcher and the second searcher can use a protocol tree format. Specifically, the input ends of the first comparator layer of the first searcher are respectively connected to the first output end of the first extraction unit 111, and the input ends of the first comparator layer include the input ends of each comparator in the first comparator layer. Taking the number of floating-point numbers included in the to-be-encoded vector as 16, that is, the number of data bits is 16, as an example, q is 16, and the first comparator layer includes 8 comparators. Figure 3 : is a structural diagram of a first searcher provided by an embodiment of the present invention, such as Figure 3 As shown, each comparator in the first comparator layer receives 16 data bits ( Figure 3 In y1~y 16Indicated). Every two comparators in the first comparator layer are connected to a comparator in the next comparator layer, and the larger of the two input values ​​obtained by comparison and its index are output as the input of the connected comparator. The comparators in the first comparator layer determine the index of the two input values ​​to be compared based on their own order. For example, c represents the order of the comparators, and the index of the input value input at the left input end can be determined as 2×c-1, and the index of the input value input at the right input end can be determined as 2×c. For any comparator layer, the order of the comparators is assigned to the comparators in sequence from left to right starting from 1. For example, for the first comparator layer, the order of the 8 comparators from left to right is: 1, 2, ..., 8. For the second comparator layer, the order of the 4 comparators from left to right is: 1, 2, ..., 4. Alternatively, the index of the data bits of each floating-point number to be encoded can be input to the comparator at the same time, and this is not limited by the embodiment of the present invention.

[0040] The first comparator layer performs pairwise comparisons, obtaining eight larger input values, which are then fed into the second comparator layer. The second comparator layer performs pairwise comparisons, obtaining four of the eight larger input values ​​and feeding them into the third comparator layer. The third comparator layer performs pairwise comparisons, obtaining two of the four larger input values ​​and feeding them into the final comparator layer. The final comparator layer performs pairwise comparisons, obtaining the larger of the two larger input values ​​as the maximum value and index. In this way, through multiple comparisons, the comparator in the final comparator layer outputs the maximum value of the 16 data bits and its index, thereby obtaining the maximum index. Similarly, the structure of the second finder is identical to that of the first finder. The comparator in the second finder outputs the smaller of the two input values ​​obtained through comparison, along with its index. Finally, the comparator in the final comparator layer outputs the minimum value of the 16 data bits and its index, thereby obtaining the minimum index.

[0041] In the embodiment of the present invention, a structure of multiple comparator layers is used as a finder, and the maximum index and the minimum index can be found through multi-level comparison, with low implementation cost.

[0042] Optionally, the comparator included in the first finder is further configured to, if one of the input data bits includes 0, use the data bit of the other input as the maximum value. The comparator included in the second finder is further configured to, if one of the input data bits includes 0, use the data bit of the other input as the minimum value. That is, the actual comparison process only involves non-zero number operations. If one of the comparator inputs is 0, the other number and its index are directly output. In this way, it is ensured that the maximum index can accurately represent the non-zero floating-point number with the largest absolute value in the vector to be encoded, and it is ensured that the minimum index can accurately represent the non-zero floating-point number with the smallest absolute value in the vector to be encoded.

[0043] The floating-point encoding component 11 further includes a first input unit and a normalization unit. The normalization unit is connected to the first input unit and the first extraction unit 111, respectively. The first input unit is configured to truncate each floating-point number to be encoded to a predetermined length, and the normalization unit is configured to normalize each floating-point number to be encoded output by the truncation unit.

[0044] Among them, the preset length can be set as needed. For example, in one application scenario, the floating-point number to be encoded can be in FP32 format, that is, a single-precision floating-point number with a length of 32 bits. Correspondingly, the preset length can be 16 bits, that is, the first input unit takes the upper 16 bits of the floating-point number to be encoded to obtain a truncated floating-point number to be encoded. In this way, truncation is equivalent to truncating it into BF16 format data, that is, a floating-point number with a length of 16 bits. The floating-point encoding component 11 in the embodiment of the present invention receives data in FP32 format, truncates it into BF16 format data, and then encodes it to meet the format requirements of the floating-point number to be encoded input by the first extraction unit 111. It should be noted that the first input unit may include q parallel truncation subunits to truncate the q floating-point numbers to be encoded in parallel. The normalization unit may include q parallel normalization subunits to normalize the q floating-point numbers to be encoded in parallel.

[0045] Among them, the value of the mantissa of the normalized floating-point number is ≥1 and <2. Since a non-normalized floating-point number may appear after the floating-point number to be encoded is truncated, it can be normalized by the normalization unit first. The specific process of converting a non-normalized floating-point number into a normalized floating-point number can be: the bit width of the exponent of the floating-point number to be encoded is expanded by one bit. If the floating-point number to be encoded itself is a normalized floating-point number, then the high bit of the exponent of the floating-point number to be encoded is directly padded with 0 to implement the normalization operation. If the floating-point number to be encoded itself is a non-normalized floating-point number, then the number of leading 0s in the mantissa of the floating-point number to be encoded is first counted, recorded as Num. For example, in the binary representation of the mantissa, the number of consecutive 0s is counted starting from the highest bit until the first 1 appears, and the number obtained by counting is Num. Then determine the displacement length as Num+1, shift the mantissa of the floating-point number to be encoded left by Num+1 bits so that the most significant bit of the mantissa is 1, obtain the normalized mantissa, use the complement representation of -Num as the normalized exponent, and then obtain the normalized floating-point number to implement the normalization operation.

[0046] The normalization unit can input the normalized floating-point numbers to be encoded into the first extraction subunit. The q parallel extraction subunits in the first extraction unit 111 extract the q floating-point numbers to be encoded in parallel to obtain the sign bit, exponent, and mantissa of each floating-point number to be encoded.

[0047] In the embodiment of the present invention, normalization processing is first performed to reduce the occupation of the representation space by non-normalized numbers. For example, for FP8 in the non-logarithmic floating point format, 6.25% of the representation space is occupied by non-normalized numbers. In the embodiment of the present invention, normalization processing is performed to reduce the occupation of the representation space. At the same time, since additional hardware logic needs to be added when performing operations on non-normalized numbers, the embodiment of the present invention provides a normalization unit for normalization processing, which can reduce hardware overhead and thus reduce hardware area and power consumption.

[0048] Optionally, the data bits of the floating point number to be encoded include an exponent and a mantissa. Figure 2 The encoding unit 113 includes a logarithm subunit 1131, a first splicing subunit 1132, an encoding subunit 1133, and a second splicing subunit 1134. The logarithm subunit 1131 is connected to the first extraction unit 111 and the first splicing subunit 1132, the first splicing subunit 1132 is connected to the first extraction unit 111, the encoding subunit 1133 is connected to the second splicing subunit 1134, the search unit 112 is connected to the first splicing subunit 1132, and the second splicing subunit 1134 is connected to the first extraction unit 111.

[0049] The logarithm subunit 1131 is configured to perform a logarithm operation on the mantissas of the floating-point numbers to be encoded to generate a logarithm operation result. The first extraction unit 111 outputs the mantissas of the floating-point numbers to be encoded to the logarithm subunit 1131 via the third output terminal. Accordingly, the logarithm subunit 1131 receives the mantissas of the floating-point numbers to be encoded output by the first extraction unit 111 and outputs the logarithm operation results of the floating-point numbers to be encoded to the first splicing subunit 1132.

[0050] The first splicing subunit 1132 is configured to splice the logarithmic representation of the floating-point number to be encoded based on the exponent of the floating-point number to be encoded and the result of the logarithmic operation. The first extraction unit 111 outputs the exponent of each floating-point number to be encoded to the first splicing subunit 1132 via the fourth output terminal, and the logarithmic subunit 1131 outputs the result of the logarithmic operation of each floating-point number to be encoded to the first splicing subunit 1132. Accordingly, the first splicing subunit 1132 receives the exponent of each floating-point number to be encoded output by the first extraction unit 111, receives the result of the logarithmic operation of each floating-point number to be encoded output by the logarithmic subunit 1131, and outputs the spliced ​​logarithmic representation of the floating-point number to be encoded to the encoding subunit 1133.

[0051] The encoding subunit 1133 is configured to determine a maximum logarithmic representation and a minimum logarithmic representation based on the logarithmic representation, the maximum index, and the minimum index of the floating-point number to be encoded, and generate target data bits corresponding to the floating-point number to be encoded based on the target number of bits n. Specifically, the target data bits corresponding to the floating-point number to be encoded can be generated based on the first coefficient corresponding to the target number of bits n, the logarithmic representation of the floating-point number to be encoded, the maximum logarithmic representation, and the minimum logarithmic representation, and output to the second splicing subunit 1134.

[0052] The second splicing sub-unit 1134 is configured to splice the sign bit of the floating-point number to be encoded with the target data bits to obtain a logarithmic floating-point format representation. The first extraction unit 111 outputs the sign bit of each floating-point number to be encoded to the second splicing sub-unit 1134 via the fifth output terminal. Accordingly, the second splicing sub-unit 1134 receives the sign bit of each floating-point number to be encoded output by the first extraction unit 111 and the target data bits output by the encoding sub-unit 1133 and splices them together to obtain a logarithmic floating-point format representation.

[0053] The search operation performed by the search unit 112 and the operations performed by the logarithm sub-unit 1131 and the first splicing sub-unit 1132 can be performed in parallel.

[0054] The logarithm subunit 1131 may include q parallel logarithm operation circuits, which perform logarithm operations on the mantissas of the q floating-point numbers to be encoded in parallel through the q parallel logarithm operation circuits. The first splicing subunit 1132 may include q parallel first splicing circuits, which perform parallel splicing on the exponents of the q floating-point numbers to be encoded and their logarithm operation results through the q parallel first splicing circuits. The second splicing subunit 1134 may include q parallel second splicing circuits, which perform parallel splicing on the sign bits and target data bits of the q floating-point numbers to be encoded in parallel through the q parallel second splicing circuits. The encoding subunit 1133 may calculate the target data bits corresponding to each of the q floating-point numbers to be encoded in parallel. In this way, through parallel processing, the overall encoding efficiency of the vector to be encoded can be improved.

[0055] Specifically, the logarithm subunit 1131 receives the mantissa of the floating-point number to be encoded output by the first extraction unit 111. Each logarithm operation circuit in the logarithm subunit 1131 includes a first iterative operation circuit and a first multiplication operation circuit. The first iterative operation circuit is used to select a corresponding iteration round as a first target iteration round based on the precision of the mantissa of the floating-point number to be encoded. Iteration is performed based on the mantissa of the floating-point number to be encoded until the first target iteration round is reached. The first multiplication operation circuit is used to calculate the product of the iteration result of the first iterative operation circuit and a preset constant corresponding to the first target iteration round, and obtain the logarithm operation result corresponding to the floating-point number to be encoded based on the product.

[0056] In an embodiment of the present invention, the process of calculating the natural logarithm by the logarithm subunit 1131 can adopt the CORDIC algorithm, and the first iterative operation circuit is an implementation circuit for implementing the CORDIC algorithm. Among them, the specific implementation method and hardware structure of the CORDIC algorithm can refer to the existing method. The number of iterations corresponding to the precision of different mantissas can be pre-set in the control node of the distributed system. For example, the mantissa precision of FP8 and BF16 can represent a maximum of 5 rounds and 13 rounds respectively, that is, when the floating point number to be encoded is FP8, the first target iteration round can be 5, and when the floating point number to be encoded is BF16, the first target iteration round can be 13.

[0057] b represents the mantissa of the floating point number to be encoded after normalization, and the logarithm subunit 1131 is used to calculate ln(b). The logarithm subunit 1131 may include a first variable register, a second variable register, and a third variable register. The first variable, the second variable, and the third variable are respectively represented by u m , v m , z m. Here, m represents the current iteration number. When performing logarithmic operations, m∈[1, L], where L represents the first target iteration round. Based on the rules of the CORDIC algorithm, the initial values ​​of the first, second, and third variables can be assigned to b+1, b-1, and 0, respectively. Specifically, the initial values ​​of the first, second, and third variable registers are in binary form.

[0058] The first variable register (recorded as register u m ), the second variable register (denoted as register v m ) and the third variable register (denoted as register z m ) is used as the input of the CORDIC algorithm to obtain the third variable output of the CORDIC algorithm. The third variable output is the third variable after L iterations (for example, z can be recorded). L ). In the iterative process, z m The change in gradually decreases, and iteration stops when the change is reduced to a point where the mantissa precision of the floating-point number to be encoded cannot be represented. Because the number of iterations in the CORDIC algorithm is fixed, and the number of iterations corresponding to different mantissa precisions is fixed, the logarithmic calculation process using the CORDIC algorithm adopts a fully pipelined design. Each beat of the pipeline represents a CORDIC algorithm iteration, and the intermediate results of the algorithm iteration are passed backward through the pipeline via registers.

[0059] Figure 4 It is an algorithm implementation circuit schematic diagram in the prior art, such as Figure 4 As shown in FIG, the implementation circuit of the CORDIC algorithm consists of multiple registers, shifters, and adders. In each iteration process, the iteration method is: u m+1 =u m +σ m ×2 -m ×v m v m+1 =v m +σ m ×2 -m ×u m z m+1 =z m -σ m ×tan -1 (2 -m ) Among them, σ m represents z m The symbol z mThe specific value of represents the specific value of the third variable register of the mth iteration. This iterative process is actually the vector (u m , v m ) The process of rotating around the origin, each iteration rotates α=tan -1 (2 -m ) angle. The higher the mantissa precision, the greater the number of required iterations. The first target iteration number can indicate how many rounds of rotation α the mantissa precision of the mantissa can represent at most.

[0060] Among them, with register u m The connected shifter is used to implement the multiplication operation so that the adder can calculate v m Plus (±2 -m ×u m ). Among them, 2 -m ×u m The specific positive or negative value of is determined by the value of the sign bit extractor input. For binary, multiplying by 2 -m The multiplication operation is equivalent to the shift operation. For example, for a binary number: 1011 multiplied by 0.1, 0.1 is 2 in binary -1 , the result of the operation is 101.1, which can be achieved by a shifter.

[0061] Furthermore, with register v m The connected shifter is used to implement the multiplication operation for the adder to calculate u m Plus (±2 -m ×v m ). Among them, 2 -m ×v m The specific positive or negative value of is determined by the value of the sign bit extractor input. m The connected adders are used to read register z separately m tan -1 (2 -m ) is added to the value of the sign bit extractor to calculate z m with -σ m ×tan -1 (2 -m ) and.

[0062] The iterative process can be viewed as a vector (u m , v m ) is the process of iteratively rotating around the origin to approach the target angle. The amount of each rotation during the iteration can be expressed as cos𝛼 (where 𝛼 represents the angle of rotation per iteration and cos𝛼 is a constant). When the CORDIC algorithm is used for logarithmic calculations, the third variable z after L iterations is LIt can be approximately expressed as: ln(b)×first target constant / 2. The first target constant can be expressed as . The first iterative operation circuit will set the last pipeline register z L The value of (i.e., the third variable obtained in the last beat) is output as the iteration result to the first multiplication circuit. The first multiplication circuit is used to calculate the product of the iteration result and the preset constant corresponding to the first target iteration round to obtain a logarithmic operation result. Among them, the preset constant corresponding to the first target iteration round is: 2 / When L takes different values, the value of the preset constant corresponding to the first target iteration round is different. The first multiplication circuit can store the preset constants corresponding to the first target iteration round obtained in different iteration rounds (i.e., when L takes different values), so that the preset constants corresponding to the corresponding first target iteration round can be used for multiplication and addition operations during calculation.

[0063] It should be noted that the direct result obtained by the CORDIC algorithm is the natural logarithm with the natural constant e as the base, and when the floating-point number to be encoded is in IEEE floating-point format, the base of the exponent is 2. Therefore, the first multiplication circuit is further configured to multiply the product of the iteration result and the preset constant corresponding to the first target iteration round by the constant used for conversion: log2e, to achieve a unified base and obtain the final logarithmic operation result.

[0064] In this embodiment of the present invention, logarithmic subunit 1131 implements logarithmic operations through a first iterative operation circuit and a first multiplication operation circuit, thereby enabling processor support for logarithmic operations and ensuring exponential operation efficiency. Of course, other methods may also be used to implement logarithmic operations, such as precalculating and storing logarithmic values ​​corresponding to different input values ​​and directly obtaining the logarithmic operation results using a lookup table (LUT), which is not limited in this embodiment of the present invention.

[0065] Furthermore, each first splicing circuit in the first splicing subunit 1132 splices the exponent of a floating-point number to be encoded with the logarithm operation result of the floating-point number to be encoded to obtain the logarithm representation corresponding to the floating-point number to be encoded. The logarithm representation of the floating-point number to be encoded is a fixed-point decimal. For example, for the above-mentioned vector to be encoded {x1, x2, ..., x q}, the logarithmic representation of q floating-point numbers to be encoded can be recorded as {a1, a2, ..., a q The encoding subunit 1133 can encode the logarithmic representation of the q floating point numbers to be encoded respectively according to the encoding specification of the logarithmic floating point format, thereby obtaining the target data bits {K1, K2, ..., K q}. The encoding subunit 1133 directly sets the target data bits of the floating-point number to be encoded to all 0s when the logarithm of the floating-point number to be encoded is 0, that is, when the exponent and mantissa of the floating-point number to be encoded are both 0 and the resulting fixed-point decimal is also 0. The number of bits of a target data bit is the specific value of the target number of bits n minus 1. For example, when n=8, a target data bit includes 7 bits, and when n=10, a target data bit includes 9 bits.

[0066] When the second splicing subunit 1134 performs splicing, if the target data bit of the floating-point number to be encoded is 0, the sign bit is set to the default value of 0, thereby splicing to obtain 0.0…0. That is, for ±0, both are encoded as n zeros (0.000…0). Because the representation of -0 and +0 does not significantly differ for deep learning and actually increases programming difficulty, in this embodiment of the present invention, when the target data bit is 0, the sign bit defaults to 0, that is, -0 and +0 are uniformly encoded as all-zero representations, thereby reducing programming difficulty.

[0067] If the target data bit of the floating point number to be encoded is not 0, the sign bit of the floating point number to be encoded is concatenated with the target data bit of the floating point number to be encoded. For example, assuming that the target data bit of the floating point number to be encoded is K k , then after splicing we get S k .K k .

[0068] It should be noted that the encoding specifications of the logarithmic floating point format also include: i Coded as S i .000…1, x j Coded as S j .111…1. Accordingly, if the index of the floating-point number being processed is the same as the maximum index or the minimum index, then if it is the same as the minimum index, 000…1 is directly output. That is, a data bit consisting of n-2 0s and the least significant 1 is output as the target data bit corresponding to the floating-point number being encoded. If it is the same as the maximum index, 11…1 is directly output. That is, a data bit consisting of n-1 1s is output as the target data bit corresponding to the floating-point number being encoded.

[0069] In an embodiment of the present invention, by setting a logarithmic subunit 1131, a first splicing subunit 1132, an encoding subunit 1133 and a second splicing subunit 1134 in the encoding unit 113 and performing processing in sequence, a logarithmic floating-point format representation of the floating-point number to be encoded can be obtained, ensuring that hardware support is provided for floating-point number encoding.

[0070] In an embodiment of the present invention, when different specific values ​​are set for the target number of bits n, the first number of iterations and the first coefficient are selected based on the specific value of n. In this way, a floating-point number in a logarithmic floating-point format of the corresponding number of bits can be generated, thereby realizing encoding that supports multiple precision floating-point types such as LogFMT-8Bit and LogFMT-10Bit.

[0071] Optionally, the data processing device in an embodiment of the present invention further includes a sending component, which is used to send the encoded vector and first decoding information of the encoded vector; wherein the first decoding information includes the target number of bits n, the maximum logarithmic representation and the minimum logarithmic representation of the vector to be encoded.

[0072] The first decoding information is the information required to decode the coded vector, and the first decoding information includes the information used in the process of encoding to obtain the coded vector. The input end of the sending component is connected to the output end of the second splicing subunit 1134 to obtain the coded vector spliced ​​by the second splicing subunit 1134. The coded vector can be {S1.K1, S2.K2, ..., S q .K q}, assuming n is 10, then the first decoding information may include n=10, a i and a j , wherein n=10 included in the first decoding information may be a Size signal.

[0073] The output of the transmission component can be connected to the input of other data processing devices in the distributed system to output the encoded vector and the first decoded information of the encoded vector to the other data processing devices in the distributed system. For example, the encoded vector and the first decoded information of the encoded vector can be transmitted based on the Internet Protocol (IP) address of the computing node where the other data processing device is located. It should be noted that in another implementation, the transmission component can be omitted and the transmission can be performed via the transmission subunit in the encoding unit 113. The transmission subunit is connected to the second splicing subunit 1134.

[0074] In an embodiment of the present invention, communication between data processing devices utilizes encoded vectors in logarithmic floating-point format. This reduces communication burdens due to the smaller data transmission bit width of the logarithmic floating-point format. Simultaneously transmitting the first decoding information required for decoding facilitates decoding of the encoded vectors by other data processing devices.

[0075] Alternatively, as Figure 5As shown, the encoding subunit 1133 includes a first selector, a first adder, a second adder, a first divider, a first multiplier, a second selector, a third adder, and a rounding circuit. The first selector is connected to the first adder and the second adder, respectively; the first adder and the second adder are connected to the first divider; the first divider is connected to the first multiplier; the first multiplier is connected to the second selector and the third adder; and the third adder is connected to the rounding circuit.

[0076] In the embodiment of the present invention, the encoding subunit 1133 can be composed of a selector, an adder, a multiplier, and a rounding circuit, and the hardware cost is relatively low.

[0077] The first selector is used to determine the maximum logarithmic representation and the minimum logarithmic representation. Specifically, the first selector receives the maximum index j and the minimum index i output by the search unit 112 and the logarithmic representations of each floating-point number to be encoded output by the first splicing subunit 1132. Based on the maximum index j and the minimum index i, the first selector selects the maximum logarithmic representation {a1, a2, ..., a q}, select a i and a j , get the minimum logarithmic representation and the maximum logarithmic representation, and output a to the first adder i and a j , output a to the second adder j .

[0078] The control signals of the first adder and the second adder are both subtraction operation signals. The first adder is used to calculate the first difference between the maximum logarithmic representation and the minimum logarithmic representation. The first difference can be a j -a i , the first adder outputs the calculated first difference to the first divider.

[0079] The second adder is used to calculate the difference between the logarithmic representation of the floating point number to be encoded and the minimum logarithmic representation to obtain a second difference corresponding to the floating point number to be encoded. The second adder can calculate the difference between the logarithmic representation of each floating point number to be encoded and the minimum logarithmic representation output by the first splicing sub-unit 1132 in parallel to obtain q second differences in parallel. For example, the second difference can be expressed as: k -a i , where 1≤k≤q, the second adder will output the calculated second differences to the first divider respectively.

[0080] The control signal is a signal in the input signal received by the adder in the circuit, which is used to control the adder to perform addition or subtraction operation, in the embodiment of the application, by inputting the subtraction operation signal as the control information for the first adder and the second adder, the adder can perform the subtraction operation, thereby saving the hardware resources.

[0081] The first divider is configured to perform division operation on the second difference and the first difference corresponding to the floating point number to be encoded output by the first adder and the second adder, and obtain a first division result corresponding to the floating point number to be encoded. Specifically, the first divider can perform the division operation on the second difference and the first difference corresponding to the q floating point numbers to be encoded respectively, and obtain q first division results. The first division result can be represented as (a k -a i ) / (a j -a i ), and the first divider outputs the calculated first division result to the first multiplier.

[0082] The second selector is configured to output the first constant corresponding to the target bit number n as the first coefficient to the first multiplier. Specifically, the second selector can select the first constant corresponding to the target bit number n as the first coefficient according to the specific value of the target bit number n carried in the control signal, and output the first constant to the first multiplier. According to the encoding specification of the logarithmic floating point format, the first constant corresponding to the target bit number n can be represented as (2 n-1 -2), and the specific value of the first constant is different when the target bit number n adopts different specific values. The second selector can define the corresponding first constant when the target bit number n adopts different specific values in advance. The second selector can find the corresponding first constant based on the specific value of the input target bit number n.

[0083] The first multiplier is configured to calculate the product of the first coefficient output by the second selector and the first division result corresponding to the floating point number to be encoded output by the first divider, and obtain the first product corresponding to the floating point number to be encoded. Specifically, the first multiplier can calculate the product of the first coefficient and the first division result corresponding to the q floating point numbers to be encoded respectively, and output q first products to the third adder. The first product can be represented as (a k -a i ) / (a j -a i )×(2 n-1 -2).

[0084] The third adder is used to calculate the sum of the first product corresponding to the floating-point number to be encoded output by the first multiplier and the second constant to obtain the first sum corresponding to the floating-point number to be encoded. The second constant is a fixed value set in advance according to the encoding specification of the logarithmic floating-point format, and the second constant is 1. The third adder can calculate the sum of the first product corresponding to each of q floating-point numbers to be encoded and the second constant in parallel, obtain q first sums and output them to the rounding circuit. The first sum can be expressed as (a k -a i ) / (a j -a i )×(2 n-1 -2)+1.

[0085] The rounding circuit is used to round the first sum value corresponding to the floating point number to be encoded to obtain the logarithmic floating point format representation corresponding to the floating point number to be encoded. The rounding circuit can use a rounding method. The rounding circuit can perform rounding operations on the first sum value corresponding to each of the q floating point numbers to be encoded in parallel to obtain q logarithmic floating point format representations. The final K k Can be expressed as: RoundToInt((a k -a i ) / (a j -a i )×(2 n-1 -2)+1).

[0086] Among them, RoundToInt means rounding the result to an integer, and the rounding method complies with the IEEE standard. (a j -a i ) / (2 n-1 -2) can be recorded as increment Step, which is used to represent the quantization interval between two adjacent floating-point numbers in the logarithmic space.

[0087] Thus, when the encoding subunit 1133 provided in the embodiment of the present invention, which is composed of the first selector, the first adder, the second adder, the first divider, the first multiplier, the second selector, the third adder, and the rounding circuit, performs encoding, it complies with the encoding specification of the logarithmic floating point format: for non-zero x k , which is encoded as S k .K k , where K k K k 'The rounded result, K k '=(a k -a i ) / (a j -a i )×(2 n-1 -2) +1, K k' is the first sum value mentioned above, thereby ensuring that the logarithmic floating point format representation can be accurately encoded.

[0088] Optionally, the floating-point encoding component 11 also includes an exception output unit; the exception output unit is used to output an exception signal when the vector to be encoded hits a preset special type; wherein the special types include infinity, non-number, and the exponent difference between the maximum floating-point number and the minimum floating-point number is greater than a preset difference threshold.

[0089] The floating point number encoding component 11 further includes a first generating unit, which is used to generate a default logarithmic floating point format representation when the floating point number to be encoded is infinite or non-number.

[0090] Special types include: infinity or non-numbers included in the vector to be encoded. The first generation unit is connected to the first extraction unit 111. The first extraction unit 111 is further configured to send the floating-point numbers to be encoded that are infinity or non-numbers in the vector to be encoded to the first generation unit. Accordingly, the first generation unit is configured to generate a default logarithmic floating-point format representation for the received floating-point numbers to be encoded. The default logarithmic floating-point format representation can be represented as 1.000…0. That is, the first generation unit uniformly encodes IEEE754 floating-point numbers of type sNaN, qNaN, and infinity as 1.000…000.

[0091] It should be noted that, when the vector to be encoded includes infinity and non-number, the encoding sub-unit 1133 can perform parallel processing on the floating-point numbers to be encoded except infinity and non-number. Since data such as non-numbers and infinity are less important in the training and reasoning of deep learning. In an embodiment of the present invention, non-numbers and infinity are uniformly encoded as a default form by the first generation unit, which can make the data format more consistent with the characteristics of deep learning and reduce the occupation of representation space by non-numbers and infinity. For example, for FP8 in the non-logarithmic floating point format, 6.25% of the representation space is occupied by non-numbers and infinity. In an embodiment of the present invention, uniformly representing non-numbers and infinity can further reduce the occupation of representation space, thereby further reducing hardware overhead, hardware area and power consumption.

[0092] Specifically, the exception output unit is connected with the first extraction unit 111. The first extraction unit 111 outputs a first identification to the exception output unit through a sixth output end in the case that the processed vector to be encoded includes infinity. The exception output unit outputs a first exception signal in response to the first identification. The first exception signal can be OVERFLOW exception, which can be displayed in the user interface. That is, the OVERFLOW exception is triggered in the embodiment of the present application when the encoded floating-point number vector contains infinity. The first extraction unit 111 outputs a second identification to the exception output unit in the case that the processed vector to be encoded includes non-number. The exception output unit outputs a second exception signal in response to the second identification, which can be displayed in the user interface. The first identification and the second identification can be set as needed, for example, the first identification and the second identification can be 1 and 0 respectively. The second exception signal can be INVALID exception. That is, the INVALID exception is triggered in the embodiment of the present application when the encoded floating-point number vector contains non-number.

[0093] The exception output unit is also connected with the lookup unit 112, and the lookup unit 112 is also used to input the maximum index and the minimum index to the exception output unit. The first extraction unit 111 also inputs each floating-point number to be encoded to the exception output unit through a seventh output end. The exception output unit is also used to find the maximum floating-point number (i.e., the above x j ) and the minimum floating-point number (i.e., the above x i ) from the floating-point numbers to be encoded based on the maximum index and the minimum index; and outputs a third exception signal in the case that the exponents of x j and x i are greater than a preset difference threshold value. That is, the preset special type of the vector to be encoded also includes that the exponents of x j and x i are greater than the preset difference threshold value.

[0094] The preset difference threshold value can be 2 d , where d represents the exponent width of the floating-point number to be encoded. The third exception signal can be UNDERFLOW exception, which can be displayed in the user interface. That is, the UNDERFLOW exception is triggered in the embodiment of the present application when the exponents of x j and x i are greater than the preset difference threshold value. The OVERFLOW exception, the INVALID exception and the UNDERFLOW exception can be collectively referred to as VZOUI exception.

[0095] Figure 6 is a schematic diagram of an encoding process provided by the embodiment of the present application, as Figure 6As shown, the first input unit can first receive the vector to be encoded, and the normalization unit can perform normalization processing on the floating-point number to be encoded. Then, the first extraction unit 111 can extract the sign bit, exponent, and mantissa of the floating-point number to be encoded. The search unit 112 can search for the maximum index and the minimum index based on the data bits consisting of the exponent and mantissa. The encoding unit 113 can perform a logarithmic operation on the mantissa of the floating-point number to be encoded to obtain the logarithmic representation of the floating-point number to be encoded. Finally, the encoded vector is generated based on the first coefficient, the logarithmic representation of the floating-point number to be encoded, the sign bit, the maximum index, and the minimum index. During the processing, if the vector to be encoded includes infinity or non-number, the exception output unit can output a first exception signal and a second exception signal respectively. If the difference in exponent between the maximum floating-point number and the minimum floating-point number is greater than a preset difference threshold, the exception output unit can output a third exception signal.

[0096] In the embodiment of the present invention, by setting an abnormal alarm mechanism, when the vector to be encoded includes infinite or non-number floating-point numbers to be encoded, an abnormal signal is output, which can facilitate subsequent adjustments.

[0097] Optionally, Figure 7 1 is a schematic diagram of the framework of another data processing device provided by an embodiment of the present invention. The data processing device in the embodiment of the present invention further includes: a floating-point decoding component 12 composed of a second extraction unit 121, a restoration unit 122, and a decoding unit 123. The second extraction unit 121 is connected to the restoration unit 122 and the decoding unit 123, and the restoration unit 122 is connected to the decoding unit 123; The second extraction unit 121 is configured to extract the sign bit and data bits of the floating-point number to be decoded from the vector to be decoded.

[0098] The restoration unit 122 is configured to restore the logarithmic representation corresponding to the floating-point number to be decoded based on the second decoding information of the vector to be decoded and the data bits of the floating-point number to be decoded.

[0099] The decoding unit 123 is used to generate a non-logarithmic floating-point format representation of the floating-point number to be decoded based on the sign bit of the floating-point number to be decoded and the logarithmic representation corresponding to the floating-point number to be decoded, obtain a decoded vector, and input the decoded vector to the data operation component 14 for operation.

[0100] The floating-point number decoding component 12 adopts a fully pipelined design. The floating-point numbers included in the vector to be decoded are the floating-point numbers to be decoded. The vector to be decoded may include p floating-point numbers, i.e., the vector to be decoded includes p floating-point numbers to be decoded. The vector to be decoded may be input externally via the data input component 13, or may be generated by other components in the data processing device, which is not limited in this embodiment of the present invention. For example, the vector to be decoded may be an encoded vector sent by another data processing device. Here, p may be the same as q. The floating-point numbers to be decoded may be floating-point numbers in LogFMT-nBit format. The second extraction unit 121 extracts the sign bit of each floating-point number to be decoded, and uses the portion of the floating-point number to be decoded other than the sign bit as the data bits of the floating-point number to be decoded. The extracted data bits of each floating-point number to be decoded are output to the restoration unit 122, and the extracted sign bits of each floating-point number to be decoded are output to the decoding unit 123.

[0101] In the embodiment of the present invention, by providing a floating-point decoding component 12, the data processing device can decode the obtained floating-point vector in the LogFMT-nBit format. In this way, when performing internal calculations, floating-point numbers in the non-logarithmic floating-point format can be used for operations, thereby improving the flexibility of the solution.

[0102] It should be noted that the data processing device may be provided with multiple floating-point decoding components 12 and multiple floating-point encoding components 11 to process multiple vectors to be decoded and multiple vectors to be encoded in parallel.

[0103] Furthermore, the second extraction unit 121 is further configured to fill the data bits of each floating point number to be decoded to a specified number of bits. The specified number of bits may be pre-set, and the specified number of bits is the maximum value of the target number of bits n supported by the floating point decoding component 12 (hereinafter referred to as n). max Specifically, assuming that the floating-point number decoding component 12 supports two formats with different n values, LogFMT-8Bit and LogFMT-10Bit, then the n supported by the floating-point number decoding component 12 is max The number of designated bits may be 10. Accordingly, the second extraction unit 121 may first extract and obtain the data bits {K1, K2, ..., K p}、The data bits of each of the p floating-point numbers to be decoded {S1, S2, ..., S p When n is 8, K1, K2, ..., K p All of them are 7 bits, and K1, K2, ..., K p The high bits of each are filled with 0 (two 0s in this example) to make K1, K2, ..., K p Both extended to n maxSubtract 1 bit. In the case that the target bit number n is 10, K1, K2, …, K p are all 9 bits, no padding is needed. In the embodiment of the present application, padding is used to enable the data processing apparatus to decode a to-be-decoded vector with different values as the target bit number n, for example, to enable the data processing apparatus provided by the embodiment of the present application to support decoding of multiple precision floating point types such as LogFMT-8Bit, LogFMT-10Bit, etc.

[0104] Optionally, Figure 8 is a circuit schematic diagram of a reduction unit 122 provided by the embodiment of the present application, as shown in Figure 8 The reduction unit 122 includes a fourth adder, a fifth adder, a second multiplier, a third selector, a third multiplier, and a sixth adder; wherein the fourth adder and the fifth adder are connected to the second multiplier, the second multiplier and the third selector are connected to the third multiplier, and the third multiplier is connected to the sixth adder. In the embodiment of the present application, the reduction unit 122 can be composed of adders, multipliers, and selectors, which have a lower hardware cost.

[0105] The fourth adder is configured to calculate a sum of the data bits of the to-be-decoded floating point number output by the second extraction unit 121 and a third constant, to obtain a second sum value corresponding to the to-be-decoded floating point number. The reduction unit 122 is a SIMD unit, and can perform the process of determining the logarithmic representation corresponding to each to-be-decoded floating point number in parallel. Specifically, the data bits of the to-be-decoded floating point number can also be referred to as a numerical part, the control signal of the fourth adder is an addition operation signal, and the third constant can be pre-set based on an encoding specification. For example, the third constant is -1. The fourth adder can be input with the data bits of each to-be-decoded floating point number and the third constant, and the fourth adder can include p parallel addition operation circuits. The p parallel addition operation circuits can respectively calculate the sum of the data bits of the p to-be-decoded floating point numbers and the third constant in parallel, to obtain p second sum values. For example, K k represents the data bits of the kth to-be-decoded floating point number, 1≤k≤p, and the second sum value can be represented as K k -1. The fourth adder inputs each second sum value calculated to the second multiplier.

[0106] The control signal of the fifth adder is a subtraction operation signal, and the fifth adder is configured to calculate a third difference value between the maximum logarithmic representation and the minimum logarithmic representation in the second decoding information of the to-be-decoded vector. For example, the third difference value can be a j -a i The fifth adder inputs the calculated third difference value to the second multiplier. In the embodiment of the present application, the fifth adder is input with the subtraction operation signal as the control information, so that the addition operation can be realized by the adder, thereby saving hardware resources.

[0107] The second multiplier is configured to calculate a product of the second sum corresponding to the to-be-decoded floating-point number and the third difference corresponding to the third difference value output by the fifth adder, to obtain a second product corresponding to the to-be-decoded floating-point number. Specifically, the second multiplier can be a two-bit Booth multiplier. The second multiplier can include p parallel multiplication operation circuits, which can respectively and in parallel calculate products of the second sum corresponding to the data bits of the p to-be-decoded floating-point numbers and the third difference, to obtain the second products corresponding to the data bits of the p to-be-decoded floating-point numbers. The second product can be expressed as (K k -1) x (a j -a i ).

[0108] The third selector is configured to select a second coefficient corresponding to the target bit number n in the second decoding information and output the second coefficient to the third multiplier. It should be noted that, in a case where the second extraction unit 121 is configured to perform the operation of filling the data bits of each to-be-decoded floating-point number to a specified bit number, a fourth constant corresponding to a maximum value of n can be selected as the second coefficient and output to the third multiplier. In a case where the second extraction unit 121 is not configured to perform the operation of filling the data bits of each to-be-decoded floating-point number to a specified bit number, a fourth constant corresponding to an actual value of n included in the second decoding information can be selected as the second coefficient and output to the third multiplier.

[0109] According to the encoding specification of the logarithmic floating-point format, the fourth constant corresponding to n can be expressed as 1 / (2 n-1 -2), and the fourth constant has different specific values when n has different specific values. The third selector can define the fourth constant corresponding to n having different values in advance. The third selector can find the fourth constant corresponding to the specific value of n based on the input specific value of n. For example, for a component supporting two values of 8 and 10, the fourth constant corresponding to each of the two values of 8 and 10 can be set in advance: 1 / (2 7 -2) and 1 / (2 9 -2). The specific value of n in the second decoding information is used as a control signal of the third selector to select the corresponding value, so as to convert the division operation into a multiplication operation and process it by using the subsequent third multiplier. Of course, in a case where the second extraction unit 121 is configured to perform the operation of filling the data bits of each to-be-decoded floating-point number to a specified bit number, the maximum value of n can be directly selected as the control information to select 1 / (2 9 -2) as the second coefficient.

[0110] The third multiplier is used to calculate the product of the second product corresponding to the floating-point number to be decoded output by the second multiplier and the second coefficient output by the third selector, to obtain the third product corresponding to the floating-point number to be decoded. Specifically, the third multiplier can be a two-bit Booth multiplier, and the third multiplier can include p parallel multiplication circuits. These p parallel multiplication circuits can calculate in parallel the product of the second product corresponding to each data bit of the p floating-point numbers to be decoded and the second coefficient, to obtain the third product corresponding to each data bit of the p floating-point numbers to be decoded. The third product can be expressed as (K k -1)×(a j -a i )×(1 / (2 n-1 -2)).

[0111] The sixth adder is used to calculate the sum of the third product corresponding to the floating-point number to be decoded output by the third multiplier and the minimum logarithmic representation in the second decoding information, to obtain the logarithmic representation corresponding to the floating-point number to be decoded. Specifically, the sixth adder may include p parallel adding circuits, which can parallelly calculate the third product corresponding to each data bit of the p floating-point numbers to be decoded and the product of the minimum logarithmic representation in the second decoding information, and then restore the logarithmic representation corresponding to each data bit of the p floating-point numbers to be decoded. Wherein, the logarithmic representation a k It can be expressed as a i + (K k -1)×(a j -a i )×(1 / (2 n-1 Specifically, due to the result K obtained by encoding k is composed of (a k -a i ) / (a j -a i )×(2 n-1 -2) +1 rounded off, we can consider K k =(a k -a i ) / (a j -a i )×(2 n-1 -2) +1.

[0112] Therefore, based on the coded K k The original numerical result can be obtained by reversing the value. k =(a k -a i ) / (a j -a i )×(2 n-1 -2) +1 reverse deduction decodes: logarithmic representation of ak =a i + (K k -1)×(a j -a i )×(1 / (2 n-1 -2)). The logarithm represents a k Can be a fixed-point decimal.

[0113] In the embodiment of the present invention, the floating point number decoding component 12 may further include a second input unit, which is connected to the second extraction unit 121 and the restoration unit 122 respectively. The second input unit is used to input the received vector to be decoded into the second extraction unit 121, and to input a in the received second decoding information into the fifth adder in the restoration unit 122. i and a j , and inputting a in the second decoded information to the sixth adder i .

[0114] Thus, when the restoration unit 122, comprising the fourth adder, fifth adder, second multiplier, third selector, third multiplier, and sixth adder provided in the embodiment of the present invention, performs decoding, it restores the logarithmic representation corresponding to each data bit of the floating-point number to be decoded. It should be noted that all operations in the restoration unit 122 are performed in fixed-point decimal format. Because both the adder and the multiplier can be fully pipelined, the process of determining the logarithmic representation by the restoration unit 122 can be fully pipelined.

[0115] Optionally, in the embodiment of the present invention, the decoding unit 123 includes a decomposition subunit, an index subunit and a third splicing subunit, the decomposition subunit is connected to the index subunit and the third splicing subunit, and the index subunit is connected to the third splicing subunit.

[0116] The decomposition subunit is used to decompose the logarithmic representation corresponding to the floating-point number to be decoded into an integer part and a decimal part. The restoration unit 122 outputs the logarithmic representation corresponding to each floating-point number to be decoded to the decomposition subunit. Correspondingly, the decomposition subunit receives the logarithmic representation corresponding to each floating-point number to be decoded output by the second extraction unit 121. The logarithmic representation can be a fixed-point decimal. The decomposition subunit can include p parallel decomposition circuits, which can decompose the data bits of the p floating-point numbers to be decoded in parallel to obtain the integer part and decimal part corresponding to each of the p floating-point numbers to be decoded, and output the integer part of each floating-point number to be decoded to the third splicing subunit, while outputting the decimal part of each floating-point number to be decoded to the exponent subunit. Since the decimal part of each floating-point number to be decoded is the mantissa of the logarithm after encoding, that is, the decimal part is obtained by taking the logarithm of the mantissa. Therefore, the floating-point number decoding component 12 performs exponential operation based on the exponential sub-unit to restore the mantissa, and outputs the mantissa to the third splicing sub-unit to generate a non-logarithmic floating-point format representation.

[0117] The exponent subunit is used to perform an exponential operation on the decimal part of the floating-point number to be decoded to obtain the mantissa of the floating-point number to be decoded. The exponent subunit outputs the mantissa of each floating-point number to be decoded to the third splicing subunit.

[0118] The third splicing subunit is used to use the integer part corresponding to the floating-point number to be decoded as the exponent corresponding to the floating-point number to be decoded, and splice it with the sign bit corresponding to the floating-point number to be decoded and the mantissa corresponding to the floating-point number to be decoded, so as to obtain a non-logarithmic floating-point format representation of the floating-point number to be decoded.

[0119] Figure 9 This is a schematic diagram of a decoding process provided by an embodiment of the present invention. Figure 9 As shown, the second input unit may first receive the vector to be decoded and the second decoding information, and then the second extraction unit 121 extracts the sign bit and the value portion. Subsequently, the restoration unit 122 restores the logarithmic representation, and the decoding unit 123 decomposes the logarithmic representation into an integer part and a decimal part, performing an exponential operation on the decimal part to obtain a mantissa. The integer part is used as an exponent and concatenated with the sign bit and the mantissa.

[0120] In this embodiment of the present invention, the second extraction unit 121 outputs the sign bit of each floating-point number to be decoded to the third splicing subunit. The exponentiation subunit may include p parallel exponential operation circuits, which perform exponential operations on the mantissas of the p floating-point numbers to be decoded in parallel. The third splicing subunit may include p parallel third splicing circuits, which perform splicing on the sign bit, exponent, and mantissa of the p floating-point numbers to be decoded in parallel. In this way, parallel processing can improve the overall encoding efficiency of the vector to be encoded and decoded.

[0121] Specifically, the exponential subunit includes a second iterative operation circuit and a second multiplication and addition operation circuit; the second iterative operation circuit is used to select the corresponding iteration round as the second target iteration round based on the mantissa precision of the target floating-point format; iterate based on the decimal part corresponding to each floating-point number to be decoded until the second target iteration round is reached; the second multiplication and addition operation circuit is used to perform multiplication and addition operations based on the iteration result of the second iterative operation circuit and the preset constant corresponding to the second target iteration round to obtain the exponential operation result corresponding to the floating-point number to be decoded.

[0122] In an embodiment of the present invention, the exponent subunit calculates the exponent in accordance with the principle of the above-mentioned method of calculating the logarithm, and the CORDIC algorithm can be used. The input and output results of the CORDIC algorithm are both fixed-point decimals, and the CORDIC algorithm calculates the exponential result in an iterative manner. The second iterative operation circuit is an implementation circuit for implementing the CORDIC algorithm. The target floating-point format is the floating-point format required to be generated after decoding. For example, if a decoded floating-point number in FP8 format needs to be generated, then the target floating-point format is FP8. If a decoded floating-point number in BF16 format needs to be generated, then the target floating-point format is BF16. Among them, for different target floating-point formats, the required number of pipeline stage beats is different.

[0123] Further, referring to the above description, the mantissa precision of FP8 and BF16 can represent 5 rounds and 13 rounds at most, respectively. That is, when the target floating point format is FP8, the second target iteration round can be 5, and when the target floating point format is BF16, the second target iteration round can be 13. During the iteration process, z m The value added to the register each time it is updated is gradually decreasing. When this value cannot be represented using the required precision, the iteration stops. Accordingly, the mantissa precision of FP8 and BF16 can represent tan of 5 and 13 rounds respectively. -1 (2 -m ) results, thus requiring 5 and 13 rounds of iterations respectively.

[0124] When performing exponential operations, u m , v m , z m The initial values ​​of can be set to 1, 0, and θ respectively. Since the exponent directly obtained by the CORDIC algorithm is the result of the natural logarithm with the natural constant e as the base, and the base of the IEEE format exponent is 2, θ represents the product of the value of the exponent to be calculated (that is, the decimal part of the floating-point number to be decoded) and the constant ln2. Let f represent the second target iteration number. When performing exponential operations, m∈[1,f]. According to the rules of the CORDIC algorithm, after f rounds of iterations, u f +v fThe result is approximately . Wherein, u f +v f is the iteration result of the second iteration operation circuit, and the exponential operation result (i.e. the mantissa corresponding to the floating point number to be decoded) is The second iteration operation circuit outputs the values (i.e. the first variable and the second variable obtained in the last stroke) of the last pipeline stage register u f and v f to the second multiplication-addition operation circuit as the iteration result. The second multiplication-addition operation circuit is used to calculate u f +v f , and then calculate the product of (u f +v f ) and the preset constant corresponding to the second target iteration number, to obtain the exponential operation result corresponding to the floating point number to be decoded as the output of the exponential subunit. Wherein, the preset constant corresponding to the second target iteration number can be expressed as , and the numerical value of the preset constant corresponding to the second target iteration number is different when f takes different values. The second multiplication-addition operation circuit can store the preset constant corresponding to the second target iteration number obtained in different iteration rounds (i.e. when f takes different values), so as to use the preset constant corresponding to the second target iteration number for multiplication-addition operation during calculation. In the embodiment of the application, the exponential subunit can realize exponential operation through the second iteration operation circuit and the second multiplication-addition operation circuit, realize the support of the processor for the exponential operation, and thus the efficiency of the exponential operation can be ensured.

[0125] In the embodiment of the application, the full-pipelining design is adopted in the actual hardware circuit design, and the iteration process is unfolded into pipeline stages. Each stroke of the pipeline represents one iteration of the CORDIC algorithm, the first pipeline stage register u0 and the register v0 are assigned as 0 and 1, and the register z0 is assigned as the result of multiplying the decimal part by the constant ln2. After 5 rounds and 13 rounds of iteration, the FP8 format mantissa and the BF16 format mantissa can be obtained respectively.

[0126] Of course, in specific implementation, other ways can also be used to realize the exponential operation, for example, the exponential operation result can be directly obtained by using the lookup table (LUT) mode through pre-computing and storing the exponential values corresponding to different input values, and the embodiment of the application does not limit this.

[0127] Furthermore, a third splicing circuit uses the integer part corresponding to a floating-point number to be decoded as an exponent, and splices it with the sign bit and mantissa corresponding to the floating-point number to be decoded, to obtain a non-logarithmic floating-point format representation of the floating-point number to be decoded. In this embodiment of the present invention, it can be pre-set that when the target number of bits n is 8, the non-logarithmic floating-point format is FP8, and when the target number of bits n is 10, the non-logarithmic floating-point format is BF16. Accordingly, when the target number of bits n in the second decoding information is 8, the third splicing circuit performs splicing according to the FP8 format requirements to generate a non-logarithmic floating-point format representation that conforms to the FP8 format. When the target number of bits n is 10, the splicing is performed according to the BF16 format requirements to generate a non-logarithmic floating-point format representation that conforms to the BF16 format.

[0128] In the embodiment of the present invention, a decomposition subunit, an exponentiation subunit, and a third splicing subunit are provided in the decoding unit 123. The decomposition subunit, the exponentiation subunit, and the third splicing subunit provided by the processor itself process in sequence to obtain a non-logarithmic floating-point format representation of the floating-point number to be decoded, thereby ensuring hardware support for floating-point number decoding.

[0129] It should be noted that the floating-point number decoding component 12 also includes a second generation unit, which is connected to the second extraction unit 121. The second extraction unit 121 is also used to send the floating-point number to be decoded in the default form of the logarithmic floating-point format representation in the vector to be decoded to the second generation unit. The second generation unit is used to generate a default form of non-logarithmic floating-point format representation for the received floating-point number to be decoded. Accordingly, when the vector to be decoded includes infinity and non-number, other units in the floating-point number decoding component 12 can perform parallel processing on the floating-point numbers to be decoded except infinity and non-number. Among them, the default form of non-logarithmic floating-point format representation can be sNaN. That is, the second generation unit can directly decode the non-number and infinite floating-point numbers to be decoded in the vector to be decoded as sNaN. This embodiment of the present invention does not limit this.

[0130] Optionally, refer to Figure 7 The data processing device further includes: a data input component 13 and a data operation component 14, the data input component 13 is connected to the floating point decoding component 12, and the floating point decoding component 12 is connected to the data operation component 14; The data input component 13 is used to send the vector to be decoded and the second decoding information to the floating-point number decoding component 12.

[0131] The data operation component 14 is used to perform data processing based on the decoded vector output by the floating-point number decoding component 12.

[0132] The hardware operation components in the data processing device are all designed with full pipelining to ensure data throughput. In the processing of a data processing device, the data input component 13 can first obtain the to-be-decoded vector and the second decoding information required for decoding from the network packet and send them to the floating-point decoding component 12. The floating-point decoding component 12 decodes and sends the decoded vector to the data operation component 14 for processing. The data operation component 14 takes the floating-point vector processed by the data processing as the to-be-encoded vector, sends it to the floating-point encoding component for encoding, obtains the encoded vector and the first decoding information, and sends them to the network of the distributed system for transmission.

[0133] In an existing manner, the model parameters are directly quantized by taking the logarithm or combined with the Posit format. The floating-point number in the Posit format is obtained by additionally adding a regime field in the IEEE754 floating-point number format to improve the representation range of the floating-point number. In the manner of directly taking the logarithm of the model, the precision loss is too large, which seriously reduces the performance of the model. In the manner of combining the logarithm with the Posit format, more errors are caused due to the difficulty in accurately performing addition and subtraction operations under the logarithmic representation.

[0134] In the embodiment of the present application, the logarithmic floating-point format is used only when communicating between different data processing devices. When the data processing device performs data operation (for example, model inference), the floating-point decoding component 12 is used to convert the logarithmic floating-point format into a non-logarithmic floating-point format, that is, the logarithmic floating-point format is not directly used when performing model inference. For example, the LogFMT-8Bit format is used in the data transmission process, and the FP32 format is decoded for operation when performing multiplication and accumulation operation. In this way, the data precision during data operation can be ensured, and the problem of excessive precision loss caused by more errors due to the difficulty in accurately performing addition and subtraction operations under the logarithmic representation when performing model inference can be avoided. Exemplarily, compared with using the FP16 or BF16 format, the data transmission bit width reaches 2 bytes, so the communication time is longer and the communication pressure is greater. In the embodiment of the present application, the encoded logarithmic floating-point format with the target bit number n=8 is used when performing data transmission. Since the data transmission bit width of the logarithmic floating-point format is small, only 8 bits, the data transmission amount in the communication process can be reduced, and the communication pressure can be reduced.

[0135] Optionally, the data operation component 14 includes p parallel operation units, and p is the number of floating-point numbers included in the decoded vector. One operation unit is configured to perform multiplication and addition operation on a target floating-point number in the local register based on a decoded floating-point number pair, and output an exception signal when the decoded floating-point number pair includes a special type of floating-point number. The two decoded floating-point numbers included in the decoded floating-point number pair come from two decoded vectors, respectively.

[0136] In one application scenario, the data operation component 14 may be a multiplication and addition component, which includes p multiplication and addition operation units. For example, a vector to be decoded including 16 floating-point numbers to be decoded may be input to each of the two floating-point decoding components 12. Accordingly, the data operation component 14 receives two decoded vectors. The multiplication and addition unit performs the multiplication and accumulation operation. There are 16 FP32 format floating-point numbers stored in the local register file in the multiplication and addition unit, and the two decoded vectors and a group of FP32 format floating-point numbers stored in the local register file are represented as {g1, g2, ..., g 16}、{h1,h2,…,h 16}, {r1, r2, …, r 16}. The multiplication and addition unit can perform operations: r t ' =g t ×h t +r t , where 1≤t≤16. After the multiplication and addition unit completes the operation, it calculates the FP32 format r t 'Stored in the original storage r t That is, the multiplication and addition component performs fused multiplication and addition operations on the two input decoded vectors and the floating-point numbers in FP32 format stored in the local accumulation register, and the intermediate multiplication results are stored in the local accumulation register in FP32 format.

[0137] In this example, the multiplication and addition unit is a SIMD unit, which may include 16 parallel multiplication and addition units, and the 16 parallel multiplication and addition units perform operations in parallel. Each pair of decoded floating-point numbers and the target floating-point number in the local register is input into the respective multiplication and addition units for execution. For example, (g t , h t ) and its corresponding target floating point number r t The input is executed in the multiplication and addition unit t. The final calculated {r1', r2', ..., r 16 '} can be used as a vector to be encoded and encoded by the floating point encoding component 11 into an encoded vector in LogFMT-8Bit or LogFMT-10Bit data format.

[0138] In an embodiment of the present invention, a number of arithmetic units is provided equal to the number p of floating-point numbers included in the decoded vectors, so that the p decoded floating-point numbers in two decoded vectors can be multiplied and added in parallel by p parallel arithmetic units, thereby improving processing efficiency. In the data processing device provided in an embodiment of the present invention, by providing a data operation component 14 including p parallel arithmetic units, a single instruction stream, multiple data (SIMD) approach can be adopted when responding to multiply-add instructions, thereby providing support for SIMD. SIMD can perform multiple multiply-add operations in parallel within a single instruction, thereby improving the parallel performance of floating-point operations and thereby increasing computational throughput and energy efficiency.

[0139] Specifically, the multiplication and addition unit may include a first extraction circuit, a multiplication operation circuit, a fourth selector, a second extraction circuit, an addition operation circuit, and a fifth selector. Figure 10 Schematic diagram of a multiplication and addition operation provided by an embodiment of the present invention, such as Figure 10 As shown, dashed lines represent pipeline stage division. In the first step, a first extraction circuit can extract the input encoded floating-point number into a sign bit, an exponent, and a mantissa. When the encoded floating-point number includes a special type of floating-point number, a special input signal is input to the fourth selector. A multiplication circuit performs a sign bit operation, an exponent operation (i.e., exponent addition), and a mantissa operation (i.e., mantissa multiplication) on the two encoded floating-point numbers. In the second step, the multiplication circuit performs a normalization shift and result rounding operation. Specifically, the normalization and result rounding subcircuit in the multiplication circuit can determine a shift length based on the number of leading zeros in the mantissa multiplication result, and perform a normalization shift based on this bit shift length. The result rounding operation is used to concatenate the sign bit operation result, the exponent addition result, and the mantissa multiplication result after normalization shifting, and then process the mantissa multiplication result according to the requirements of the desired floating-point format (e.g., FP32). Next, a multiplication result of the two encoded floating-point numbers in the encoded floating-point number pair after the result rounding operation is input to the fourth selector. After receiving the multiplication result, the fourth selector outputs the multiplication result as an intermediate result to the second extraction circuit. After receiving the special input signal, an exception signal is output to the second extraction circuit. Exemplarily, when the special input signal indicates that the encoded floating-point number pair includes infinity, a first exception signal is output; when the special input signal indicates that the encoded floating-point number pair includes non-number, a second exception signal is output.

[0140] It should be noted that, in the first beat, before extracting the sign bit, exponent, and mantissa, if the encoded floating-point number is not in BF16 format, the mantissa of the encoded floating-point number may be zero-extended at the low bit based on the number of leading 0s in the mantissa of the encoded floating-point number, and the exponent offset difference may be added to the exponent of the encoded floating-point number. For example, the exponent value is added by 2. 7 -2 4 = 112. Then, the displacement length corresponding to the number of leading zeros is subtracted. This allows alignment with the BF16 format, allowing floating-point numbers in FP8 and BF16 formats to use the same data path to produce a multiplication result in BF16 format. Because the exponent and exponent offset of floating-point numbers in BF16 and FP32 formats are the same, the result rounding step only needs to retain the calculated mantissa to the number of mantissas required for the FP32 format.

[0141] Furthermore, when the second extraction circuit receives the multiplication result (i.e., the intermediate result) output by the fourth selector in the third beat, it extracts the target floating-point number and the intermediate result in the local accumulator register into the sign bit, exponent, and mantissa, respectively. Assume that the sign bit, exponent, and mantissa of the intermediate result are sign bit 1, exponent 1, and mantissa 1, respectively, and the sign bit, exponent, and mantissa of the target floating-point number are sign bit 2, exponent 2, and mantissa 2, respectively. The addition circuit in the multiplication-addition unit calculates the absolute value of the exponent difference between exponent 1 and exponent 2 in the third beat. If this absolute value is greater than a preset absolute value threshold, the subcircuit corresponding to the first bit shift path aligns exponents 1 and 2, unifying the exponents. The mantissa corresponding to the smaller exponent of exponents 1 and 2 is then shifted accordingly to achieve mantissa alignment. For example, if the exponent difference is 3, the mantissa corresponding to the smaller exponent can be right-shifted by 3 bits.

[0142] Otherwise, the subcircuit corresponding to the second shift path is used to align the orders 1 and 2, unifying the orders. The mantissa corresponding to the smaller order, order 1 or order 2, is then shifted accordingly to achieve mantissa alignment. The shift length of the shifter in the first shift path is greater than the shift length of the shifter in the second shift path. The preset absolute value threshold can be set as needed, for example, 2. The maximum value of the mantissa shifted by the second shift path does not exceed the preset absolute value threshold. Therefore, the shift length of the shifter used is short, the circuit structure is simple, and the delay is short.

[0143] Next, the subcircuit used performs the operations of adding the mantissa, normalizing the shift, and rounding the result. It should be noted that when the subcircuit corresponding to the first shift path is used, the shift is performed in the third beat, and the mantissa addition, normalizing the shift, and rounding the result are performed in the fourth beat ( Figure 10(The case of using the sub-circuit corresponding to the first bit shift path is used as an example.) When the sub-circuit corresponding to the second bit shift path is used, the shift and mantissa addition operations are performed in the third beat, and the normalization shift and result rounding operations are performed in the fourth beat.

[0144] It should be noted that the first and second shift paths correspond to different sub-circuits in the fourth beat. For the first shift path, because the absolute value of the order difference is greater than the preset absolute value threshold, the order of magnitude difference between the shifted mantissas 1 and 2 is large. Therefore, the order of magnitude of the result after adding the mantissas does not differ significantly from the shifted mantissas 1 and 2, resulting in a small shift length for the normalized shift in the fourth beat. Conversely, for the second shift path, because the absolute value of the order difference is not greater than the preset absolute value threshold, the order of magnitude difference between the shifted mantissas 1 and 2 is small. Therefore, the order of magnitude of the result after adding the mantissas does not differ significantly from the shifted mantissas 1 and 2, resulting in a large shift length for the normalized shift in the fourth beat. Therefore, the sub-circuit corresponding to the first shift path has a simple structure and short delay. The delay of mantissa addition and normalized shift is substantially consistent with the delay of normalized shift in the sub-circuit corresponding to the second shift path.

[0145] It should be noted that the fifth selector can be connected to the subcircuit corresponding to the first shift path and the subcircuit corresponding to the second shift path. The absolute value of the order difference serves as a control signal for the fifth selector. If the control signal is greater than a preset absolute value threshold, the output value of the subcircuit corresponding to the first shift path is selected as the final calculation result. Otherwise, the output value of the subcircuit corresponding to the second shift path is selected as the final calculation result. After obtaining the final calculation result, the local accumulator register is updated. Furthermore, if the second extraction circuit receives an abnormal signal output by the fourth selector in the third beat, it is further configured to input a special input signal to the fifth selector. The fifth selector is further configured to output an abnormal signal after receiving the special input signal.

[0146] In an embodiment of the present invention, the vector to be decoded can be an encoded vector generated by the data processing device, or an encoded vector sent by another data processing device. For example, in one application scenario, a computing node in a distributed system may include a data processing device. The model in the distributed system may be used for image processing. The data processing device in the computing node may extract image vectors from each of the two images to be fused as the vector to be encoded, where the vector to be encoded includes the light intensity value of each pixel in the image to be fused. After encoding, two encoded vectors are obtained. These two encoded vectors can be sent to the next data processing device. Because the data transmission bit width of the logarithmic floating-point format is relatively small, the amount of data transmitted during the communication process can be reduced, thereby alleviating communication pressure. The next data processing device may decode the two received encoded vectors as two vectors to be decoded, thereby obtaining two decoded vectors. Subsequently, the internal data processing component 14 performs a fused multiplication-addition operation on the two decoded vectors. For example, the fused multiplication-addition operation is performed on the decoded vectors decoded to FP32 format. In this way, the data accuracy can be ensured during data calculations, avoiding the problem of difficulty in accurately performing addition and subtraction operations under logarithmic representation, which will bring more errors and excessive loss of accuracy. The final calculation result is the fused image.

[0147] Due to the high computational load and long computational delays in the model operation process, some implementations use neural network quantization to reduce the accuracy of computational data, achieve model compression, and reduce the model's memory usage, computing resource consumption, and inference delay. Neural network quantization is a technology that converts floating-point numbers into low-bit representations (e.g., INT8, INT4, etc.). However, this approach can lead to vanishing and exploding model gradients due to low precision. In an embodiment of the present invention, a high-precision data format, such as the FP32 format, is used during data operations in the combine phase. This format has a wider data representation range, can meet precision requirements, and avoid the problem of vanishing and exploding model gradients due to low precision, which in turn leads to a decrease in model expressiveness and generalization capabilities. At the same time, it can avoid the limitation of extremely fine-grained processing and control of training weights to prevent data overflow.

[0148] Compared to relying solely on software encoding and decoding, software encoding and decoding places greater pressure on registers, leading to lower actual processing efficiency. When encoding vectors to be encoded into a logarithmic floating-point format and decoding vectors to be decoded into a non-logarithmic floating-point format, the data processing device in the processor provided by the embodiments of the present invention, as a native component provided in the processor, enables the processor to support encoding and decoding operations for the logarithmic floating-point format, improving encoding and decoding efficiency, thereby increasing the communication and inference speed of each computing node in the distributed system and reducing the operational latency of the distributed system.

[0149] Different data processing devices can be set in different processor cores of a processor, for example, in the tensor core of a GPU. In this way, the hardware encoding component and the decoding component can complete the encoding and decoding process within the processor core. When performing data operations within the GPU core, the logarithmic floating-point format is not used. When different data processing devices between GPU cores communicate, they are sent in the form of encoded vectors, thereby improving the GPU's support for encoding and decoding operations, reducing the latency of the encoding and decoding process, and thus improving the overall processing speed without affecting accuracy. In this way, large language model programs deployed in distributed systems can run with high efficiency, low latency, and low communication pressure.

[0150] Figure 11 is a flowchart of a data processing method provided by an embodiment of the present invention, which can be applied to any of the aforementioned data processing devices, such as Figure 11 As shown, the method includes: Step 201: Extract the sign bit and data bits of the floating-point number to be encoded in the vector to be encoded.

[0151] Step 202: Based on the data bits of the floating-point number to be encoded, search for the indexes corresponding to the maximum floating-point number and the minimum floating-point number in the vector to be encoded respectively to obtain the maximum index and the minimum index.

[0152] Step 203: Based on the target number of bits n, the sign bit of the floating-point number to be encoded, the data bits of the floating-point number to be encoded, the maximum index, and the minimum index, encode the floating-point number to be encoded into an n-bit logarithmic floating-point format representation to obtain an encoded vector.

[0153] Optionally, the above step 202 may include: Step 2021: Based on the data bits of the floating-point number to be encoded, search for the index corresponding to the maximum floating-point number in the vector to be encoded to obtain the maximum index.

[0154] Step 2022: Based on the data bits of the floating-point number to be encoded, search for the index corresponding to the minimum floating-point number in the vector to be encoded to obtain the minimum index.

[0155] Optionally, the above step 203 may include: Step 2031: Perform a logarithmic operation on the mantissa of the floating-point number to be encoded to generate a logarithmic operation result.

[0156] Step 2032: Based on the exponent of the floating-point number to be encoded and the logarithmic operation result, obtain the logarithmic representation of the floating-point number to be encoded.

[0157] Step 2033: Determine the maximum logarithmic representation and the minimum logarithmic representation based on the logarithmic representation of the floating-point number to be encoded, the maximum index, and the minimum index, and generate target data bits corresponding to the floating-point number to be encoded based on the target number of bits n.

[0158] Step 2034: concatenate the sign bit of the floating-point number to be encoded with the target data bit to obtain the logarithmic floating-point format representation.

[0159] Optionally, the method may further include: Step S21 : extracting the sign bit and data bits of the floating-point number to be decoded in the vector to be decoded.

[0160] Step S22: Restore the logarithmic representation corresponding to the floating-point number to be decoded based on the second decoding information of the vector to be decoded and the data bits of the floating-point number to be decoded.

[0161] Step S23: Generate a non-logarithmic floating-point format representation of the floating-point number to be decoded based on the sign bit of the floating-point number to be decoded and the logarithmic representation corresponding to the floating-point number to be decoded, to obtain a decoded vector.

[0162] Optionally, the above step S23 may specifically include: Step S231: Decompose the logarithmic representation corresponding to the floating-point number to be decoded into an integer part and a decimal part.

[0163] Step S232: Perform an exponential operation on the decimal part to obtain a mantissa corresponding to the floating-point number to be decoded.

[0164] Step S233 uses the integer part corresponding to the floating-point number to be decoded as the exponent corresponding to the floating-point number to be decoded, and concatenates it with the sign bit corresponding to the floating-point number to be decoded and the mantissa corresponding to the floating-point number to be decoded, to obtain the non-logarithmic floating-point format representation.

[0165] The specific implementation process of the data processing method provided by the embodiment of the present invention and the technical effects that can be achieved can be referred to the relevant description of the above-mentioned data processing device embodiment, which will not be repeated here.

[0166] Reference Figure 12 , is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 12 As shown, the electronic device includes: a processor, a memory, a communication interface and a communication bus.

[0167] The processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, which enables the processor to execute the data processing method of the above embodiment. The executable instructions can form a program.

[0168] An embodiment of the present invention provides a processor including the above-mentioned data processing device.

[0169] An embodiment of the present invention provides a machine-readable medium having instructions stored thereon. When executed by one or more processors, the processors are enabled to execute the data processing method of the aforementioned embodiment.

[0170] The data processing device, method, processor and electronic device provided by the present invention are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A data processing device, characterized in that: include: A floating point number encoding component consisting of a first extraction unit, a search unit, and an encoding unit; The first extraction unit is used to extract the sign bit and data bit of the floating point number to be encoded in the vector to be encoded; The search unit is configured to search for the indexes corresponding to the maximum floating point number and the minimum floating point number in the vector to be encoded based on the data bits of the floating point number to be encoded, respectively, to obtain the maximum index and the minimum index; The encoding unit is used to encode the floating-point number to be encoded into an n-bit logarithmic floating-point format representation based on a target number of bits n, a sign bit of the floating-point number to be encoded, a data bit of the floating-point number to be encoded, the maximum index, and the minimum index to obtain an encoded vector.

2. The device according to claim 1, characterized in that The search unit includes a first searcher and a second searcher; The first finder is configured to find an index corresponding to a maximum floating-point number in the vector to be encoded based on data bits of the floating-point number to be encoded, to obtain the maximum index; the maximum floating-point number represents a maximum value in the vector to be encoded; The second finder is used to find the index corresponding to the minimum floating-point number in the vector to be encoded based on the data bits of the floating-point number to be encoded, so as to obtain the minimum index; the minimum floating-point number represents the minimum value in the vector to be encoded.

3. The device according to claim 2, characterized in that The first finder and the second finder each include a plurality of comparator layers; The two input ends of each comparator in the first comparator layer respectively receive the data bits of the two floating-point numbers to be encoded; the two input ends of each comparator in the next comparator layer are respectively connected to the output ends of the two comparators in the current comparator layer; The comparator in the first finder is used to output the maximum value of the data bits of the two input floating-point numbers to be encoded and the index of the maximum value; the comparator in the second finder is used to output the minimum value of the data bits of the two input floating-point numbers to be encoded and the index of the minimum value.

4. The device according to claim 1, characterized in that The data bits of the floating-point number to be encoded include an exponent and a mantissa; the encoding unit includes a logarithm subunit, a first splicing subunit, an encoding subunit and a second splicing subunit; The logarithm subunit is used to perform a logarithm operation on the mantissa of the floating-point number to be encoded to generate a logarithm operation result; The first splicing subunit is configured to splice the logarithmic representation of the floating-point number to be encoded based on the exponent of the floating-point number to be encoded and the logarithmic operation result; The encoding subunit is configured to determine a maximum logarithmic representation and a minimum logarithmic representation based on the logarithmic representation of the floating-point number to be encoded, the maximum index, and the minimum index, and generate a target data bit corresponding to the floating-point number to be encoded based on the target number of bits n; The second splicing subunit is used to splice the sign bit of the floating-point number to be encoded with the target data bit to obtain the logarithmic floating-point format representation.

5. The device according to claim 4, characterized in that The encoding subunit includes a first selector, a first adder, a second adder, a first divider, a first multiplier, a second selector, a third adder, and a rounding circuit; The first selector is used to determine the maximum logarithmic representation and the minimum logarithmic representation; The first adder is configured to calculate a first difference between the maximum logarithmic representation and the minimum logarithmic representation; The second adder is configured to calculate a difference between the logarithmic representation of the floating-point number to be encoded and the minimum logarithmic representation to obtain a second difference corresponding to the floating-point number to be encoded; The first divider is configured to perform a division operation on the second difference and the first difference to obtain a first division result corresponding to the floating-point number to be encoded; The second selector is configured to use a first constant corresponding to the target number of bits n as a first coefficient and output the first constant to the first multiplier; The first multiplier is configured to calculate a product of the first coefficient and the first division result to obtain a first product corresponding to the floating-point number to be encoded; The third adder is used to calculate the sum of the first product and a second constant to obtain a first sum corresponding to the floating-point number to be encoded; The rounding circuit is used to round the first sum value to obtain the target data bits corresponding to the floating-point number to be encoded.

6. The device according to claim 1, characterized in that The device further comprises: a floating point decoding component consisting of a second extraction unit, a restoration unit and a decoding unit; The second extraction unit is used to extract the sign bit and data bit of the floating point number to be decoded in the vector to be decoded; The restoration unit is configured to restore the logarithmic representation corresponding to the floating-point number to be decoded based on the second decoding information of the vector to be decoded and the data bits of the floating-point number to be decoded; The decoding unit is configured to generate a non-logarithmic floating-point format representation of the floating-point number to be decoded based on a sign bit of the floating-point number to be decoded and a logarithmic representation corresponding to the floating-point number to be decoded, thereby obtaining a decoded vector.

7. The device according to claim 6, characterized in that The restoration unit includes a fourth adder, a fifth adder, a second multiplier, a third selector, a third multiplier and a sixth adder; The fourth adder is used to calculate the sum of the data bits of the floating-point number to be decoded and a third constant to obtain a second sum; The fifth adder is configured to calculate a third difference between the maximum logarithmic representation and the minimum logarithmic representation in the second decoded information; The second multiplier is configured to calculate a product of the second sum and the third difference to obtain a second product; The third selector is configured to select a corresponding second coefficient according to a target number n of bits in the second decoded information and output the selected second coefficient to the third multiplier; The third multiplier is used to calculate the product of the second product and the second coefficient to obtain a third product; The sixth adder is used to calculate the sum of the third product and the minimum logarithmic representation in the second decoding information to obtain the logarithmic representation corresponding to the floating-point number to be decoded.

8. The device according to claim 6, characterized in that The decoding unit includes a decomposition subunit, an index subunit and a third splicing subunit; The decomposition subunit is used to decompose the logarithmic representation corresponding to the floating-point number to be decoded into an integer part and a decimal part; The exponent subunit is used to perform an exponential operation on the decimal part to obtain a mantissa corresponding to the floating-point number to be decoded; The third splicing subunit is used to use the integer part corresponding to the floating-point number to be decoded as the exponent corresponding to the floating-point number to be decoded, and splice it with the sign bit corresponding to the floating-point number to be decoded and the mantissa corresponding to the floating-point number to be decoded to obtain the non-logarithmic floating-point format representation.

9. A data processing method, characterized in that: Applied to the device of any one of claims 1 to 8, the method comprises: Extracting the sign bit and data bit of the floating point number to be encoded in the vector to be encoded; Based on the data bits of the floating-point number to be encoded, searching for the indexes corresponding to the maximum floating-point number and the minimum floating-point number in the vector to be encoded, respectively, to obtain the maximum index and the minimum index; Based on a target number of bits n, a sign bit of the floating-point number to be encoded, data bits of the floating-point number to be encoded, the maximum index, and the minimum index, the floating-point number to be encoded is encoded into an n-bit logarithmic floating-point format representation to obtain an encoded vector.

10. A processor, characterized in that: A device comprising any one of claims 1 to 8.

11. An electronic device, characterized in that: include: The processor, memory, communication interface and communication bus according to claim 10, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store executable instructions, and the executable instructions enable the processor to execute the method according to claim 9.

Images