Data processing apparatus, method, processor and electronic device

Abstract

Embodiments of the present application provide a kind of data processing device, method, processor and electronic equipment, it is related to computer technical field, in the device, including by first extraction unit, look-up unit and encoding unit consist of floating point number encoding component.First extraction unit, for extracting the sign bit and data bit of the floating point number to be encoded in the vector to be encoded.Look-up unit, for based on the data bit of the floating point number to be encoded, respectively look up the respective corresponding index of the maximum floating point number and the minimum floating point number in the vector to be encoded, obtain maximum index and minimum index.Encoding unit, for based on target bit number n, the sign bit of the floating point number to be encoded, the data bit of the floating point number to be encoded, maximum index and minimum index, the floating point number to be encoded is encoded into n-bit logarithmic floating point format representation, obtain the encoded vector.In this way, it is realized to high efficiency to the floating point number of logarithmic floating point format is represented.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data processing device and method, a processor and an electronic device. BACKGROUND

[0002] The floating point format refers to the specific structure of the floating point number stored and represented in the computer. The floating point format includes the logarithmic floating point format (LogFMT-nBit) and the non-logarithmic floating point format. The non-logarithmic floating point format is the floating point number format defined in the IEEE754 standard, such as TF32, BF16, FP8, etc. The logarithmic floating point format is a logarithmic floating point format defined to reduce the communication bandwidth while maintaining high model accuracy. The logarithmic floating point format can map the floating point number to the logarithmic space, significantly reducing the data transmission bit width while maintaining high model accuracy.

[0003] In order to improve the transmission efficiency of data, how to efficiently represent the floating point number in the logarithmic floating point format in the data processing process has become a technical problem to be solved. SUMMARY

[0004] The embodiments of the present application provide a data processing device and method, a processor and an electronic device, which can solve the problem of how to efficiently represent the floating point number in the logarithmic floating point format.

[0005] In order to solve the above problem, the embodiments of the present application disclose a data processing device, comprising: a floating point number encoding component composed of a first extraction unit, a lookup unit and an encoding unit;

[0006] The first extraction unit is configured to extract the sign bit and the data bit of the floating point number to be encoded in the vector to be encoded.

[0007] The lookup unit is configured to respectively look up the respective indexes corresponding to the maximum floating point number and the minimum floating point number in the vector to be encoded based on the data bit of the floating point number to be encoded, to obtain the maximum index and the minimum index.

[0008] The encoding unit is configured to encode the floating point number to be encoded into the logarithmic floating point format representation of n bits based on the target bit number n, the sign bit of the floating point number to be encoded, the data bit of the floating point number to be encoded, the maximum index and the minimum index, to obtain the encoded vector.

[0009] On the other hand, the embodiments of the present application disclose a data processing method applied to the above-mentioned data processing device, the method comprising:

[0010] extracting the sign bit and the data bit of the floating point number to be encoded in the vector to be encoded;

[0011] finding indexes corresponding to the maximum floating-point number and the minimum floating-point number in the to-be-encoded vector respectively based on the data bits of the to-be-encoded floating-point number, to obtain a maximum index and a minimum index;

[0012] encoding the to-be-encoded floating-point number into an n-bit logarithmic floating-point format representation based on the target bit number n, the sign bit of the to-be-encoded floating-point number, the data bits of the to-be-encoded floating-point number, the maximum index and the minimum index, to obtain an encoded vector.

[0013] The embodiment of the present application further discloses a processor comprising the data processing apparatus.

[0014] In another aspect, the embodiment of the present application discloses an electronic device comprising the processor, the memory, the communication interface and the communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction makes the processor execute the foregoing method.

[0015] The embodiment of the present application further discloses a machine readable medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method as described above.

[0016] The embodiment of the present application has the following advantages: the data processing apparatus applied to the processor provided by the embodiment of the present application comprises a floating-point number encoding component composed of a first extraction unit, a finding unit and an encoding unit. The first extraction unit is used for extracting the sign bit and the data bit of a to-be-encoded floating-point number in a to-be-encoded vector. The finding unit is used for finding indexes corresponding to the maximum floating-point number and the minimum floating-point number in the to-be-encoded vector respectively based on the data bits of the to-be-encoded floating-point number, to obtain a maximum index and a minimum index. The encoding unit is used for encoding the to-be-encoded floating-point number into an n-bit logarithmic floating-point format representation based on the target bit number n, the sign bit of the to-be-encoded floating-point number, the data bits of the to-be-encoded floating-point number, the maximum index and the minimum index, to obtain an encoded vector. In this way, the data processing apparatus provided as hardware supports the encoding operation for the logarithmic floating-point format, and the efficiency of encoding is improved to a certain extent, and the logarithmic floating-point format floating-point number is represented efficiently.

[0017] The embodiment of the present application uses the logarithmic floating-point format when communicating between different data processing apparatuses, and uses the floating-point number decoding component to convert the logarithmic floating-point format into the non-logarithmic floating-point format when the data processing apparatus performs data operation, so as to ensure the data precision when the data operation is performed. Meanwhile, the floating-point number decoding component is arranged to make the processor support the decoding operation for the logarithmic floating-point format, and the efficiency of decoding is improved to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application.

[0019] Figure 1 is a frame schematic diagram of a data processing device provided by the embodiments of the present application;

[0020] Figure 2 is a structure schematic diagram of a floating point number encoding component provided by the embodiments of the present application;

[0021] Figure 3 is a structure schematic diagram of a first finder provided by the embodiments of the present application;

[0022] Figure 4 is a circuit principle diagram of an algorithm implementation circuit in the prior art;

[0023] Figure 5 is a circuit schematic diagram of an encoding subunit provided by the embodiments of the present application;

[0024] Figure 6 is a schematic diagram of an encoding process provided by the embodiments of the present application;

[0025] Figure 7 is a frame schematic diagram of another data processing device provided by the embodiments of the present application;

[0026] Figure 8 is a circuit schematic diagram of a restoring unit provided by the embodiments of the present application;

[0027] Figure 9 is a decoding process schematic diagram provided by the embodiments of the present application;

[0028] Figure 10 is a multiplication-addition operation schematic diagram provided by the embodiments of the present application;

[0029] Figure 11 is a step flow chart of a data processing method provided by the embodiments of the present application;

[0030] Figure 12 is a structure schematic diagram of an electronic device provided by the embodiments of the present application.

[0031] Reference signs: 11-floating point number encoding component, 111-first extraction unit, 112-finder unit, 113-encoding unit, 1131-logarithm subunit, 1132-first splicing subunit, 1133-encoding subunit, 1134-second splicing subunit, 121-second extraction unit, 122-restoring unit, 123-decoding unit, 12-floating point decoding component, 13-data input component, 14-data operation component. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

[0033] The data processing apparatus provided in the embodiments of the present application can be applied to any computing node in a distributed system, and the computing node can be an electronic device such as a computer or a server. A deep learning model such as a large language model (LLM) can be deployed in the distributed system. After data processing by each computing node, the obtained data needs to be transmitted to other computing nodes.

[0034] For example, the model can be a large language model BLOOM-176B based on a Transformer architecture, containing 176 billion parameters, and 48 computing nodes are required to run the inference process, and data transmission is required between the computing nodes. The transmitted data can be the intermediate operation result of the large language model, and the transmission mode can be that the hidden layer data blocks of the model are operated on different computing nodes and then collected to a certain node for further transmission and operation. Alternatively, the hidden layer of the model can be divided into multiple groups, each group is operated using a computing node, and a pipeline is formed between different computing nodes to transmit the intermediate operation result from front to back.

[0035] The transmitted data can be a vector composed of floating-point numbers, and there are multiple floating-point formats of floating-point numbers available for use. For example, TF32, BF16, FP8, and a logarithmic floating-point format (LogFMT-nBit) proposed for the model to maintain high model accuracy while reducing communication bandwidth. The LogFMT-nBit can map the floating-point number to a logarithmic space, and since the data transmission bit width of the logarithmic floating-point format is smaller, n bits, the data transmission bit width is significantly reduced while maintaining high model accuracy. However, the current processor does not provide a function component for encoding and decoding the logarithmic floating-point format to efficiently represent the floating-point number of the logarithmic floating-point format, i.e., to efficiently convert the floating-point number of the logarithmic floating-point format. Therefore, the embodiments of the present application provide a data processing apparatus.

[0036] Figure 1 is a schematic diagram of the framework of a data processing apparatus provided by the embodiments of the present application, which can be applied to a processor such as Figure 1 As shown in the figure, the apparatus includes a floating-point number encoding component 11 composed of a first extraction unit 111, a lookup unit 112, and an encoding unit 113. The first extraction unit 111 is connected to the lookup unit 112 and the encoding unit 113, and the lookup unit 112 is connected to the encoding unit 113.

[0037] The first extraction unit 111 is configured to extract the sign bit and the data bit of the floating point number to be encoded in the vector to be encoded.

[0038] The searching unit 112 is configured to search the index of the maximum floating point number and the index of the minimum floating point number in the vector to be encoded respectively based on the data bit of the floating point number to be encoded, and obtain the maximum index and the minimum index.

[0039] The encoding unit 113 is configured to encode the floating point number to be encoded into the logarithmic floating point format representation with n bits based on the target bit number n, the sign bit of the floating point number to be encoded, the data bit of the floating point number to be encoded, the maximum index and the minimum index, and obtain the encoded vector.

[0040] The floating point number encoding component 11 provided by the embodiment of the present application provides a specific hardware implementation scheme. The floating point number included in the vector to be encoded is the floating point number to be encoded. The vector to be encoded can include q floating point numbers, i.e., the vector to be encoded includes q floating point numbers to be encoded. The vector to be encoded can be externally input or generated by other components in the data processing device, and the embodiment of the present application does not limit this. The floating point number to be encoded can be an IEEE standard floating point number.

[0041] The first extraction unit 111 extracts the sign bit of each floating point number to be encoded, and takes the part other than the sign bit as an independent data bit. The data bit of each floating point number to be encoded extracted is output to the searching unit 112, and the sign bit and the data bit of each floating point number to be encoded extracted are output to the encoding unit 113.

[0042] Further, the maximum floating point number is used to represent the maximum value in the vector to be encoded, i.e., the maximum floating point number refers to the non-zero floating point number with the maximum absolute value in the vector to be encoded. The minimum floating point number is used to represent the minimum value in the vector to be encoded, i.e., the minimum floating point number refers to the non-zero floating point number with the minimum absolute value in the vector to be encoded. The maximum index and the minimum index are used to represent the relative position of the maximum floating point number and the minimum floating point number in the vector to be encoded, i.e., to represent the position of the maximum floating point number and the minimum floating point number in the vector to be encoded.

[0043] For a floating point number to be encoded, the data bit of the floating point number to be encoded represents the size of the floating point number to be encoded. The searching unit 112 receives the data bit of the floating point number to be encoded input by the first extraction unit 111. Based on the data bit of each floating point number to be encoded in the vector to be encoded, the maximum floating point number and the minimum floating point number are searched, and the index of the maximum floating point number and the index of the minimum floating point number are obtained respectively.

[0044] Exemplarily, the vector to be encoded can be represented as {x1, x2, …, xq}, where x1, x2, …, xq represent the floating point numbers to be encoded in the vector to be encoded. q} The encoding specification of the logarithmic floating-point format can include: among the q floating-point numbers to be encoded in the vector to be encoded, the non-zero floating-point number with the smallest absolute value is x i , and the non-zero floating-point number with the largest absolute value is x j . Correspondingly, the minimum index and the maximum index can be i and j respectively.

[0045] After obtaining the maximum index j and the minimum index i, the searching unit 112 can output the maximum index and the minimum index to the encoding unit 113. Correspondingly, the encoding unit 113 receives the sign bit and the data bit of the floating-point number to be encoded output by the first extracting unit 111, and receives the maximum index and the minimum index output by the searching unit 112. The encoding unit 113 can determine the maximum logarithmic representation and the minimum logarithmic representation of the vector to be encoded required for encoding based on the maximum index and the minimum index. Wherein, the maximum logarithmic representation max of the vector to be encoded can be denoted as a j = ln |x j |, and the minimum logarithmic representation min of the vector to be encoded can be denoted as a i = ln |x i |.

[0046] Wherein, the specific value of the target bit number n can be preset or externally input. The first constant corresponding to each value of n can be pre-stored in the encoding unit 113, and the first constant corresponding to the specific value of the target bit number n is the first coefficient. Wherein, the specific value of the target bit number n is determined in advance according to the encoding specification of the logarithmic floating-point format, and the specific value of n represents the total number of bits included in each floating-point number in the encoded vector. For example, the specific value of n can be pre-defined in the floating-point number encoding component 11 of the data processing device of each computing node in the distributed system to facilitate encoding.

[0047] The logarithmic floating-point format representation obtained by encoding a single floating-point number to be encoded according to the logarithmic floating-point format is the encoded floating-point number. That is, the encoded floating-point number is the logarithmic floating-point format floating-point number representation obtained by converting the floating-point number to be encoded. The encoded vector includes q encoded floating-point numbers obtained by encoding q floating-point numbers to be encoded respectively. Wherein, a single encoded floating-point number contains n bits. For example, when the specific value of n is 8, a single encoded floating-point number includes 8 bits, and when the specific value of n is 10, a single encoded floating-point number includes 10 bits. The logarithmic floating-point format of a single encoded floating-point number includes 1 bit of sign bit (denoted as S) and n-1 bits of scale bit (i.e. data bit) for reflecting the scale of linear space and logarithmic space.

[0048] In summary, the data processing apparatus applied to the processor provided by the embodiment of the present application comprises a floating-point number encoding component 11 composed of a first extraction unit 111, a lookup unit 112 and an encoding unit 113. The first extraction unit 111 is used to extract the sign bit and the data bit of the floating-point number to be encoded in the vector to be encoded. The lookup unit 112 is used to find the index corresponding to the maximum floating-point number and the index corresponding to the minimum floating-point number in the vector to be encoded respectively based on the data bit of the floating-point number to be encoded, so as to obtain the maximum index and the minimum index. The encoding unit 113 is used to encode the floating-point number to be encoded into the logarithmic floating-point format representation with n bits based on the target bit number n, the sign bit of the floating-point number to be encoded, the data bit of the floating-point number to be encoded, the maximum index and the minimum index, so as to obtain the encoded vector. In this way, the data processing apparatus as hardware supports the encoding operation for the logarithmic floating-point format, and further improves the encoding efficiency to a certain extent, and realizes efficient representation of the floating-point number in the logarithmic floating-point format.

[0049] And one vector to be encoded is equivalent to one data block, and when the LogFMT-nBit format is used for encoding, the maximum logarithmic representation and the minimum logarithmic representation are calculated locally for the vector to be encoded itself, and the encoding is performed based on the maximum logarithmic representation and the minimum logarithmic representation, and the size of the data block can be flexibly set according to actual requirements. In this way, for data blocks of different sizes, the representation range can be flexibly adjusted, so that the encoding processing of data blocks of different sizes can be met in a distributed environment to adapt to different data distribution.

[0050] Optionally, Figure 2 is a structural schematic diagram of the floating-point number encoding component 11 provided by the embodiment of the present application, and the lookup unit 112 comprises a first finder and a second finder. The input ends of the first finder and the second finder are respectively connected with the output ends of the first extraction unit 111, and the output ends of the first finder and the second finder are respectively connected with the input ends of the encoding unit 113.

[0051] The first finder is used to find the index corresponding to the maximum floating-point number in the vector to be encoded based on the data bit of the floating-point number to be encoded, so as to obtain the maximum index.

[0052] The second finder is used to find the index corresponding to the minimum floating-point number in the vector to be encoded based on the data bit of the floating-point number to be encoded, so as to obtain the minimum index.

[0053] The first extraction unit 111 can comprise a first output end and a second output end. The first output end connected with the input end of the first finder is used to input the extracted data bit of each floating-point number to be encoded to the first finder for finding the maximum index. And the second output end connected with the input end of the second finder is used to input the extracted data bit of each floating-point number to be encoded to the second finder for finding the minimum index.

[0054] In the embodiment of the present application, two look-up tables, i.e., the first look-up table and the second look-up table, are respectively arranged, so that the first look-up table and the second look-up table can respectively search the maximum index and the minimum index based on the data bits of all the floating-point numbers output by the first extraction unit 111 in parallel, thereby improving the coding efficiency.

[0055] Optionally, the first look-up table and the second look-up table each include a plurality of comparator layers. The first comparator layer includes q / 2 comparators, where q is the number of floating-point numbers included in the vector to be coded. Two input terminals of each comparator in the first comparator layer respectively receive the data bits of two floating-point numbers to be coded. Two input terminals of each comparator in the next comparator layer are respectively connected to the output terminals of two comparators in the current comparator layer. The next comparator layer is any comparator layer except the first comparator layer in the plurality of comparator layers, and the current comparator layer is the previous comparator layer of the next comparator layer. Correspondingly, the number of comparators in the current comparator layer is twice the number of comparators in the next comparator layer.

[0056] The comparators in the first look-up table are used to output the maximum value in the data bits of the two floating-point numbers to be coded and the index of the maximum value. The comparators in the second look-up table are used to output the minimum value in the data bits of the two floating-point numbers to be coded and the index of the minimum value. The comparators included in the first look-up table are used to select the maximum value in the data bits of the two floating-point numbers to be coded and output the maximum value and the index of the maximum value. The comparators included in the second look-up table are used to select the minimum value in the data bits of the two floating-point numbers to be coded and output the minimum value and the index of the minimum value.

[0057] The first look-up table and the second look-up table can use the format of a reduction tree. Specifically, the input terminals of the first comparator layer of the first look-up table are respectively connected to the first output terminals of the first extraction unit 111, and the input terminals of the first comparator layer include the input terminals of the comparators in the first comparator layer. Taking the number of floating-point numbers included in the vector to be coded as 16, i.e., the number of data bits as 16, for example, q is 16, and the first comparator layer includes 8 comparators. Figure 3 is a structural schematic diagram of a first look-up table provided by the embodiment of the present application, as shown in Figure 3 The comparators in the first comparator layer respectively receive the 16 data bits output by the first extraction unit 111. Figure 3 y1~y 16The first comparator layer is connected to the next comparator layer, and the larger of the two input values and the index of the larger value are outputted as the input of the connected comparator. The index of the two input values is determined by the order of the comparator in the first comparator layer. For example, the index of the input value inputted from the left input end is determined as 2×c-1, and the index of the input value inputted from the right input end is determined as 2×c, where c represents the order of the comparator. For any comparator layer, the order of the comparator is assigned from left to right. For example, the order of the 8 comparators in the first comparator layer is 1, 2, …, 8 from left to right. For the second comparator layer, the order of the 4 comparators is 1, 2, …, 4 from left to right. Alternatively, the index of the data bit of the floating-point number to be encoded can also be inputted to the comparator, which is not limited in the embodiment of the application.

[0058] The two-by-two comparison is performed by the first comparator layer, and the larger of the 8 input values is inputted to the second comparator layer. The two-by-two comparison is performed by the second comparator layer, and the larger of the 4 input values is inputted to the third comparator layer. The two-by-two comparison is performed by the third comparator layer, and the larger of the 2 input values is inputted to the last comparator layer. The two-by-two comparison is performed by the last comparator layer, and the larger of the 1 input value is outputted as the maximum value and the index. In this way, the maximum value and the index of the 16 data bits are outputted by the comparator in the last comparator layer, and the maximum index is obtained. Similarly, the structure of the second lookup device is the same as that of the first lookup device, and the smaller of the two input values and the index of the smaller value are outputted by the comparator in the second lookup device. In this way, the minimum value and the index of the 16 data bits are outputted by the comparator in the last comparator layer, and the minimum index is obtained.

[0059] In the embodiment of the application, the structure of the lookup device is composed of multiple comparator layers, and the maximum index and the minimum index are obtained by multi-stage comparison, which is low in cost.

[0060] Optionally, the comparator included in the first finder is further configured to take the other input data bit as the maximum value if the one input data bit includes 0. The comparator included in the second finder is further configured to take the other input data bit as the minimum value if the one input data bit includes 0. That is, the actual comparison process only involves non-0 number operation, and if one input number of the comparator is 0, the other number and its index are directly output. In this way, it is ensured that the maximum index can accurately represent the non-zero floating point number with the largest absolute value in the to-be-encoded vector, and it is ensured that the minimum index can accurately represent the non-zero floating point number with the smallest absolute value in the to-be-encoded vector.

[0061] The floating point number encoding component 11 described above further includes a first input unit and a normalization unit. The normalization unit is connected with the first input unit and the first extraction unit 111 respectively. The first input unit is configured to truncate each to-be-encoded floating point number to a preset length, and the normalization unit is configured to normalize each to-be-encoded floating point number output by the truncation unit.

[0062] The preset length can be set as needed. For example, in an application scenario, the to-be-encoded floating point number can be in FP32 format, i.e., a single-precision floating point number with a length of 32 bits. Correspondingly, the preset length can be 16 bits, i.e., the first input unit takes the upper 16 bits of the to-be-encoded floating point number to obtain the truncated to-be-encoded floating point number. In this way, the truncation is equivalent to truncating it into BF16 format data, i.e., a brain floating point number with a length of 16 bits. The floating point number encoding component 11 in the embodiment receives data in FP32 format, encodes it after truncating it into BF16 format data, so as to meet the format requirement of the to-be-encoded floating point number input by the first extraction unit 111. It should be noted that the first input unit can include q parallel truncation sub-units to truncate q to-be-encoded floating point numbers in parallel. The normalization unit can include q parallel normalization sub-units to normalize q to-be-encoded floating point numbers in parallel.

[0063] The value of the mantissa of the normalized floating-point number is greater than or equal to 1 and less than 2. Since the floating-point number to be encoded may be a non-normalized floating-point number after truncation, the normalization unit can be used to perform normalization processing. The specific process of converting the non-normalized floating-point number into the normalized floating-point number can be: extending the bit width of the exponent of the floating-point number to be encoded by one bit. If the floating-point number to be encoded is a normalized floating-point number itself, 0 is directly filled in the high bit of the exponent of the floating-point number to be encoded to perform normalization. If the floating-point number to be encoded is a non-normalized floating-point number itself, the number of leading 0s in the mantissa of the floating-point number to be encoded is first counted, denoted as Num. For example, the number of continuous 0s from the highest bit in the binary representation of the mantissa is counted until the first 1 appears, and the counted number is Num. Then, the shift length is determined as Num+1, and the mantissa of the floating-point number to be encoded is left shifted by Num+1 bits, so that the most significant bit of the mantissa is 1, to obtain a normalized mantissa. The complement representation of -Num is used as a normalized exponent, and then a normalized floating-point number is obtained, to perform normalization.

[0064] The normalization unit can input the normalized floating-point numbers to the first extraction subunit. The q parallel extraction subunits in the first extraction unit 111 perform parallel extraction on the q floating-point numbers to be encoded, to obtain the sign bit, the exponent, and the mantissa of each floating-point number to be encoded.

[0065] In the embodiment of the present application, normalization processing is performed first, and then the occupation of the representation space by non-normalized numbers is reduced. For example, for FP8 in the non-logarithmic floating-point format, 6.25% of the representation space is occupied by non-normalized numbers. In the embodiment of the present application, normalization processing is performed to reduce the occupation of the representation space. At the same time, additional hardware logic is needed when performing operations on non-normalized numbers, and in the embodiment of the present application, the normalization unit is arranged to perform normalization processing, thereby reducing hardware overhead and further reducing hardware area and power consumption.

[0066] Optionally, the data bits of the floating-point number to be encoded include the exponent and the mantissa. Referring to Figure 2 The encoding unit 113 includes a logarithm subunit 1131, a first splicing subunit 1132, an encoding subunit 1133, and a second splicing subunit 1134. The logarithm subunit 1131 is connected with the first extraction unit 111 and the first splicing subunit 1132, the first splicing subunit 1132 is connected with the first extraction unit 111, the encoding subunit 1133 is connected with the second splicing subunit 1134, the lookup unit 112, and the first splicing subunit 1132, and the second splicing subunit 1134 is connected with the first extraction unit 111.

[0067] The logarithmic subunit 1131 is used to perform logarithmic operations on the mantissa of the floating-point number to be encoded, generating a logarithmic result. Specifically, the first extraction unit 111 outputs the mantissa of each floating-point number to be encoded to the logarithmic subunit 1131 through its third output terminal. Correspondingly, the logarithmic subunit 1131 receives the mantissa of each floating-point number to be encoded output by the first extraction unit 111 and outputs the logarithmic result of each floating-point number to the first concatenation subunit 1132.

[0068] The first concatenation subunit 1132 is used to concatenate the exponents of the floating-point numbers to be encoded and the logarithmic operation results to obtain the logarithmic representation of the floating-point numbers to be encoded. Specifically, the first extraction unit 111 outputs the exponents of each floating-point number to be encoded to the first concatenation subunit 1132 through its fourth output terminal, and the logarithmic subunit 1131 outputs the logarithmic operation results of each floating-point number to be encoded to the first concatenation subunit 1132. Correspondingly, the first concatenation subunit 1132 receives the exponents of each floating-point number to be encoded output by the first extraction unit 111, receives the logarithmic operation results of each floating-point number to be encoded output by the logarithmic subunit 1131, and outputs the concatenated logarithmic representation of the floating-point number to be encoded to the encoding subunit 1133.

[0069] The encoding subunit 1133 is used to determine the maximum logarithmic representation and the minimum logarithmic representation based on the logarithmic representation, the maximum index, and the minimum index of the floating-point number to be encoded, and to generate the target data bits corresponding to the floating-point number to be encoded based on the target number of bits n. Specifically, the target data bits corresponding to the floating-point number to be encoded can be generated based on the first coefficient corresponding to the target number of bits n, the logarithmic representation, the maximum logarithmic representation, and the minimum logarithmic representation of the floating-point number to be encoded, and then output to the second concatenation subunit 1134.

[0070] The second concatenation subunit 1134 is used to concatenate the sign bit of the floating-point number to be encoded with the target data bits to obtain a logarithmic floating-point format representation. Specifically, the first extraction unit 111 outputs the sign bit of each floating-point number to be encoded to the second concatenation subunit 1134 through its fifth output terminal. Correspondingly, the second concatenation subunit 1134 receives the sign bit of each floating-point number to be encoded output by the first extraction unit 111 and the target data bits output by the encoding subunit 1133, and concatenates them to obtain the logarithmic floating-point format representation.

[0071] The search operation performed by the search unit 112 can be performed in parallel with the operations performed by the logarithmic subunit 1131 and the first splicing subunit 1132.

[0072] The logarithm sub-unit 1131 can include q parallel logarithm operation circuits, and the logarithm operation circuits are used to perform logarithm operation on the mantissas of the q floating-point numbers to be encoded in parallel. The first splicing sub-unit 1132 can include q parallel first splicing circuits, and the first splicing circuits are used to splice the exponents of the q floating-point numbers to be encoded and the logarithm operation results in parallel. The second splicing sub-unit 1134 can include q parallel second splicing circuits, and the second splicing circuits are used to splice the sign bits of the q floating-point numbers to be encoded and the target data bits in parallel. The encoding sub-unit 1133 can calculate the target data bits corresponding to the q floating-point numbers to be encoded in parallel. In this way, the overall encoding efficiency of the vector to be encoded can be improved through parallel processing.

[0073] Specifically, the logarithm sub-unit 1131 receives the mantissas of the floating-point numbers to be encoded output by the first extraction sub-unit 111. Each logarithm operation circuit in the logarithm sub-unit 1131 includes a first iteration operation circuit and a first multiplication operation circuit. The first iteration operation circuit is used to select a corresponding iteration round as a first target iteration round based on the precision of the mantissa of the floating-point number to be encoded. Iteration is performed based on the mantissa of the floating-point number to be encoded until the first target iteration round is reached. The first multiplication operation circuit is used to calculate the product of the iteration result of the first iteration operation circuit and a preset constant corresponding to the first target iteration round, and obtain the logarithm operation result corresponding to the floating-point number to be encoded according to the product.

[0074] In the embodiment of the application, the process of calculating the natural logarithm by the logarithm sub-unit 1131 can adopt the CORDIC algorithm, and the first iteration operation circuit is an implementation circuit for implementing the CORDIC algorithm. The specific implementation method and hardware structure of the CORDIC algorithm can refer to the existing method. The iteration rounds corresponding to the precisions of different mantissas can be pre-set in the control node of the distributed system. For example, the mantissa precisions of FP8 and BF16 can be represented by a maximum of 5 rounds and 13 rounds respectively, that is, the first target iteration round can be 5 when the floating-point number to be encoded is FP8, and the first target iteration round can be 13 when the floating-point number to be encoded is BF16.

[0075] Let b represent the mantissa of the floating-point number to be encoded after normalization processing, and the logarithm sub-unit 1131 is used to calculate ln(b). The logarithm sub-unit 1131 can include a first variable register, a second variable register and a third variable register, and the first variable, the second variable and the third variable are respectively represented by u m , v m , z mLet m represent the current iteration number. When performing logarithmic operations, m ∈ [1, L], and L represents the first target iteration round. Based on the rules of the CORDIC algorithm, the initial values ​​of the first, second, and third variables can be assigned to b+1, b-1, and 0, respectively. Specifically, the initial values ​​of the first variable register, the second variable register, and the third variable register are in binary form.

[0076] The first variable register (denoted as register u) m ), the second variable register (denoted as register v) m ) and the third variable register (denoted as register z) m The value of z is used as the input to the CORDIC algorithm to obtain the third variable output. The third variable output is the result of L iterations (for example, z can be recorded). L ), where z, during the iteration process m The change gradually decreases, and the iteration stops when the change decreases to the point where the mantissa precision of the floating-point number to be encoded cannot be represented. Since the number of iterations in the CORDIC algorithm is fixed, and the number of iteration rounds corresponding to different mantissa precisions is fixed, the process of performing logarithmic operations using the CORDIC algorithm adopts a fully pipelining design. Each step of the pipeline represents one CORDIC algorithm iteration, and the intermediate results of the algorithm iteration are passed forward through registers along the pipeline.

[0077] Figure 4 This is a schematic diagram of an algorithm implementation circuit in existing technology, such as... Figure 4 As shown, the implementation circuit of the CORDIC algorithm consists of multiple registers, shifters, and adders. In each iteration, the iteration method is as follows:

[0078] u m+1 =u m +σ m ×2 -m ×v m

[0079] v m+1 =v m +σ m ×2 -m ×u m

[0080] z m+1 =z m -σ m ×tan -1 (2) -m )

[0081] Where, σ m Indicate z m The symbol, zm The specific value represents the value of the third variable register in the m-th iteration. This iteration process is actually the vector (u) in the XOY coordinate system. m v m The process of rotating around the origin involves each iteration with a rotation α = tan -1 (2) -m The angle is defined as follows: The higher the precision of the mantissa, the more iterations are required. The first target iteration round can represent the maximum number of rotations α that the precision of the mantissa can represent.

[0082] Among them, with register u m The connected shifter is used to perform multiplication, so that the adder can calculate v. m Add (±2) -m ×u m ), of which 2 -m ×u m The specific positive or negative value is determined by the value input to the sign bit extractor. For binary, multiplying by 2... -m Multiplication is essentially a bitwise shift operation. For example, for the binary number 1011 multiplied by 0.1, 0.1 represents 2 in binary. -1 The result obtained is 101.1, which can be achieved using a shifter.

[0083] Furthermore, with register v m The connected shifter is used to perform multiplication, so that the adder can calculate u. m Add (±2) -m ×v m ), of which 2 -m ×v m The specific positive or negative value is determined by the value input to the sign bit extractor. (This is related to register z.) m The connected adder is used to read register z respectively m tan -1 (2) -m Add the value of the sign bit extractor to calculate z. m With -σ m ×tan -1 (2) -m ) and.

[0084] This iterative process can be viewed as a vector (u) m v m The process of iteratively rotating around the origin to approximate the target angle can be represented by cosφ in each iteration (where φ represents the angle of rotation in each iteration, and cosφ is a constant). When performing logarithmic operations using the CORDIC algorithm, the third variable z after L iterations... Lwhich can be approximately expressed as: ln(b) x first target constant / 2. Wherein, the first target constant can be expressed as .

[0085] The first iteration operation circuit outputs the value of the last pipeline register z L (i.e. the third variable obtained in the last beat) as an iteration result to the first multiplication operation circuit. The first multiplication operation circuit is configured to calculate the product of the iteration result and the preset constant corresponding to the first target iteration round, to obtain a logarithm operation result. Wherein, the preset constant corresponding to the first target iteration round is: 2 The preset constant corresponding to the first target iteration round has different values in the case of different values of L. Wherein, the first multiplication operation circuit can store the preset constant corresponding to the first target iteration round obtained in different iteration rounds (i.e. when L takes different values), so as to facilitate multiplication and addition operation using the corresponding preset constant corresponding to the first target iteration round in calculation.

[0086] It should be noted that since the direct result obtained by the CORDIC algorithm is the natural logarithm with the natural constant e as the base, and the base of the exponent in the case of the IEEE floating point format of the floating point number to be encoded is 2. Therefore, the first multiplication operation circuit is further configured to multiply the product of the iteration result and the preset constant corresponding to the first target iteration round by the constant for conversion: log2e, to realize the unification of the base, and obtain the final logarithm operation result.

[0087] In the embodiment of the present application, the logarithm subunit 1131 can realize the logarithm operation through the first iteration operation circuit and the first multiplication operation circuit, realize the support of the processor for the logarithm operation, and further ensure the efficiency of the exponential operation. Of course, in specific implementation, other ways can also be used to realize the logarithm operation, for example, the logarithm operation result can be directly obtained by using the lookup table (LUT) by pre-computing and storing the logarithm values corresponding to different input values, and the embodiment of the present application does not limit this.

[0088] Further, each first splicing circuit in the first splicing subunit 1132 splices the exponent of a floating point number to be encoded and the logarithm operation result of the floating point number to be encoded, to obtain the logarithmic representation of the floating point number to be encoded. Wherein, the logarithmic representation of the floating point number to be encoded is a fixed-point decimal. Exemplarily, for the above-mentioned to-be-encoded vector {x1, x2, …, x q}, the logarithmic representations of the q floating point numbers to be encoded can be denoted as {a1, a2, …, a q}. The encoding subunit 1133 can encode based on the logarithmic representations of the q floating point numbers to be encoded according to the encoding specification of the logarithmic floating point format, to obtain the target data bits {K1, K2, …, K qIn this process, encoding subunit 1133 directly sets the target data bits of the floating-point number to be encoded to all zeros when the logarithmic representation of the floating-point number to be encoded is 0 (i.e., both the exponent and mantissa of the floating-point number to be encoded are 0, and the resulting fixed-point fractional part is also 0). The number of bits in one target data bit is the specific value of the target number of bits n minus 1. For example, when n=8, one target data bit consists of 7 bits; when n=10, one target data bit consists of 9 bits.

[0089] When the second concatenation subunit 1134 performs concatenation, if the target data bit of the floating-point number to be encoded is 0, the sign bit is set to the default value of 0, and thus 0.0…0 is obtained. That is, ±0 is encoded as n zeros (0.000…0). Since the representation of -0 and +0 does not make a significant difference for deep learning, and may even increase the programming difficulty, in this embodiment of the invention, when the target data bit is 0, the sign bit is defaulted to 0, that is, -0 and +0 are uniformly encoded as all zeros, thereby reducing the programming difficulty.

[0090] If the target data bit of the floating-point number to be encoded is not 0, then the sign bit of the floating-point number to be encoded is concatenated with the target data bit. For example, assume the target data bit of the floating-point number to be encoded is K. k Then, after splicing, we get S. k .K k .

[0091] It should be noted that the encoding standard for logarithmic floating-point format also includes: x i Encoded as S i .000…1, x j Encoded as S j 0.111…1. Correspondingly, if the index of the floating-point number to be encoded is the same as the maximum or minimum index, then if it is the same as the minimum index, 000…1 is directly output. That is, the data bits consisting of n-2 zeros and the least significant 1 are output as the target data bits corresponding to the floating-point number to be encoded. If it is the same as the maximum index, 11…1 is directly output. That is, the data bits consisting of n-1 ones are output as the target data bits corresponding to the floating-point number to be encoded.

[0092] In this embodiment of the invention, by setting a logarithmic subunit 1131, a first splicing subunit 1132, an encoding subunit 1133, and a second splicing subunit 1134 in the encoding unit 113 for sequential processing, the logarithmic floating-point format representation of the floating-point number to be encoded can be obtained, ensuring hardware support for floating-point number encoding.

[0093] In the embodiment of the present application, when the target bit number n is set to different specific values, the first iteration number and the first coefficient are selected based on the specific value of n, so that the floating point numbers in the corresponding bit number logarithmic floating point format can be generated, and the encoding of the floating point types with multiple precisions such as LogFMT-8Bit and LogFMT-10Bit can be implemented.

[0094] Optionally, the data processing apparatus in the embodiment of the present application further comprises a sending component, configured to send the encoded vector and the first decoding information of the encoded vector; wherein the first decoding information comprises the target bit number n, the maximum logarithmic representation and the minimum logarithmic representation of the to-be-encoded vector.

[0095] The first decoding information is the information required for decoding the encoded vector, and the first decoding information comprises the information used in the process of obtaining the encoded vector by encoding. The input end of the sending component is connected with the output end of the second splicing sub-unit 1134, so as to obtain the encoded vector obtained by the second splicing sub-unit 1134. The encoded vector can be {S1.K1, S2.K2, …, S q .K q}, assuming that n is 10, the first decoding information can comprise n=10, a i and a j , wherein n=10 included in the first decoding information can be the Size signal.

[0096] The output end of the sending component can be connected with the input end of other data processing apparatuses in the distributed system, so as to output the encoded vector and the first decoding information of the encoded vector to other data processing apparatuses in the distributed system. For example, the encoded vector and the first decoding information of the encoded vector can be sent based on the Internet Protocol (IP) address of the computing node where the other data processing apparatus is located. It should be noted that in another implementation manner, the sending component can not be provided, and the sending can be performed by the sending sub-unit in the encoding unit 113. The sending sub-unit is connected with the second splicing sub-unit 1134.

[0097] In the embodiment of the present application, the communication is performed between the data processing apparatuses, so that the encoded vector in the logarithmic floating point format is used to transfer the data, the data transmission bit width of the logarithmic floating point format is small, and the communication pressure can be relieved to a certain extent. Meanwhile, the first decoding information required for decoding is sent, so that the other data processing apparatuses can conveniently decode the encoded vector.

[0098] Optionally, as Figure 5As shown, the encoding subunit 1133 includes a first selector, a first adder, a second adder, a first divider, a first multiplier, a second selector, a third adder, and a rounding circuit. The first selector is connected to both the first and second adders; the first and second adders are connected to the first divider; the first divider is connected to the first multiplier; the first multiplier is connected to both the second selector and the third adder; and the third adder is connected to the rounding circuit.

[0099] In this embodiment of the invention, the encoding subunit 1133 can be formed by several devices, including a selector, an adder, a multiplier, and a rounding circuit, resulting in low hardware cost.

[0100] The first selector is used to determine the maximum and minimum logarithmic representations. Specifically, the first selector receives the maximum index j and minimum index i output by the lookup unit 112, and the logarithmic representations of each floating-point number to be encoded output by the first concatenation subunit 1132. Based on the maximum index j and minimum index i, it selects from all logarithmic representations {a1, a2, ..., a...} q In}, select a i and a j The minimum logarithmic representation and the maximum logarithmic representation are obtained, and 'a' is output to the first adder. i and a j Output a to the second adder j .

[0101] The control signals for both the first and second adders are subtraction operation signals. The first adder is used to calculate the first difference between the maximum logarithmic representation and the minimum logarithmic representation. This first difference can be a. j -a i The first adder will output the calculated first difference to the first divider.

[0102] The second adder is used to calculate the difference between the logarithmic representation and the minimum logarithmic representation of the floating-point number to be encoded, thereby obtaining the second difference value corresponding to the floating-point number to be encoded. Specifically, this second adder can calculate the difference between the logarithmic representation and the minimum logarithmic representation of each floating-point number output by the first concatenation subunit 1132 in parallel, to obtain q second difference values ​​in parallel. For example, the second difference value can be represented as: a k -a i Where 1≤k≤q, the second adder will output the calculated second difference values ​​to the first divider respectively.

[0103] The control signal is a signal in the input signal received by the adder in the circuit, which is used to control the adder to perform addition or subtraction operation, in the embodiment of the application, by inputting the subtraction operation signal as the control information for the first adder and the second adder, the adder can perform the subtraction operation, thereby saving the hardware resources.

[0104] The first divider is configured to perform division operation on the second difference and the first difference corresponding to the floating point number to be encoded output by the first adder and the second adder, and obtain a first division result corresponding to the floating point number to be encoded. Specifically, the first divider can perform the division operation on the second difference and the first difference corresponding to the q floating point numbers to be encoded respectively, and obtain q first division results. The first division result can be represented as (a k -a i ) / (a j -a i ), and the first divider outputs the calculated first division result to the first multiplier.

[0105] The second selector is configured to output a first constant corresponding to the target bit number n as a first coefficient to the first multiplier. Specifically, the second selector can select the first constant corresponding to the target bit number n as the first coefficient according to the specific value of the target bit number n carried in the control signal, and output the first constant to the first multiplier. According to the encoding specification of the logarithmic floating point format, the first constant corresponding to the target bit number n can be represented as (2 n-1 -2), and the specific value of the first constant is different when the target bit number n adopts different specific values. The second selector can define the first constant corresponding to the target bit number n when the target bit number n adopts different specific values in advance. The second selector can find the first constant corresponding to the target bit number n based on the specific value of the target bit number n input.

[0106] The first multiplier is configured to calculate the product of the first coefficient output by the second selector and the first division result corresponding to the floating point number to be encoded output by the first divider, and obtain a first product corresponding to the floating point number to be encoded. Specifically, the first multiplier can calculate the product of the first coefficient and the first division result corresponding to the q floating point numbers to be encoded respectively, and output q first products to the third adder. The first product can be represented as (a k -a i ) / (a j -a i )×(2 n-1 -2).

[0107] The third adder calculates the sum of the first product and the second constant corresponding to the floating-point number to be encoded output by the first multiplier, obtaining the first sum value corresponding to the floating-point number to be encoded. The second constant is a fixed value pre-set according to the encoding standard of logarithmic floating-point format, and is 1. The third adder can calculate the sum of the first product and the second constant corresponding to q floating-point numbers to be encoded in parallel, obtaining q first sum values ​​and outputting them to the rounding circuit. The first sum value can be represented as (a... k -a i ) / (a j -a i )×(2 n-1 -2)+1.

[0108] The rounding circuit is used to round the first sum of the floating-point numbers to be encoded, obtaining the logarithmic floating-point representation of the floating-point numbers. The rounding circuit can use a rounding method (rounding to the nearest whole number). The rounding circuit can perform rounding operations on the first sum of q floating-point numbers to be encoded in parallel, obtaining q logarithmic floating-point representations. The final K... k This can be represented as: RoundToInt((a k -a i ) / (a j -a i )×(2 n-1 -2)+1).

[0109] Here, RoundToInt indicates that the result is rounded to an integer, and the rounding method conforms to the IEEE standard. (a) j -a i ) / (2 n-1 -2) can be denoted as the increment Step, which is used to represent the quantization interval between two adjacent floating-point numbers in the logarithmic space.

[0110] Thus, when the encoding subunit 1133, composed of the first selector, first adder, second adder, first divider, first multiplier, second selector, third adder, and rounding circuit provided in this embodiment of the invention, performs encoding, it conforms to the encoding specification of logarithmic floating-point format: for non-zero x... k Encode it as S k .K k , where K k For K k The rounding result of ', K k '=(a k -a i ) / (a j -a i )×(2 n-1 -2)+1,K k, which is the first sum value, and further ensures that the logarithmic floating-point format representation can be accurately encoded.

[0111] Optionally, the floating-point number encoding component 11 further comprises an exception output unit; the exception output unit is configured to output an exception signal in a case where the to-be-encoded vector hits a preset special type; and the special type includes infinity, non-number, and a difference between a mantissa of a maximum floating-point number and a mantissa of a minimum floating-point number being greater than a preset difference threshold.

[0112] The floating-point number encoding component 11 further comprises a first generation unit, which is configured to generate a default form of logarithmic floating-point format representation in a case where the to-be-encoded floating-point number is infinity or non-number.

[0113] The special type includes infinity and non-number in the to-be-encoded vector. The first generation unit is connected with the first extraction unit 111, and the first extraction unit 111 is further configured to send, to the first generation unit, the to-be-encoded floating-point number that is infinity or non-number in the to-be-encoded vector. Correspondingly, the first generation unit is configured to generate the default form of logarithmic floating-point format representation for the received to-be-encoded floating-point number. The default form of logarithmic floating-point format representation can be represented as 1.000…0. That is, the first generation unit uniformly encodes the to-be-encoded IEEE754 floating-point number type sNaN, qNaN, and infinity as 1.000…000.

[0114] It should be noted that, in a case where the to-be-encoded vector includes infinity and non-number, the encoding subunit 1133 can perform parallel processing on the to-be-encoded floating-point numbers other than infinity and non-number. Since non-number and infinity have relatively low importance in deep learning training and reasoning, in the embodiment of the present application, the non-number and infinity are uniformly encoded as the default form by the first generation unit, so that the data format is more suitable for the characteristics of deep learning, and the occupation of the representation space by non-number and infinity is reduced. For example, for FP8 in the non-logarithmic floating-point format, 6.25% of the representation space is occupied by non-number and infinity. In the embodiment of the present application, the non-number and infinity are uniformly represented, which can further reduce the occupation of the representation space, and further reduce hardware overhead, hardware area, and power consumption.

[0115] Specifically, the exception output unit is connected with the first extraction unit 111. The first extraction unit 111 outputs a first identification to the exception output unit through a sixth output end in the case that the processed vector to be encoded includes infinity. The exception output unit outputs a first exception signal in response to the first identification. The first exception signal can be OVERFLOW exception, which can be displayed in the user interface. That is, the OVERFLOW exception is triggered in the embodiment of the present application when the encoded floating-point number vector contains infinity. The first extraction unit 111 outputs a second identification to the exception output unit in the case that the processed vector to be encoded includes non-number. The exception output unit outputs a second exception signal in response to the second identification, which can be displayed in the user interface. The first identification and the second identification can be set as needed, for example, the first identification and the second identification can be 1 and 0 respectively. The second exception signal can be INVALID exception. That is, the INVALID exception is triggered in the embodiment of the present application when the encoded floating-point number vector contains non-number.

[0116] The exception output unit is also connected with the lookup unit 112, and the lookup unit 112 is also used to input the maximum index and the minimum index to the exception output unit. The first extraction unit 111 also inputs each floating-point number to be encoded to the exception output unit through a seventh output end. The exception output unit is also used to find the maximum floating-point number (i.e., the above x j ) and the minimum floating-point number (i.e., the above x i ) from the floating-point numbers to be encoded based on the maximum index and the minimum index; and outputs a third exception signal in the case that the exponents of x j and x i are greater than a preset difference threshold value. That is, the preset special type of the vector to be encoded also includes that the exponents of x j and x i are greater than the preset difference threshold value.

[0117] The preset difference threshold value can be 2 d , where d represents the exponent width of the floating-point number to be encoded. The third exception signal can be UNDERFLOW exception, which can be displayed in the user interface. That is, the UNDERFLOW exception is triggered in the embodiment of the present application when the exponents of x j and x i are greater than the preset difference threshold value. The OVERFLOW exception, the INVALID exception and the UNDERFLOW exception can be collectively referred to as VZOUI exception.

[0118] Figure 6 is a schematic diagram of an encoding process provided by the embodiment of the present application, as Figure 6As shown, the first input unit can first receive the to-be-encoded vector, and the normalization unit can normalize the to-be-encoded floating-point number. Then, the first extraction unit 111 can extract the sign bit, the exponent, and the mantissa of the to-be-encoded floating-point number. The lookup unit 112 can look up the maximum index and the minimum index based on the data bits composed of the exponent and the mantissa. The encoding unit 113 can perform logarithmic operation on the mantissa of the to-be-encoded floating-point number to obtain the logarithmic representation of the to-be-encoded floating-point number. Finally, the encoded vector is generated based on the first coefficient, the logarithmic representation of the to-be-encoded floating-point number, the sign bit, the maximum index, and the minimum index. During the processing, if the to-be-encoded vector includes an infinite or non-number, the exception output unit can output a first exception signal and a second exception signal, respectively. If the difference between the exponents of the maximum floating-point number and the minimum floating-point number is greater than a preset difference threshold, the exception output unit can output a third exception signal.

[0119] In the embodiment of the present application, the abnormal alarm mechanism is set, and the exception signal is output in the case where the to-be-encoded floating-point number in the to-be-encoded vector is infinite or non-number, so that subsequent adjustment can be facilitated.

[0120] Optionally, Figure 7 is another frame diagram of a data processing apparatus provided by the embodiment of the present application, and the data processing apparatus in the embodiment of the present application further includes a floating-point decoding component 12 composed of a second extraction unit 121, a restoration unit 122, and a decoding unit 123. The second extraction unit 121 is connected with the restoration unit 122 and the decoding unit 123, and the restoration unit 122 is connected with the decoding unit 123.

[0121] The second extraction unit 121 is configured to extract the sign bit and the data bit of the to-be-decoded floating-point number in the to-be-decoded vector.

[0122] The restoration unit 122 is configured to restore the logarithmic representation corresponding to the to-be-decoded floating-point number based on the second decoding information of the to-be-decoded vector and the data bit of the to-be-decoded floating-point number.

[0123] The decoding unit 123 is configured to generate the non-logarithmic floating-point format representation of the to-be-decoded floating-point number based on the sign bit of the to-be-decoded floating-point number and the logarithmic representation corresponding to the to-be-decoded floating-point number, obtain the decoded vector, and input the decoded vector to the data operation component 14 for operation.

[0124] The floating point number decoding component 12 is designed in full pipelining, and the floating point numbers included in the to-be-decoded vector are the to-be-decoded floating point numbers. The to-be-decoded vector can include p floating point numbers, that is, the to-be-decoded vector includes p to-be-decoded floating point numbers. The to-be-decoded vector can be input from outside through the data input component 13, or be generated by other components in the data processing device, and the embodiments of the present application do not limit this. Exemplarily, the to-be-decoded vector can be an encoded vector sent by another data processing device. The p can be the same as the q. The to-be-decoded floating point number can be a floating point number in the LogFMT-nBit format. The second extraction unit 121 extracts the sign bit of each to-be-decoded floating point number, and takes the part of the to-be-decoded floating point number except the sign bit as the data bit of the to-be-decoded floating point number, outputs the data bit of each to-be-decoded floating point number extracted to the restoring unit 122, and outputs the sign bit of each to-be-decoded floating point number extracted to the decoding unit 123.

[0125] In the embodiments of the present application, the floating point number decoding component 12 is arranged, so that the data processing device can decode the obtained floating point number vector in the LogFMT-nBit format. In this way, when performing internal calculation, the floating point number in the non-logarithmic floating point format can be used for operation, and the flexibility of the scheme is further improved.

[0126] It should be noted that a plurality of floating point number decoding components 12 and a plurality of floating point number encoding components 11 can be arranged in the data processing device, so as to process a plurality of to-be-decoded vectors and a plurality of to-be-encoded vectors respectively in parallel.

[0127] Further, the second extraction unit 121 is further configured to pad the data bit of each to-be-decoded floating point number to a specified bit number. The specified bit number can be pre-set, and the specified bit number is the maximum value of the target bit number n supported by the floating point number decoding component 12 (hereinafter referred to as n max max) minus 1. Specifically, assuming that the floating point number decoding component 12 supports two formats with different values of n, that is, LogFMT-8Bit and LogFMT-10Bit, the n max supported by the floating point number decoding component 12 can be 10. Accordingly, the specified bit number can be 9. The second extraction unit 121 can first extract the data bit {K1, K2, …, K p} of each of the p to-be-decoded floating point numbers, and the data bit {S1, S2, …, S p} of each of the p to-be-decoded floating point numbers. In the case where n is 8, K1, K2, …, K p are all 7 bits, the high bits of K1, K2, …, K p can be padded with 0 (two bits of 0 are padded in this example) respectively, so as to expand K1, K2, …, K p to n maxSubtract 1 bit. In the case that the target bit number n is 10, K1, K2, …, K p are all 9 bits, no padding is needed. In the embodiment of the present application, padding is used to enable the data processing apparatus to decode a to-be-decoded vector with different values as the target bit number n, for example, to enable the data processing apparatus provided by the embodiment of the present application to support decoding of multiple precision floating point types such as LogFMT-8Bit, LogFMT-10Bit, etc.

[0128] Optionally, Figure 8 is a circuit schematic diagram of a reduction unit 122 provided by the embodiment of the present application, as shown in Figure 8 The reduction unit 122 includes a fourth adder, a fifth adder, a second multiplier, a third selector, a third multiplier, and a sixth adder. The fourth adder and the fifth adder are connected to the second multiplier, the second multiplier and the third selector are connected to the third multiplier, and the third multiplier is connected to the sixth adder. In the embodiment of the present application, the reduction unit 122 can be composed of adders, multipliers, and selectors, which have a lower hardware cost.

[0129] The fourth adder is configured to calculate a sum of the data bits of the to-be-decoded floating point number output by the second extraction unit 121 and a third constant, to obtain a second sum value corresponding to the to-be-decoded floating point number. The reduction unit 122 is a SIMD unit, and can perform the process of determining the logarithmic representation corresponding to each to-be-decoded floating point number in parallel. Specifically, the data bits of the to-be-decoded floating point number can also be referred to as a numerical part, the control signal of the fourth adder is an addition operation signal, and the third constant can be pre-set based on an encoding specification. For example, the third constant is -1. The fourth adder can be input with the data bits of each to-be-decoded floating point number and the third constant. The fourth adder can include p parallel addition operation circuits, which can respectively calculate the sum of the data bits of p to-be-decoded floating point numbers and the third constant to obtain p second sum values. For example, K k represents the data bits of the kth to-be-decoded floating point number, 1≤k≤p, and the second sum value can be represented as K k -1. The fourth adder inputs each second sum value calculated to the second multiplier.

[0130] The control signal of the fifth adder is a subtraction operation signal, and the fifth adder is configured to calculate a third difference value between the maximum logarithmic representation and the minimum logarithmic representation in the second decoding information of the to-be-decoded vector. For example, the third difference value can be a j -a i The fifth adder inputs the calculated third difference value to the second multiplier. In the embodiment of the present application, the fifth adder is input with the subtraction operation signal as the control information, so that the addition operation can be realized by the adder, thereby saving hardware resources.

[0131] The second multiplier is configured to calculate a product of the second sum corresponding to the to-be-decoded floating-point number and the third difference corresponding to the third difference value output by the fifth adder, to obtain a second product corresponding to the to-be-decoded floating-point number. Specifically, the second multiplier can be a two-bit Booth multiplier. The second multiplier can include p parallel multiplication operation circuits, which can respectively and in parallel calculate products of the second sum corresponding to the data bits of the p to-be-decoded floating-point numbers and the third difference, to obtain the second products corresponding to the data bits of the p to-be-decoded floating-point numbers. The second product can be expressed as (K k -1) x (a j -a i ).

[0132] The third selector is configured to select a second coefficient corresponding to the target bit number n in the second decoding information and output the second coefficient to the third multiplier. It should be noted that, in a case where the second extraction unit 121 is configured to perform the operation of padding the data bits of each to-be-decoded floating-point number to a specified bit number, a fourth constant corresponding to a maximum value of n can be selected as the second coefficient and output to the third multiplier. In a case where the second extraction unit 121 is not configured to perform the operation of padding the data bits of each to-be-decoded floating-point number to a specified bit number, a fourth constant corresponding to an actual value of n included in the second decoding information can be selected as the second coefficient and output to the third multiplier.

[0133] According to the encoding specification of the logarithmic floating-point format, the fourth constant corresponding to n can be expressed as 1 / (2 n-1 -2), and the fourth constant has different specific values when n has different specific values. The third selector can define the fourth constant corresponding to n having different values in advance. The third selector can find the fourth constant corresponding to the specific value of n based on the input specific value of n. For example, for a component supporting two values of 8 and 10, the fourth constant corresponding to each of the two values of 8 and 10 can be set in advance: 1 / (2 7 -2) and 1 / (2 9 -2). The specific value of n in the second decoding information is used as a control signal of the third selector to select the corresponding value, so as to convert the division operation into a multiplication operation and process the multiplication operation by using the subsequent third multiplier. Of course, in a case where the second extraction unit 121 is configured to perform the operation of padding the data bits of each to-be-decoded floating-point number to a specified bit number, the maximum value of n can be directly selected as the control information to select 1 / (2 9 -2) as the second coefficient.

[0134] The third multiplier calculates the product of the second product (output of the second multiplier) and the second coefficient (output of the third selector) of the floating-point number to be decoded, thus obtaining the third product of the floating-point number. Specifically, the third multiplier can be a two-bit Booth multiplier, which may include p parallel multiplication circuits. These p parallel multiplication circuits can calculate in parallel the product of the second product and the second coefficient for each of the data bits of the p floating-point numbers to be decoded, thus obtaining the third product for each of the data bits of the p floating-point numbers to be decoded. The third product can be represented as (K... k -1)×(a) j -a i )×(1 / (2) n-1 -2).

[0135] The sixth adder calculates the sum of the third product of the floating-point number to be decoded (output by the third multiplier) and the minimum logarithmic representation in the second decoded information, thus obtaining the logarithmic representation of the floating-point number to be decoded. Specifically, the sixth adder may include p parallel addition circuits. These p parallel addition circuits can calculate in parallel the product of the third product corresponding to each data bit of the p floating-point numbers to be decoded and the minimum logarithmic representation in the second decoded information, thereby reconstructing the logarithmic representation of each data bit of the p floating-point numbers to be decoded. Wherein, the logarithmic representation a... k It can be represented as a i + (K) k -1)×(a) j -a i )×(1 / (2) n-1 -2). Specifically, due to the encoding result K k It is composed of (a) k -a i ) / (a j -a i )×(2 n-1 -2) +1, rounded down, can be considered as K k =(a k -a i ) / (a j -a i )×(2 n-1 -2)+1.

[0136] Therefore, based on the encoded K k The original numerical result can be obtained by reversing the value, which can be done based on K. k =(a k -a i ) / (a j -a i )×(2 n-1 -2) +1 reverse derivation decodes: logarithmic representation ak = a i + (K k - 1) x (a j - a i ) x (1 / (2 n-1 - 2)). Wherein, the logarithm represents a k which can be a fixed-point decimal.

[0137] In the embodiment of the present application, the floating-point number decoding component 12 can further comprise a second input unit, which is connected with the second extraction unit 121 and the restoring unit 122 respectively. The second input unit is used to input the received to-be-decoded vector to the second extraction unit 121, and input a i and a j in the received second decoding information to the fifth adder in the restoring unit 122, and input a i in the second decoding information to the sixth adder.

[0138] Thus, when the restoring unit 122 composed of the fourth adder, the fifth adder, the second multiplier, the third selector, the third multiplier and the sixth adder in the embodiment of the present application performs decoding, the data bits of each to-be-decoded floating-point number corresponding to the logarithmic representation are obtained. It should be noted that all operations of the restoring unit 122 are performed in the fixed-point decimal format. Since the adder and the multiplier can be designed in full-pipe, the process of determining the logarithmic representation by the restoring unit 122 can be implemented in full-pipe.

[0139] Optionally, in the embodiment of the present application, the decoding unit 123 comprises a decomposition sub-unit, an index sub-unit and a third splicing sub-unit, the decomposition sub-unit is connected with the index sub-unit and the third splicing sub-unit, and the index sub-unit is connected with the third splicing sub-unit.

[0140] The decomposing sub-unit is configured to decompose the logarithmic representation of each to-be-decoded floating-point number into an integer part and a decimal part. The restoring unit 122 outputs the logarithmic representation of each to-be-decoded floating-point number to the decomposing sub-unit. The decomposing sub-unit receives the logarithmic representation of each to-be-decoded floating-point number output by the second extracting unit 121. The logarithmic representation can be a fixed-point decimal. The decomposing sub-unit can include p parallel decomposing circuits. The p parallel decomposing circuits can decompose the data bits of the p to-be-decoded floating-point numbers in parallel to obtain the integer part and the decimal part of each to-be-decoded floating-point number. The decomposing sub-unit outputs the integer part of each to-be-decoded floating-point number to the third splicing sub-unit and outputs the decimal part of each to-be-decoded floating-point number to the exponent sub-unit. Since the to-be-decoded floating-point numbers are encoded, the decimal part of each to-be-decoded floating-point number is the mantissa after logarithm, i.e., the decimal part is obtained by taking logarithm of the mantissa. Therefore, the floating-point decoding component 12 restores the mantissa based on the exponent sub-unit to perform exponent operation and outputs the mantissa to the third splicing sub-unit to generate the non-logarithmic floating-point format representation.

[0141] The exponent sub-unit is configured to perform exponent operation on the decimal part of each to-be-decoded floating-point number to obtain the mantissa of each to-be-decoded floating-point number. The exponent sub-unit outputs the mantissa of each to-be-decoded floating-point number to the third splicing sub-unit.

[0142] The third splicing sub-unit is configured to splice the integer part of each to-be-decoded floating-point number as the exponent of each to-be-decoded floating-point number with the sign bit of each to-be-decoded floating-point number and the mantissa of each to-be-decoded floating-point number to obtain the non-logarithmic floating-point format representation of each to-be-decoded floating-point number.

[0143] Figure 9 FIG. 1 is a schematic diagram of a decoding process according to an embodiment of the present application. As shown in FIG. 1, the second input unit can receive a to-be-decoded vector and second decoding information. The second extracting unit 121 extracts the sign bit and the value part of each to-be-decoded floating-point number. The restoring unit 122 restores the logarithmic representation. The decoding unit 123 decomposes the logarithmic representation into an integer part and a decimal part and performs exponent operation on the decimal part to obtain the mantissa. The integer part is spliced with the sign bit and the mantissa as the exponent. Figure 9

[0144] In the embodiment of the present application, the second extracting unit 121 outputs the sign bit of each to-be-decoded floating-point number to the third splicing sub-unit. The exponent sub-unit can include p parallel exponent operation circuits to perform exponent operation on the mantissa of the p to-be-decoded floating-point numbers in parallel. The third splicing sub-unit can include p parallel third splicing circuits to splice the sign bit, the exponent and the mantissa of the p to-be-decoded floating-point numbers in parallel. In this way, parallel processing can improve the overall encoding efficiency of the to-be-encoded and decoded vector.​

[0145] Specifically, the exponent subunit comprises a second iteration operation circuit and a second multiply-add operation circuit; the second iteration operation circuit is configured to select a corresponding iteration round as a second target iteration round based on a mantissa precision of a target floating-point format; and the second iteration operation circuit is configured to perform iteration based on a fractional part of each to-be-decoded floating-point number until the second target iteration round is reached; and the second multiply-add operation circuit is configured to perform multiply-add operation on an iteration result of the second iteration operation circuit and a preset constant corresponding to the second target iteration round to obtain an exponent operation result corresponding to the to-be-decoded floating-point number.

[0146] In the embodiment of the present application, the exponent subunit calculates the exponent in the same way as the above-mentioned logarithm calculation, and can use the CORDIC algorithm. The input and output results of the CORDIC algorithm are both fixed-point decimals, and the CORDIC algorithm calculates the exponent result in an iterative manner. The second iteration operation circuit is an implementation circuit of the CORDIC algorithm. The target floating-point format is a floating-point format required to be generated after decoding. For example, if an FP8 format decoded floating-point number is required to be generated, the target floating-point format is FP8. If a BF16 format decoded floating-point number is required to be generated, the target floating-point format is BF16. The number of pipeline stages required is different for different target floating-point formats.

[0147] Further, referring to the foregoing related description, the mantissa precision of FP8 and BF16 can represent a maximum of 5 rounds and 13 rounds, respectively. That is, in the case of the target floating-point format being FP8, the second target iteration round can be 5, and in the case of the target floating-point format being BF16, the second target iteration round can be 13. In the iteration process, z m The value added to the register each time is gradually reduced, and when the value cannot be represented using the required precision, the iteration is stopped. Accordingly, the mantissa precision of FP8 and BF16 can represent a maximum of 5 rounds and 13 rounds of tan -1 (2 -m ) respectively.

[0148] In the exponent operation, u m , v m , and z m can be set to 1, 0, and θ respectively. Since the exponent directly obtained by the CORDIC algorithm is the result of the natural logarithm with the base of the natural constant e, and the base of the IEEE format exponent is 2, θ represents the product of the value of the to-be-solved exponent (i.e., the fractional part of the to-be-decoded floating-point number) and the constant ln2. Let f represent the second target iteration number, and m∈[1, f] in the exponent operation. According to the rules of the CORDIC algorithm, after f rounds of iteration, u f +v fThe result is approximately . Wherein, u f +v f is the iteration result of the second iteration operation circuit, and the exponential operation result (i.e. the mantissa corresponding to the floating point number to be decoded) is The second iteration operation circuit outputs the values (i.e. the first variable and the second variable obtained in the last stroke) of the last pipeline stage register u f and v f to the second multiplication-addition operation circuit as the iteration result. The second multiplication-addition operation circuit is used to calculate u f +v f , and then calculate the product of (u f +v f ) and the preset constant corresponding to the second target iteration number, to obtain the exponential operation result corresponding to the floating point number to be decoded as the output of the exponential subunit. Wherein, the preset constant corresponding to the second target iteration number can be expressed as , and the numerical value of the preset constant corresponding to the second target iteration number is different when f takes different values. The second multiplication-addition operation circuit can store the preset constant corresponding to the second target iteration number obtained in different iteration rounds (i.e. when f takes different values), so as to use the preset constant corresponding to the second target iteration number for multiplication-addition operation during calculation. In the embodiment of the application, the exponential subunit can realize exponential operation through the second iteration operation circuit and the second multiplication-addition operation circuit, realize the support of the processor for the exponential operation, and thus the efficiency of the exponential operation can be ensured.

[0149] In the embodiment of the application, the full-pipelining design is adopted in the actual hardware circuit design, and the iteration process is unfolded into pipeline stages. Each stroke of the pipeline represents one iteration of the CORDIC algorithm, the first pipeline stage register u0 and the register v0 are assigned as 0 and 1, and the register z0 is assigned as the result of multiplying the decimal part by the constant ln2. After 5 rounds and 13 rounds of iteration, the FP8 format mantissa and the BF16 format mantissa can be obtained respectively.

[0150] Of course, in specific implementation, other ways can also be used to realize the exponential operation, for example, the exponential operation result can be directly obtained by using the lookup table (LUT) mode through pre-computing and storing the exponential values corresponding to different input values, and the embodiment of the application does not limit this.

[0151] Further, a third splicing circuit splices an integer part corresponding to the to-be-decoded floating point number as an exponent, splices a sign bit corresponding to the to-be-decoded floating point number, and splices a mantissa to obtain a non-logarithmic floating point format representation of the to-be-decoded floating point number. In the embodiment of the present application, the non-logarithmic floating point format is FP8 when the target bit number n is 8, and the non-logarithmic floating point format is BF16 when the target bit number n is 10. Correspondingly, the third splicing circuit splices according to the requirement of the FP8 format when the target bit number n in the second decoding information is 8, to generate a non-logarithmic floating point format representation conforming to the FP8 format. The third splicing circuit splices according to the requirement of the BF16 format when the target bit number n is 10, to generate a non-logarithmic floating point format representation conforming to the BF16 format.

[0152] In the embodiment of the present application, the decomposition subunit, the exponent subunit and the third splicing subunit are arranged in the decoding unit 123. The decomposition subunit, the exponent subunit and the third splicing subunit provided by the processor itself are sequentially processed, so that the non-logarithmic floating point format representation of the to-be-decoded floating point number is obtained, and hardware support for floating point number decoding is ensured.

[0153] It should be noted that the floating point number decoding component 12 further comprises a second generating unit connected with the second extracting unit 121. The second extracting unit 121 is further configured to send the to-be-decoded floating point number in the to-be-decoded vector in the default form of the logarithmic floating point format representation to the second generating unit. The second generating unit is configured to generate the default form of the non-logarithmic floating point format representation for the received to-be-decoded floating point number. Correspondingly, in the case that the to-be-decoded vector includes infinity and non-number, other units in the floating point number decoding component 12 can perform parallel processing on the to-be-decoded floating point numbers except for the infinity and non-number. The default form of the non-logarithmic floating point format representation can be sNaN. That is, the second generating unit can directly decode the to-be-decoded floating point number of the non-number and the infinity in the to-be-decoded vector into sNaN, and the embodiment of the present application does not limit this.

[0154] Optionally, referring to Figure 7 , the data processing apparatus further comprises a data input component 13 and a data operation component 14. The data input component 13 is connected with the floating point number decoding component 12, and the floating point number decoding component 12 is connected with the data operation component 14.

[0155] The data input component 13 is configured to send the to-be-decoded vector and the second decoding information to the floating point number decoding component 12.

[0156] The data operation component 14 is configured to perform data processing based on the decoded vector output by the floating point number decoding component 12.

[0157] The hardware operation components in the data processing device are all designed with full pipelining to ensure data throughput. In the processing of a data processing device, the data input component 13 can first obtain the to-be-decoded vector and the second decoding information required for decoding from the network packet and send them to the floating-point decoding component 12. The floating-point decoding component 12 decodes and sends the decoded vector to the data operation component 14 for processing. The data operation component 14 takes the floating-point vector processed by the data processing as the to-be-encoded vector, sends it to the floating-point encoding component for encoding, obtains the encoded vector and the first decoding information, and sends them to the network of the distributed system for transmission.

[0158] In an existing manner, the model parameters are directly quantized by taking logarithm or combined with the Posit format. The floating-point number in the Posit format is obtained by additionally adding a regime field in the IEEE754 floating-point number format to improve the representation range of the floating-point number. In the manner of directly taking logarithm of the model, the precision loss is too large, which seriously reduces the performance of the model. In the manner of combining the logarithm with the Posit format, more errors are caused due to the difficulty in accurately performing addition and subtraction operations under the logarithmic representation.

[0159] In the embodiment of the present application, the logarithmic floating-point format is used only when communicating between different data processing devices, and the floating-point decoding component 12 is used to convert the logarithmic floating-point format into a non-logarithmic floating-point format when the data processing device performs data operation (for example, model inference), that is, the logarithmic floating-point format is not directly used when performing model inference. For example, the LogFMT-8Bit format is used in the data transmission process, and the FP32 format is decoded for operation when performing multiplication and accumulation operation. In this way, the data precision during data operation can be ensured, and the problem of excessive precision loss caused by more errors due to the difficulty in accurately performing addition and subtraction operations under the logarithmic representation when performing model inference can be avoided. Exemplarily, compared with the FP16 or BF16 format, the data transmission bit width reaches 2 bytes, so the communication time is longer and the communication pressure is greater. In the embodiment of the present application, the encoded logarithmic floating-point format with the target bit number n=8 is used during data transmission, and the data transmission bit width of the logarithmic floating-point format is smaller, only 8 bits, so the data transmission amount in the communication process can be reduced and the communication pressure can be reduced.

[0160] Optionally, the data operation component 14 includes p parallel operation units, and p is the number of floating-point numbers included in the decoded vector. One operation unit is configured to perform multiplication and addition operation on a target floating-point number in the local register based on a decoded floating-point number pair, and output an exception signal when the decoded floating-point number pair includes a special type of floating-point number. The two decoded floating-point numbers included in the decoded floating-point number pair come from two decoded vectors, respectively.

[0161] In one application scenario, the data operation component 14 can be a multiply-add component, which includes p multiply-add operation units. Exemplarily, two decoded vectors each including 16 to-be-decoded floating-point numbers can be input to the decoded floating-point number decoding component 12. Correspondingly, the data operation component 14 receives the two decoded vectors. The multiply-add units complete multiply-accumulate operations. There are 16 FP32 format floating-point numbers stored in the local register file in the multiply-add units, and the two decoded vectors and a set of FP32 format floating-point numbers stored in the local register file are respectively represented as {g1, g2, …, g 16}, {h1, h2, …, h 16}, {r1, r2, …, r 16}. The multiply-add units can perform operations: r t ’ = g t × h t + r t , where 1≤t≤16. After the operations are completed, the multiply-add units store the calculated FP32 format r t ’ in the register where r t was originally stored. That is, the multiply-add component performs a fused multiply-add operation on the two input decoded vectors and the FP32 format floating-point numbers stored in the local accumulation register, and the intermediate multiplication result is stored in the local accumulation register using the FP32 format.

[0162] In this example, the multiply-add component is a SIMD component, and the multiply-add component can include 16 parallel multiply-add units that perform operations in parallel. Each pair of decoded floating-point numbers and the target floating-point number in the local register is input into a respective multiply-add unit for execution. For example, (g t , h t ) and its corresponding target floating-point number r t are input into multiply-add unit t for execution. The finally calculated {r1’, r2’, …, r 16 ’} can be used as a to-be-encoded vector and encoded by the floating-point number encoding component 11 into a LogFMT-8Bit or LogFMT-10Bit data format encoded vector.

[0163] The number of operation units in the embodiment of the application is equal to the number of floating-point numbers included in the decoded vectors, so that the p decoded floating-point numbers in the two decoded vectors can be operated in parallel through the p parallel operation units, thereby improving the processing efficiency. The data processing device provided in the embodiment of the application includes the data operation component 14 including the p parallel operation units, so that the single instruction stream multi-data stream (SIMD) mode can be used in response to the multiply-add instruction, and the support for the SIMD is provided. The SIMD can perform multiple multiply-add operations in parallel in a single instruction, so as to improve the parallel performance of the floating-point operation, thereby improving the operation throughput and energy efficiency ratio.

[0164] Specifically, the multiply-add unit can include a first extraction circuit, a multiplication operation circuit, a fourth selector, a second extraction circuit, an addition operation circuit, and a fifth selector. Figure 10 is a schematic diagram of the multiply-add operation provided in the embodiment of the application, as Figure 10 shown, the dashed line represents the pipeline division. The first extraction circuit can extract the input encoded floating-point numbers into a sign bit, an exponent, and a mantissa in the first beat. When the encoded floating-point numbers include special type floating-point numbers, a special input signal is input to the fourth selector. The multiplication operation circuit performs sign bit operation, exponent operation (i.e., exponent addition), and mantissa operation (i.e., mantissa multiplication) on the two encoded floating-point numbers. The normalization bit shift and result rounding operation are performed by the multiplication operation circuit in the second beat. Specifically, the normalization and result rounding sub-circuit in the multiplication operation circuit can determine the bit shift length based on the number of leading 0s in the mantissa multiplication result, and perform normalization bit shift based on the bit shift length. The result rounding operation is used to splice the sign bit operation result, the exponent addition result, and the mantissa multiplication result after the normalization bit shift, and process the mantissa multiplication result according to the requirements of the floating-point format (e.g., FP32) to be obtained. Then, the multiplication result of the two encoded floating-point numbers after the result rounding operation is input to the fourth selector. The fourth selector outputs the multiplication result as an intermediate result to the second extraction circuit after receiving the multiplication result, and outputs an exception signal to the second extraction circuit after receiving the special input signal. For example, when the special input signal represents that the encoded floating-point number pair includes infinity, a first exception signal is output, and when the special input signal represents that the encoded floating-point number pair includes non-number, a second exception signal is output.

[0165] It should be noted that, in the first beat, before extracting the sign bit, the exponent and the mantissa, in the case that the encoded floating point number is not in the BF16 format, the leading zeros in the mantissa of the encoded floating point number can be first zero-extended in the low bit, and the exponent offset difference can be added to the exponent of the encoded floating point number. For example, the value of the exponent is added by 2 7 -2 4 = 112. Then, the length of the displacement corresponding to the number of leading zeros is subtracted. In this way, alignment with the BF16 format can be achieved, so that the FP8 format and the BF16 format floating point numbers can use the same data path to obtain the multiplication result in the BF16 format. Since the number of exponent bits and the exponent offset of the BF16 and FP32 format floating point numbers are the same, the result rounding link only needs to retain the calculated mantissa to the number of mantissa bits required by the FP32 format.

[0166] Further, in the case that the second extraction circuit receives the multiplication result (i.e., the intermediate result) output by the fourth selector in the third beat, the target floating point number in the local accumulation register and the intermediate result are extracted as the sign bit, the exponent and the mantissa, respectively. Assuming that the sign bit, the exponent and the mantissa of the intermediate result are sign bit 1, exponent 1 and mantissa 1, respectively, and the sign bit, the exponent and the mantissa of the target floating point number are sign bit 2, exponent 2 and mantissa 2, respectively. The absolute value of the exponent difference between exponent 1 and exponent 2 can be calculated by the addition operation circuit in the multiplication and addition unit in the third beat. If the absolute value is greater than a preset absolute value threshold, the sub-circuit corresponding to the first displacement path is used to align the exponents of exponent 1 and exponent 2, and then the mantissas corresponding to the smaller exponents in exponent 1 and exponent 2 are shifted to achieve mantissa alignment. For example, in the case that the exponent difference is 3, the mantissa corresponding to the smaller exponent is right-shifted by 3 bits.

[0167] Otherwise, the sub-circuit corresponding to the second displacement path is used to align the exponents of exponent 1 and exponent 2, and then the mantissas corresponding to the smaller exponents in exponent 1 and exponent 2 are shifted to achieve mantissa alignment. The moving length of the shifter in the first displacement path is greater than the moving length of the shifter in the second displacement path, and the preset absolute value threshold can be set as needed, for example, the preset absolute value threshold can be 2. The maximum value of the shift of the mantissa in the second displacement path does not exceed the preset absolute value threshold, so the shifter used has a small shift length, and the circuit structure is simple and has short delay.

[0168] Then, the sub-circuit used performs mantissa addition, normalization displacement and result rounding operation. It should be noted that, in the case that the sub-circuit corresponding to the first displacement path is used, the shift is performed in the third beat, and the mantissa addition, normalization displacement and result rounding operation are performed in the fourth beat (the fourth beat is the next beat after the third beat) Figure 10The first displacement path corresponds to the sub-circuit in the first case (as shown in the first case in FIG. 6). The second displacement path corresponds to the sub-circuit in the second case (as shown in the second case in FIG. 6). In the first case, the shift and the addition of the mantissas are performed in the third clock cycle, and the normalization displacement and the rounding of the result are performed in the fourth clock cycle. In the second case, the shift and the addition of the mantissas are performed in the third clock cycle, and the normalization displacement and the rounding of the result are performed in the fourth clock cycle.

[0169] It should be noted that the first displacement path and the second displacement path correspond to different sub-circuits in the fourth clock cycle. For the first displacement path, since the absolute value of the exponent difference is greater than the preset absolute value threshold, the difference in the order of magnitude between the shifted mantissa 1 and the shifted mantissa 2 is large, and thus the order of magnitude of the result after the addition of the mantissas will not be greatly different from that of the shifted mantissa 1 and the shifted mantissa 2. In this case, the moving length of the normalization displacement in the fourth clock cycle is small. Conversely, for the second displacement path, since the absolute value of the exponent difference is not greater than the preset absolute value threshold, the difference in the order of magnitude between the shifted mantissa 1 and the shifted mantissa 2 is small, and thus the order of magnitude of the result after the addition of the mantissas will not be greatly different from that of the shifted mantissa 1 and the shifted mantissa 2. In this case, the moving length of the normalization displacement in the fourth clock cycle is large. Therefore, the sub-circuit corresponding to the first displacement path has a simple structure and a short delay, and the delay of the addition of the mantissas and the normalization displacement is substantially the same as that of the normalization displacement of the sub-circuit corresponding to the second displacement path.

[0170] It should be noted that the fifth selector can be connected to the sub-circuit corresponding to the first displacement path and the sub-circuit corresponding to the second displacement path, and the absolute value of the exponent difference is used as the control signal of the fifth selector. When the control signal is greater than the preset absolute value threshold, the output value of the sub-circuit corresponding to the first displacement path is selected as the final operation result. Otherwise, the output value of the sub-circuit corresponding to the second displacement path is selected as the final operation result. After obtaining the final operation result, the local accumulation register is updated. In the case where the fourth selector outputs the abnormal signal in the third clock cycle, the second extraction circuit is further configured to input a special input signal to the fifth selector, and the fifth selector is further configured to output an abnormal signal after receiving the special input signal.

[0171] In the embodiment of the present application, the to-be-decoded vector can be an encoded vector generated by the data processing apparatus or an encoded vector sent by another data processing apparatus. For example, in an application scenario, a data processing apparatus can be included in a computing node in a distributed system, and a model in the distributed system can be used for image processing. The data processing apparatus in the computing node can extract image vectors of two to-be-fused images as to-be-encoded vectors, wherein the to-be-encoded vectors include light intensity values of pixels in the to-be-fused images. After encoding, two encoded vectors are obtained. The two encoded vectors can be sent to the next data processing apparatus. Since the data transmission bit width of the logarithmic floating-point format is small, the data transmission amount in the communication process can be reduced, and the communication pressure can be reduced. The next data processing apparatus can obtain two decoded vectors by decoding the two received encoded vectors as two to-be-decoded vectors. Then, the internal data operation component 14 is used to perform fusion multiplication on the two decoded vectors. For example, the decoded vectors in the FP32 format are subjected to fusion multiplication operation. In this way, the data accuracy during data operation can be ensured, the problem of inaccurate addition and subtraction operation in logarithmic representation, which brings more errors and excessive precision loss, can be avoided, and the final operation result is the fused image.

[0172] Since there is a problem of high operation amount and long operation delay in the model operation process. In some implementations, the neural network quantization is used to reduce the operation data precision, realize model compression, and reduce the memory occupation, calculation resource consumption and inference delay of the model. Among them, the neural network quantization is a technology of converting floating point to low bit representation (for example, INT8, INT4, etc.). However, in this way, the problem of model gradient disappearance and gradient explosion caused by low precision may occur. In the embodiment of the present application, when performing data operation in the combining (Combine) stage, a high-precision data format such as FP32 format is used, the data representation range is larger, the precision requirement can be met, the problem of model gradient disappearance and gradient explosion caused by low precision can be avoided, and the problem of decline of model expression ability and generalization ability caused by low precision can be avoided. At the same time, the limitation that the training weight must be controlled extremely finely to prevent data overflow can be avoided.

[0173] Compared with relying only on software coding and decoding, the register pressure is large during software coding and decoding, which may cause the actual processing efficiency to be low. When encoding the to-be-encoded vector into the logarithmic floating-point format and decoding the to-be-decoded vector into the non-logarithmic floating-point format, the data processing apparatus in the processor provided in the embodiment of the present application is a component originally provided in the processor, so that the processor can support the coding and decoding operation for the logarithmic floating-point format, improve the coding and decoding efficiency, and further improve the communication and inference speed of each computing node in the distributed system, and reduce the running delay of the distributed system.

[0174] Different data processing devices can be arranged in different processor cores of the processor, for example, in tensor cores of the GPU. In this way, the hardware encoding component and the decoding component can complete the encoding and decoding process within the processor core. When the GPU core performs data operation, the logarithmic floating point format is not used, and when the different data processing devices between the GPU cores communicate, the encoded vector in the encoded form is sent, thereby improving the support degree of the GPU for the encoding and decoding operation, reducing the delay of the encoding and decoding process, and thereby improving the overall processing speed without affecting the accuracy. In this way, the large language model program deployed in the distributed system can run with high efficiency, low latency and low communication pressure.

[0175] Figure 11 is a step flowchart of a data processing method provided by an embodiment of the present application. The method can be applied to any of the data processing devices described above, as shown in Figure 11 The method comprises the following steps:

[0176] Step 201, extracting the sign bit and the data bit of the floating point number to be encoded in the vector to be encoded.

[0177] Step 202, based on the data bit of the floating point number to be encoded, respectively finding the index corresponding to the maximum floating point number and the minimum floating point number in the vector to be encoded, to obtain the maximum index and the minimum index.

[0178] Step 203, based on the target bit number n, the sign bit of the floating point number to be encoded, the data bit of the floating point number to be encoded, the maximum index and the minimum index, encoding the floating point number to be encoded into an n-bit logarithmic floating point format representation, to obtain an encoded vector.

[0179] Optionally, the above step 202 can comprise:

[0180] Step 2021, based on the data bit of the floating point number to be encoded, finding the index corresponding to the maximum floating point number in the vector to be encoded, to obtain the maximum index.

[0181] Step 2022, based on the data bit of the floating point number to be encoded, finding the index corresponding to the minimum floating point number in the vector to be encoded, to obtain the minimum index.

[0182] Optionally, the above step 203 can comprise:

[0183] Step 2031, performing logarithmic operation on the mantissa of the floating point number to be encoded to generate a logarithmic operation result.

[0184] Step 2032, based on the exponent of the floating point number to be encoded and the logarithmic operation result, splicing to obtain the logarithmic representation of the floating point number to be encoded.

[0185] Step 2033, determining a maximum logarithmic representation and a minimum logarithmic representation based on the logarithmic representation of the floating point number to be encoded, the maximum index and the minimum index, and generating target data bits corresponding to the floating point number to be encoded based on the target bit number n.

[0186] Step 2034, splicing the sign bit of the floating point number to be encoded with the target data bits to obtain the logarithmic floating point format representation.

[0187] Optionally, the method can further include:

[0188] Step S21, extracting the sign bit and the data bit of the floating point number to be decoded in the vector to be decoded.

[0189] Step S22, restoring the logarithmic representation corresponding to the floating point number to be decoded based on the second decoding information of the vector to be decoded and the data bit of the floating point number to be decoded.

[0190] Step S23, generating the non-logarithmic floating point format representation of the floating point number to be decoded based on the sign bit of the floating point number to be decoded and the logarithmic representation corresponding to the floating point number to be decoded, to obtain a decoded vector.

[0191] Optionally, the step S23 can specifically include:

[0192] Step S231, decomposing the logarithmic representation corresponding to the floating point number to be decoded into an integer part and a decimal part.

[0193] Step S232, performing an exponential operation on the decimal part to obtain a mantissa corresponding to the floating point number to be decoded.

[0194] Step S233, taking the integer part corresponding to the floating point number to be decoded as an exponent corresponding to the floating point number to be decoded, and splicing the sign bit corresponding to the floating point number to be decoded and the mantissa corresponding to the floating point number to be decoded to obtain the non-logarithmic floating point format representation.

[0195] The specific implementation process of the data processing method provided by the embodiment of the application and the technical effects that can be achieved can be referred to the related description of the data processing device embodiment described above, which will not be repeated here.

[0196] Referring to Figure 12 is a structural schematic diagram of an electronic device provided by the embodiment of the application. As shown in Figure 12 , the electronic device includes a processor, a memory, a communication interface and a communication bus.

[0197] The processor, the memory and the communication interface complete communication with each other through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction makes the processor execute the data processing method of the foregoing embodiments. The executable instruction can constitute a program.

[0198] The embodiment of the present application provides a processor, comprising the data processing device.

[0199] The embodiment of the present application provides a machine readable medium, which stores instructions, and when the instructions are executed by one or more processors, the processors can execute the data processing method of the foregoing embodiments.

[0200] The data processing device, method, processor and electronic equipment provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples in this paper. The above embodiment is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as the limitation of the present application.

Claims

1. A data processing apparatus, characterized by, The application relates to a floating-point number encoding component. The first extraction unit is used for extracting the sign bit and the data bit of a floating-point number to be encoded in a vector to be encoded. The lookup unit is used for respectively searching the index corresponding to the maximum floating-point number and the minimum floating-point number in the vector to be encoded based on the data bit of the floating-point number to be encoded, so as to obtain the maximum index and the minimum index. The encoding unit is used for determining the logarithmic representation of the floating-point number to be encoded based on the data bit of the floating-point number to be encoded. The maximum index and the minimum index are used for determining the target data bit corresponding to the floating-point number to be encoded based on the logarithmic representation of the floating-point number to be encoded, the target bit number n, the maximum index and the minimum index; and the sign bit of the floating-point number to be encoded is spliced with the target data bit to obtain the logarithmic floating-point format representation as an encoded vector. The lookup unit comprises a first searcher and a second searcher.

2. The apparatus of claim 1, wherein, The first searcher is used for searching the index corresponding to the maximum floating-point number in the vector to be encoded based on the data bit of the floating-point number to be encoded, so as to obtain the maximum index; the maximum floating-point number represents the maximum value in the vector to be encoded. The second searcher is used for searching the index corresponding to the minimum floating-point number in the vector to be encoded based on the data bit of the floating-point number to be encoded, so as to obtain the minimum index; the minimum floating-point number represents the minimum value in the vector to be encoded. The first searcher and the second searcher each comprise a plurality of comparator layers.

3. The apparatus of claim 2, wherein, The two input ends of each comparator in the first comparator layer respectively receive the data bit of each of the two floating-point numbers to be encoded; the two input ends of each comparator in the next comparator layer are respectively connected to the output ends of two comparators in the current comparator layer. The comparators in the first searcher are used for outputting the maximum value in the data bit of the two floating-point numbers to be encoded and the index of the maximum value; the comparators in the second searcher are used for outputting the minimum value in the data bit of the two floating-point numbers to be encoded and the index of the minimum value. The data bit of the floating-point number to be encoded comprises a mantissa and a tail number; the encoding unit comprises a logarithmic subunit, a first splicing subunit, an encoding subunit and a second splicing subunit.

4. The apparatus of claim 1, wherein, The logarithmic subunit is used for performing logarithmic operation on the tail number of the floating-point number to be encoded to generate a logarithmic operation result. The first splicing subunit is used for splicing the mantissa of the floating-point number to be encoded and the logarithmic operation result to obtain the logarithmic representation of the floating-point number to be encoded. The encoding subunit is used for determining the maximum logarithmic representation and the minimum logarithmic representation based on the logarithmic representation of the floating-point number to be encoded, the maximum index and the minimum index, and generating the target data bit corresponding to the floating-point number to be encoded based on the target bit number n. The second splicing subunit is used for splicing the sign bit of the floating-point number to be encoded with the target data bit to obtain the logarithmic floating-point format representation. The encoding subunit comprises a first selector, a first adder, a second adder, a first divider, a first multiplier, a second selector, a third adder and a rounding circuit.

5. The apparatus of claim 4, wherein, ​ The first selector is configured to determine the maximum logarithm representation and the minimum logarithm representation; The first adder is configured to calculate a first difference value between the maximum logarithm representation and the minimum logarithm representation; The second adder is configured to calculate a difference value between the logarithm representation of the floating-point number to be encoded and the minimum logarithm representation, to obtain a second difference value corresponding to the floating-point number to be encoded; The first divider is configured to perform a division operation on the second difference value and the first difference value, to obtain a first division result corresponding to the floating-point number to be encoded; The second selector is configured to output a first constant corresponding to the target bit number n as a first coefficient to the first multiplier; The first multiplier is configured to calculate a product of the first coefficient and the first division result, to obtain a first product corresponding to the floating-point number to be encoded; The third adder is configured to calculate a sum value of the first product and a second constant, to obtain a first sum value corresponding to the floating-point number to be encoded; The rounding circuit is configured to round the first sum value, to obtain a target data bit corresponding to the floating-point number to be encoded.

6. The apparatus of claim 1, wherein, The device further comprises a floating-point decoding component composed of a second extraction unit, a restoration unit and a decoding unit; The second extraction unit is configured to extract a sign bit and a data bit of a floating-point number to be decoded in a vector to be decoded; The restoration unit is configured to restore a logarithm representation corresponding to the floating-point number to be decoded based on second decoding information of the vector to be decoded and the data bit of the floating-point number to be decoded; The decoding unit is configured to generate a non-logarithm floating-point format representation of the floating-point number to be decoded based on the sign bit of the floating-point number to be decoded and the logarithm representation corresponding to the floating-point number to be decoded, to obtain a decoded vector.

7. The apparatus of claim 6, wherein, The restoration unit comprises a fourth adder, a fifth adder, a second multiplier, a third selector, a third multiplier and a sixth adder; The fourth adder is configured to calculate a sum value of the data bit of the floating-point number to be decoded and a third constant, to obtain a second sum value; The fifth adder is configured to calculate a third difference value between a maximum logarithm representation and a minimum logarithm representation in the second decoding information; The second multiplier is configured to calculate a product of the second sum value and the third difference value, to obtain a second product; The third selector is configured to select a second coefficient corresponding to a target bit number n in the second decoding information and output the second coefficient to the third multiplier; The third multiplier is configured to calculate a product of the second product and the second coefficient, to obtain a third product; The sixth adder is configured to calculate a sum value of the third product and the minimum logarithm representation in the second decoding information, to obtain the logarithm representation corresponding to the floating-point number to be decoded.

8. The apparatus of claim 6, wherein, The decoding unit comprises a decomposition subunit, an exponent subunit and a third splicing subunit; The decomposition subunit is configured to decompose the logarithm representation corresponding to the floating-point number to be decoded into an integer part and a decimal part; The exponent subunit is configured to perform an exponent operation on the decimal part, to obtain a mantissa corresponding to the floating-point number to be decoded; The third splicing unit is configured to splice an integer part corresponding to the to-be-decoded floating-point number as an exponent corresponding to the to-be-decoded floating-point number, and splice a sign bit corresponding to the to-be-decoded floating-point number and a mantissa corresponding to the to-be-decoded floating-point number, to obtain the non-logarithmic floating-point format representation.

9. A data processing method, characterized by, The method is applied to the device in any one of claims 1 to 8, and the method comprises: extracting a sign bit and a data bit of a to-be-encoded floating-point number in a to-be-encoded vector; finding indexes corresponding to a maximum floating-point number and a minimum floating-point number in the to-be-encoded vector respectively based on the data bit of the to-be-encoded floating-point number, to obtain a maximum index and a minimum index; determining a logarithmic representation of the to-be-encoded floating-point number based on the data bit of the to-be-encoded floating-point number, determining a target data bit corresponding to the to-be-encoded floating-point number based on the logarithmic representation of the to-be-encoded floating-point number, the target bit number n, the maximum index and the minimum index, and splicing the sign bit of the to-be-encoded floating-point number and the target data bit to obtain a logarithmic floating-point format representation as an encoded vector.

10. A processor, comprising: The device in any one of claims 1 to 8.

11. An electronic device, comprising: comprises: the processor, the memory, the communication interface and the communication bus in claim 10, the processor, the memory and the communication interface complete communication with each other through the communication bus; the memory is configured to store executable instructions, and the executable instructions enable the processor to execute the method in claim 9.

Images