Data processing methods and apparatus, processors, electronic devices, storage media

Abstract

A data processing method, data processing apparatus, processor, electronic device, and non-transient storage medium are disclosed. The data processing method includes: acquiring multiple input tensors as input parameters for computational processing; representing each input tensor using a combination of M input sub-tensors; replacing the input tensor with the combination of the M input sub-tensors for each input tensor; performing computational processing; and obtaining a computational result. This data processing method expands the applicable scenarios of computational processing, effectively utilizes the powerful computing capabilities of the inherently low-precision floating-point numbers, and significantly improves the overall efficiency of computation.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data processing method, a data processing apparatus, a processor, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] A tensor is a multilinear mapping defined on the Cartesian product of some vector spaces and some dual spaces. For example, a scalar can be viewed as a 0-dimensional tensor, a vector as a one-dimensional tensor, and a matrix as a two-dimensional tensor. Tensor operations are widely used in processors such as parallel processors.

[0003] With the development of artificial intelligence and machine learning, new requirements have been placed on numerous parallel processor devices, such as multi-core processors, graphics processing units (GPUs), and digital signal processors (DSPs). Tensor operations in parallel processors can include general matrix multiplication (GEMM) or convolutional multiplication. For example, in neural network processing frequently used in fields such as artificial intelligence, such as convolutional neural networks, matrix multiply and accumulate (MACC) calculations are often required. MACC is also a type of tensor operation; for example, it involves multiplying corresponding elements of two matrices and then summing the results to obtain a final value. Summary of the Invention

[0004] At least one embodiment of this disclosure provides a data processing method, comprising: acquiring a plurality of input tensors as input parameters for computational processing, wherein the plurality of input tensors are all of a first precision type; for each input tensor, representing the input tensor by combining M input sub-tensors, wherein the M input sub-tensors have at least two different precision types, the at least two precision types being different from the first precision type, and M being an integer greater than 1; for each input tensor, replacing the input tensor with the M input sub-tensors that combine to represent the input tensor, performing the computational processing, and obtaining a computation result.

[0005] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein for each input tensor, the M input sub-tensors have the same shape as the input tensor, and for each input tensor, the input tensor is represented by a combination of the M input sub-tensors, including: for each parameter element in the input tensor, splitting the parameter element into M sub-elements, wherein the M sub-elements are elements in the M input sub-tensors that have the same position as the parameter element in the input tensor, and the parameter element is represented as a sum of the M sub-elements.

[0006] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein for each parameter element in the input tensor, the parameter element is split into M sub-elements, including: determining that the exponent and sign bit of the first sub-element among the M sub-elements are the same as the exponent and sign bit of the parameter element, and that the mantissa portion of the first sub-element is the same as the first high-order significant bit portion of the mantissa significant bit of the parameter element; determining the other M-1 sub-elements among the M sub-elements excluding the first sub-element, wherein the sum of the other M-1 sub-elements is the difference between the parameter element and the first sub-element.

[0007] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein the at least two precision types include a second precision type, the precision type of the first sub-element is the second precision type, the total number of bits of the second precision type is N2, N2 is a positive integer, and the binary representation of the first sub-element is the first N2 bits of the binary representation of the parameter element.

[0008] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein the mantissa significant bits of the parameter element, excluding the first high-order significant bits, are divided into M-1 consecutive segments, and the other M-1 sub-elements correspond to the M-1 segments respectively. The number of significant bits in each segment is less than or equal to the number of mantissa significant bits in the sub-element corresponding to the segment. Determining the other M-1 sub-elements excluding the first sub-element includes: determining that the mantissa significant bits of each of the other M-1 sub-elements are respectively the M-1 segments; determining that the exponent of each of the other M-1 sub-elements is P-Qi, where P is the exponent of the parameter element, Qi is the bit difference between the highest bit of the segment corresponding to the sub-element and the highest bit of the mantissa significant bits of the parameter element, and P and Qi are integers.

[0009] For example, in at least one embodiment of this disclosure, a data processing method is provided where the number of significant mantissa bits for the first precision type is F1, the at least two precision types include a second precision type and a third precision type, the number of significant mantissa bits for the second precision type is F2, the number of significant mantissa bits for the third precision type is F3, F1, F2, and F3 are positive integers, and the other M-1 sub-elements include a second sub-element, the precision type of which is the third precision type. The determination of the other M-1 sub-elements... The method of using the M-1 segments as their respective mantissas includes: in response to F1-F2 being less than or equal to F3, determining that M-1 is 1, and determining that the high F1-F2 bits of the mantissa of the binary representation of the second sub-element are the same as the F1-F2-1 bits to the 0th bit of the mantissa portion of the binary representation of the parameter element; in response to F1-F2 being greater than F3, determining that the mantissa of the binary representation of the second sub-element is the same as the F1-F2-F3-1 bits to the F1-F2-1 bits of the mantissa portion of the binary representation of the parameter element.

[0010] For example, in at least one embodiment of this disclosure, a data processing method is provided in which, in response to F1-F2 being greater than F3, F1 is equal to the sum of the number of mantissa significant bits in the precision types of the M sub-elements.

[0011] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein the computational processing includes at least convolution operation or matrix multiplication operation, the number of significant mantissa digits of the first precision type is F1, where F1 is a positive integer, the at least two precision types include a second precision type and a third precision type, for each input tensor, the input tensor is replaced by M input sub-tensors that combine to represent the input tensor, the computational processing is performed, and a computational result is obtained, including: according to the computational processing, replacing each input tensor in the computational processing with the sum of M input sub-tensors that combine to represent the input tensor, expanding to obtain L first intermediate results, wherein each first intermediate result is represented as the product or convolution of two input sub-tensors, and L is a positive integer greater than 1; determining the L exponents corresponding to the L first intermediate results and the largest exponent among the L exponents; selecting at least one first intermediate result from the L first intermediate results according to the L exponents, wherein the absolute value of the difference between the exponent of the at least one first intermediate result and the largest exponent is less than or equal to F1; and taking the sum of the at least one first intermediate result as the computational result.

[0012] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein the computational processing includes at least convolution or matrix multiplication, the number of significant mantissa digits of the first precision type is F1, where F1 is a positive integer, and the at least two precision types include a second precision type and a third precision type. For each input tensor, the input tensor is replaced by M input sub-tensors that combine to represent the input tensor, and the computational processing is performed to obtain a computational result, including: according to the computational processing, replacing each input tensor in the computational processing with the sum of M input sub-tensors that combine to represent the input tensor, expanding to obtain L first intermediate results, wherein each first intermediate result is represented as the product or convolution of two input sub-tensors. The method involves: for each input subtensor of type 2 precision among the L first intermediate results, representing the input subtensor of type 2 precision using a combination of W intermediate subtensors of type 3 precision to obtain U second intermediate results, where L and U are positive integers; determining U exponents corresponding to the U second intermediate results and the largest exponent among the U exponents; selecting at least one second intermediate result from the U second intermediate results based on the U exponents, wherein the absolute value of the difference between the exponent of the at least one second intermediate result and the largest exponent is less than or equal to F1; and summing the at least one second intermediate result as the calculation result.

[0013] For example, in at least one embodiment of this disclosure, a data processing method is provided. For each input sub-tensor of at least a portion of the input sub-tensors of type second precision among the L first intermediate results, the input sub-tensor of type second precision is represented by a combination of W intermediate sub-tensors of type third precision to obtain U second intermediate results. This includes: determining L exponents corresponding to the L first intermediate results respectively; selecting the maximum value from the L exponents; determining L-1 first intermediate results other than the first intermediate result corresponding to the maximum value among the L first intermediate results; and for V first intermediate results including input sub-tensors of type second precision among the L-1 first intermediate results. As a result, for each of the V first intermediate results, the input sub-tensor of type second precision is replaced by the sum of W intermediate sub-tensors of type third precision, and expanded to obtain W third intermediate results corresponding to each of the V first intermediate results. The third intermediate results are represented as the product or convolution of the input sub-tensor and the intermediate sub-tensor of type third precision, where V is a positive integer. All the third intermediate results corresponding to the V first intermediate results, the first intermediate result corresponding to the maximum value, and the other L-1-V first intermediate results (excluding the V first intermediate results) among the L-1 first intermediate results are taken as the U second intermediate results.

[0014] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein the first precision type and the at least two precision types are both floating-point types.

[0015] For example, in at least one embodiment of this disclosure, a data processing method is provided in which the number of exponent bits in the first precision type is the same as the number of exponent bits in the at least two precision types.

[0016] For example, in at least one embodiment of this disclosure, a data processing method is provided in which the accuracy of the first accuracy type is higher than the accuracy of any one of the at least two accuracy types.

[0017] For example, in at least one embodiment of this disclosure, a data processing method is provided, wherein the first precision type is FP32, and the at least two precision types include BF16 and BF24. The exponent part of BF16 has 8 bits, and the mantissa part of BF16 has 8 significant bits. The exponent part of BF24 has 8 bits, and the mantissa part of BF24 has 16 significant bits. Each input tensor is represented by a combination of one input subtensor of type BF16 and one input subtensor of type BF24.

[0018] At least one embodiment of this disclosure also provides a data processing method, comprising: receiving first data, wherein the first data is of a first precision type; representing the first data using a combination of M sub-data; and using the combination of the M sub-data to replace the first data for subsequent processing, wherein the M data have at least two different precision types, the at least two precision types are different from the first precision type, M is an integer greater than 1, the first precision type and the at least two precision types are both floating-point types, the number of exponent bits in the first precision type is the same as the number of exponent bits in the at least two precision types, and the precision of the first precision type is higher than the precision of any one of the at least two precision types.

[0019] At least one embodiment of this disclosure also provides a data processing apparatus, comprising: an acquisition module configured to acquire a plurality of input tensors as input parameters for computational processing, wherein the plurality of input tensors are all of a first precision type; a first processing module configured to, for each input tensor, represent the input tensor by combining M input sub-tensors, wherein the M input sub-tensors have at least two different precision types, the at least two precision types are different from the first precision type, and M is an integer greater than 1; and a second processing module configured to, for each input tensor, replace the input tensor with the M input sub-tensors that represent the input tensor, perform the computational processing, and obtain a computational result, wherein the first precision type and the at least two precision types are both floating-point types, the number of exponent bits in the first precision type is the same as the number of exponent bits in the at least two precision types, and the precision of the first precision type is higher than the precision of any one of the at least two precision types.

[0020] At least one embodiment of this disclosure also provides a processor, including the data processing apparatus as described in any embodiment of this disclosure.

[0021] At least one embodiment of this disclosure also provides a data processing method, comprising: receiving a data calculation instruction, wherein the data calculation instruction includes a plurality of input tensors as calculation input parameters; parsing the data calculation instruction and then executing the data calculation instruction using a data processing unit, wherein executing the data calculation instruction using the data processing unit includes: acquiring a plurality of input tensors as input parameters for calculation processing, wherein the plurality of input tensors are all of a first precision type; for each input tensor, representing the input tensor using a combination of M input sub-tensors, wherein the M input sub-tensors have at least two different precision types, the at least two precision types are different from the first precision type, and M is an integer greater than 1; for each input tensor, replacing the input tensor with the M input sub-tensors that represent the input tensor, performing the calculation processing, and obtaining a calculation result, wherein the first precision type and the at least two precision types are both floating-point types, the number of exponent bits in the first precision type is the same as the number of exponent bits in the at least two precision types, and the precision of the first precision type is higher than the precision of any one of the at least two precision types.

[0022] At least one embodiment of this disclosure also provides a processor, including an instruction parsing unit and a data processing unit, wherein the instruction parsing unit is used to receive and parse data calculation instructions, wherein the data calculation instructions include a plurality of input tensors as calculation input parameters, and the data processing unit executes the data processing method according to any embodiment of this disclosure after the instruction parsing unit parses the data calculation instructions.

[0023] At least one embodiment of this disclosure also provides an electronic device, including: a memory that non-transitoryly stores computer-executable instructions; and a processor configured to run the computer-executable instructions, wherein the computer-executable instructions are executed by the processor to implement the data processing method according to any embodiment of this disclosure.

[0024] At least one embodiment of this disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the data processing method according to any embodiment of this disclosure. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0026] Figure 1A schematic flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0027] Figure 2A This is an illustrative representation of the data format of parameter elements provided in at least one embodiment of the present disclosure;

[0028] Figure 2B This is an illustrative representation of the data format of parameter elements provided in at least one embodiment of the present disclosure;

[0029] Figure 3 This is a schematic diagram illustrating the relationship between parameter elements and sub-elements provided in at least one embodiment of this disclosure;

[0030] Figure 4A A schematic flowchart illustrating step S30 provided for at least one embodiment of this disclosure;

[0031] Figure 4B A schematic diagram illustrating a mixed-precision simulation process provided in at least one embodiment of this disclosure;

[0032] Figure 5A A schematic flowchart illustrating step S30 provided for at least one embodiment of this disclosure;

[0033] Figure 5B A schematic diagram illustrating the processing steps of mixed-precision simulation provided in at least one embodiment of this disclosure;

[0034] Figure 6 A flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0035] Figure 7A A schematic block diagram of a data processing apparatus provided in at least one embodiment of this disclosure;

[0036] Figure 7B A schematic block diagram of a data processor provided for at least one embodiment of this disclosure;

[0037] Figure 8 A schematic structural diagram of a processor provided for at least one embodiment of this disclosure;

[0038] Figure 9 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of this disclosure;

[0039] Figure 10 This is a schematic block diagram of an electronic device provided in one embodiment of the present disclosure. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0041] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0042] To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and known components have been omitted.

[0043] Floating-point numbers (FP) are primarily used to represent decimals and typically consist of three parts: a sign bit, an exponent, and a mantissa. The exponent part can also be called the exponent code. For example, a floating-point number V can usually be represented in the following form:

[0044] V = (-1) s ×M×2 E

[0045] In this context, the sign bit s can be 1 bit, determining whether the floating-point number V is negative or positive; M represents the mantissa, which can include multiple bits and is a binary fractional form, defining the precision of the floating-point number; E represents the exponent (also called the exponent value), used to weight the floating-point number, reflecting the position of the decimal point in the floating-point number V, and defining the range of values ​​for the floating-point number.

[0046] For example, traditional floating-point numbers typically include three formats: half-precision floating-point numbers (FP16), single-precision floating-point numbers (FP32), and double-precision floating-point numbers (FP64), with different numbers of bits in their exponent and mantissa parts.

[0047] For normalized floating-point numbers, the number of significant mantissa bits is the number of bits in the mantissa plus 1. For example, for single-precision floating-point numbers, the mantissa consists of 23 bits, with 24 significant mantissa bits and the most significant bit being 1.

[0048] GPUs (Graphics Processing Units) and AI accelerators are widely used for deep learning model training. GPU manufacturers have made specific optimizations in their hardware and software designs to accelerate computation, particularly for tensor operations common in deep learning models. For example, some GPU and AI accelerator manufacturers provide dedicated data processing units to optimize tensor computations. These data processing units may include Tensor Cores, which significantly increase data throughput and improve computational efficiency.

[0049] For example, data processing devices using Tensor Cores support various computational operations, such as regular numerical operations, matrix multiplication, and convolution multiplication. Furthermore, these devices have optimized various floating-point data formats for fields like artificial intelligence and deep learning, such as BF16 (brain floating point 16, 16-bit width), BF24 (brain floating point 24, 24-bit width), and TF32 (Tensor Float 32, 19-bit width). These data formats can significantly reduce the computational resources and power consumption required for operations, especially matrix multiplication or convolution multiplication. In addition, the devices also support some common floating-point types, such as half-precision floating-point numbers (FP16, 16-bit width) or double-precision floating-point numbers (FP64, 64-bit width).

[0050] However, single-precision floating-point numbers (FP32, with a bit width of 32 bits) are a common data type, and data processing devices, such as Tensor Cores, do not directly support the single-precision floating-point format. However, performing computational processing using single-precision floating-point numbers is a very important basic operation in high-performance computing such as artificial intelligence and data analysis. If GPUs, AI accelerators, and other devices cannot support this type of operation, it will affect the applicability of these devices.

[0051] This disclosure provides at least one embodiment of a data processing method, a data processing apparatus, a processor, an electronic device, and a non-transitory computer-readable storage medium. The data processing method includes: acquiring a plurality of input tensors as input parameters for computational processing, wherein the plurality of input tensors are all of a first precision type; for each input tensor, representing the input tensor using a combination of M input sub-tensors, wherein the M input sub-tensors have at least two different precision types, and both of the at least two precision types are different from the first precision type, and M is an integer greater than 1; for each input tensor, replacing the input tensor with the M input sub-tensors that represent the input tensor, performing computational processing, and obtaining a computational result.

[0052] In the data processing method provided in at least one embodiment of this disclosure, multiple low-precision input sub-tensors of mixed precision type are used to simulate high-precision input tensors. This enables processors using the data processing method to support calculations of high-precision data formats that might not have been supported previously, expanding the applicable scenarios for computational processing, improving the applicability of processors using the data processing method, and effectively utilizing the powerful computing power of the low-precision floating-point numbers provided. This not only does not increase the overall computation time of computational processing but also significantly improves the overall efficiency of computation. Furthermore, during the execution of the data processing method, it is decoupled from upper-layer application software, such as artificial intelligence applications. The upper-layer application is not aware of the specific process of the data processing method, thereby greatly reducing the cost of software adaptation.

[0053] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0054] Figure 1 This is a schematic flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0055] like Figure 1 As shown, the data processing method provided in this embodiment includes steps S10 to S30.

[0056] In step S10, multiple input tensors are obtained as input parameters for computation.

[0057] For example, multiple input tensors are of the first precision type.

[0058] For example, the input tensor can be obtained by reading the input parameters of the computation instruction, or it can be obtained by reading pre-stored data as the input tensor, etc. This disclosure does not impose any specific restrictions on this.

[0059] For example, the input tensor can be a 0-dimensional tensor, i.e., a single floating-point number, or a 1-dimensional tensor, i.e., a scalar (array) of type floating-point number, or a 2-dimensional tensor (matrix) or a higher-dimensional tensor. This disclosure does not impose any specific restrictions on this.

[0060] In step S20, for each input tensor, the input tensor is represented by a combination of M input sub-tensors.

[0061] For example, M input subtensors have at least two different precision types, both of which are different from the first precision type, where M is an integer greater than 1.

[0062] In step S30, for each input tensor, the input tensor is replaced by a combination of M input sub-tensors, and the calculation process is performed to obtain the calculation result.

[0063] For example, computational processing may include matrix multiplication, convolution, and conventional arithmetic operations, and the embodiments of this disclosure do not impose specific limitations on these. For example, when the input tensor is a two-dimensional tensor, the computational processing may be matrix multiplication; when the input tensor is a multi-dimensional tensor, the computational processing may be convolution; conventional arithmetic operations may include addition, subtraction, etc.

[0064] Depending on the computational process, the number of input tensors can be adjusted. For example, there can be two input tensors, and the computational process can be to perform a convolution operation on the two input tensors. Of course, there can be more input tensors. For example, the computational process can be to perform a convolution operation on the two input tensors and add the convolution result to other input tensors, etc. The embodiments of this disclosure do not impose specific limitations on this.

[0065] For example, the first precision type and the above at least two precision types are both floating-point types. The number of exponent bits (i.e., the number of bits in the exponent part or the exponent part) in the first precision type is the same as the number of exponent bits in the at least two precision types. The precision of the first precision type is higher than the precision of any of the above at least two precision types.

[0066] For example, since the number of bits of the exponent in the first precision type is the same as the number of bits of the exponent in at least the two precision types mentioned above, the range of data that it can represent is also the same, and there will be no data overflow problem.

[0067] For example, "the precision of the first precision type" is higher than "the precision of any one of the at least two precision types" means that the number of significant bits in the mantissa of the first precision type is greater than the number of significant bits in the mantissa of any one of the at least two precision types.

[0068] For example, at least two different precision types are represented, assuming that at least two different precision types include n precision types, the number of bits in the exponent part of these n precision types is the same, but the number of significant bits in the mantissa part of these n precision types is different, where n is a positive integer and greater than or equal to 2.

[0069] Of course, the first precision type and the at least two precision types must also be different data formats.

[0070] For example, the first precision type can be a single-precision floating-point number FP32. Correspondingly, the above-mentioned at least two precision types include BF24 and BF16, and their data formats are shown in Table 1 below:

[0071] Table 1 Data Format

[0072] Data format Total number of bits sign bit Index section Last digits Number of significant digits in the mantissa FP32 32 1 8 23 24 BF24 24 1 8 15 16 BF16 16 1 8 7 8

[0073] As shown in Table 1, the first precision type is FP32, which has a total of 32 bits, including 1 sign bit, 8 bits for the exponent (i.e., the exponent code), 23 bits for the mantissa, and 23 + 1 = 24 significant mantissa bits. The at least two precision types include BF24 and BF16. BF24 has a total of 24 bits, including 1 sign bit, 8 bits for the exponent (i.e., the exponent code), 15 bits for the mantissa, and 15 + 1 = 16 significant mantissa bits. BF16 has a total of 16 bits, including 1 sign bit, 8 bits for the exponent (i.e., the exponent code), 7 bits for the mantissa, and 7 + 1 = 8 significant mantissa bits.

[0074] Of course, the embodiments disclosed herein are not limited to this. For example, the first precision type may be FP32, and the at least two precision types may include BF16 and TF32, etc. In this disclosure, the first precision type and the various low-precision types simulating the first precision type may adopt any feasible combination of floating-point types that satisfy the format requirements of the first precision type and the at least two precision types as described above, and this disclosure does not impose specific limitations on this.

[0075] In the data processing method provided in at least one embodiment of this disclosure, a variety of low-precision floating-point tensors are used to simulate high-precision floating-point tensors, thereby expanding the applicable scenarios of computational processing, improving the applicability of processors using this data processing method, effectively utilizing the powerful computing power of the low-precision floating-point numbers themselves, without increasing the overall computation time of computational processing, and can also significantly improve the overall efficiency of computation.

[0076] The execution process of the data processing method provided in at least one embodiment of this disclosure will be described in detail below with reference to the accompanying drawings.

[0077] For example, for each input tensor, the M input sub-tensors that combine to represent the input tensor have the same shape as the input tensor, that is, they have the same dimensions and the same length on each axis. For example, if the input tensor is a two-dimensional tensor, then the input tensor and the M input sub-tensors are both matrices of type a rows and b columns, where a and b are positive integers and both greater than 1.

[0078] Step S20 may include: for each parameter element in the input tensor, splitting the parameter element into M sub-elements, where the M sub-elements are elements in the M input sub-tensors that have the same position as the parameter element in the input tensor, and the parameter element is represented as the sum of the M sub-elements. It should be noted that in at least one embodiment of this disclosure, the "sum" of A and B may include A+B, and also AB; for the latter, it is equivalent to A+(-B).

[0079] For example, for an input tensor A among multiple input tensors, it can be split into A = A1 + A2 + ... + A M Where A1, A2, ..., A M To combine M input subtensors A1, A2, ..., A3 to represent the input tensor A. M It has at least two different precision types, for example, A1, A2, ..., A M There can be M different precision types, that is, their respective precision types are different from each other. For example, A1 and A2 have a second precision type, and other input subtensors have a third precision type, etc. The embodiments of this disclosure do not impose specific limitations on this.

[0080] For example, if the input tensor A is a two-dimensional tensor, that is, the input tensor A is a matrix with a row and b columns, then the parameter element p located in the i-th row and j-th column of the input tensor A will be... ij It is split into M sub-elements, where each M sub-element is the element p located in the i-th row and j-th column of the input sub-tensor A1. 1ij The element p in the i-th row and j-th column of the input subtensor A2 2ij ..., Input subtensor A M The element p located in the i-th row and j-th column Mij And parameter element p ij =p 1ij +p 2ij +...+p Mij Here, i is a positive integer less than or equal to a, and j is a positive integer less than or equal to b.

[0081] For example, for each parameter element in the input tensor, splitting the parameter element into M sub-elements may include: determining that the exponent and sign bit of the first sub-element among the M sub-elements are the same as the exponent and sign bit of the parameter element, and that the mantissa of the first sub-element is the same as the first high-order significant bit of the mantissa of the parameter element; determining the other M-1 sub-elements among the M sub-elements excluding the first sub-element, where the sum of the other M-1 sub-elements is the difference between the parameter element and the first sub-element.

[0082] For example, the at least two precision types include a second precision type, the precision type of the first sub-element is the second precision type, the total number of bits of the second precision type is N2, N2 is a positive integer, and the binary representation of the first sub-element is the first N2 bits of the binary representation of the parameter element.

[0083] For example, if the total number of bits in the first precision type is N1, where N1 is a positive integer, and element p 1ij If p is the first child element, then the first child element is p. 1ij It can be obtained through the following formula:

[0084] p 1ij =p ij &A (Formula 1)

[0085] In this context, "&" represents AND (i.e., bitwise AND). A = "0b111...100...0" indicates that it is a binary representation, consisting of N2 consecutive 1s in the high-order bits and N1-N2 consecutive 0s in the low-order bits. The total number of bits in A is N1.

[0086] For example, the first N2 bits of a parameter element include one sign bit, c exponent bits indicating the exponent of the parameter element, and d mantissa significant bits in the most significant bits, i.e., the most significant bits, where N2 = 1 + c + d. For instance, extracting the first N2 bits of the binary representation of the parameter element as the first sub-element, since the first and second precision types have the same number of exponent bits, the sign bit of the first sub-element is the same as the sign bit of the parameter element, the exponent of the first sub-element is the same as the exponent of the parameter element (i.e., the c bits of the exponent part of the first sub-element are the same as the c bits of the exponent part of the parameter element), and the mantissa of the first sub-element is the same as the most significant bits in the most significant bits of the mantissa of the parameter element, i.e., the d mantissa significant bits of the first sub-element are the same as the most significant bits in the most significant bits of the most significant bits of the parameter element.

[0087] For example, the mantissa significant bits of a parameter element, excluding the first few significant bits, are divided into M-1 consecutive segments. Each of these M-1 sub-elements corresponds to one of these M-1 segments, where the number of significant bits in each segment is less than or equal to the number of significant mantissa bits in the corresponding sub-element. In other words, the other mantissa significant bits of the parameter element are segmented according to the number of significant mantissa bits for at least two precision types, and each segment is used as the mantissa portion of its corresponding sub-element. Furthermore, in the embodiments of this disclosure, the sum of the number of significant mantissa bits in the M sub-elements must be greater than or equal to the number of significant mantissa bits in the parameter element.

[0088] For example, determining the other M-1 child elements among M child elements (excluding the first child element) can include: determining that the mantissa of each of the other M-1 child elements is divided into M-1 segments; determining that the exponent of each of the other M-1 child elements is P-Qi, where P is the exponent of the parameter element, Qi is the difference in number of bits between the highest bit of the segment corresponding to the child element and the highest bit of the mantissa of the parameter element, and P and Qi are integers.

[0089] For example, the number of significant digits in the mantissa of the first precision type is F1, the M sub-elements correspond to at least two precision types including the second precision type and the third precision type, the number of significant digits in the mantissa of the second precision type is F2, the number of significant digits in the mantissa of the third precision type is F3, F1, F2 and F3 are positive integers, and the other M-1 sub-elements include the second sub-element, the precision type of the second sub-element is the third precision type.

[0090] For example, determining that the mantissa of each of the other M-1 sub-elements is divided into M-1 segments can include: in response to F1-F2 being less than or equal to F3, determining that M-1 is 1, and determining that the high F1-F2 bits of the mantissa of the binary representation of the second sub-element are the same as the F1-F2-1 bits to the 0th bit of the mantissa of the binary representation of the parameter element; in response to F1-F2 being greater than F3, determining that the mantissa of the binary representation of the second sub-element is the same as the F1-F2-F3-1 bits to the F1-F2-1th bit of the mantissa of the binary representation of the parameter element.

[0091] For example, in some embodiments, in response to F1-F2 being greater than F3, F1 is equal to the sum of the number of mantissa significant bits in the precision types of the M sub-elements. In this case, the number of significant bits in each segment is equal to the number of mantissa significant bits in the corresponding sub-element. For example, in a specific example, the data format of the parameter element is as follows: Figure 2AAs shown, it includes 1 sign bit, c exponent bits, and 32 mantissa significant bits, i.e., F1 = 32; M = 3. For the second precision type, the number of mantissa significant bits is F2 = 16, and for the third precision type, the number of mantissa significant bits is F3 = 8. At this time, Figure 2A The 16th to 31st bits are the most significant bits, including the segment 1 of the 8th to 15th bits of the mantissa as the mantissa of a sub-element of type 3 precision, and the segment 2 of the 0th to 7th bits of the mantissa as the mantissa of another sub-element of type 3 precision.

[0092] For example, in some other embodiments, F1-F2 is greater than F3, but F1 is not exactly equal to the sum of the number of mantissa significant bits in the precision types of the M sub-elements. In this case, one of the M-1 segments has a number of significant bits less than the number of mantissa significant bits in the corresponding sub-element. For example, in a specific example, such as Figure 2B As shown, M = 3, and F1 = 24, F2 = 11, F3 = 8. At this time, Figure 2B The 13th to 23rd bits are the most significant bits, including the segment 1 of the 2nd to 12th bits of the mantissa as the mantissa of the sub-element of type second precision, and the segment 2 of the 1st to 0th bits of the mantissa as the F3-1st to F3-2nd bits of the mantissa of the sub-element of type third precision.

[0093] For example, when F1-F2 is greater than F3, M is greater than 2, that is, M sub-elements may also include third sub-elements, fourth sub-elements, etc. The third sub-elements, etc., can be of the second precision type or the third precision type. Of course, they can also be other precision types lower than the first precision type that are different from the second precision type and the third precision type. This disclosure does not impose specific limitations on this.

[0094] For example, in some embodiments, if F1-F2 is less than or equal to F3, then M = 2, meaning the M sub-elements include the first and second sub-elements. When F1-F2 = F3, the mantissa of the second sub-element is the same as the 0th to F3-1th bits of the parameter element. When F1-F2 is less than F3, the high F1-F2 bits (i.e., the F3-1th to F3-1-(F1-F2)th bits) of the mantissa of the second sub-element are the same as the F1-F2-1th to 0th bits of the parameter element. Simply put, in this case, the second sub-element is the difference between the parameter element and the first sub-element.

[0095] Since the child element actually represents a portion of the mantissa significant bits of the parameter element, its exponent also needs to be adjusted accordingly. For example, the exponent of the child element is P-Qi, where P is the exponent of the parameter element, and Qi is the difference in number of bits between the highest bit of the segment corresponding to the child element and the highest bit of the mantissa significant bits of the parameter element. For example, with... Figure 2A For example, the Qi of the sub-element corresponding to segment 1 is the difference in number of bits between the most significant bit (15th bit) of segment 1 and the most significant bit (31st bit) of the mantissa of the parameter element, which is 31-15=16. Therefore, the exponent of the sub-element corresponding to segment 1 is P-16. Similarly, the exponent of the sub-element corresponding to segment 2 is P-(31-7)=P-24. Typically, the processor will further adjust the exponent of the sub-element during calculation to ensure that the most significant bit of the mantissa is 1. If all mantissa bits are 0, the exponent is adjusted to 0.

[0096] Figure 3 This is a schematic diagram illustrating the relationship between parameter elements and sub-elements provided in at least one embodiment of this disclosure. The following is in conjunction with... Figure 3 This section provides a detailed explanation of the process for determining child elements.

[0097] like Figure 3 As shown, parameter element p ij Including N1 bits, where N1 = 1 + c + F1, i.e., parameter element p ij It includes 1 sign bit, c exponent bits, and F1 mantissa significant bits. For example, as... Figure 3 As shown, the sign bit is the most significant bit of the parameter element, such as the (N1-1)th bit, and the last significant bit of the mantissa is the least significant bit of the parameter element, such as the 0th bit. Of course, Figure 3 The illustration shown is for illustrative purposes only. The position of the mantissa, exponent, and sign bit in the parameter elements can be adjusted accordingly, and this disclosure does not impose any restrictions on this.

[0098] like Figure 3 As shown, parameter element p ij It can be divided into M child elements, including the first child element p. 1ij Second child element p 2ij Third child element p 3ij ...etc. Of course, more sub-elements may be included as needed, or only the first and second sub-elements (M=2) may be included. The embodiments disclosed herein do not impose specific limitations on this.

[0099] For example, the first child element p 1ij The binary representation of the parameter element p ij The first N2 bits of the binary representation of the parameter element p ij The sign bit is used as the first child element p 1ij The sign bit of the parameter element pij The exponent part of the first child element is used as the exponent part of the parameter element p. ij The F2 most significant mantissa bits closest to the most significant bit are taken as the first child element p. 1ij The last digit.

[0100] For example, except for the first child element p ij The other M-1 sub-elements are used to represent the difference between the parameter element and the first sub-element.

[0101] For example, the parameter element p ij The F1-F2 mantissa significant bits, excluding the most significant bits, are divided into M-1 segments. Each M-1 segment corresponds one-to-one with the M-1 sub-elements. Furthermore, the number of significant bits in each segment is less than or equal to the number of mantissa significant bits in the corresponding sub-element.

[0102] For example, such as Figure 3 As shown, F1-F2 is greater than F3, so the parameter element p... ij In the binary representation of the mantissa, bits F1-F2-F3-1 to F1-F2-1 are taken as segment 1, and segment 1 is taken as the second child element p. 2ij The mantissa, that is, the mantissa part of the second sub-element, is the same as the F1-F2-F3-1 to F1-F2-1 bits of the mantissa part of the binary representation of the parameter element.

[0103] For example, such as Figure 3 As shown, the M child elements also include a third child element p of type fourth precision. 3ij The fourth precision type has F4 significant digits in its mantissa, where F4 is a positive integer. Assuming F1 - F2 - F3 = F4, then the parameter element p... ij In the binary representation of p, bits F4-1 to F0 are used as segment 2, and segment 2 is used as the third child element p. 3ij The last digit, that is, the third child element p. 3ij The mantissa part and parameter element p ij In the binary representation, bits F4-1 through F0 are the same in the mantissa.

[0104] Similar to the preceding content, the exponent of each sub-element is also adaptively adjusted. For example, for the second sub-element in Figure 2, its Qi = F2, therefore the exponent of the second sub-element should be P - F2. For example, for Figure 3 The third child element has a Qi = F3 + F2, therefore the exponent of the second child element should be P - F2 - F3. For example, for Figure 3 The fourth sub-element in the equation has a value of Qi = F3 + F2 + F4, therefore the exponent of the second sub-element should be P - F2 - F3 - F4.

[0105] For example, if expressed as a formula, when the parameter element p ij From the first child element p 1ij Second child element p 2ij and the third child element p 3ij When represented as a combination, the first child element p 1ij Second child element p 2ij and the third child element p 3ij The calculation formula is as follows:

[0106]

[0107] The meanings of the parameters “&” and “A” in Formula 2 are similar to those in Formula 1, and will not be repeated here.

[0108] For example, in a specific example, if the first precision type is FP32, the second precision type is BF24, and the third precision type is BF16, then F1 = 24, F2 = 16, F3 = 8, and thus F1 = F2 + F3. The parameter element can be represented by two sub-elements. The first sub-element type can be BF24, and the second sub-element type can be BF16. Of course, the first sub-element type can also be BF16, and the second sub-element type can be BF24.

[0109] For example, at this time the first child element p 1ij and the first child element p 2ij It can be calculated using the following formula 3:

[0110] p 1ij =p ij &0xFFFFFF00

[0111] p 2ij =p ij -p 1ij (Formula 3)

[0112] That is, the binary representation of the first sub-element is the first 24 bits of the binary representation of the parameter element, and the second sub-element is the difference between the parameter element and the first sub-element.

[0113] It should be noted that, depending on the type of low-precision floating-point number used and the type of first precision, a combination of multiple sub-elements of different low-precision floating-point number types can be used to represent the input tensor. This disclosure does not impose specific restrictions on the types of at least two different precision types used and the number M of input sub-tensors M that can be combined to represent the input tensor, and can be set according to actual needs.

[0114] In the data processing method provided in the above embodiments of this disclosure, multiple low-precision floating-point numbers are mixed to simulate high-precision floating-point numbers, thereby improving the applicability of computational processing and enabling processors, chips, etc., that apply this data processing method to have a wider range of applicable scenarios; and it can make full use of the processing performance of the low-precision floating-point numbers originally provided by the data processing device to improve the computing power and computational efficiency of simulating high-precision data formats.

[0115] For example, matrix multiplication and convolution are two common computational operations in tensor operations. After the parameter elements are split into multiple sub-elements, the exponent of the product of some sub-elements is very small. Removing the multiplication of these sub-elements not only has no impact on the calculation result, but also reduces the number of multiplications, thereby improving computational power and processing efficiency.

[0116] Figure 4A This is a schematic flowchart illustrating step S30 provided for at least one embodiment of the present disclosure.

[0117] like Figure 4A As shown, step S30 may include steps S301-S304.

[0118] In step S301, according to the calculation process, each input tensor in the calculation process is replaced with the sum of M input sub-tensors that represent the input tensor, and expanded to obtain L first intermediate results.

[0119] For example, each first intermediate result is represented as the product or convolution of two input subtensors, where L is a positive integer greater than 1.

[0120] In step S302, determine the L indices corresponding to the L first intermediate results and the largest index among the L indices.

[0121] In step S303, based on L indices, at least one first intermediate result is selected from the L first intermediate results, wherein the absolute value of the difference between the index of at least one first intermediate result and the maximum index is less than or equal to F1.

[0122] In step S304, the sum of at least one first intermediate result is used as the calculation result.

[0123] For example, the following describes the specific execution process of steps S301-S304, taking input tensors including input tensor A and input tensor B as an example. Of course, this disclosure is not limited to this; similar calculations of multiplication and convolution of more input tensors can also be performed.

[0124] For matrix multiplication or convolution operations, the calculation result is C = A × B, where × indicates that depending on the dimension of the input tensor, it can represent matrix multiplication or convolution multiplication. This disclosure does not impose any specific restrictions on this.

[0125] For example, according to step S20, M input sub-tensors A1, ..., A2 that represent the input tensor A can be obtained. M And the combination of M input subtensors B1, ..., B1, representing the input tensor B. M The specific process is as described in step S20, and will not be repeated here.

[0126] First, in step S301, the input tensor A in the calculation process A×B is replaced with A1+...+A M The input tensor B is replaced with B1+...+B M That is, C = A × B = (A1 + ... + A M )×(B1+...+B M Expanding this, we get C = A1 × B1 + ... + A1 × B M +..+A M ×B1+..+A M ×B M There are a total of L first intermediate results, for example, A1×B1, A1×B M A M ×B1、A M ×B M All of these are first intermediate results, and when expanded, they are summed into L first intermediate results.

[0127] Then, in step S302, calculate the L indices corresponding to the L first intermediate results and the largest index among the L indices.

[0128] For example, if A1 is an input subtensor composed of the first sub-elements corresponding to the parameter elements in input tensor A, and B1 is an input subtensor composed of the first sub-elements corresponding to the parameter elements in input tensor B, then the exponent corresponding to the first intermediate result A1×B1 is the maximum exponent.

[0129] It should be noted that, since this disclosure is concerned with the relative magnitude relationship between exponents, the exponents can be described from the perspective of elements. For example, let g be the exponent of any parameter element in input tensor A, and h be the exponent of any parameter element in input tensor B, then the exponent of A1×B1 can be expressed as g+h. The concepts below are similar and will not be repeated.

[0130] Then, in step S303, from A1×B1, ..., A1×B M A M ×B1、...、A M ×B M Select one or more first intermediate results whose absolute value of the difference between the exponent and the maximum exponent is less than or equal to F1.

[0131] For example, if the first intermediate result is selected where the absolute value of the difference between the exponent and the maximum exponent is less than or equal to F1, the calculation results obtained based on these first intermediate results will have virtually no loss of accuracy, and the potential impact on accuracy will be limited to 2. -F1 Within the corresponding decimal range, the difference from the actual calculation result is negligible. Even if the absolute value of the difference between the exponent and the maximum exponent is ignored and equal to the first intermediate result of F1, it will not affect the accuracy of the final result.

[0132] Finally, in step S304, the sum of these selected first intermediate results is calculated as the result C.

[0133] Figure 4B This is a schematic diagram of a mixed-precision simulation process provided in at least one embodiment of the present disclosure.

[0134] like Figure 4B As shown, in a specific example, if M = 2, and the first precision type is FP32, the second precision type is BF24, and the third precision type is BF16, referring to the content described above, after step S20, the two input sub-tensors A1 and A2 corresponding to the input tensor A can be obtained. For example, if the input subtensor A1 is of second precision type, the input subtensor... For the third precision type, A1 = A & 0xFFFFFF00. Similarly, we can also obtain the two input sub-tensors B1 and B2 corresponding to the input tensor B.

[0135] Through step S301, four first intermediate results are calculated, namely A1×B1,

[0136] Through step S302, the indices of the four first intermediate results are obtained, as shown in Table 2:

[0137] Table 2 Index of the first intermediate result

[0138]

[0139] For example, referring to the description of step S20, The exponent is h-16, therefore The exponent is g+h-16; similarly, The index is g-16. The exponent is h-16, therefore The exponent is g+h-32.

[0140] In step S303, the first intermediate result A1×B1 is selected where the absolute value of the difference between the exponent and the maximum exponent g+h is less than or equal to 24.

[0141] Finally, in step S304, calculate Thus, the calculation result C is obtained.

[0142] Therefore, in the data processing method provided in at least one embodiment of this disclosure, the precision of FP32 multiplication can be simulated by using one BF24 multiplication and two mixed-precision BF24 and BF16 multiplications, thereby enabling the execution of FP32 multiplication calculations and expanding the applicable scenarios of computational processing. Furthermore, by effectively utilizing the powerful computing power of the originally provided low-precision floating-point numbers BF16 and BF24, the overall computation time of computational processing is not increased, and the overall efficiency of computation is significantly improved, with low resource consumption, high computing power, and good performance.

[0143] In some data processing devices, such as in a Tensor core, the computational cost of mixed-precision multiplication of BF24 and BF16 is between that of multiplication of BF16 and multiplication of BF24.

[0144] Therefore, to further improve computational efficiency, the input subtensor of type second precision in the first intermediate result can be further decomposed into multiple intermediate subtensors of type third precision. For example, the input subtensor of type second precision in mixed-precision multiplication can be further decomposed into multiple intermediate subtensors of type third precision. In this case, the precision of the second precision type is higher than that of the third precision type, that is, the number of significant mantissa bits of the second precision type is greater than that of the third precision type. Of course, if the precision of the second precision type is lower than that of the third precision type, that is, the number of significant mantissa bits of the second precision type is less than that of the third precision type, then the input subtensor of type third precision in the first intermediate result can be further decomposed into multiple intermediate subtensors of type second precision. This disclosure does not impose specific limitations on this.

[0145] Figure 5A This is a schematic flowchart illustrating step S30 provided for at least one embodiment of the present disclosure.

[0146] like Figure 5A As shown, in some other embodiments, step S30 includes steps S305-S309.

[0147] In step S305, according to the calculation process, each input tensor in the calculation process is replaced with the sum of M input sub-tensors that represent the input tensor, and expanded to obtain L first intermediate results.

[0148] For example, each first intermediate result is represented as the product or convolution of two input subtensors.

[0149] In step S306, for each input sub-tensor of type 2 precision among the L first intermediate results, the input sub-tensor of type 2 precision is represented by a combination of W intermediate sub-tensors of type 3 precision to obtain U second intermediate results, where L and U are positive integers.

[0150] In step S307, determine the U indices corresponding to the U second intermediate results and the largest index among the U indices.

[0151] In step S308, based on U indices, at least one second intermediate result is selected from U second intermediate results, wherein the absolute value of the difference between the index of at least one second intermediate result and the maximum index is less than or equal to F1.

[0152] In step S309, the sum of at least one second intermediate result is used as the calculation result.

[0153] For example, in some embodiments, the partial input subtensors of type second precision in the L first intermediate results can be replaced with the sum of W input subtensors of type third precision.

[0154] At this point, step S306 may include: determining the L exponents corresponding to the L first intermediate results; selecting the maximum value from the L exponents to determine the L-1 first intermediate results other than the first intermediate result corresponding to the maximum value among the L first intermediate results; for the V first intermediate results among the L-1 first intermediate results that include input sub-tensors of type second precision, replacing the input sub-tensors of type second precision in each of the V first intermediate results with the summation of W intermediate sub-tensors of type third precision, expanding to obtain the W third intermediate results corresponding to each of the V first intermediate results, where the third intermediate results are represented as the product or convolution of the input sub-tensors and intermediate sub-tensors of type third precision, and V is a positive integer; taking all the third intermediate results corresponding to the V first intermediate results, the first intermediate result corresponding to the maximum value, and the other L-1-V first intermediate results other than the V first intermediate results among the L-1 first intermediate results as U second intermediate results.

[0155] For example, in some other embodiments, the input subtensors of type second precision in the L first intermediate results can be replaced with the sum of W input subtensors of type third precision.

[0156] For example, the following example, taking input tensors A and B as input tensors, illustrates the specific execution process of steps S305-S309 when calculating A×B.

[0157] First, L first intermediate results A1×B1 and A1×B1 are obtained through step S305. M A M ×B1、A M ×B M For details, please refer to step S301. Repeated steps will not be repeated here.

[0158] Then, in step S306, the input subtensors of type second precision in the L-1 first intermediate results (excluding A1×B1, which has the maximum exponent) are replaced by the sum of W intermediate subtensors. For example, assuming A1 and B1 are of type second precision and the other M-1 input subtensors are of type third precision, then referring to the process in step S20, W intermediate subtensors A'1, ..., A'' representing A1 are obtained. w And the combination of W intermediate subtensors B'1, ..., B' of B1. w A'1、...、A' w and B'1、...、B' w All are of third precision type. For the L-1 first intermediate results (excluding A1×B1) that include both A1 and B1, V first intermediate results are used to represent A1 in these first intermediate results using A'1+...+A'. w Replace B1 with B'1+...+B' w Substitution and expansion yield the third intermediate result corresponding to each of the V first intermediate results. Let A1×B M For example, replace A1 with A'1+...+A' w Therefore, A1×B M = (A'1+...+A') w )×B M =A'1×B M +...+A' w ×B M Thus, the first intermediate result A1×B is obtained. M The corresponding W third intermediate results A'1×B M ... A' w ×B M .

[0159] The first intermediate result A1×B1, the V third intermediate results corresponding to the V first intermediate results including A1 and B1, and the L-1-V first intermediate results that do not include A1 or B1 among the L intermediate results are taken as U second intermediate results, where U = V*W + LV.

[0160] Next, in step S307, the U exponents corresponding to the U second intermediate results and the largest exponent among them are determined. The specific process can be referred to in step S302, and will not be repeated here.

[0161] Then, in step S308, at least one second intermediate result is selected from the U second intermediate results whose absolute value of the difference between the exponent and the maximum exponent is less than or equal to F1.

[0162] Finally, in step S309, the sum of the at least one second intermediate result is calculated as the final calculation result.

[0163] Therefore, compared to the calculation results obtained in steps S301-S304, the calculation process in steps S305-S309 removes mixed-precision multiplication and replaces all the original mixed-precision multiplications with multiplications of the same precision type. This further improves computing power, increases computational efficiency, and reduces resource consumption without affecting precision. Furthermore, since the two second-precision type multiplication operations corresponding to the largest exponent are retained, and the second-precision type input subtensors in other first intermediate results are replaced with the sum of third-precision type intermediate subtensors, W*W third-precision type multiplication operations can be completed using two second-precision type multiplication operations, thus reducing the number of multiplication operations. Compared to replacing all input subtensors with the sum of third-precision type intermediate subtensors, this further improves computing power, increases computational efficiency, and reduces resource consumption.

[0164] Figure 5B This is a schematic diagram of the processing procedure for mixed-precision simulation provided in at least one embodiment of the present disclosure.

[0165] like Figure 5B As shown, in a specific example, if the first precision type is FP32, the second precision type is BF24, and the third precision type is BF16, referring to the previously described content, after step S20, the two input sub-tensors A1 and A2 corresponding to the input tensor A can be obtained. For example, if the input subtensor A1 is of second precision type, the input subtensor... This is the third precision type. Similarly, we can also obtain the two input sub-tensors B1 and B2 corresponding to the input tensor B.

[0166] Through step S305, four first intermediate results are calculated, namely A1×B1,

[0167] Step S306 yields the exponents of the four first intermediate results, as shown in Table 2. Referring to step S20, two intermediate subtensors of type BF16 representing the combined representation A1 are obtained. and in, Similarly, we obtain two intermediate subtensors of type BF16 representing the combination B1. and Next, A1 in Replacement, B1 Substitution and expansion yield the following formula 4:

[0168]

[0169] This yields six second intermediate results, namely A1B1, and

[0170] Next, in step S306, the indices of the six second intermediate results are determined, as shown in Table 3:

[0171] Table 3 Index of the second intermediate results

[0172]

[0173] For example, referring to the description of step S20, The exponent is g. The exponent is h-16, therefore The exponent is g+h-16; similarly, The index is g-16. The exponent is h-8, therefore The exponent is g+h-24.

[0174] Next, in step S308, at least one second intermediate result is selected from the six second intermediate results whose absolute value of the difference between the exponent and the maximum exponent is less than or equal to F1. For example, as mentioned earlier, since the number of significant bits in the mantissa of FP32 is 24, second intermediate results whose exponent is less than or equal to g+h-24 have virtually no impact on the final calculation result, thus A1×B1 can be selected. For the final calculation, it can ensure accuracy while further reducing the number of multiplications, achieving a balance between calculation accuracy and computing power.

[0175] Finally, in step S309, calculate Thus, the final calculation result C is obtained.

[0176] Therefore, in the data processing method provided in at least one embodiment of this disclosure, the accuracy of FP32 multiplication can be simulated using one BF24 multiplication and two BF16 multiplications, which further reduces resource consumption, increases computing power, and improves performance.

[0177] For example, in some chips or processors, the computational power consumption of BF24 multiplication is twice that of BF16, meaning that performing one BF24 multiplication is equivalent to performing two BF16 multiplications. If A1 and B1 in the first intermediate result are both split into the sum of two intermediate subtensors of BF16, then six BF16 multiplications are ultimately required. However, in this disclosure, one BF24 multiplication replaces four BF16 multiplications. Even though the computational power consumption of BF24 multiplication is twice that of BF16, the processing method provided by this disclosure reduces the resource consumption of computational processing, resulting in higher efficiency, better performance, and stronger theoretical computing power.

[0178] At least one embodiment of this disclosure also provides a data processing method; Figure 6 This is a flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0179] like Figure 6 As shown, the data processing method includes at least steps S40-S60.

[0180] In step S40, first data is received. For example, the first data is of a first precision type.

[0181] In step S50, the first data is represented by a combination of M sub-data.

[0182] For example, the first data is the sum of M sub-data.

[0183] For example, M data have at least two different precision types, both of which are different from the first precision type, M is an integer greater than 1, the first precision type and the at least two precision types are both floating-point types, the number of bits of the exponent in the first precision type is the same as the number of bits of the exponent in the at least two precision types, and the precision of the first precision type is higher than the precision of any one of the at least two precision types.

[0184] For the specific execution process of step S50, please refer to the relevant description of step S20 above. Repeated parts will not be repeated here.

[0185] In step S60, the combination of M sub-data is used to replace the first data for subsequent processing.

[0186] For example, the subsequent processing may include the aforementioned calculation processing, or it may include any other processing required during the use of the first data. This disclosure does not limit the specific operations of the "subsequent processing".

[0187] In the data processing method provided in at least one embodiment of the present disclosure, multiple low-precision sub-data of mixed precision type are used to simulate high-precision first data, thereby enabling processors and the like using the data processing method to support calculations of high-precision data formats that may not have been supported originally. This expands the applicable scenarios of computational processing, improves the applicability of processors and the like using the data processing method, and effectively utilizes the powerful computing power of the low-precision floating-point numbers provided. It does not increase the overall computation time of computational processing and can also significantly improve the overall efficiency of computation.

[0188] At least one embodiment of this disclosure also provides a data processing apparatus. Figure 7A This is a schematic block diagram of a data processing apparatus provided for at least one embodiment of the present disclosure.

[0189] like Figure 7A As shown, the data processing device 100 includes an acquisition module 101, a first processing module 102, and a second processing module 103.

[0190] For example, the acquisition module 101 is configured to acquire multiple input tensors as input parameters for computation processing, wherein the multiple input tensors are all of the first precision type;

[0191] For example, the first processing module 102 is configured to represent each input tensor using a combination of M input sub-tensors, wherein the M input sub-tensors have at least two different precision types, both of which are different from the first precision type, and M is an integer greater than 1;

[0192] For example, the second processing module 103 is configured to replace the input tensor with M input sub-tensors that represent the input tensor in combination for each input tensor, and perform the calculation process to obtain the calculation result.

[0193] For example, the first precision type and the at least two precision types are both floating-point types, the number of bits of the exponent in the first precision type is the same as the number of bits of the exponent in the at least two precision types, and the precision of the first precision type is higher than the precision of any one of the at least two precision types.

[0194] For example, the calculation results can be directly output from the data processing device 100 and transmitted to other components that need to use the calculation results, such as storage devices or other computing devices.

[0195] For example, the acquisition module 101, the first processing module 102, and the second processing module 103 include code and programs stored in memory. The acquisition module 101, the first processing module 102, and the second processing module 103 are implemented, for example, as a central processing unit (CPU) or other processing units with data processing capabilities and / or instruction execution capabilities. The processing unit can be a general-purpose processor, and also a microcontroller, microprocessor, digital signal processor, dedicated image processing chip, or field-programmable logic array, etc. The acquisition module 101, the first processing module 102, and the second processing module 103 execute the code and programs to implement some or all of the functions of the acquisition module 101, the first processing module 102, and the second processing module 103 as described above. For example, the acquisition module 101, the first processing module 102, and the second processing module 103 can be a single circuit board or a combination of multiple circuit boards for implementing the functions described above. In the embodiments of this application, the single circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) processor-executable firmware stored in memory.

[0196] It should be noted that the acquisition module 101 can be used to implement Figure 1 In step S10 shown, the first processing module 102 can be used to implement... Figure 1 As shown in step S20, the second processing module 103 can be used to implement... Figure 1 The step S30 is shown. Therefore, for a detailed description of the functions that the acquisition module 101, the first processing module 102, and the second processing module 103 can be achieved, please refer to the relevant descriptions of steps S10 to S30 in the embodiments of the above data processing method; repeated details will not be repeated here. Furthermore, the data processing device 100 can achieve similar technical effects to the aforementioned data processing method, which will not be described further here.

[0197] It should be noted that in at least one embodiment of this disclosure, the data processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be constructed from analog devices, digital chips, or other suitable methods according to circuit principles.

[0198] For example, in some embodiments, the data processing device 100 may be a tensor core. Of course, the data processing device 100 may also be implemented as other chips, processors, etc. that need to perform computational processing, including but not limited to image processing units (GPUs), data processing units (DPUs), tensor processing units (TPUs), neural network processing units (NPUs), AI accelerators, etc. This disclosure does not impose specific limitations in this regard.

[0199] At least one embodiment of this disclosure also provides a processor. Figure 7B This is a schematic block diagram of a data processor provided for at least one embodiment of the present disclosure.

[0200] like Figure 7B As shown, the processor 200 includes a data processing device 201 as described in any embodiment of this disclosure. The structure, function, and technical effects of the data processing device 201 are described above with reference to the data device 100, and will not be repeated here.

[0201] For example, processor 200 may also include storage device 202, which is configured to input multiple input tensors into acquisition module 101.

[0202] For example, storage device 203 is also configured to receive and store calculation results.

[0203] For example, storage device 203 may include storage devices of any structure capable of storing data, such as memory and cache.

[0204] Of course, depending on actual needs, the processor 200 may also include other components for performing subsequent processing of the calculation results, and this disclosure does not impose specific limitations on this.

[0205] For example, the processor 200 can be implemented as a single-chip package (e.g., a SOC chip), a multi-chip package (e.g., a Chiplet), etc., depending on actual needs, and this disclosure does not limit it.

[0206] For example, in one embodiment, the data processing 200 can be a GPU, and the data processing device 201 can be a tensor core.

[0207] At least one embodiment of this disclosure also provides a data processing method. For example, the data processing method includes: receiving a data calculation instruction, wherein the data calculation instruction includes a plurality of input tensors as calculation input parameters; parsing the data calculation instruction and then executing the data calculation instruction using a data processing unit.

[0208] For example, executing data calculation instructions using a data processing unit includes: acquiring multiple input tensors as input parameters for calculation processing, wherein all input tensors are of a first precision type; for each input tensor, representing the input tensor using a combination of M input sub-tensors, wherein the M input sub-tensors have at least two different precision types, and at least two precision types are different from the first precision type, and M is an integer greater than 1; for each input tensor, replacing the input tensor with the M input sub-tensors that represent the input tensor in combination, performing calculation processing, and obtaining a calculation result.

[0209] For example, the first precision type and the at least two precision types are both floating-point types, the number of exponent bits in the first precision type is the same as the number of exponent bits in the at least two precision types, and the precision of the first precision type is higher than the precision of any one of the at least two precision types.

[0210] For example, the data processing method provided in at least one embodiment of this disclosure can be applied to... Figure 7A The processor 200 shown.

[0211] For example, in at least one embodiment of the data processing method provided in this disclosure, a data calculation instruction is provided, which includes multiple tensors as input parameters for the calculation process. For example, after receiving the data calculation instruction, the processor parses the data calculation instruction, such as decoding the data calculation instruction, generates microinstructions, and sends the microinstructions to the instruction allocation unit; the instruction allocation unit sends the microinstructions to the corresponding scheduling queue according to the microinstruction category; in response to the microinstructions, after multiple input tensors (all or the required portion) are prepared, the data processing unit reads the data and executes the relevant operations of the data calculation instruction.

[0212] For the specific process of using the data processing unit to execute data calculation instructions, please refer to steps S10-S30 in the data processing method described above. Repeated steps will not be repeated here.

[0213] Figure 8 This is a schematic structural diagram of a processor provided for at least one embodiment of the present disclosure. For example... Figure 8 As shown, the processor 300 includes an instruction parsing unit 301 and a data processing unit 302.

[0214] For example, the instruction parsing unit 301 is used to receive and parse data calculation instructions, wherein the data calculation instructions include multiple input tensors as calculation input parameters.

[0215] For example, after the instruction parsing unit parses the data calculation instruction, the data processing unit 302 executes the data processing method according to any embodiment of the present disclosure.

[0216] Specifically, when processor-based upper-layer software (such as AI applications, HPC applications, and scientific computing applications) sends data computation instructions for computation processing to the processor (e.g., CPU or GPU) through a unified encapsulated function library, these instructions can carry input tensors. When the processor receives the data computation instructions, the instruction parsing unit 301 parses the instructions to obtain the input tensors. The processor then schedules the data processing unit to execute the computation task on the input tensors. For example, after parsing the data computation instructions, the processor can store the input tensors in the instructions into registers or memory. This allows the data processing unit to retrieve multiple input tensors as computation input parameters from the registers or memory when performing computation.

[0217] For the specific process of using the data processing unit to execute data calculation instructions, please refer to steps S10-S30 in the data processing method described above. Repeated steps will not be repeated here.

[0218] Figure 9 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. For example, such as Figure 9 As shown, storage medium 400 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 401 can be stored non-transitory. For example, when the computer-readable instructions 401 are executed by a processor, one or more steps in the data processing method described above can be performed.

[0219] For example, the storage medium 400 can be applied to the processor 200 described above. For example, the storage medium 400 may include the storage device 202 in the processor 200.

[0220] For example, a storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and a processor may execute the computer-readable instructions to implement various functions of the processor. Various application programs and various data may also be stored in the storage medium.

[0221] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0222] Figure 10 This is a schematic block diagram of an electronic device provided according to an embodiment of the present disclosure. Figure 10 As shown, the electronic device 500 is, for example, suitable for implementing the data processing method provided in the embodiments of this disclosure. It should be noted that... Figure 10 The components of the electronic device 500 shown are merely exemplary and not limiting. The electronic device 500 may have other components as needed for the actual application.

[0223] like Figure 10 As shown, electronic device 500 may include processing device (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions.

[0224] For example, when the computer-readable instructions are executed by the processing device 501, one or more steps in the data processing described in any of the above embodiments can be performed. It should be noted that a detailed description of the data processing procedure can be found in the relevant descriptions in the above data processing embodiments, and repetitions will not be repeated.

[0225] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 503 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 508 into RAM 503 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 502, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0226] For example, the processing device 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0227] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, flash memory, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 10 An electronic device 500 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 500 may alternatively implement or possess more or fewer devices. For example, a processor 501 can control other components in the electronic device 500 to perform desired functions. The processor 501 may be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), a tensor processor (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) may be an x86, ARM, RISC-V architecture, etc. The GPU may be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0228] The following points should be noted regarding this disclosure:

[0229] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0230] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0231] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A processor comprising an instruction parsing unit, an instruction distribution unit and a data processing unit, The instruction analysis unit is configured to receive and analyze data computation instructions, wherein, the data computing instruction comprises a plurality of input tensors as input parameters of a computing process, each of the plurality of input tensors is of a first precision type, the instruction parsing unit, when receiving and parsing the data computing instruction, comprises decoding the data computing instruction, generating micro-instructions and sending the micro-instructions to the instruction distribution unit; the instruction distribution unit is configured to send the micro-instructions to corresponding scheduling queues according to categories of the micro-instructions; the data processing unit is configured to read the plurality of input tensors from registers or memories in response to the micro-instructions, and perform the following operations: for each input tensor, the input tensor is represented by a combination of M input sub-tensors, wherein the M input sub-tensors are of at least two precision types different from each other and from the first precision type, and M is an integer greater than 1; for each input tensor, the computing process is performed using the combination of the M input sub-tensors to replace the input tensor, to obtain a computing result; wherein the computing process comprises at least a convolution operation or a matrix multiplication operation, the number of significant digits of the first precision type is F1, and F1 is a positive integer, for each input tensor, the computing process is performed using the combination of the M input sub-tensors to replace the input tensor, to obtain a computing result, comprising: according to the computing process, each input tensor in the computing process is replaced by a sum form of the M input sub-tensors, to obtain L first intermediate results, wherein each first intermediate result is represented in the form of multiplication or convolution of two input sub-tensors, and L is a positive integer greater than 1; based on the L first intermediate results, a plurality of exponents and a target maximum exponent in the plurality of exponents are determined; based on the plurality of exponents, at least one target intermediate result is determined, wherein the absolute value of the difference between the exponent of the at least one target intermediate result and the target maximum exponent is less than or equal to F1; the sum of the at least one target intermediate result is taken as the computing result.

2. The processor of claim 1, wherein, for each input tensor, the M input sub-tensors have the same shape as the input tensor, for each input tensor, the input tensor is represented by the combination of the M input sub-tensors, comprising: for each parameter element in the input tensor, the parameter element is split into M sub-elements, wherein the M sub-elements are respectively elements in the M input sub-tensors that have the same position as the parameter element in the input tensor, and the parameter element is represented in the sum form of the M sub-elements.

3. The processor of claim 2, wherein, for each parameter element in the input tensor, the parameter element is split into M sub-elements, comprising: the exponent, the sign bit and the significant digits of a first sub-element in the M sub-elements are determined to be the same as those of the parameter element, and the significant digits of the first sub-element are determined to be the same as the leading significant digits of the significant digits of the parameter element. determining M-1 sub-elements other than the first sub-element in the M sub-elements, wherein a sum of the M-1 sub-elements is a difference between the parameter element and the first sub-element.

4. The processor of claim 3, wherein, the at least two precision types include a second precision type, a precision type of the first sub-element is the second precision type, a total number of bits of the second precision type is N2, N2 is a positive integer, a binary representation of the first sub-element is a first N2 bits of a binary representation of the parameter element.

5. The processor of claim 3, wherein, a part of significant bits other than the front part of the most significant bits in the significant bits of the mantissa of the parameter element is divided into M-1 continuous segments, the M-1 sub-elements correspond to the M-1 segments respectively, wherein a number of significant bits included in each segment is less than or equal to a number of significant bits of the mantissa of the sub-element corresponding to the segment, determining M-1 sub-elements other than the first sub-element in the M sub-elements, includes: determining the significant bits of the mantissa of each of the M-1 sub-elements as the M-1 segments respectively; determining an exponent of each of the M-1 sub-elements as P-Qi, wherein P is an exponent of the parameter element, Qi is a bit difference between the highest bit of the segment corresponding to the sub-element and the highest bit of the significant bits of the mantissa of the parameter element, P and Qi are integers.

6. The processor of claim 5, wherein, a number of significant bits of the first precision type is F1, the at least two precision types include a second precision type and a third precision type, a number of significant bits of the second precision type is F2, a number of significant bits of the third precision type is F3, F1, F2 and F3 are positive integers, the M-1 sub-elements include a second sub-element, a precision type of the second sub-element is the third precision type, wherein determining the significant bits of the mantissa of each of the M-1 sub-elements as the M-1 segments respectively, includes: in response to F1-F2 being less than or equal to F3, determining M-1 as 1, and determining that the F1-F2 most significant bits of the binary representation of the mantissa of the second sub-element are the same as the F1-F2-1th bit to the 0th bit of the binary representation of the mantissa of the parameter element; in response to F1-F2 being greater than F3, determining that the binary representation of the mantissa of the second sub-element is the same as the F1-F2-F3-1th bit to the F1-F2-1th bit of the binary representation of the mantissa of the parameter element.

7. The processor of claim 6, wherein, in response to F1-F2 being greater than F3, F1 is equal to a sum of the numbers of significant bits of the precision types of the M sub-elements respectively.

8. The processor of claim 1, wherein, determining a plurality of exponents and a target maximum exponent in the plurality of exponents based on the L first intermediate results includes: determining L exponents corresponding to the L first intermediate results respectively as the plurality of exponents, and determining a maximum exponent in the L exponents as the target maximum exponent; According to the plurality of indexes, determining at least one target intermediate result, comprising: according to the L indexes, selecting at least one first intermediate result from the L first intermediate results as the at least one target intermediate result, wherein the absolute value of the difference between the index of the at least one first intermediate result and the target maximum index is less than or equal to F1.

9. The processor of claim 1, wherein, The at least two precision types include a second precision type and a third precision type, Based on the L first intermediate results, determining a plurality of indexes and a target maximum index in the plurality of indexes, comprising: For each input sub-tensor in at least part of the input sub-tensors of the L first intermediate results of the second precision type, using a combination of W intermediate sub-tensors of the third precision type to represent the input sub-tensor of the second precision type, to obtain U second intermediate results, L and U are positive integers; Determine the U indexes corresponding to the U second intermediate results respectively as the plurality of indexes, and determine the maximum index in the U indexes as the target maximum index; According to the plurality of indexes, determining at least one target intermediate result, comprising: according to the U indexes, selecting at least one second intermediate result from the U second intermediate results as the at least one target intermediate result, wherein the absolute value of the difference between the index of the at least one second intermediate result and the target maximum index is less than or equal to F1.

10. The processor of claim 9, wherein, For each input sub-tensor in at least part of the input sub-tensors of the L first intermediate results of the second precision type, using a combination of W intermediate sub-tensors of the third precision type to represent the input sub-tensor of the second precision type, to obtain U second intermediate results, comprising: Determine the L indexes corresponding to the L first intermediate results respectively; Select the maximum value from the L indexes, and determine L-1 first intermediate results except the first intermediate result corresponding to the maximum value in the L first intermediate results; For V first intermediate results including input sub-tensors of the second precision type in the L-1 first intermediate results, use the sum form of the W intermediate sub-tensors to replace the input sub-tensors of the second precision type in each of the V first intermediate results, to obtain W third intermediate results corresponding to each of the V first intermediate results, wherein the third intermediate result is represented in the form of multiplication or convolution of input sub-tensors and intermediate sub-tensors of the third precision type, V is a positive integer; All third intermediate results corresponding to the V first intermediate results, the first intermediate result corresponding to the maximum value, and other L-1-V first intermediate results except the V first intermediate results in the L-1 first intermediate results are taken as the U second intermediate results.

11. The processor of any one of claims 1-10, wherein, The first precision type and the at least two precision types are both floating point types.

12. The processor of any one of claims 1-10, wherein, The number of index bits in the first precision type and the number of index bits in the at least two precision types are the same.

13. The processor of any one of claims 1-10, wherein, The precision of the first precision type is higher than the precision of any of the at least two precision types.

14. The processor of any one of claims 1-10, wherein, The first precision type is FP32, the at least two precision types include BF16 and BF24, the BF16 has 8 exponent bits, the BF16 has 8 significand significant bits, the BF24 has 8 exponent bits, and the BF24 has 16 significand significant bits, Each of the input tensors is represented by a combination of one input sub-tensor of BF16 type and one input sub-tensor of BF24 type.

15. A data processing method, comprising: receiving a data computation instruction, wherein the data computation instruction includes a plurality of input tensors as computation input parameters, parsing the data computation instruction and then executing the data computation instruction using a data processing unit, wherein executing the data computation instruction using the data processing unit comprises: obtaining the plurality of input tensors from memory or registers, wherein the plurality of input tensors are all of a first precision type; for each input tensor, representing the input tensor by a combination of M input sub-tensors, wherein the M input sub-tensors have at least two precision types, the at least two precision types are different from each other and different from the first precision type, and M is an integer greater than 1; for each of the input tensors, performing a computation process using the M input sub-tensors to replace the input tensor to obtain a computation result; wherein the computation process at least includes convolution operation or matrix multiplication operation, the first precision type has F1 significand significant bits, and F1 is a positive integer, for each of the input tensors, performing the computation process using the combination of the M input sub-tensors to replace the input tensor to obtain a computation result, comprising: According to the computation process, each input tensor in the computation process is replaced by the sum of the M input sub-tensors to obtain L first intermediate results, wherein each first intermediate result is represented in the form of multiplication or convolution of two input sub-tensors, and L is a positive integer greater than 1; Based on the L first intermediate results, determine a plurality of exponents and a target maximum exponent among the plurality of exponents; According to the plurality of exponents, determine at least one target intermediate result, wherein the absolute value of the difference between the exponent of the at least one target intermediate result and the target maximum exponent is less than or equal to F1; The sum of the at least one target intermediate result is taken as the computation result.

16. An electronic device, comprising: a memory that non-transiently stores computer executable instructions; a processor configured to run the computer executable instructions, wherein the computer executable instructions, when executed by the processor, implement the data processing method according to claim 15.

17. A non-transitory computer-readable storage medium, wherein, The non-transitory computer readable storage medium stores computer executable instructions, the computer executable instructions, when executed by the processor, implement the data processing method according to claim 15.

Images