A tensor computation method and related apparatus

Abstract

This application provides a tensor computation method and related apparatus. Through this method, a unified preprocessing is used to obtain the corresponding sign bit, exponent part, and mantissa part for tensors of different data types. Then, through subsequent general exponent accumulation, mantissa multiplication, comparison, shifting, and addition processing, the above unified processing logic can be adapted to the computational needs of different data tensors. Compared with the design idea of ​​equipping each data type with dedicated integrated circuits, this fundamentally eliminates circuit redundancy, thereby reducing chip area, reducing power consumption, and improving flexibility.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a tensor computation method and related apparatus. Background Technology

[0002] Tensor computing devices, as dedicated matrix accelerators in general-purpose computing on graphics processing units (GPGPUs), play a central role in modern artificial intelligence, deep learning, and high-performance computing. With the diversification of application scenarios, tensor computing devices need to support computations on various data types in practical applications.

[0003] In related technologies, tensor computing devices typically design independent application-specific integrated circuits (ASICs) for each data type to support multiple data types, with little or no interoperability or shared logic between the different data types. In other words, dedicated computing circuits are designed for different data types, and the ASICs corresponding to different data types are not interchangeable.

[0004] However, this design approach of equipping each data type with dedicated integrated circuits (DICs) presents several problems in practical applications: First, equipping each data type with a DIC leads to the duplication of data paths for computational units with the same function, significantly increasing chip area. Second, from a power efficiency perspective, there is a serious resource utilization problem; during computational tasks, typically only one or some data type circuits are actively operating, while DICs for other data types remain idle, resulting in significant power waste. Third, when a new data type is needed, the hardware architecture often requires redesigning and adding new DICs, leading to poor flexibility. Therefore, there is an urgent need to design a tensor computation method and related devices applicable to different data types. Summary of the Invention

[0005] This application provides at least one tensor computation method and related apparatus. For tensors of different data types, a unified preprocessing method is used to obtain the corresponding sign bit, exponent part, and mantissa part. Through subsequent general exponent accumulation, mantissa multiplication, comparison, shifting, and addition processing, the above unified processing logic can be adapted to the computational needs of different data tensors. Compared with the design idea of ​​equipping each data type with dedicated integrated circuits, this fundamentally eliminates circuit redundancy, thereby reducing chip area, reducing power consumption, and improving flexibility.

[0006] Firstly, this application provides a tensor computation method, which includes: Obtain M tensor sets, where each of the M tensor sets contains a first tensor and a second tensor, where M is an integer and M > 1; For the i-th tensor group among M tensor groups, determine the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group, where i is an integer and M≥i≥1; Based on the first tensor data type, the first tensor is preprocessed to obtain the first sign bit, the first tensor exponent, and the first tensor mantissa; based on the second tensor data type, the second tensor is preprocessed to obtain the second sign bit, the second tensor exponent, and the second tensor mantissa. Based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, the first tensor exponent and the second tensor exponent are accumulated to obtain the exponent accumulation result corresponding to the i-th tensor group; Based on the first tensor data type and the second tensor data type, the mantissa of the first tensor and the mantissa of the second tensor are multiplied by a dot to obtain the mantissa dot product result corresponding to the i-th tensor group; Compare the summation results of the M exponents corresponding to the M tensor sets one-to-one to determine the shift value; Referring to the first and second sign bits corresponding to the M tensor groups respectively, the M mantissa dot product results corresponding to the M tensor groups are shifted according to the shift value to obtain the M shifted mantissa dot product results. Based on the first and second tensor data types corresponding to the M tensor groups respectively, the M shift mantissa dot product results are added to obtain the tensor calculation results corresponding to the M tensor groups.

[0007] Secondly, this application also provides a tensor computing device, which includes: The acquisition unit is used to acquire M tensor groups, each of which includes a first tensor and a second tensor. M is an integer, M>1; The determining unit is used to determine, for the i-th tensor in the M tensor groups, the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group. The first tensor preprocessing unit is used to preprocess the first tensor based on the first tensor data type to obtain the first sign bit, the first tensor exponent, and the first tensor mantissa of the first tensor. The second tensor preprocessing unit is used to preprocess the second tensor based on the second tensor data type to obtain the second sign bit, the second tensor exponent, and the second tensor mantissa of the second tensor. A multi-head adder is used to accumulate the exponents of the first tensor and the second tensor based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, to obtain the exponent accumulation result corresponding to the i-th tensor group. A multi-head multiplier is used to perform dot multiplication on the mantissas of the first and second tensors based on the first and second tensor data types, to obtain the dot multiplication result of the mantissas of the i-th tensor group. A comparator is used to compare the summation results of the M exponents corresponding to the M tensor sets one-to-one to determine the shift value; A shifter is used to refer to the first and second sign bits of M tensor groups respectively, and according to the shift value, to shift the M mantissa dot product results of the M tensor groups one by one, so as to obtain M shifted mantissa dot product results. An addition tree is used to perform addition on the result of M shifted mantissa multiplications based on the first and second tensor data types of M tensor groups respectively, to obtain the tensor calculation result corresponding to the M tensor groups.

[0008] Thirdly, this application also provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the tensor calculation method provided in this application is executed.

[0009] Fourthly, this application also provides a computer-readable storage medium storing a computer program that is executed by a processor to perform the tensor computation method provided in this application.

[0010] Fifthly, this application also provides a computer program product, including a computer program that is executed by a processor to perform the tensor computation method provided in this application.

[0011] In summary, this application provides a tensor computation method and related apparatus. The method includes: acquiring M tensor groups, each of the M tensor groups including a first tensor and a second tensor, where M is an integer and M > 1; for the i-th tensor group in the M tensor groups, determining the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group; preprocessing the first tensor based on the first tensor data type to obtain the first tensor sign bit, the first tensor exponent, and the first tensor mantissa corresponding to the first tensor; preprocessing the second tensor based on the second tensor data type to obtain the second tensor sign bit, the second tensor exponent, and the second tensor mantissa corresponding to the second tensor; and preprocessing the second tensor based on the first tensor data type, the first tensor scaling factor, and the first tensor scaling factor. Using two tensor data types and a second tensor scaling factor, the exponents of the first and second tensors are accumulated to obtain the exponent accumulation result corresponding to the i-th tensor group. Based on the first and second tensor data types, the mantissas of the first and second tensors are multiplied by a dot, resulting in the mantissa dot product result corresponding to the i-th tensor group. The M exponent accumulation results corresponding to each of the M tensor groups are compared to determine the shift value. Referring to the first and second sign bits corresponding to the M tensor groups respectively, the M mantissa dot product results corresponding to the M tensor groups are shifted according to the shift value to obtain M shifted mantissa dot product results. Based on the first and second tensor data types corresponding to the M tensor groups respectively, the M shifted mantissa dot product results are added to obtain the tensor calculation result corresponding to the M tensor groups. Using the above method, a unified preprocessing is used to obtain the corresponding sign bit, exponent part, and mantissa part for tensors of different data types. Then, through subsequent general exponent accumulation, mantissa multiplication, comparison, shifting, and addition processing, the above unified processing logic can be adapted to the calculation requirements of different data tensors. Compared with the design idea of ​​equipping each data type with dedicated integrated circuits, this fundamentally eliminates circuit redundancy, thereby reducing chip area, reducing power consumption, and improving flexibility.

[0012] Other advantages of this application will be explained in more detail in conjunction with the following description and figures.

[0013] It should be understood that the above description is merely an overview of the technical solution of this application, so as to enable a general understanding of the technical means of this application and to implement it in accordance with the contents of the specification. In order to make the above and other objects, features and advantages of this application more apparent and understandable, specific embodiments of this application are illustrated below. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. The accompanying drawings are incorporated in and constitute a part of this specification. These drawings illustrate embodiments conforming to this application and are used together with the specification to explain the technical solutions of this application. It should be understood that the drawings only illustrate certain embodiments of this application and should not be considered as a limitation on the scope of protection. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. Furthermore, the same reference numerals denote the same components throughout the drawings. In the drawings: Figure 1 A flowchart of a tensor calculation method provided in this application embodiment; Figure 2 A flowchart illustrating the entire process of a tensor computation method provided in this application embodiment; Figure 3 A 4×4 dot product structure diagram provided for an embodiment of this application; Figure 4 A schematic diagram of a clock gating system provided in an embodiment of this application; Figure 5 This is a schematic diagram of a tensor computing device provided in an embodiment of this application. Detailed Implementation

[0015] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0016] In the description of embodiments of this application, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of the disclosed features, figures, steps, behaviors, components, portions or combinations thereof in this specification, and do not exclude the possibility of the presence of one or more other features, figures, steps, behaviors, components, portions or combinations thereof.

[0017] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A alone, A and B at the same time, and B alone.

[0018] The terms "first," "second," etc., are used only for ease of description to distinguish identical or similar technical features and should not be construed as indicating or implying the relative importance or number of these technical features. Therefore, a feature defined by "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, the term "multiple" means two or more.

[0019] In related technologies, tensor computing devices typically design independent application-specific integrated circuits (ASICs) for each data type to support multiple data types, with little or no interoperability or shared logic between the different data types. In other words, dedicated computing circuits are designed for different data types, and the ASICs corresponding to different data types are not interchangeable.

[0020] However, this design approach of equipping each data type with dedicated integrated circuits (DICs) presents several problems in practical applications: First, equipping each data type with a DIC leads to the duplication of data paths for computational units with the same function, significantly increasing chip area. Second, from a power efficiency perspective, there is a serious resource utilization problem; during computational tasks, typically only one or some data type circuits are actively operating, while DICs for other data types remain idle, resulting in significant power waste. Third, when a new data type is needed, the hardware architecture often requires redesigning and adding new DICs, leading to poor flexibility. Therefore, there is an urgent need to design a tensor computation method and related devices applicable to different data types.

[0021] In view of this, this application provides a tensor calculation method and related apparatus. Through the above method, a unified preprocessing is used to obtain the corresponding sign bit, exponent part and mantissa part for tensors of different data types. Then, through the subsequent general exponent accumulation, mantissa multiplication, comparison, shift and addition processing, the above unified processing logic can be adapted to the calculation requirements of different data tensors. Compared with the design idea of ​​equipping each data type with dedicated integrated circuits, this fundamentally eliminates circuit redundancy, thereby reducing chip area, reducing power consumption and improving flexibility.

[0022] The tensor computation method provided in this application can be implemented using a computer device, which can be a terminal device or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal devices include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this connection.

[0023] The tensor computation method provided in this application will be described below through method embodiments, such as... Figure 1 As shown, Figure 1 This application provides a flowchart of a tensor computation method, wherein the aforementioned computer device can be a server, and the method includes: S101. Obtain M tensor groups, each of which includes a first tensor and a second tensor.

[0024] Specifically, a tensor group refers to a group of tensors to be processed. In order to facilitate the subsequent pairwise processing of tensors, each tensor group includes a first tensor and a second tensor. The first tensor and the second tensor refer to the two initial tensors to be processed included in the tensor group. In this embodiment, the server can obtain M tensor groups, where M is an integer and M>1.

[0025] S102. For the i-th tensor group among the M tensor groups, determine the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group.

[0026] Specifically, we need to perform general processing on each of the M tensor groups. The following explanation will take the i-th tensor group among the M tensor groups as an example, where i is an integer, M≥i≥1, and the i-th tensor group is any tensor group among the M tensor groups.

[0027] Considering that the tensor computation method provided in this application is applied to different data types, for subsequent data processing, for the first tensor and the second tensor in the i-th tensor group, the server can determine the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group. Here, the first tensor data type is used to represent the specific data type of the first tensor, and the first tensor scaling factor is used to represent the scaling factor that needs to be scaled on the exponent part of the first tensor in subsequent processing. The second tensor data type is used to represent the specific data type of the second tensor, and the second tensor scaling factor is used to represent the scaling factor that needs to be scaled on the exponent part of the second tensor in subsequent processing.

[0028] like Figure 2 As shown, the scaling factor processing unit is used to determine and store the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor. The first tensor unit is used to store the first tensor, and the second tensor unit is used to store the second tensor.

[0029] S103. Based on the first tensor data type, preprocess the first tensor to obtain the first sign bit, the first tensor exponent, and the first tensor mantissa of the first tensor; based on the second tensor data type, preprocess the second tensor to obtain the second sign bit, the second tensor exponent, and the second tensor mantissa of the second tensor.

[0030] Specifically, to accommodate different data types, the server needs to employ a unified preprocessing method to obtain the corresponding sign bit, exponent, and mantissa. For the first tensor, based on its data type, preprocessing can be performed to obtain the first sign bit, exponent, and mantissa. For the second tensor, based on its data type, preprocessing can be performed to obtain the second sign bit, exponent, and mantissa.

[0031] Through the above preprocessing, tensors corresponding to different data types can be organized into general data for subsequent processing.

[0032] like Figure 2 As shown, the first tensor preprocessing unit can preprocess the first tensor according to the first tensor data type, and the second tensor preprocessing unit can preprocess the second tensor according to the second tensor data type.

[0033] S104. Based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, perform cumulative processing on the first tensor exponent and the second tensor exponent to obtain the exponent accumulation result corresponding to the i-th tensor group.

[0034] Specifically, after obtaining the first tensor exponent and the second tensor exponent in S103, the first tensor exponent and the second tensor exponent are accumulated based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, so as to obtain the exponent accumulation result corresponding to the i-th tensor group. Similarly, the server can obtain the exponent accumulation results corresponding to M tensor groups based on the above process.

[0035] like Figure 2 As shown, the multi-head adder can be used to perform exponential addition on the first tensor exponent and the second tensor exponent, respectively, in this module.

[0036] S105. Based on the first tensor data type and the second tensor data type, perform a dot product on the mantissa of the first tensor and the mantissa of the second tensor to obtain the dot product result of the mantissa of the i-th tensor group.

[0037] Specifically, after obtaining the mantissas of the first and second tensors in S103, the mantissas of the first and second tensors are multiplied by a dot based on the data types of the first and second tensors to obtain the mantissa dot product result corresponding to the i-th tensor group. Similarly, the server can obtain the mantissa dot product results corresponding to M tensor groups based on the above process.

[0038] like Figure 2 As shown, the multi-head multiplier can be used to perform mantissa multiplication in this module, corresponding to the mantissas of the first and second tensors respectively.

[0039] In practical applications, the first and second tensors can be multiplied using an N×J dot product. This dot product enhances the data result, where N and J are arbitrary positive integers, and their values ​​can be set as needed. For example... Figure 3 As shown, a 4×4 dot product can be used for the first tensor stored in the first tensor unit and the second tensor stored in the second tensor unit.

[0040] S106. Compare the summation results of the M exponents corresponding to the M tensor groups one by one to determine the shift value.

[0041] Specifically, the server processes the M tensor groups according to the aforementioned process to obtain the exponential summation results corresponding to the M tensor groups respectively. After obtaining the M exponential summation results corresponding to the M tensor groups one by one, the server can compare the M exponential summation results to determine the shift value. For example, the server can use the maximum value among the M exponential summation results as the shift value.

[0042] like Figure 2As shown, when there are 3 tensor sets, the corresponding exponential summation results can be obtained from the 3 tensor sets respectively. The comparator can compare the exponential summation results corresponding to the 3 tensor sets respectively to determine the shift value.

[0043] S107. Referring to the first and second sign bits corresponding to the M tensor groups respectively, according to the shift value, the M mantissa dot product results corresponding to the M tensor groups are shifted respectively to obtain the M shifted mantissa dot product results.

[0044] Specifically, the server processes the M tensor groups according to the aforementioned process to obtain the mantissa dot product results corresponding to the M tensor groups respectively. After obtaining the M mantissa dot product results corresponding to the M tensor groups, the server can perform shift processing on the M mantissa dot product results corresponding to the M tensor groups respectively, based on the first sign bit and the second sign bit corresponding to the M tensor groups, according to the shift value, thereby obtaining M shifted mantissa dot product results.

[0045] like Figure 2 As shown, when there are 3 tensor groups, the corresponding mantissa dot product results can be obtained from the 3 tensor groups respectively. The shifter can shift the mantissa dot product results of the 3 tensor groups respectively to obtain 3 shifted mantissa dot product results.

[0046] S108. Based on the first tensor data type and the second tensor data type corresponding to the M tensor groups respectively, perform addition processing on the M shift mantissa dot product results to obtain the tensor calculation results corresponding to the M tensor groups.

[0047] Specifically, after obtaining the M shifted mantissa dot product results, the server can perform addition on the M shifted mantissa dot product results based on the first tensor data type and the second tensor data type corresponding to the M tensor groups respectively. In practical applications, the number of dot products involved in the addition operation is different depending on the different data types, thus obtaining the final tensor calculation result.

[0048] like Figure 2 As shown, the addition tree can be used to add the results of the dot product of the three shifted mantissas to obtain the final tensor calculation result for output.

[0049] Through the above steps, a unified preprocessing method is used to obtain the corresponding exponent and mantissa parts for tensors of different data types. Then, through subsequent general exponent accumulation, mantissa multiplication, comparison, shifting and addition processing, the above unified processing logic can be adapted to the calculation requirements of different data tensors. Compared with the design idea of ​​equipping each data type with dedicated integrated circuits, this fundamentally eliminates circuit redundancy, thereby reducing chip area, reducing power consumption and improving flexibility.

[0050] In one possible implementation, S104 performs a cumulative summation of the first tensor exponent and the second tensor exponent based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor to obtain the exponent accumulation result corresponding to the i-th tensor group, including: The first tensor index and the second tensor index are scaled based on the first tensor scaling factor and the second tensor scaling factor, respectively, to obtain the first tensor scaling index and the second tensor scaling index. Based on the first tensor data type and the second tensor data type, the scaling exponents of the first tensor and the second tensor are summed to obtain the sum of the exponents corresponding to the i-th tensor group.

[0051] Specifically, the server can first scale the first tensor index and the second tensor index based on the first tensor scaling factor and the second tensor scaling factor, respectively. Then, based on the first tensor data type and the second tensor data type, the scaled first tensor scaling index and the second tensor scaling index are accumulated to obtain the accumulated index result corresponding to the i-th tensor group.

[0052] In one possible implementation, in S107, referring to the first and second sign bits corresponding to the M tensor groups respectively, the M mantissa dot product results corresponding to the M tensor groups are shifted according to the shift value to obtain M shifted mantissa dot product results, including: For the i-th tensor group, if the first and second sign bits of the i-th tensor group are positive, then the mantissa dot product result of the i-th tensor group is directly shifted according to the shift value to obtain the shifted mantissa dot product result of the i-th tensor group; if the first and second sign bits of the i-th tensor group are negative, then the mantissa dot product result of the i-th tensor group is first processed in two's complement according to the shift value, and then the mantissa dot product result after two's complement is shifted to obtain the shifted mantissa dot product result of the i-th tensor group.

[0053] Specifically, during the shift process, the sign bit needs to be checked first. If the sign bit is positive, the shift process is performed directly. If the sign bit is negative, the two's complement is performed first, and then the shift process is performed.

[0054] In one possible implementation, to ensure the reliability of tensor computation, the method further includes: Obtain the offset tensors corresponding to the M tensor groups, and determine the data type of the offset tensors. Based on the offset tensor data type, the offset tensor is preprocessed to obtain the offset sign bit, offset tensor exponent, and offset tensor mantissa of the offset tensor. In S106, the M exponent sums corresponding to the M tensor sets are compared to determine the shift value, including: The shift value is determined by comparing the sum of the M exponents corresponding to the M tensor sets with the offset tensor exponent. In S108, based on the first and second tensor data types corresponding to the M tensor groups respectively, the M shifted mantissa dot product results are added to obtain the tensor calculation results corresponding to the M tensor groups, including: Based on the first tensor data type, the second tensor data type, and the offset tensor data type corresponding to the M tensor groups respectively, the dot product results of the M shift mantissas and the mantissas of the offset tensor are added to obtain the tensor calculation results.

[0055] Specifically, to ensure the reliability of tensor calculation, an offset tensor is introduced in this embodiment, and a general preprocessing is performed on the offset tensor to obtain the offset sign bit, offset tensor exponent, and offset tensor mantissa. For the comparison step, the sum of the M exponents corresponding to the M tensor groups is compared with the offset tensor exponent to determine the shift value. For the addition step, the dot product of the M shift mantissas is added with the offset tensor mantissa to obtain the tensor calculation result.

[0056] like Figure 2 As shown, the offset unit is used to store the offset tensor, and the offset unit preprocessing unit is used to preprocess the offset tensor. When an offset tensor exists, the comparator will also add the offset tensor exponent to the comparison, and the addition tree will add the offset tensor mantissa to the addition process.

[0057] In one possible implementation, the method further includes: Identify whether the first tensor is special data; If the first tensor is determined to be special data, then the first tensor is output directly; Accordingly, identify whether the second tensor is special data; If the second tensor is determined to be special data, then the second tensor is output directly.

[0058] Specifically, regarding the aforementioned preprocessing steps, for the first tensor and the second tensor, if the first tensor is determined to be special data, the above calculation steps can be skipped and the first tensor can be output directly. Similarly, if the second tensor is determined to be special data, the above calculation steps can also be skipped and the second tensor can be output directly. Special data can include various special cases in actual tensor calculations, such as zero, infinity, and non-normal numbers.

[0059] In one possible implementation, the method also includes regularizing the tensor computation results.

[0060] Specifically, such as Figure 2 As shown, before outputting the tensor calculation results, the tensor calculation results can be regularized through multi-data regularization, thereby achieving regularized output of the tensor calculation results.

[0061] In one possible implementation, the method is executed by a tensor computing device, which includes a control pipeline and a data pipeline to further reduce power consumption. The control pipeline corresponds to the master clock and is used to transmit control signals, including clock gating enable signals. The data pipeline corresponds to the slave clock and is used to perform tensor calculations. The slave clock is generated by the master clock based on the clock gating enable signal.

[0062] Specifically, to further reduce power consumption, the tensor computation device includes two pipelines: a control pipeline and a data pipeline. The control pipeline corresponds to the master clock and is used to transmit control signals, while the data pipeline corresponds to the slave clock and is used to perform tensor computations. In other words, the data pipeline is the primary computation pipeline, used to perform tensor computations, while the control pipeline only transmits control signals. Precise clock gating of the data pipeline is achieved through a clock gating enable signal, thereby avoiding invalid signal flips. In this embodiment, by isolating the control path from the data path, the control signal typically changes less, while the data signal flips frequently. Precise gating of the data pipeline is achieved through the master clock, thereby maximizing power efficiency while ensuring correct functionality.

[0063] like Figure 4 As shown, Figure 4 The special pipeline in the pipeline is the control pipeline, which uses the master clock; the pipeline is the data pipeline, which uses the slave clock. The master clock controls the clock of the data pipeline through the clock gating enable signal.

[0064] In one possible implementation, the clock gating enable signal is generated by combining the control pipeline enable signal with a special data flag; the special data flag is used to indicate whether there is currently any data input to the data pipeline.

[0065] Specifically, for the data pipeline, in order to shut down the clock corresponding to the idle part and eliminate the power consumption of the idle part, the clock gating enable signal can be generated by combining the control pipeline enable signal with a special data flag. The special data flag is used to indicate whether there is currently data input to the data pipeline, so that the clock corresponding to the idle part can be shut down when the data is idle.

[0066] like Figure 4As shown, when a special data flag indicates that no data is currently being input to the data pipeline, the idle portion of the data pipeline can be closed by controlling the pipeline transmission clock gating enable signal.

[0067] In summary, this application provides a tensor computation method, which includes: obtaining M tensor groups, each of the M tensor groups including a first tensor and a second tensor, where M is an integer and M > 1; for the i-th tensor group in the M tensor groups, determining the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group; based on the first tensor data type, preprocessing the first tensor to obtain the first sign bit, the first tensor exponent, and the first tensor mantissa corresponding to the first tensor; based on the second tensor data type, preprocessing the second tensor to obtain the second sign bit, the second tensor exponent, and the second tensor mantissa corresponding to the second tensor; based on the first tensor data type, the first tensor scaling factor, and the second tensor... The data type and the second tensor scaling factor are used to accumulate the exponents of the first and second tensors to obtain the exponent accumulation result corresponding to the i-th tensor group. Based on the data type of the first and second tensors, the mantissas of the first and second tensors are multiplied by a dot to obtain the mantissa multiplication result corresponding to the i-th tensor group. The M exponent accumulation results corresponding to the M tensor groups are compared to determine the shift value. Referring to the first and second sign bits corresponding to the M tensor groups respectively, the M mantissa multiplication results corresponding to the M tensor groups are shifted according to the shift value to obtain M shifted mantissa multiplication results. Based on the data type of the first and second tensors corresponding to the M tensor groups respectively, the M shifted mantissa multiplication results are added to obtain the tensor calculation result corresponding to the M tensor groups. Using the above method, a unified preprocessing is used to obtain the corresponding sign bit, exponent part, and mantissa part for tensors of different data types. Then, through subsequent general exponent accumulation, mantissa multiplication, comparison, shifting, and addition processing, the above unified processing logic can be adapted to the calculation requirements of different data tensors. Compared with the design idea of ​​equipping each data type with dedicated integrated circuits, this fundamentally eliminates circuit redundancy, thereby reducing chip area, reducing power consumption, and improving flexibility.

[0068] In the description of this specification, references to terms such as "some possible implementations," "some implementations," "example," "specific example," or "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with that implementation or example is included in at least one implementation or example of this application, and the aforementioned terms do not necessarily refer to the same implementation or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more implementations or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different implementations or examples described in this specification, as well as the features of different implementations or examples.

[0069] The method flowcharts for embodiments of this application describe certain operations as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, or some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., by software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0070] Those skilled in the art will understand that in the methods described in the above specific embodiments, the order in which the steps are written does not imply a strict execution order, and the specific execution order of each step should be determined by its function and possible internal logic.

[0071] Based on the foregoing Figures 1-4 The tensor computing device provided in this application will be described below through device embodiments. Figure 5 This is a schematic diagram of a tensor computing device provided in an embodiment of this application, as shown below. Figure 5 As shown, the tensor computing device 500 includes: Acquisition unit 501 is used to acquire M tensor groups, each of the M tensor groups includes a first tensor and a second tensor, M is an integer, M>1; The determining unit 502 is used to determine, for the i-th tensor in the M tensor groups, the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group. The first tensor preprocessing unit 503 is used to preprocess the first tensor based on the first tensor data type to obtain the first sign bit, the first tensor exponent, and the first tensor mantissa corresponding to the first tensor. The second tensor preprocessing unit 504 is used to preprocess the second tensor based on the second tensor data type to obtain the second sign bit, the second tensor exponent, and the second tensor mantissa of the second tensor. The multi-head adder 505 is used to accumulate the first tensor exponent and the second tensor exponent based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, to obtain the exponent accumulation result corresponding to the i-th tensor group. The multi-head multiplier 506 is used to perform dot multiplication on the mantissa of the first tensor and the mantissa of the second tensor based on the first tensor data type and the second tensor data type, so as to obtain the dot multiplication result of the mantissa of the i-th tensor group. Comparator 507 is used to compare the summation results of the M exponents corresponding to the M tensor sets one by one to determine the shift value; The shifter 508 is used to refer to the first sign bit and the second sign bit corresponding to the M tensor groups respectively, and according to the shift value, to perform shift processing on the M mantissa dot product results corresponding to the M tensor groups respectively, so as to obtain M shifted mantissa dot product results. Addition Tree 509 is used to perform addition on the dot product results of M shifted mantissas based on the first and second tensor data types corresponding to M tensor groups, respectively, to obtain the tensor calculation results corresponding to the M tensor groups.

[0072] In one possible implementation, the multi-head adder 505 is used for: The first tensor index and the second tensor index are scaled based on the first tensor scaling factor and the second tensor scaling factor, respectively, to obtain the first tensor scaling index and the second tensor scaling index. Based on the first tensor data type and the second tensor data type, the scaling exponents of the first tensor and the second tensor are summed to obtain the sum of the exponents corresponding to the i-th tensor group.

[0073] In one possible implementation, shifter 508 is used for: For the i-th tensor group, if the first and second sign bits of the i-th tensor group are positive, then the mantissa dot product result of the i-th tensor group is directly shifted according to the shift value to obtain the shifted mantissa dot product result of the i-th tensor group; if the first and second sign bits of the i-th tensor group are negative, then the mantissa dot product result of the i-th tensor group is first processed in two's complement according to the shift value, and then the mantissa dot product result after two's complement is shifted to obtain the shifted mantissa dot product result of the i-th tensor group.

[0074] In one possible implementation, the acquisition unit 501 is also used for: Obtain the offset tensors corresponding to the M tensor groups, and determine the data type of the offset tensors. The device also includes an offset unit preprocessing unit, used for: Based on the offset tensor data type, the offset tensor is preprocessed to obtain the offset sign bit, offset tensor exponent, and offset tensor mantissa of the offset tensor. Comparator 507 is used for: The shift value is determined by comparing the sum of the M exponents corresponding to the M tensor sets with the offset tensor exponent. Addition tree 509, used for: Based on the first tensor data type, the second tensor data type, and the offset tensor data type corresponding to the M tensor groups respectively, the dot product results of the M shift mantissas and the mantissas of the offset tensor are added to obtain the tensor calculation results.

[0075] In one possible implementation, the first tensor preprocessing unit 503 is further configured to: Identify whether the first tensor is special data; If the first tensor is determined to be special data, then the first tensor is output directly. Correspondingly, the second tensor preprocessing unit 504 is also used for: Identify whether the second tensor is special data; If the second tensor is determined to be special data, then the second tensor is output directly.

[0076] In one possible implementation, the device also includes multi-data regularization; Multi-data regularization is used to regularize the results of tensor calculations.

[0077] In one possible implementation, the device includes a control pipeline and a data pipeline; The control pipeline corresponds to the master clock and is used to transmit control signals, including clock gating enable signals. The data pipeline corresponds to the slave clock and is used to perform tensor calculations. The slave clock is generated by the master clock based on the clock gating enable signal.

[0078] In one possible implementation, the clock gating enable signal is generated by combining the control pipeline enable signal with a special data flag; the special data flag is used to indicate whether there is currently any data input to the data pipeline.

[0079] It should be noted that the apparatus in the embodiments of this application can implement each process of the aforementioned method and achieve the same effect and function, which will not be elaborated here.

[0080] This application also provides an electronic device, including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the following processing is performed: Obtain M initial tensors and divide the M initial tensors into M tensor groups; each of the M tensor groups contains the initial tensor, M is an even number, M>2; For the i-th tensor group among M tensor groups, determine the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group, where i is an integer and M≥i≥1; Based on the first tensor data type, the first tensor is preprocessed to obtain the first tensor exponent and the first tensor mantissa; based on the second tensor data type, the second tensor is preprocessed to obtain the second tensor exponent and the second tensor mantissa. Based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, the first tensor exponent and the second tensor exponent are accumulated to obtain the exponent accumulation result corresponding to the i-th tensor group; Based on the first tensor data type and the second tensor data type, the mantissa of the first tensor and the mantissa of the second tensor are multiplied by a dot to obtain the mantissa dot product result corresponding to the i-th tensor group; Compare the summation results of the M exponents corresponding to the M tensor sets one-to-one to determine the shift value; Based on the shift value, the M mantissa dot product results corresponding to the M tensor groups are shifted respectively to obtain the M shifted mantissa dot product results. Based on the first and second tensor data types corresponding to the M tensor groups respectively, the M shift mantissa dot product results are added to obtain the tensor calculation results corresponding to the M tensor groups.

[0081] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the tensor computation method described in the above-described method embodiments. The storage medium can be either volatile or non-volatile computer-readable storage.

[0082] This application also provides a computer program product, including a computer program carrying program code. The instructions included in the program code can be used to execute the steps of the tensor calculation method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0083] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0084] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the descriptions of the apparatus, device, and computer-readable storage medium embodiments are simplified because they are substantially similar to the method embodiments; relevant details can be found in the descriptions of the method embodiments.

[0085] The apparatus, device, and computer-readable storage medium provided in the embodiments of this application correspond one-to-one with the method. Therefore, the apparatus, device, and computer-readable storage medium also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the apparatus, device, and computer-readable storage medium will not be repeated here.

[0086] While the spirit and principles of this application have been described above with reference to several specific embodiments, it should be understood that this application is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined. This application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A tensor computation method, characterized in that, The method includes: Obtain M tensor groups, each of the M tensor groups including a first tensor and a second tensor, where M is an integer and M > 1; For the i-th tensor group among M tensor groups, determine the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group, where i is an integer and M≥i≥1; Based on the first tensor data type, the first tensor is preprocessed to obtain the first sign bit, the first tensor exponent, and the first tensor mantissa corresponding to the first tensor; based on the second tensor data type, the second tensor is preprocessed to obtain the second sign bit, the second tensor exponent, and the second tensor mantissa corresponding to the second tensor. Based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, the first tensor exponent and the second tensor exponent are accumulated to obtain the exponent accumulation result corresponding to the i-th tensor group; Based on the first tensor data type and the second tensor data type, the mantissa of the first tensor and the mantissa of the second tensor are multiplied by a dot to obtain the mantissa dot product result corresponding to the i-th tensor group; The shift value is determined by comparing the M exponent accumulation results corresponding one-to-one with the M tensor groups; Referring to the first sign bit and the second sign bit corresponding to the M tensor groups respectively, and according to the shift value, the M mantissa dot product results corresponding to the M tensor groups are shifted respectively to obtain M shifted mantissa dot product results; Based on the first tensor data type and the second tensor data type corresponding to the M tensor groups respectively, the M shift mantissa dot product results are added to obtain the tensor calculation results corresponding to the M tensor groups.

2. The method according to claim 1, characterized in that, The step of accumulating the first tensor exponent and the second tensor exponent based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor to obtain the exponent accumulation result corresponding to the i-th tensor group includes: Based on the first tensor scaling factor and the second tensor scaling factor, the first tensor exponent and the second tensor exponent are scaled respectively to obtain the first tensor scaling exponent and the second tensor scaling exponent. Based on the first tensor data type and the second tensor data type, the first tensor scaling index and the second tensor scaling index are accumulated to obtain the index accumulation result corresponding to the i-th tensor group.

3. The method according to claim 1, characterized in that, Referring to the first and second sign bits corresponding to the M tensor groups respectively, and based on the shift value, the M mantissa dot product results corresponding to the M tensor groups are shifted respectively to obtain M shifted mantissa dot product results, including: For the i-th tensor group, if the first sign bit and the second sign bit corresponding to the i-th tensor group are positive, then according to the shift value, the mantissa dot product result corresponding to the i-th tensor group is directly shifted to obtain the shifted mantissa dot product result corresponding to the i-th tensor group; if the first sign bit and the second sign bit corresponding to the i-th tensor group are negative, then according to the shift value, the mantissa dot product result corresponding to the i-th tensor group is first processed by two's complement, and then the mantissa dot product result after two's complement is shifted to obtain the shifted mantissa dot product result corresponding to the i-th tensor group.

4. The method according to claim 1, characterized in that, The method further includes: Obtain the offset tensors corresponding to the M tensor groups, and determine the data type of the offset tensors corresponding to the offset tensors; Based on the offset tensor data type, the offset tensor is preprocessed to obtain the offset sign bit, offset tensor exponent, and offset tensor mantissa corresponding to the offset tensor. The step of comparing the summation results of the M exponents corresponding to the M tensor groups to determine the shift value includes: The shift value is determined by comparing the cumulative sum of the M exponents corresponding to the M tensor groups with the offset tensor exponent. The addition process is performed on the M shift mantissa dot product results based on the first tensor data type and the second tensor data type corresponding to the M tensor groups, respectively, to obtain the tensor calculation results corresponding to the M initial tensors, including: Based on the first tensor data type and the second tensor data type corresponding to the M tensor groups respectively, as well as the offset tensor data type, the dot product results of the M shift mantissas and the mantissa of the offset tensor are added to obtain the tensor calculation result.

5. The method according to claim 1, characterized in that, The method further includes: Identify whether the first tensor is special data; If the first tensor is determined to be special data, then the first tensor is output directly; Accordingly, it is determined whether the second tensor is special data; If the second tensor is determined to be special data, then the second tensor is output directly.

6. The method according to claim 1, characterized in that, The method further includes regularizing the tensor calculation results.

7. The method according to claim 1, characterized in that, The method is executed by a tensor computing device, which includes a control pipeline and a data pipeline; The control pipeline corresponds to the master clock and is used to transmit control signals, including clock gating enable signals. The data pipeline corresponds to the slave clock and is used to perform tensor calculations. The slave clock is generated by the master clock based on the clock gating enable signal.

8. The method according to claim 7, characterized in that, The clock gating enable signal is generated by combining the control pipeline enable signal with a special data flag; the special data flag is used to indicate whether there is currently any data input to the data pipeline.

9. A tensor computing device, characterized in that, The device includes: An acquisition unit is used to acquire M tensor groups, each of the M tensor groups including a first tensor and a second tensor, where M is an integer and M>1; The determining unit is used to determine, for the i-th tensor group in M ​​tensor groups, the first tensor data type and the first tensor scaling factor corresponding to the first tensor in the i-th tensor group, and the second tensor data type and the second tensor scaling factor corresponding to the second tensor in the i-th tensor group. The first tensor preprocessing unit is used to preprocess the first tensor based on the first tensor data type to obtain the first sign bit, the first tensor exponent, and the first tensor mantissa corresponding to the first tensor. The second tensor preprocessing unit is used to preprocess the second tensor based on the second tensor data type to obtain the second sign bit, the second tensor exponent, and the second tensor mantissa corresponding to the second tensor. A multi-head adder is used to accumulate the first tensor exponent and the second tensor exponent based on the first tensor data type, the first tensor scaling factor, the second tensor data type, and the second tensor scaling factor, to obtain the exponent accumulation result corresponding to the i-th tensor group; A multi-head multiplier is used to perform dot product processing on the mantissa of the first tensor and the mantissa of the second tensor based on the first tensor data type and the second tensor data type, to obtain the mantissa dot product result corresponding to the i-th tensor group; A comparator is used to compare the summation results of the M exponents corresponding to the M tensor groups one by one to determine the shift value; A shifter is used to shift the M mantissa multiplication results corresponding to the M tensor groups one by one according to the shift value, with reference to the first sign bit and the second sign bit corresponding to the M tensor groups respectively, to obtain M shifted mantissa multiplication results. An addition tree is used to perform addition processing on the M shift mantissa dot product results based on the first tensor data type and the second tensor data type corresponding to the M tensor groups respectively, to obtain the tensor calculation results corresponding to the M tensor groups.

10. The apparatus according to claim 9, characterized in that, The multi-head adder is used for: Based on the first tensor scaling factor and the second tensor scaling factor, the first tensor exponent and the second tensor exponent are scaled respectively to obtain the first tensor scaling exponent and the second tensor scaling exponent. Based on the first tensor data type and the second tensor data type, the first tensor scaling index and the second tensor scaling index are accumulated to obtain the index accumulation result corresponding to the i-th tensor group.

11. The apparatus according to claim 9, characterized in that, Shifters are used for: For the i-th tensor group, if the first sign bit and the second sign bit corresponding to the i-th tensor group are positive, then according to the shift value, the mantissa dot product result corresponding to the i-th tensor group is directly shifted to obtain the shifted mantissa dot product result corresponding to the i-th tensor group; if the first sign bit and the second sign bit corresponding to the i-th tensor group are negative, then according to the shift value, the mantissa dot product result corresponding to the i-th tensor group is first processed by two's complement, and then the mantissa dot product result after two's complement is shifted to obtain the shifted mantissa dot product result corresponding to the i-th tensor group.

12. The apparatus according to claim 9, characterized in that, The acquisition unit is further configured to: Obtain the offset tensors corresponding to the M tensor groups, and determine the data type of the offset tensors corresponding to the offset tensors; The device further includes an offset unit preprocessing unit for: Based on the offset tensor data type, the offset tensor is preprocessed to obtain the offset sign bit, offset tensor exponent, and offset tensor mantissa corresponding to the offset tensor. The comparator is used for: The shift value is determined by comparing the cumulative sum of the M exponents corresponding to the M tensor groups with the offset tensor exponent. The addition tree is used for: Based on the first tensor data type and the second tensor data type corresponding to the M tensor groups respectively, as well as the offset tensor data type, the dot product results of the M shift mantissas and the mantissa of the offset tensor are added to obtain the tensor calculation result.

13. The apparatus according to claim 9, characterized in that, The first tensor preprocessing unit is further configured to: Identify whether the first tensor is special data; If the first tensor is determined to be special data, then the first tensor is output directly; Accordingly, the second tensor preprocessing unit is also used for: Identify whether the second tensor is special data; If the second tensor is determined to be special data, then the second tensor is output directly.

14. The apparatus according to claim 9, characterized in that, The device also includes multi-data regularization; The multi-data regularization is used to perform regularization processing on the tensor calculation results.

15. The apparatus according to claim 9, characterized in that, The device includes a control pipeline and a data pipeline; The control pipeline corresponds to the master clock and is used to transmit control signals, including clock gating enable signals. The data pipeline corresponds to the slave clock and is used to perform tensor calculations. The slave clock is generated by the master clock based on the clock gating enable signal.

16. The apparatus according to claim 15, characterized in that, The clock gating enable signal is generated by combining the control pipeline enable signal with a special data flag; the special data flag is used to indicate whether there is currently any data input to the data pipeline.

17. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the tensor computation method as described in any one of claims 1 to 8 is performed.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the tensor computation method as described in any one of claims 1 to 8.

19. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, performs the tensor computation method as described in any one of claims 1 to 8.

Images