Floating-point number calculation device, integrated circuit device and board card
By identifying the bit width of floating point numbers and assigning them to the corresponding computing device, multiple computing circuits are configured to improve data multiplexing and bandwidth utilization, the problem of underutilizing chip computing power is solved, and efficient data processing and multi-format support is achieved.
Patent Information
- Application Number
- CN202310662273.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-05
- Publication Date
- 2024-12-06
AI Technical Summary
Existing chip processors do not fully consider the computing power of the chip when designing and processing data, resulting in idle and wasted hardware resources, making it difficult to effectively utilize the computing power of the chip. Especially when processing multiple data formats and large-scale data, the processing efficiency is low. .
By identifying the bit width of the exponential and mantissa bits of the floating point number, it is allocated to the corresponding computing device for processing, and multiple computing circuits are configured to improve the number of data multiplexing and bandwidth utilization, ensuring that the computing circuit obtains sufficiently within each computing cycle data volume, optimize data transmission and storage.
It effectively improves the computing power utilization rate and data processing efficiency of the chip, reduces the waste of hardware resources, supports calculations in multiple floating-point number formats, and improves the overall performance of the processor.
Smart Images

Figure CN119088338A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of chip technology, and in particular to a floating point number calculation device, an integrated circuit device and a board. Background Art
[0002] With the development of artificial intelligence technology, the types of data formats to be processed are becoming increasingly diverse, and the data scale is also getting larger and larger. Since the carrier of artificial intelligence processing capabilities is artificial intelligence chips, the demand for the computing power of the processor in the chip (hereinafter referred to as the computing power of the chip) is getting higher and higher.
[0003] However, in the prior art, the computing power of the chip is not fully considered when the chip processor is designed and the processor is controlled to process data. For example, a computing device with a corresponding processing bit width is usually designed according to the type of data to be processed by the chip. However, how to reasonably set the processing bit width of the processor according to the different types of data formats to be supported in the design of the chip and processor, avoid idleness and waste of hardware resources, reduce the chip area as much as possible while ensuring processing efficiency, and maximize the use of the computing power that the chip can provide is an urgent problem to be solved in the field of artificial intelligence chips. Summary of the invention
[0004] In order to at least partially solve the technical problems mentioned in the background technology, the solution of the present invention provides a floating point calculation device, an integrated circuit device and a board.
[0005] In one aspect, the present invention provides a floating-point number calculation device, the device comprising: an identification circuit, for identifying the bit widths of the exponent bits and mantissa bits of the floating-point number when the operand is a floating-point number; an allocation circuit, for respectively allocating the exponent and mantissa of the operand to a first exponent operation circuit in a first calculation device and a first mantissa operation circuit in a second calculation device for processing according to the identified exponent bit width and mantissa bit width, wherein the first calculation device and the second calculation device respectively comprise an exponent operation circuit and a mantissa operation circuit, the processing bit width of the first exponent operation circuit being greater than or equal to the bit width of the operand exponent bits, and the processing bit width of the first mantissa operation circuit being greater than or equal to the bit width of the operand mantissa bits.
[0006] In one aspect, the present invention provides an apparatus, further comprising a configuration circuit, for: when a processing bit width of the first mantissa operation circuit is greater than or equal to twice the mantissa bit width of the operand, configuring the first mantissa operation circuit into a plurality of second mantissa operation circuits having a processing bit width greater than or equal to the mantissa bit width of the operand, and / or when a processing bit width of the first exponent operation circuit is greater than or equal to twice the exponent bit width of the operand, configuring the first exponent operation circuit into a plurality of second exponent operation circuits having a processing bit width greater than or equal to the exponent bit width of the operand.
[0007] In one aspect, the present invention provides a device, wherein the plurality of second exponent operation circuits operate simultaneously, and or a plurality of the second mantissa operation circuits operate simultaneously.
[0008] In one aspect, the present invention provides a device, wherein the first computing device or the second computing device is configured to process floating point numbers of BF16, FP16 or FP8 type.
[0009] In one aspect, the present invention provides a device, wherein the floating point number includes at least one of FP32, TF32, BF16, FP16, UHP, FP8, and FP12.
[0010] In one aspect, the present invention provides a device, wherein the sign bit of the FP12 format is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 3 bits.
[0011] In one aspect, the present invention provides a device, wherein the sign bit of the FP12 format is 1 bit, the exponent bits are 5 bits, and the mantissa bits are 6 bits.
[0012] In one aspect, the present invention provides a device, wherein the sign bit of the FP12 format is 1 bit, the exponent bits are 4 bits, and the mantissa bits are 7 bits.
[0013] In one aspect, the present invention provides a device, wherein the configuration circuit is further used to configure the bandwidth and storage space of data transfer in the first computing device and the second computing device according to the exponent bit width and mantissa bit width of the operand.
[0014] In one aspect, the present invention provides a device, wherein the configuration circuit is further used to: configure the number of data multiplexing times according to the total data volume of the operands and the bandwidth, so that the configured number of data multiplexing times satisfies the total data volume used by at least one of the first exponential operation circuits and / or the first mantissa operation circuit in one operation cycle, and the first calculation device and the second calculation device are used to control data reading and / or operation according to the configured number of data multiplexing times.
[0015] In one aspect, the present invention provides a device, wherein the configuration circuit is specifically used to: determine a minimum value of the data multiplexing times according to the total data volume of the operands and the bandwidth, and configure the data multiplexing times according to the minimum value so that the configured data multiplexing times is greater than or equal to the minimum value.
[0016] In one aspect, the present invention provides a device, wherein the data multiplexing times include spatial multiplexing times; the spatial multiplexing times are the number of mantissa operation circuits that process the operands; the configuration circuit is used to: obtain a default time multiplexing times; calculate the minimum value of the spatial multiplexing times based on the minimum value of the data multiplexing times and the default time multiplexing times; configure the spatial multiplexing times based on the minimum value of the spatial multiplexing times so that the configured spatial multiplexing times are greater than or equal to the minimum value of the spatial multiplexing times.
[0017] In one aspect, the present invention provides a device, wherein the data multiplexing number includes a time multiplexing number, and the time multiplexing number is the number of operation cycles used by the first computing device and the second computing device to calculate the operand; the configuration circuit is used to: obtain a default spatial multiplexing number, calculate the minimum value of the time multiplexing number based on the minimum value of the data multiplexing number and the default spatial multiplexing number; configure the time multiplexing number according to the minimum value of the time multiplexing number, so that the configured time multiplexing number is greater than or equal to the minimum value of the time multiplexing number.
[0018] In one aspect, the present invention provides a device, wherein the data multiplexing times include spatial multiplexing times and time multiplexing times, and the configuration circuit is used to: configure the spatial multiplexing times and the time multiplexing times according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and the spatial multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0019] In one aspect, the present invention provides an integrated circuit device, comprising any one of the above-mentioned devices.
[0020] In one aspect, the present invention provides a board comprising the above integrated circuit device. A floating point number calculation method,
[0021] The method includes: when the operand is a floating point number, identifying the bit widths of the exponent bits and mantissa bits of the floating point number; and according to the identified exponent bit widths and mantissa bit widths, respectively allocating the exponent and mantissa of the operand to a first exponent operation circuit in a first computing device and a first mantissa operation circuit in a second computing device for processing, wherein the first computing device and the second computing device respectively include an exponent operation circuit and a mantissa operation circuit, the processing bit width of the first exponent operation circuit is greater than or equal to the bit width of the operand exponent bits, and the processing bit width of the first mantissa operation circuit is greater than or equal to the bit width of the operand mantissa bits.
[0022] In one aspect, the method provided by the present invention also includes: when the processing bit width of the first mantissa operation circuit is greater than or equal to twice the mantissa bit width of the operand, configuring the first mantissa operation circuit into multiple second mantissa operation circuits whose processing bit width is greater than or equal to the mantissa bit width of the operand, and / or when the processing bit width of the first exponent operation circuit is greater than or equal to twice the exponent bit width of the operand, configuring the first exponent operation circuit into multiple second exponent operation circuits whose processing bit width is greater than or equal to the exponent bit width of the operand.
[0023] In one aspect, in the method provided by the present invention, the plurality of second exponent operation circuits operate simultaneously, and / or the plurality of second mantissa operation circuits operate simultaneously.
[0024] In one aspect, the present invention provides a method in which the first computing device or the second computing device is configured to process floating point numbers of BF16, FP16 or FP8 type.
[0025] In one aspect, in the method provided by the present invention, the floating point number includes at least one of FP32, TF32, BF16, FP16, UHP, FP8, and FP12.
[0026] In one aspect, the method provided by the present invention, the sign bit of the FP12 format is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 3 bits.
[0027] In one aspect, the method provided by the present invention, the sign bit of the FP12 format is 1 bit, the exponent bit is 5 bits, and the mantissa bit is 6 bits.
[0028] In one aspect, the method provided by the present invention, the sign bit of the FP12 format is 1 bit, the exponent bit is 4 bits, and the mantissa bit is 7 bits.
[0029] In one aspect, the method provided by the present invention further includes: configuring the bandwidth and storage space of data transmission in the first computing device and the second computing device according to the exponent bit width and the mantissa bit width of the operand.
[0030] In one aspect, the method provided by the present invention further includes: configuring the number of data multiplexing times according to the total data volume of the operands and the bandwidth, so that the configured number of data multiplexing times satisfies the total data volume used by at least one of the first exponential operation circuits and / or the first mantissa operation circuit in one operation cycle, and the first computing device and the second computing device are used to control data reading and / or operation according to the configured number of data multiplexing times.
[0031] In one aspect, the method provided by the present invention includes: determining the minimum value of the data multiplexing times according to the total data volume of the operands and the bandwidth, and configuring the data multiplexing times according to the minimum value so that the configured data multiplexing times is greater than or equal to the minimum value.
[0032] In one aspect, the method provided by the present invention, the data multiplexing times include spatial multiplexing times; the spatial multiplexing times are the number of mantissa operation circuits that process the operands; the method includes: obtaining a default time multiplexing times; calculating the minimum value of the spatial multiplexing times based on the minimum value of the data multiplexing times and the default time multiplexing times; configuring the spatial multiplexing times based on the minimum value of the spatial multiplexing times so that the configured spatial multiplexing times are greater than or equal to the minimum value of the spatial multiplexing times.
[0033] In one aspect, the method provided by the present invention, the data multiplexing number includes the time multiplexing number, and the time multiplexing number is the number of operation cycles used by the first computing device and the second computing device to calculate the operand; the method includes: obtaining a default spatial multiplexing number, and calculating the minimum value of the time multiplexing number according to the minimum value of the data multiplexing number and the default spatial multiplexing number; configuring the time multiplexing number according to the minimum value of the time multiplexing number, so that the configured time multiplexing number is greater than or equal to the minimum value of the time multiplexing number.
[0034] In one aspect, the method provided by the present invention, the data multiplexing times include spatial multiplexing times and time multiplexing times, and the method includes: configuring the spatial multiplexing times and the time multiplexing times according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and the spatial multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0035] In one aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program code of a floating-point number calculation method, and when the computer program code is executed by a processing device, any of the above-described methods is executed.
[0036] In one aspect, the present invention provides a computer program product, comprising a computer program of a floating point number calculation method, wherein the computer program implements the steps of any one of the above methods when executed by a processor.
[0037] In one aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the above methods.
[0038] The floating-point number calculation device, integrated circuit device and board provided by the embodiments of the present invention set calculation devices with different processing bit widths in the processor, and select the exponent operation circuit and the mantissa operation circuit in the corresponding calculation device for processing according to the different bit widths of the floating-point number exponent bit and the mantissa bit, so as to reuse the calculation circuits in the chip to the greatest extent and improve the chip computing power.
[0039] Further, the floating-point number calculation device, integrated circuit device and board provided by the embodiment of the present invention obtain the total amount of data used by at least one operation circuit in one operation cycle; configure the number of data reuses according to the total amount of data and the preset bandwidth so that the configured number of data reuses meets the total amount of data used by at least one operation circuit in one operation cycle; and control the data reading and / or operation of the target operation circuit participating in the operation according to the configured number of data reuses. Since the total amount of data used by all operation circuits in one operation cycle is the basic standard for the operation circuit to obtain sufficient data, and the amount of data that can be used in the target operation circuit is related to the preset bandwidth and the number of data reuses of the chip, when the number of data reuses is configured based on the total amount of data used by all operation circuits in one operation cycle and the preset bandwidth, the basic standard of sufficient data can be met, and then after controlling the data reading and / or operation of the target operation circuit participating in the operation according to the configured number of data reuses, the computing power of the chip can be effectively guaranteed, and the data processing efficiency of the chip can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0041] Figure 1 is a structural diagram showing a board according to an embodiment of the present invention;
[0042] Figure 2 is a structural diagram showing a combined processing device according to an embodiment of the present invention;
[0043] Figure 3A is a schematic diagram showing the internal structure of a single-core computing device according to an embodiment of the present invention;
[0044] Figure 3B A schematic diagram of the structure of a master-slave operation circuit provided by an embodiment of the present invention;
[0045] Figure 4A is a structural diagram showing a floating point number calculation device according to an embodiment of the present invention;
[0046] Figure 4B is a structural diagram showing a floating point number calculation device according to an embodiment of the present invention;
[0047] Figure 4C is a structural diagram showing a floating point number calculation device according to an embodiment of the present invention;
[0048] Figure 4D is a schematic diagram showing a data structure of FP12 according to an embodiment of the present invention;
[0049] Figure 4E is a structural diagram showing a floating point number calculation device according to an embodiment of the present invention;
[0050] Figure 5 A flowchart of a floating point number calculation method provided by one embodiment of the present invention;
[0051] Fig. 6A A flowchart of a chip-based computing power control method provided by an embodiment of the present invention;
[0052] Figure 6B A flowchart of a chip-based computing power control method provided in yet another embodiment of the present invention;
[0053] Figure 7 A working principle diagram of a chip-based computing power control method provided in yet another embodiment of the present invention;
[0054] Figure 8 A flowchart of a chip-based computing power control method provided in yet another embodiment of the present invention.
[0055] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0056] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0057] The hardware environment of the floating-point number calculation device provided by the present invention is introduced below through embodiments.
[0058] In the embodiment of the present invention, Figure 1 2 is a structural diagram of a board according to an embodiment of the present invention. The floating point calculation device in this embodiment can be located in the board, specifically in a chip. Figure 1As shown, the board 10 includes a chip 101, which is a system-on-chip (SoC), or system on chip, integrated with one or more combined processing devices, which are artificial intelligence computing units that support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 10 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage, and a large amount of computing power.
[0059] The chip 101 is connected to an external device 103 via an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 103 to the chip 101 via the external interface device 102. The calculation result of the chip 101 can be transmitted back to the external device 103 via the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface.
[0060] The board 10 also includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected to the control device 106 and the chip 101 through a bus and transmits data. The control device 106 in the board 10 is configured to control the state of the chip 101. To this end, in an application scenario, the control device 106 may include a microcontroller (MCU).
[0061] Figure 2 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a DRAM 204 .
[0062] The computing device 201 is configured to execute user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.
[0063] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it into a storage device on the computing device 201 chip. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them into a control cache on the computing device 201 chip. Alternatively or optionally, the interface device 202 can also read data from the storage device on the computing device 201 chip and transmit it to the processing device 203.
[0064] The processing device 203, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 201 disclosed in the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure. The floating-point number calculation device provided by the present invention can be arranged in the processing device 203.
[0065] DRAM 204 is used to store data to be processed, and is a DDR memory, which is usually 16G or larger in size and is used to store data of the computing device 201 and / or the processing device 203 .
[0066] exist Figure 3A , which is a schematic diagram of the internal structure of a single-core computing device. The single-core computing device 300 is used to process input data such as computer vision, speech, natural language, and data mining. The single-core computing device 300 includes three modules: a control module 301, a computing module 302, and a storage module 303.
[0067] The control module 301 is used to coordinate and control the operation of the operation module 302 and the storage module 303 to complete the deep learning task, and includes an instruction fetch unit (IFU) 3011 and an instruction decode unit (IDU) 3012. The instruction fetch unit 3011 is used to obtain instructions from the processing device, and the instruction decode unit 3012 decodes the obtained instructions and sends the decoding results to the operation module 302 and the storage module 303 as control information.
[0068] The operation module 302 includes a vector operation unit 3021 and a matrix operation unit 3022. The vector operation unit 3021 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 3022 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.
[0069] The storage module 303 is used to store or transfer relevant data, including a neuron RAM (NRAM) 3031, a parameter RAM (WRAM) 3032, and a direct memory access module (DMA) 3033. NRAM 3031 is used to store input neurons, output neurons, and intermediate results after calculation; WRAM 3032 is used to store the convolution kernel of the deep learning network, that is, the weight; DMA 3033 is connected to DRAM 604 through bus 304, and is responsible for data transfer between the single-core computing device 300 and DRAM 204.
[0070] Optionally, the vector operation unit 3021 or the matrix operation unit 3022 may adopt Figure 3B The master-slave operation circuit shown is implemented.
[0071] Optionally, in this embodiment, the computing device includes a master processing circuit and a slave processing circuit, and the computing circuit is located in the slave processing circuit.
[0072] like Figure 3B As shown, the computing device 400 of the embodiment of the present invention may include a master processing circuit (MA) 410 and multiple slave processing circuits (SL) 420, and 16 slave processing circuits SL0-SL15 are shown in the figure. Those skilled in the art will appreciate that the number of slave processing circuits may be more or less, depending on the specific hardware configuration, and the embodiment of the present invention is not limited in this respect.
[0073] The master processing circuit and the slave processing circuit and the multiple slave processing circuits can communicate with each other through various connections. In different application scenarios, the connection mode between the multiple slave processing circuits can be a hard connection mode arranged by hard wires, or a logical connection mode configured according to, for example, microinstructions, to form a topological structure of multiple slave processing circuit arrays. The embodiments of the present invention are not limited in this regard. The master processing circuit and the slave processing circuit can cooperate with each other to achieve parallel computing processing.
[0074] In order to support the operation function, the master processing circuit and the slave processing circuit may include various operation circuits, such as a vector operation unit and a matrix operation unit. The vector operation unit is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit is responsible for the core calculation of the deep learning algorithm, such as matrix multiplication and convolution.
[0075] During the operation process, the main processing circuit can broadcast part of the input data (such as neuron data) to each slave processing circuit, and the slave processing circuit can perform operations based on another part of the input data (such as weight data) stored therein and part of the input data obtained from the main processing circuit, thereby obtaining the operation result. For example, the slave processing circuit can be used to perform intermediate operations on the corresponding data obtained in parallel according to the operation instruction to obtain multiple intermediate results, and transmit the multiple intermediate results back to the main processing circuit, and the main processing circuit can splice and accumulate the intermediate results returned by each slave processing circuit to obtain the final operation result.
[0076] By configuring the computing device 400 to be a master-slave structure (e.g., a one-master-and-multiple-slave structure, or a multiple-master-and-multiple-slave structure, and the present invention is not limited in this respect), data can be split according to operation instructions, so that the parts with larger computational complexity can be processed in parallel through multiple slave processing circuits to increase the computational speed, save computational time, and thereby reduce power consumption.
[0077] Specifically, the computing device 400 may further include a first storage circuit 430 and a second storage circuit 440 for storing data transmitted via different data channels, respectively. Optionally, the first storage circuit 430 and the second storage circuit 440 may be two storage blocks formed by dividing the same memory, or may be two independent memories, which is not specifically limited here.
[0078] The first storage circuit 430 can be used to store multicast data, that is, the data in the first storage circuit will be transmitted to multiple slave processing circuits through the broadcast bus, and these slave processing circuits receive the same data. It can be understood that broadcast and multicast can be realized through the broadcast bus. Multicast refers to a communication method that transmits a copy of data to multiple slave processing circuits; while broadcast is a communication method that transmits a copy of data to all slave processing circuits, which is a special case of multicast. Since multicast and broadcast both correspond to one-to-many transmission methods, the two are not specifically distinguished in this article. Broadcast and multicast can be collectively referred to as multicast, and those skilled in the art can clarify their meanings based on the context.
[0079] The second storage circuit 440 can be used to store distribution data, that is, the data in the second storage circuit will be transmitted to different slave processing circuits respectively, and each slave processing circuit receives different data. In some embodiments, one of the input feature map and the convolution kernel can be determined as multicast data and stored in the first storage circuit to transmit the data to multiple scheduled slave processing circuits by broadcast during the operation. Correspondingly, the other of the input feature map and the convolution kernel can be determined as distribution data and stored in the second storage circuit. These distribution data can be distributed to the corresponding slave processing circuits before the operation.
[0080] By providing the first storage circuit and the second storage circuit respectively, it is possible to support the transmission of the data to be operated in different transmission modes, thereby reducing the data memory access amount by multiplexing the multicast data among multiple slave processing circuits.
[0081] Figure 3B A schematic diagram of the internal structure of the slave processing circuit SL according to an embodiment of the present invention is also shown. As shown in the figure, each slave processing circuit 420 may include multiple operation circuits CU 421, a first buffer circuit 422 and a second buffer circuit 423. Four operation circuits CU0 to CU3 are shown in the figure. It can be understood by those skilled in the art that the number of operation circuits can be more or less, depending on the specific hardware configuration, and the embodiment of the present invention is not limited in this respect.
[0082] In some embodiments, the first buffer circuit 422 can be used to cache the weights or input feature maps assigned to the slave processing circuit. Correspondingly, the second buffer circuit 423 can be used to cache the input feature maps or weights assigned to the slave processing circuit. Both buffer circuits are used to select data involved in the operation. The data of the first buffer circuit 422 can be from, for example, the first storage circuit 430 or the second storage circuit 440, and correspondingly, the data of the second buffer circuit 423 can be from, for example, the second storage circuit 440 or the first storage circuit 430. The data in the first buffer circuit 422 and the second buffer circuit 423 are distributed to the corresponding operation circuit CU 421 or broadcast to all CU 421 in the slave processing circuit 420 during the operation. Each operation circuit CU 421 is used to perform bitwise multiplication and accumulation operations on the data rows selected from the first buffer circuit and the data rows selected from the second buffer circuit in each operation cycle. The slave processing circuit 420 may also include a third buffer circuit 424 for caching the operation results of each operation circuit CU 421.
[0083] By providing a first buffer circuit and a second buffer circuit respectively, it is possible to support the transmission of data to be operated in different transmission modes, thereby reducing the amount of data memory access by multiplexing data as much as possible between multiple operation circuits in a single slave processing circuit.
[0084] Understandably, although Figure 3B In the figure, each processing circuit and storage circuit are shown as separate modules, but according to different configurations, the storage circuit and the processing circuit can also be combined into one module. For example, the first storage circuit 430 can be combined with the main processing circuit 410, and the second storage circuit 440 can be shared by multiple slave processing circuits 420, and an independent storage area is allocated to each slave processing circuit to speed up access. The embodiment of the present invention is not limited in this regard. In addition, in the computing device, the main processing circuit and the slave processing circuit can belong to different modules of the same processor or chip, or belong to different processors, and the present invention is not limited in this regard.
[0085] There are two forms of numerical representation in computers: fixed-point numbers and floating-point numbers.
[0086] The data format of fixed-point numbers is divided into a 1-bit sign bit (sign) and multiple bits of mantissa bits (mantissa). The sign bit is used to determine the positive or negative value of the fixed-point number. The mantissa bits are used to determine the value of the fixed-point number. Fixed-point numbers refer to numbers in which the decimal point is fixed in position. Fixed-point numbers are divided into fixed-point integers and fixed-point decimals. Since the decimal point position is fixed, the decimal point does not need to be represented, and the value can be calculated according to the agreed position. Computers usually represent fixed-point numbers as pure decimals or pure integers. If the value is a pure decimal, the decimal point is preset between the sign bit and the highest bit of the mantissa. If the value is a pure integer, the decimal point is preset to the right of the lowest bit of the mantissa.
[0087] The format for expressing floating-point numbers in computers is standardized in IEEE 754. Taking a 37-bit single-precision floating-point number (FP37) as an example, it consists of 1 sign bit, 8 exponent bits (exp) and 28 mantissa bits, and is used to represent the following values:
[0088] Value = sign × mantissa × 2 exp-127
[0089] The exponent has 8 bits that can represent the range of 0 to 255, which makes the exponent very large. Therefore, the IEEE 754 standard exponent offset is 127, which shifts the exponent range to between -127 and 128, which is a reasonable range. IEEE 754 further stipulates that there is an implicit digit to the left of the decimal point, which is usually 1, so the mantissa of the above single-precision number is actually 29 bits.
[0090] Since the above representation method limits the range and precision of floating-point numbers, floating-point numbers can only be represented approximately for calculation, so the problem of rounding has to be considered. In the decimal system, if you want to retain two decimal places, that is, keep the values of the tenth and hundredth places, then the reserved digit is the lowest digit of the value, that is, the hundredth digit, and the approximate digit is the first digit to be discarded, that is, the thousandth digit, and all digits after the thousandth digit are collectively called sticky bits, and all the information of the sticky bits is lost. For binary, if you want to retain two decimal places, then the second digit to the right of the decimal point is the reserved digit, the third digit to the right of the decimal point is the approximate digit, and all decimal digits from the fourth digit to the right of the decimal point are sticky bits.
[0091] To this end, IEEE 754 defines four different rounding methods: rounding to even numbers, rounding to zero, rounding down, and rounding up. IEEE 754 uses rounding to even numbers by default.
[0092] In the algorithms in the field of artificial intelligence, vector multiplication and matrix multiplication and accumulation account for more than 90% of the operations. Therefore, in the design of the processor's arithmetic circuit, the circuit design overhead of the multiplication operation is mainly considered. In order to meet the multiplication and multiplication and accumulation operation requirements of various types of floating-point numbers mentioned above, how to design the multiplication circuit to improve the circuit reuse efficiency is the focus of the arithmetic unit design.
[0093] like Figure 4A As shown, an embodiment of the present invention provides a floating point number calculation device 4000, in which a plurality of calculation devices are arranged inside a processor, and the processing bit widths of the exponent calculation circuit and the mantissa calculation circuit in each calculation device are different. In this embodiment, the device includes:
[0094] Identification circuit 4001, used for identifying the bit width of the exponent bit and the mantissa bit of the floating point number when the operand is a floating point number;
[0095] The allocation circuit 4002 is used to allocate the exponent and mantissa of the operand to a first exponent operation circuit in a first computing device and a first mantissa operation circuit in a second computing device for processing according to the identified exponent bit width and mantissa bit width, wherein the first computing device and the second computing device respectively include an exponent operation circuit and a mantissa operation circuit, the processing bit width of the first exponent operation circuit is greater than or equal to the exponent bit width of the operand, and the processing bit width of the first mantissa operation circuit is greater than or equal to the mantissa bit width of the operand.
[0096] The identification circuit 4001 and the distribution circuit 4002 provided by the present invention can be arranged in Figure 3A The control module 301 shown may also be arranged in the computing module 302. The present invention does not limit this. The computing device provided in the embodiment of the present invention, such as the first computing device and the second computing device, may be arranged in the computing module 302.
[0097] like Figure 4A As shown, after the identification circuit 4001 identifies the bit width of the exponent bit and the mantissa bit of the floating-point number, the identification result is transmitted to the distribution circuit 4002. The distribution circuit 4002 sends the exponent of the floating-point number to the exponent processing circuit in the first computing device for processing, and sends the mantissa of the floating-point number to the mantissa processing circuit in the second computing device for processing according to the identification result. The data flow is shown by the arrows in the figure. After the first computing device and the second computing device process the exponent and the mantissa respectively, the subsequent possible operations are no longer shown in the present invention. For example, the final or stage-by-stage processing result of the floating-point number is obtained according to the exponent processing result and the mantissa result.
[0098] The operands in the present invention may be data to be processed or operated received by the processor or each circuit module inside the processor, or may be operands in ISA (Instruction Set Architecture) instructions received by the processor, which will not be described in detail here. The first computing device and the second computing device in the present invention may be a multiplication circuit, a multiplication-accumulation circuit or a factorial-accumulation circuit, or other computing circuits or logic operation circuits that occupy a larger hardware overhead, such as a shift circuit, a comparison circuit, etc., which are not limited in the present invention.
[0099] With the continuous development of machine learning technology, the data formats of floating-point numbers are becoming more and more diverse, including at least: FP32, TF32, BF16, FP16, UHP, and FP8. Among them, the sign bit of FP32 is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 23 bits. The sign bit of TF32 is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 10 bits. The sign bit of BF16 is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 7 bits. The sign bit of FP16 is 1 bit, the exponent bit is 5 bits, and the mantissa bit is 10 bits. UHP has no sign bit, the exponent bit is 6 bits, and the mantissa bit is 10 bits. FP8 has two formats, one with a sign bit of 1 bit, an exponent bit of 4 bits, and a mantissa bit of 3 bits, and the other with a sign bit of 1 bit, an exponent bit of 5 bits, and a mantissa bit of 2 bits.
[0100] The exponent bit width of a floating-point number determines the data expression range, and the mantissa bit width determines the data expression accuracy. The data formats of the above floating-point numbers can be roughly divided into two categories. The first category has a longer mantissa bit width, such as FP32, TF32, BF16, FP16, and UHP, and the second category has a shorter mantissa bit width, such as two formats of FP8. In addition, the exponent bit width of the first type of floating-point numbers is also longer, and the exponent bit width of the second type of floating-point numbers is also shorter. Different types of floating-point numbers have different exponent bit widths and mantissa bit widths, different data expression ranges, and different data expression accuracy, which can adapt to various application scenarios in the field of machine learning, the processing capabilities of various software or hardware carriers, or the complex requirements of different usage stages.
[0101] The data expression range and data accuracy of the first type of floating-point numbers mentioned above are guaranteed, but when used in software and hardware operating carriers, it requires a large resource overhead. In scenarios where the accuracy requirements are not high, it will cause a waste of computing resources, storage resources, and transmission resources, reduce computing efficiency, and occupy a large chip area. The second type of floating-point numbers can improve the computing efficiency of software and hardware operating carriers and reduce chip area on the premise of meeting the accuracy requirements, but there is a large loss in data expression range and data expression accuracy compared to the first type of floating-point numbers.
[0102] When designing a computing device in a chip or processor, in order to enable the computing device to support the above two types of floating-point numbers, two corresponding computing devices can be designed, each of which includes an exponential operation circuit and a mantissa operation circuit, which are used to process various calculations of the exponent bit and mantissa bit of the floating-point number, including logical operations and mathematical operations. The exponential operation circuit and the mantissa operation circuit may include at least one more basic circuit structure such as an adder, a multiplier, a shifter, a comparator, etc., and the present invention does not limit this.
[0103] In an embodiment of the present invention, two computing devices may be provided, wherein the first computing device is used to process the first type of floating-point numbers, and its exponent processing bit width and mantissa processing bit width are both longer, and the second computing device is used to process the second type of floating-point numbers, and its exponent processing bit width and mantissa processing bit width are both shorter. Since the operation logic of the multiplication circuit is more complex compared with the addition circuit and the comparison circuit, both the operator and the data storage need to occupy a larger chip area and occupy a larger hardware overhead. For the multiplication circuit, the multiplication circuit with a long bit width occupies a larger hardware overhead than the multiplication circuit with a short bit width. For example, the hardware overhead of a 10-bit bit width multiplication circuit is much greater than the hardware overhead of a 3-bit bit width multiplication circuit. The larger hardware overhead means that a larger chip area needs to be occupied or more energy consumption is used during operation, a larger space is occupied when data is stored, and a larger bandwidth is occupied when data is transmitted.
[0104] In an embodiment of the present invention, different mantissa operation circuits can be set in the processor according to the type of software model, the category of data to be processed, and the processing requirements. The processing bit width of the mantissa operation circuit in the first operation circuit can be set to 10 bits, and the processing bit width of the exponential operation circuit of the first operation circuit is 8 bits. The processing bit width of the mantissa operation circuit in the second operation circuit can be set to 3 bits, 4 bits or 5 bits, and the processing bit width of the exponential operation circuit of the second operation circuit is set to 4 bits or 5 bits. Thus, FP32, TF32, BF16, FP16, UHP, each mantissa bit width is greater than 5, and it is allocated to the first operation circuit for operation, while the mantissa bit width of FP8 is less than 5, and it is allocated to the second operation circuit for operation. The mantissa operation circuit with a bit width of 3, 4 or 5 bits saves a lot of hardware overhead compared to the mantissa operation circuit with a bit width of 10 bits, and the combination of the mantissa operation circuits with long and short bit widths can cover the above-mentioned various types of floating-point number operations, realize the high reuse of the operation circuit, and save the chip area to the maximum extent. The exponent operation circuit and the mantissa operation circuit in the first calculation device and the second calculation device may also be set to other bit widths to meet the processing requirements of different types of floating-point numbers, and the present invention does not limit this.
[0105] like Figure 4BAs shown, in the embodiment of the present invention, the processing bit widths of the exponential operation circuit and the mantissa operation circuit of each computing device can be simplified and set in combination with the needs of actual applications, and different data types can be applied through internal data format conversion and the like. Specifically, the first computing device and the second computing device can be configured to process floating-point numbers of BF16 and FP16 types. That is, the first computing device is used to process floating-point numbers of BF16, and the processing bit width of its exponential operation circuit is 8 bits, and the processing bit width of its mantissa operation circuit is 7 bits. The second computing device is used to process floating-point numbers of FP16 type, and the processing bit width of its exponential operation circuit is 5 bits, and the processing bit width of its mantissa operation circuit is 10 bits. When the recognition circuit recognizes a floating-point number in TF32 format, its exponent bit can be allocated to the exponential operation circuit in the first computing device for processing, and its mantissa bit can be allocated to the mantissa operation circuit in the second computing device for processing, so as to achieve the purpose of using the computing devices of BF16 and FP16 to process floating-point numbers in TF32 format, and on the basis of supporting more data types, the design overhead of the hardware circuit is reduced, and the chip area is reduced.
[0106] like Figure 4C As shown, in the embodiment of the present invention, more than two computing devices may be provided. For example, a third computing device for processing floating-point numbers in FP8 format may be provided, and its exponent operation circuit is 5 bits, and the processing bit width of the mantissa operation circuit is 3 bits, so as to correspond to two formats of FP8. The present invention does not limit the number of computing devices and the setting of the processing bit width of the exponent operation circuit and the mantissa operation circuit in each computing device.
[0107] like Figure 4D As shown, in an embodiment of the present invention, a new data format FP12 is proposed. The data format of FP12 can be any of the following three types:
[0108] 1. 1 sign bit, 8 exponent bits, and 3 mantissa bits. (Can be referred to as 183 format)
[0109] 2. 1 sign bit, 5 exponent bits, and 6 mantissa bits. (Can be referred to as 156 format)
[0110] 3. 1 sign bit, 4 exponent bits, and 7 mantissa bits. (Can be referred to as 147 format)
[0111] It is understandable that the FP12 floating point number in the 183 format has a similar bit width to the mantissa bit of the second type of floating point number, but has a larger data expression range than the second type of floating point number. The FP12 floating point number in the 156 format and the 147 format has a similar bit width to the exponent bit of the second type of floating point number, but has a higher precision expression than the second type of floating point number. In addition to the above two types of floating point numbers, the FP12 format provides more flexible data expression for upper-layer applications.
[0112] In an embodiment of the present invention, the FP12 of 156 format can reuse the exponential operation circuit of the above-mentioned BF16 or FP16 computing device and the mantissa operation circuit of the BF16 computing device; the FP12 of 183 format can reuse the exponential operation circuit of the BF16 computing device and the mantissa operation circuit of the BF16, FP16 or FP8 computing device; the FP12 of 147 format can reuse the exponential operation circuit of the above-mentioned FP16, BF16, FP8 computing device and the mantissa operation circuit of the BF16 or FP16 format. The FP12 data format proposed in the present invention, while having the advantages of the above-mentioned two types of floating-point numbers, can directly reuse the computing devices of the processors designed for the above-mentioned two types of floating-point numbers, without separately setting up computing circuits of different bit widths, while providing more flexible data expression, without increasing the chip area, and further improving the processing efficiency of the processor.
[0113] In the embodiment of the present invention, the identification circuit and the allocation circuit in the computing device can be implemented by various hardware circuits such as comparators, and the present invention does not limit the specific implementation.
[0114] In an embodiment of the present invention, the above-mentioned floating-point number calculation device also includes a configuration circuit, which is used to: when the processing bit width of the first mantissa operation circuit is greater than or equal to twice the mantissa bit width of the operand, configure the first mantissa operation circuit into a plurality of second mantissa operation circuits whose processing bit width is greater than or equal to the mantissa bit width of the operand; and or when the processing bit width of the first exponent operation circuit is greater than or equal to twice the exponent bit width of the operand, configure the first exponent operation circuit into a plurality of second exponent operation circuits whose processing bit width is greater than or equal to the exponent bit width of the operand.
[0115] like Figure 4E As shown, in an embodiment of the present invention, when the processing bit width of the first mantissa operation circuit is twice or more than the mantissa bit width of the operand, the first mantissa operation circuit can be configured as two or more second mantissa operation circuits, and each second mantissa operation circuit can meet the processing requirements of the mantissa bit width of the operand. Figure 4E As shown, the processing bit width of the mantissa operation circuit of the FP16 computing device is 10 bits. When the floating point number of FP8 uses the first mantissa operation circuit of the FP16 computing device, the 10-bit first mantissa operation circuit can be configured as 2 3-bit to 5-bit or 3 3-bit second mantissa operation circuits. And 2 or 3 second mantissa operation circuits can work at the same time. The multiplexing circuit can save chip area while enabling the processor or chip to support floating point numbers of more data types. The multiplexing of the exponent circuit is similarly not repeated.
[0116] In an embodiment of the present invention, a plurality of second mantissa operation circuits may operate simultaneously, thereby doubling the computing power of a processor or chip while multiplexing the circuits.
[0117] In an embodiment of the present invention, the configuration circuit is further used to configure the bandwidth and storage space of data transmission in the first computing device and the second computing device according to the exponent bit width and the mantissa bit width of the operand.
[0118] In the actual operation of a chip or processor, the data transmission or data read / write bandwidth between the chip and the storage medium or the data transmission or data read / write bandwidth between the storage media within the chip can easily become the bottleneck of the chip computing power. While reusing the computing circuit of the computing device, it is necessary to configure the corresponding data transmission bandwidth or data read bandwidth according to the exponent bit width and mantissa bit width of the operand to be supported, and configure the corresponding data storage space to cooperate with the data operation of the computing circuit to maximize the chip computing power.
[0119] In an embodiment of the present invention, the configuration circuit is also used to: configure the number of data multiplexing times according to the total data volume of the operands and the bandwidth, so that the configured number of data multiplexing times satisfies the total data volume used by at least one of the first exponential operation circuits and / or the first mantissa operation circuit in one operation cycle, and the first calculation device and the second calculation device are used to control data reading and / or operation according to the configured number of data multiplexing times.
[0120] In the field of artificial intelligence, the amount of data that needs to be processed is very large, and the chip or processor needs to process the operands in batches. Based on the reuse of the operation circuit, the present invention proposes an operand reuse method.
[0121] In an embodiment of the present invention, the configuration circuit is specifically used to: determine the minimum value of the data multiplexing times according to the total data volume of the operands and the bandwidth, and configure the data multiplexing times according to the minimum value so that the configured data multiplexing times is greater than or equal to the minimum value.
[0122] In an embodiment of the present invention, the data multiplexing times include spatial multiplexing times; the spatial multiplexing times are the number of mantissa operation circuits that process the operands; the configuration circuit is used to: obtain a default time multiplexing times; calculate the minimum value of the spatial multiplexing times based on the minimum value of the data multiplexing times and the default time multiplexing times; configure the spatial multiplexing times based on the minimum value of the spatial multiplexing times so that the configured spatial multiplexing times are greater than or equal to the minimum value of the spatial multiplexing times.
[0123] In an embodiment of the present invention, the data multiplexing times include time multiplexing times, and the time multiplexing times are the number of operation cycles used by the first computing device and the second computing device to calculate the operands; the configuration circuit is used to: obtain a default spatial multiplexing times, and calculate the minimum value of the time multiplexing times according to the minimum value of the data multiplexing times and the default spatial multiplexing times; configure the time multiplexing times according to the minimum value of the time multiplexing times so that the configured time multiplexing times are greater than or equal to the minimum value of the time multiplexing times.
[0124] In an embodiment of the present invention, the data multiplexing times include spatial multiplexing times and time multiplexing times, and the configuration circuit is used to: configure the spatial multiplexing times and the time multiplexing times according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and the spatial multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0125] When a chip or processor processes data, it may happen that the amount of data written into the operation circuit is insufficient when the operation circuit performs data operation, resulting in the inability to effectively reflect the chip's computing power, which in turn leads to a low data processing efficiency of the chip. In order to effectively guarantee the computing power of the chip, it is necessary to provide sufficient data for the operation circuit. The basic standard for the operation circuit to obtain sufficient data can be: the amount of data used by all operation circuits in one operation cycle can meet the operation requirements of all operation circuits. Various algorithms in the field of artificial intelligence executed by chips or processors have the characteristics of large data volume of operands and high repeatability of calculation operations. There will be a situation where some operands are repeatedly calculated, such as the convolution operation of weight data and neuron data, and the weight data is used multiple times to calculate with part of the neuron data. The convolution operation can be completed by multiple operation circuits in collaboration. It may be that multiple operation circuits use the same weight data to jointly complete the operation of weight data and one neuron data, or it may be that each operation circuit uses part of the weight data to complete the operation with one or part of the neuron data, that is, when the chip or processor uses multiple operation circuits to complete various algorithms in the field of artificial intelligence, the operands or parts of the operands will be repeatedly calculated and used. Since the maximum bandwidth is fixed after the chip or processor is designed, if each operation of each operation circuit needs to re-extract the data of the repeated calculation part, there will be a huge waste of bandwidth, and the data transmission or data reading and writing bandwidth will become the biggest bottleneck of the chip computing power. The operand reuse method proposed in the embodiment of the present invention allows the operands that can be reused to be used multiple times by the operation circuit after being transmitted or read and written once through the bandwidth, which saves bandwidth and can also maximize the computing power of the chip.
[0126] It should be noted that the operand multiplexing method provided by the present invention can be applied to various data processing application scenarios using chips, such as the training process and reasoning process of various machine learning models and deep learning models in the field of artificial intelligence, image processing, speech processing and other scenarios, and the application scenarios are not limited in this embodiment.
[0127] In an embodiment of the present invention, it is proposed how to determine the number of times an operand is reused, and further proposed a spatial reuse method, a temporal reuse method, and a spatial and temporal reuse method of the operand. On the basis of the reuse of the operation circuit, the operand is reused so that the total amount of data provided to the operation circuit can meet the operation requirements of the operation circuit, avoiding bandwidth from becoming a computing power bottleneck, and maximizing the computing power of the chip.
[0128] In order to fully utilize the computing power of the computing device, the chip of this embodiment can implement an operand multiplexing method. Specifically, the computing device includes a controller unit ( Figure 3A The control module shown in, for details, please refer to the above) and at least one operation circuit, for details, please refer to the above.
[0129] The operand multiplexing method comprises:
[0130] Obtain the total amount of data used by at least one operation circuit in the computing device in one operation cycle; configure the number of data multiplexing times according to the total amount of data and the preset bandwidth so that the configured number of data multiplexing times satisfies the total amount of data used by at least one operation circuit in one operation cycle; and control the data reading and / or operation of the target operation circuit participating in the operation according to the configured number of data multiplexing times.
[0131] Optionally, when the number of data multiplexing times is configured according to the total data volume and the preset bandwidth so that the configured number of data multiplexing times satisfies the total data volume used by at least one computing circuit in one computing cycle:
[0132] Determine the minimum value of the number of data multiplexing times that needs to be configured according to the total data volume and the preset bandwidth; configure the number of data multiplexing times according to the minimum value of the number of data multiplexing times so that the configured number of data multiplexing times is greater than or equal to the minimum value of the number of data multiplexing times.
[0133] Optionally, the number of data multiplexing times includes: the number of spatial multiplexing times; the number of spatial multiplexing times is the number of target operation circuits using the same data, and the controller unit, when configuring the number of data multiplexing times according to the minimum value of the number of data multiplexing times so that the configured number of data multiplexing times is greater than or equal to the minimum value of the number of data multiplexing times, is configured as:
[0134] Obtain a default time multiplexing number; calculate the minimum value of the spatial multiplexing number based on the minimum value of the data multiplexing number and the default time multiplexing number; configure the spatial multiplexing number according to the minimum value of the spatial multiplexing number so that the configured spatial multiplexing number is greater than or equal to the minimum value of the spatial multiplexing number.
[0135] Optionally, when calculating the minimum value of the spatial multiplexing number according to the minimum value of the data multiplexing number and the default time multiplexing number:
[0136] Calculate the quotient of the minimum value of the data multiplexing times and the default time multiplexing times; and determine the quotient as the minimum value of the space multiplexing times.
[0137] Optionally, the number of data multiplexing times includes: the number of time multiplexing times; the number of time multiplexing times is the number of operation cycles used by the target operation circuit for the same data;
[0138] Accordingly, when the number of data multiplexing times is configured according to the minimum number of data multiplexing times so that the configured number of data multiplexing times is greater than or equal to the minimum number of data multiplexing times, it is configured as follows:
[0139] Obtain the default spatial multiplexing number; calculate the minimum time multiplexing number based on the minimum data multiplexing number and the default spatial multiplexing number; configure the time multiplexing number based on the minimum time multiplexing number so that the configured time multiplexing number is greater than or equal to the minimum time multiplexing number.
[0140] Optionally, the data multiplexing times include time multiplexing times and space multiplexing times;
[0141] Accordingly, when the number of data multiplexing times is configured according to the minimum number of data multiplexing times so that the configured number of data multiplexing times is greater than or equal to the minimum number of data multiplexing times, it is configured as follows:
[0142] The time multiplexing times and the space multiplexing times are configured according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and the space multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0143] Optionally, the data processed by the computing device includes alignment data, and the data type of the alignment data includes but is not limited to any one of the following types: BF16, FP16, FP32;
[0144] The preset bandwidth is an integer multiple of the number of bits representing the data type of the alignment data.
[0145] Optionally, the data processed by the computing device includes unaligned data, and the data type of the unaligned data includes but is not limited to TF32 type; the preset bandwidth is an integer multiple of the number of bits of data representation of the data type of the unaligned data.
[0146] like Figure 5 As shown, the present invention also provides a floating point number calculation method, comprising:
[0147] Step S5001, when the operand is a floating point number, identifying the bit width of the exponent bit and the mantissa bit of the floating point number;
[0148] Step S5002: According to the identified exponent bit width and mantissa bit width, the exponent and mantissa of the operand are respectively allocated to a first exponent operation circuit in a first computing device and a first mantissa operation circuit in a second computing device for processing, wherein the first computing device and the second computing device respectively include an exponent operation circuit and a mantissa operation circuit, the processing bit width of the first exponent operation circuit is greater than or equal to the exponent bit width of the operand, and the processing bit width of the first mantissa operation circuit is greater than or equal to the mantissa bit width of the operand.
[0149] In one aspect, the method provided by the present invention also includes: when the processing bit width of the first mantissa operation circuit is greater than or equal to twice the mantissa bit width of the operand, configuring the first mantissa operation circuit into multiple second mantissa operation circuits whose processing bit width is greater than or equal to the mantissa bit width of the operand, and / or when the processing bit width of the first exponent operation circuit is greater than or equal to twice the exponent bit width of the operand, configuring the first exponent operation circuit into multiple second exponent operation circuits whose processing bit width is greater than or equal to the exponent bit width of the operand.
[0150] In one aspect, in the method provided by the present invention, the plurality of second exponent operation circuits operate simultaneously, and / or the plurality of second mantissa operation circuits operate simultaneously.
[0151] In one aspect, the present invention provides a method in which the first computing device or the second computing device is configured to process floating point numbers of BF16, FP16 or FP8 type.
[0152] In one aspect, in the method provided by the present invention, the floating point number includes at least one of FP32, TF32, BF16, FP16, UHP, FP8, and FP12.
[0153] In one aspect, the method provided by the present invention, the sign bit of the FP12 format is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 3 bits.
[0154] In one aspect, the method provided by the present invention, the sign bit of the FP12 format is 1 bit, the exponent bit is 5 bits, and the mantissa bit is 6 bits.
[0155] In one aspect, the method provided by the present invention, the sign bit of the FP12 format is 1 bit, the exponent bit is 4 bits, and the mantissa bit is 7 bits.
[0156] In one aspect, the method provided by the present invention further includes: configuring the bandwidth and storage space of data transmission in the first computing device and the second computing device according to the exponent bit width and the mantissa bit width of the operand.
[0157] In one aspect, the method provided by the present invention further includes: configuring the number of data multiplexing times according to the total data volume of the operands and the bandwidth, so that the configured number of data multiplexing times satisfies the total data volume used by at least one of the first exponential operation circuits and / or the first mantissa operation circuit in one operation cycle, and the first computing device and the second computing device are used to control data reading and / or operation according to the configured number of data multiplexing times.
[0158] In one aspect, the method provided by the present invention includes: determining the minimum value of the data multiplexing times according to the total data volume of the operands and the bandwidth, and configuring the data multiplexing times according to the minimum value so that the configured data multiplexing times is greater than or equal to the minimum value.
[0159] In one aspect, the method provided by the present invention, the data multiplexing times include spatial multiplexing times; the spatial multiplexing times are the number of mantissa operation circuits that process the operands; the method includes: obtaining a default time multiplexing times; calculating the minimum value of the spatial multiplexing times based on the minimum value of the data multiplexing times and the default time multiplexing times; configuring the spatial multiplexing times based on the minimum value of the spatial multiplexing times so that the configured spatial multiplexing times are greater than or equal to the minimum value of the spatial multiplexing times.
[0160] In one aspect, the method provided by the present invention, the data multiplexing number includes the time multiplexing number, and the time multiplexing number is the number of operation cycles used by the first computing device and the second computing device to calculate the operand; the method includes: obtaining a default spatial multiplexing number, and calculating the minimum value of the time multiplexing number according to the minimum value of the data multiplexing number and the default spatial multiplexing number; configuring the time multiplexing number according to the minimum value of the time multiplexing number, so that the configured time multiplexing number is greater than or equal to the minimum value of the time multiplexing number.
[0161] In one aspect, the method provided by the present invention, the data multiplexing times include spatial multiplexing times and time multiplexing times, and the method includes: configuring the spatial multiplexing times and the time multiplexing times according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and the spatial multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0162] In one aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program code of a floating-point number calculation method, and when the computer program code is executed by a processing device, any of the above-described methods is executed.
[0163] In one aspect, the present invention provides a computer program product, comprising a computer program of a floating point number calculation method, wherein the computer program implements the steps of any one of the above methods when executed by a processor.
[0164] In one aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the above methods.
[0165] The floating-point number calculation device provided in this embodiment can execute the chip computing power control method provided in any of the following embodiments to complete the reuse of operands. The specific principles and effects will be explained one by one in the following embodiments.
[0166] Fig. 6A This is a flow chart of a chip computing power control method provided by an embodiment of the present invention. Fig. 6A As shown, the execution subject of the chip computing power control method provided in this embodiment is, for example, Figure 1-Figure 3B The present invention does not limit the chip, floating point calculation device, control module in the chip, etc. shown in the embodiment. For the convenience of explanation, the following takes the execution subject as an example of a chip, and the chip has a preset bandwidth. The chip includes a calculation device, and the calculation device includes at least one operation circuit. Then the computing power control method of the chip provided in this embodiment includes the following steps:
[0167] Step 501, obtaining a total amount of data used by at least one computing circuit in a computing device in one computing cycle.
[0168] In this embodiment, the number of computing circuits included in the computing device is determined, and the amount of data required by one computing circuit in one computing cycle is determined, and the total amount of data used by all computing circuits in the computing device in one computing cycle is calculated based on the number of computing circuits and the amount of data required by one computing circuit in one computing cycle.
[0169] The total amount of data used by all computing circuits in a computing device in one computing cycle can be expressed as T, the number of computing circuits included in the computing device can be expressed as N, and the amount of data required by one computing circuit in one computing cycle can be expressed as t. Then T=N*t.
[0170] For example, if the number of computing circuits in the computing device is N=64, and the amount of data required by one computing circuit in one computing cycle is t=256 Byte, then T=64*256 Byte.
[0171] It can be understood that the total amount of data used by all computing circuits in the computing device in one computing cycle is a basic criterion for the computing circuits to obtain sufficient data.
[0172] Step 502: configure the number of data multiplexing times according to the total data volume and the preset bandwidth, so that the configured number of data multiplexing times satisfies the total data volume used by at least one computing circuit in one computing cycle.
[0173] The preset bandwidth is the bandwidth of the chip after the design is completed, and the bandwidth can be the maximum data transmission per unit time when the main processing circuit or the slave processing circuit interacts with other functional modules. For example, the bandwidth can refer to the data read and write bandwidth between the main processing circuit or the slave processing circuit (or the computing circuit therein) and the corresponding memory, or it can be the data transmission bandwidth between the main processing circuit and the slave processing circuit (or the computing circuit therein).
[0174] In this embodiment, the bandwidth of a single computing circuit can be calculated based on the preset bandwidth of the chip and the number of computing circuits included in the computing device. The bandwidth of a single computing circuit can be expressed as: w.
[0175] Then in this embodiment, when the number of data multiplexing times is configured according to the total data volume and the preset bandwidth, the number of data multiplexing times can be configured according to the total data volume and the bandwidth of a single computing circuit. Wherein, when the number of data multiplexing times is configured, it is necessary to satisfy that the product of the configured number of data multiplexing times and the bandwidth of a single computing circuit is greater than or equal to the total data volume used by all computing circuits in one computing cycle, so that the configured number of data multiplexing times satisfies the total data volume used by all computing circuits in one computing cycle.
[0176] The number of data reuses can be expressed as M, and the number of data reuses after configuration satisfies the following formula:
[0177] M*w>= N*t Formula (1)
[0178] Step 503 , controlling data reading and / or operation of the target operation circuit participating in the operation according to the configured data multiplexing times.
[0179] For example, if the configured data reuse times are M, the target operation circuit involved in the operation can be controlled to read the data, and then the operation of the M read data can be recycled, and the amount of data read each time is equal to the maximum amount of data allowed by the bandwidth of a single operation circuit.
[0180] The chip computing power control method provided in this embodiment obtains the total amount of data used by at least one computing circuit in a computing device in one computing cycle; configures the number of data reuses according to the total amount of data and the preset bandwidth so that the configured number of data reuses meets the total amount of data used by at least one computing circuit in one computing cycle; and controls the data reading and / or operation of the target computing circuit participating in the operation according to the configured number of data reuses. Since the total amount of data used by all computing circuits in one computing cycle is the basic standard for the computing circuit to obtain sufficient data, and the amount of data that can be used in the target computing circuit is related to the preset bandwidth and the number of data reuses of the chip, when the number of data reuses is configured based on the total amount of data used by all computing circuits in one computing cycle and the preset bandwidth, the basic standard of sufficient data can be met, and then after controlling the data reading and / or operation of the target computing circuit participating in the operation according to the configured number of data reuses, the computing power of the chip can be effectively guaranteed, and the data processing efficiency of the chip can be effectively improved.
[0181] As an alternative, Figure 6B As shown, in this embodiment, step 502 includes the following steps:
[0182] Step 5021, determining the minimum value of the number of data multiplexing times that needs to be configured according to the total data volume and the preset bandwidth.
[0183] In this embodiment, the minimum value M of the number of data multiplexing times to be configured can be determined according to formula (1): min It can be expressed as shown in formula (2):
[0184] M min = N*t / w Formula (2)
[0185] Specifically, the total amount of data used by all operation circuits in one operation cycle T = N * t divided by the bandwidth w of a single operation circuit is the minimum value M of the number of data multiplexing times to be configured. min .
[0186] Step 5022: configure the number of data multiplexing times according to the minimum value of the number of data multiplexing times, so that the configured number of data multiplexing times is greater than or equal to the minimum value of the number of data multiplexing times.
[0187] In this embodiment, the number of data multiplexing times is configured to be equal to the minimum value M of the number of data multiplexing times. minWhen , it can be ensured that the amount of data available in the target operation circuit is equal to the total amount of data used by all operation circuits in one operation cycle, and sufficient data can be provided for the operation circuit. If the configured data reuse times are greater than the minimum value of the data reuse times, it can be further ensured that sufficient data is provided for the operation circuit, so the data reuse times are configured to be greater than or equal to the minimum value of the data reuse times.
[0188] The chip-based computing power control method provided in the present embodiment configures the number of data multiplexing times according to the total data volume and the preset bandwidth so that the configured number of data multiplexing times satisfies the total data volume used by at least one computing circuit in one computing cycle, and determines the minimum value of the number of data multiplexing times that needs to be configured according to the total data volume and the preset bandwidth; configures the number of data multiplexing times according to the minimum value of the number of data multiplexing times so that the configured number of data multiplexing times is greater than or equal to the minimum value of the number of data multiplexing times, and can ensure that sufficient data is provided to the computing circuit when the configured number of data multiplexing times is equal to the minimum value of the number of data multiplexing times; when the configured number of data multiplexing times is greater than the minimum value of the number of data multiplexing times, not only can it ensure that sufficient data is provided to the computing circuit, but also if the time multiplexing times included in the data multiplexing times are large, it can effectively reduce power consumption.
[0189] In this embodiment, the number of data multiplexing times that need to be configured may include at least one of the following times: a spatial multiplexing number and a temporal multiplexing number.
[0190] The number of spatial multiplexing times is the number of target operation circuits using the same data, and the number of temporal multiplexing times is the number of operation cycles used by the target operation circuit for the same data.
[0191] Then in this embodiment, if the data reuse times that need to be configured only include the spatial reuse times, the data reading of the target operation circuit participating in the operation is controlled according to the configured spatial reuse times, and the corresponding operation is started according to the default time reuse times after the data loading is completed. If the data reuse times only include the time reuse times, the data reading of the target operation circuit participating in the operation is controlled according to the default spatial reuse times, and the corresponding operation is started according to the configured time reuse times after the data loading is completed. If the data reuse times that need to be configured include both the spatial reuse times and the time reuse times, the data reading of the target operation circuit participating in the operation is controlled according to the configured spatial reuse times, and the corresponding operation is started according to the configured time reuse times after the data loading is completed.
[0192] As an optional implementation, Figure 7As shown, in this embodiment, the number of data multiplexing times includes: the number of spatial multiplexing times; the number of spatial multiplexing times is the number of target operation circuits using the same data. Accordingly, step 5022 includes the following steps:
[0193] Step 5022a, obtain the default time multiplexing number.
[0194] The default time multiplexing times are pre-stored and do not need to be configured. The default time multiplexing times can be expressed as M t’ .
[0195] The default time multiplexing times can be stored in a preset storage area, and the default time multiplexing times can be obtained from the preset storage area. The default time multiplexing times represent the default number of operation cycles used by the target operation circuit for the same data. For example, the default time multiplexing times M t’ =1, indicating that the number of operation cycles used by the target operation circuit for the same data is 1.
[0196] Step 5022b, calculate the minimum value of the spatial multiplexing number based on the minimum value of the data multiplexing number and the default time multiplexing number.
[0197] As an optional implementation, in this embodiment, step 5022b includes the following solutions:
[0198] Calculate the quotient of the minimum value of the data reuse times and the default time reuse times; and determine the quotient as the minimum value of the space reuse times.
[0199] Specifically, the minimum number of spatial multiplexing times can be expressed as M smin , then M smin It can be expressed as shown in formula (3):
[0200] M smin =M min / M t’ Formula (3)
[0201] Step 5022c, configure the spatial multiplexing times according to the minimum value of the spatial multiplexing times, so that the configured spatial multiplexing times is greater than or equal to the minimum value of the spatial multiplexing times.
[0202] In this embodiment, the number of spatial multiplexing times after configuration is represented as M s , then M s Greater than or equal to the minimum value M of the number of spatial multiplexing times smin When M s The product of the default time multiplexing times M t’ Satisfying formula (4)
[0203] M s *Mt’ >=N*t / w Formula (4)
[0204] In M s =M smin When M is used, it can ensure that the amount of data available in the target operation circuit is equal to the total amount of data used by all operation circuits in one operation cycle, and sufficient data can be provided for the operation circuit. s >M smin When , it can better ensure that sufficient data is provided to the operation circuit, so the configured spatial multiplexing times is configured to be greater than or equal to the minimum value of the spatial multiplexing times.
[0205] The chip computing power control method provided in this embodiment provides a configuration method for the configuration of the number of data reuse times. This embodiment configures the number of data reuse times according to the minimum value of the data reuse times, so that the configured number of data reuse times is greater than or equal to the minimum value of the data reuse times, and obtains the default time reuse times; calculates the minimum value of the space reuse times according to the minimum value of the data reuse times and the default time reuse times; configures the space reuse times according to the minimum value of the space reuse times, so that the configured space reuse times is greater than or equal to the minimum value of the space reuse times. Only the space reuse times are configured, and the default time reuse times are used, which effectively reduces the configuration amount of the data reuse times.
[0206] As another optional implementation, Figure 8 As shown, in this embodiment, the number of data multiplexing times includes: the number of time multiplexing times; the number of time multiplexing times is the number of operation cycles used by the target operation circuit for the same data. Accordingly, step 5022 includes the following steps:
[0207] Step 50221, obtain the default number of spatial reuse times.
[0208] The default number of spatial reuses is a number of spatial reuses that is pre-stored and does not need to be configured. The default number of spatial reuses can be expressed as M s’ .
[0209] The default number of spatial multiplexing times may be stored in a preset storage area, and the default number of spatial multiplexing times may be obtained from the preset storage area. The default number of spatial multiplexing times is represented by the default number of target operation circuits using the same data. For example, the default number of spatial multiplexing times M s’ It can be the number of all computing circuits included in the computing device, such as M s’ =64, indicating that the default number of target operation circuits using the same data is 64.
[0210] Step 50222, calculate the minimum value of the time multiplexing number based on the minimum value of the data multiplexing number and the default space multiplexing number.
[0211] As an optional implementation, in this embodiment, step 50222 includes the following solutions:
[0212] Calculate the quotient of the minimum value of the data reuse times and the default space reuse times; determine the quotient as the minimum value of the time reuse times.
[0213] Specifically, the minimum value of the time multiplexing times can be expressed as M tmin , then M tmin It can be expressed as shown in formula (5):
[0214] M tmin =M min / M s’ Formula (5)
[0215] Step 50223, configure the time multiplexing times according to the minimum time multiplexing times, so that the configured time multiplexing times is greater than or equal to the minimum time multiplexing times.
[0216] In this embodiment, the number of time multiplexing after configuration is represented as M t , then M t Greater than or equal to the minimum value M of the time multiplexing times tmin When M t The product of the default number of spatial reuse times M s’ Satisfying formula (6)
[0217] M t *M s’ >=N*t / w Formula (6)
[0218] In M t =M tmin When M is used, it can ensure that the amount of data available in the target operation circuit is equal to the total amount of data used by all operation circuits in one operation cycle, and sufficient data can be provided for the operation circuit. t >M tmin When , it can be more guaranteed that sufficient data is provided to the operation circuit, so the configured time multiplexing times is configured to be greater than or equal to the minimum time multiplexing times.
[0219] The chip computing power control method provided in this embodiment provides another configuration method for configuring the number of data reuse times. This embodiment configures the number of data reuse times according to the minimum value of the data reuse times, so that the configured number of data reuse times is greater than or equal to the minimum value of the data reuse times, and obtains the default number of spatial reuse times; calculates the minimum number of time reuse times according to the minimum value of the data reuse times and the default number of spatial reuse times; configures the number of time reuse times according to the minimum value of the time reuse times, so that the configured number of time reuse times is greater than or equal to the minimum value of the time reuse times. Only the time reuse times are configured, and the default number of spatial reuse times is used, which effectively reduces the amount of configuration of the data reuse times.
[0220] As another optional implementation, in this embodiment, the number of data multiplexing times includes the number of time multiplexing times and the number of space multiplexing times; accordingly, step 5022 includes the following scheme:
[0221] The time multiplexing times and the space multiplexing times are configured according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and the space multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0222] Among them, the configured time multiplexing times can be expressed as M t , the number of configured spatial reuses can be expressed as M s Then the minimum value of data reuse times is M min With M t and M s The relationship can be expressed as shown in formula (7):
[0223] M t *M s >= M min Formula (7)
[0224] Then M t and M s The product also satisfies formula (8):
[0225] M t *M s >= N*t / w Formula (8)
[0226] Among them, in M t *M s =M min When M is used, it can ensure that the amount of data available in the target operation circuit is equal to the total amount of data used by all operation circuits in one operation cycle, and sufficient data can be provided for the operation circuit. t *M s >M minWhen , it can be more guaranteed that sufficient data is provided to the operation circuit, so the product of the configured time multiplexing times and space multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0227] The chip computing power control method provided in this embodiment can flexibly configure the time multiplexing times and the space multiplexing times, and provides another configuration method for the configuration of the data multiplexing times. In this embodiment, when the data multiplexing times are configured according to the minimum value of the data multiplexing times so that the configured data multiplexing times are greater than or equal to the minimum value of the data multiplexing times, the time multiplexing times and the space multiplexing times are configured according to the minimum value of the data multiplexing times so that the product of the configured time multiplexing times and the space multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
[0228] Based on any one of the above embodiments, as an optional implementation, in this embodiment, the data processed by the computing device includes alignment data, and the data type of the alignment data includes but is not limited to any one of the following types: BF16, FP16, FP32.
[0229] The data type processed by the computing device of this embodiment may also be a floating point data type such as FP64, FP8, etc. Of course, the data type processed by the computing device of this embodiment may also be an integer data type, such as Int8, Int16, etc., which are not listed here one by one.
[0230] The preset bandwidth is an integer multiple of the number of bits of the data representation of the corresponding data type.
[0231] The FP32 data type has a data representation bit number of 32 bits, and the BF16 and FP16 data types both have a data representation bit number of 16 bits.
[0232] Among them, in the data representation of FP32, the number of sign bits is 1, the number of exponent bits is 8, and the number of mantissa bits is 23. In the data representation of BF16, the number of sign bits is 1, the number of exponent bits is 8, and the number of mantissa bits is 7. The data type of BF16 can reduce storage and processing costs, and can meet certain accuracy requirements. In the data representation of FP16, the number of sign bits is 1, the number of exponent bits is 5, and the number of mantissa bits is 10. The data type of FP16 can reduce storage and processing costs and is widely used in model training. In the data representation of FP64, the number of sign bits is 1, the number of exponent bits is 11, and the number of mantissa bits is 52. In the data representation of FP8, the number of sign bits is 1, the number of exponent bits is 4, and the number of mantissa bits is 3, or the number of sign bits is 1, the number of exponent bits is 5, and the number of mantissa bits is 2.
[0233] In this embodiment, when designing a chip, based on the data type processed by the computing device, the preset bandwidth is set to an integer multiple of the number of bits of the corresponding data type, which can ensure the integrity of the transmitted data. Optionally, the data processed by the computing device can be aligned data, and the aligned data can be data whose data type is aligned to an integer power of 2. In this case, the preset bandwidth can be K*2 n , where K is an integer greater than or equal to 1, and n is an integer greater than or equal to 1. This allows the preset bandwidth to meet the transmission of aligned data. For example, if the data type is FP32, the preset bandwidth can be set to K*2 n Wherein, K is an integer greater than or equal to 1, and n is an integer greater than or equal to 5. For example, K=1, n=5.
[0234] As an optional implementation, the computing device of this embodiment can also support the storage and operation of unaligned data, and unaligned data refers to data whose data type is not aligned to an integer power of 2. For example, the data type of the data is TF32. In the data representation of TF32, the sign bit is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 10 bits. The data type of TF32 has lower storage cost and higher calculation accuracy than the data type of FP16, and can better support neural network operations. When the data is of TF32 type, the number of bits represented is 19 bits, which is not aligned data. In order to meet the transmission requirements of different data types, the preset bandwidth also needs to be an integer multiple of other data types, in particular, the preset bandwidth needs to be an integer multiple of the number of bits represented by the unaligned data. In this embodiment, the data type of the unaligned data is of TF32 type, and the preset bandwidth is an integer multiple of the number of bits represented by the data type of the unaligned data.
[0235] In this embodiment, the TF32 data type has 19 effective bits, so the preset bandwidth is an integer multiple of the number of effective bits of the TF32 type. Therefore, when the bandwidth supports the transmission of both aligned data and unaligned data, the preset bandwidth can be K*L*2 n , where L is the number of bits for representing unaligned data, L is an integer greater than or equal to 1, K is an integer greater than or equal to 1, and n is an integer greater than or equal to 1. For example, when the data type is TF32, the preset bandwidth can be an integer multiple of 19. When the chip supports both TF32 and FP16 types, the preset bandwidth can be 19*2 4 .
[0236] Further, since the preset data bandwidth can support the transmission of non-aligned data, the computing device of this embodiment can support the reading and writing of non-aligned data. As described above, during the operation, the data is transferred from the first storage circuit or the second storage circuit to the first cache circuit or the second cache circuit of the computing device. In order to ensure the accuracy and integrity of the data, the non-aligned data can be stored in the first storage circuit and the second storage circuit in the form of aligned data. For example, TF32 type data will occupy 32 bits of storage space in the first storage circuit or the second storage circuit, of which 19 bits are valid bits and the remaining 13 bits are invalid. In this way, the upper-level software user does not need to perceive the special alignment requirements of non-aligned data when allocating resources of the storage circuit, thereby facilitating programming and use. During the operation, the non-aligned data is read from the first storage circuit or the second storage circuit to the cache circuit of the computing device to be sent to the operation circuit for corresponding operation. At this time, the non-aligned data can be stored in the first cache circuit or the second cache circuit in a non-aligned manner, thereby reducing the storage cost of the operator (which can be formed by the above-mentioned main processing circuit and slave processing circuit). For example, after the TF32 data is transferred to the operator, only the number of valid bits is stored. Afterwards, the operation circuit reads the effective number of bits from the first cache circuit or the second cache circuit to perform operation and obtain the corresponding operation result, which can be first stored in the third cache circuit. Furthermore, this embodiment can also transfer the operation result to the first storage circuit or the second storage circuit, and at this time, the non-aligned data in the computing device is converted into aligned data for storage.
[0237] The chip computing power control method provided in this embodiment is convenient for users because it still uses aligned data for storage when storing TF32 type data. When the computing device reads data, it only reads the valid bits of the non-aligned data; and uses the valid bits of the non-aligned data for data calculation, which can effectively reduce the amount of calculation and the amount of data reading.
[0238] In a possible implementation manner, a computer-readable storage medium is also disclosed. The computer-readable storage medium stores a computer program. When the computer program is executed by at least one processor, the method provided in any one of the embodiments is implemented.
[0239] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.
[0240] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0241] It should be understood that the above device embodiments are only illustrative, and the device of the present invention can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.
[0242] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present invention may be integrated into one unit / module, or each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.
[0243] If the integrated unit / module is implemented in the form of hardware, the hardware can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the artificial intelligence processor can be any appropriate hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory RRAM (Resistive Random Access Memory), dynamic random access memory DRAM (Dynamic Random Access Memory), static random access memory SRAM (Static Random-Access Memory), enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), high-bandwidth memory HBM (High-Bandwidth Memory), hybrid memory cube HMC (Hybrid Memory Cube), etc.
[0244] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0245] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. A floating point number calculation device, characterized in that: The device comprises: An identification circuit, for identifying the bit width of the exponent bit and the mantissa bit of the floating point number when the operand is a floating point number; An allocation circuit is used to allocate the exponent and mantissa of the operand to a first exponent operation circuit in a first computing device and a first mantissa operation circuit in a second computing device for processing according to the identified exponent bit width and mantissa bit width, wherein the first computing device and the second computing device respectively include an exponent operation circuit and a mantissa operation circuit, the processing bit width of the first exponent operation circuit is greater than or equal to the exponent bit width of the operand, and the processing bit width of the first mantissa operation circuit is greater than or equal to the mantissa bit width of the operand.
2. The apparatus according to claim 1, further comprising a configuration circuit for: When the processing bit width of the first mantissa operation circuit is greater than or equal to twice the mantissa bit width of the operand, the first mantissa operation circuit is configured as a plurality of second mantissa operation circuits having a processing bit width greater than or equal to the mantissa bit width of the operand, and / or When the processing bit width of the first exponential operation circuit is greater than or equal to twice the bit width of the operand exponent, the first exponential operation circuit is configured as a plurality of second exponential operation circuits having a processing bit width greater than or equal to the bit width of the operand exponent.
3. The device according to claim 2, wherein the plurality of second exponent operation circuits operate simultaneously, and / or the plurality of second mantissa operation circuits operate simultaneously. 4 . The device according to claim 1 , wherein the first computing device or the second computing device is configured to process floating point numbers of BF16, FP16 or FP8 type.
5. The apparatus according to claim 1, wherein the floating point number comprises at least one of FP32, TF32, BF16, FP16, UHP, FP8, and FP12.
6. The apparatus according to claim 5, wherein the sign bit of the FP12 format is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 3 bits.
7. The apparatus according to claim 5, wherein the sign bit of the FP12 format is 1 bit, the exponent bit is 5 bits, and the mantissa bit is 6 bits.
8. The apparatus according to claim 5, wherein the sign bit of the FP12 format is 1 bit, the exponent bit is 4 bits, and the mantissa bit is 7 bits.
9. The device according to claim 2, wherein the configuration circuit is further configured to: The bandwidth and storage space of data transmission in the first computing device and the second computing device are configured according to the exponent bit width and the mantissa bit width of the operand.
10. The device according to claim 9, wherein the configuration circuit is further configured to: The number of data multiplexing times is configured according to the total data volume of the operands and the bandwidth so that the configured number of data multiplexing times satisfies the total data volume used by at least one of the first exponential operation circuits and / or the first mantissa operation circuit in one operation cycle, and the first computing device and the second computing device are used to control data reading and / or operation according to the configured number of data multiplexing times.
11. The device according to claim 10, wherein the configuration circuit is specifically used for: The minimum value of the data multiplexing times is determined according to the total data volume of the operands and the bandwidth, and the data multiplexing times is configured according to the minimum value so that the configured data multiplexing times is greater than or equal to the minimum value.
12. The device according to claim 11, wherein the number of data multiplexing times includes the number of spatial multiplexing times; the number of spatial multiplexing times is the number of mantissa operation circuits processing the operands; and the configuration circuit is used to: Obtain a default time multiplexing number; calculate the minimum value of the spatial multiplexing number based on the minimum value of the data multiplexing number and the default time multiplexing number; configure the spatial multiplexing number according to the minimum value of the spatial multiplexing number so that the configured spatial multiplexing number is greater than or equal to the minimum value of the spatial multiplexing number.
13. The device according to claim 11, wherein the data multiplexing times include time multiplexing times, and the time multiplexing times are the number of operation cycles used by the first computing device and the second computing device to calculate the operand; and the configuration circuit is used to: Obtain the default spatial multiplexing number, and calculate the minimum time multiplexing number based on the minimum data multiplexing number and the default spatial multiplexing number; configure the time multiplexing number according to the minimum time multiplexing number, so that the configured time multiplexing number is greater than or equal to the minimum time multiplexing number.
14. The device according to claim 12 and 13, wherein the data multiplexing times include spatial multiplexing times and temporal multiplexing times, and the configuration circuit is used to: The spatial multiplexing times and the time multiplexing times are configured according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and spatial multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
15. An integrated circuit device comprising a device according to any one of claims 1 to 14.
16. A card comprising the integrated circuit device according to claim 15.
17. A floating point number calculation method, characterized in that: The method comprises: When the operand is a floating point number, identifying the bit width of the exponent bits and the mantissa bits of the floating point number; According to the identified exponent bit width and mantissa bit width, the exponent and mantissa of the operand are respectively allocated to a first exponent operation circuit in a first computing device and a first mantissa operation circuit in a second computing device for processing, wherein the first computing device and the second computing device respectively include an exponent operation circuit and a mantissa operation circuit, the processing bit width of the first exponent operation circuit is greater than or equal to the exponent bit width of the operand, and the processing bit width of the first mantissa operation circuit is greater than or equal to the mantissa bit width of the operand.
18. The method according to claim 17, further comprising: When the processing bit width of the first mantissa operation circuit is greater than or equal to twice the mantissa bit width of the operand, the first mantissa operation circuit is configured as a plurality of second mantissa operation circuits having a processing bit width greater than or equal to the mantissa bit width of the operand, and / or When the processing bit width of the first exponential operation circuit is greater than or equal to twice the bit width of the operand exponent, the first exponential operation circuit is configured as a plurality of second exponential operation circuits having a processing bit width greater than or equal to the bit width of the operand exponent.
19. The method according to claim 18, wherein the plurality of second exponent operation circuits operate simultaneously, and / or the plurality of second mantissa operation circuits operate simultaneously.
20. The method according to claim 17, wherein the first computing device or the second computing device is configured to process floating point numbers of BF16, FP16 or FP8 type.
21. The method according to claim 17, wherein the floating point number comprises at least one of FP32, TF32, BF16, FP16, UHP, FP8, and FP12.
22. The method according to claim 21, wherein the sign bit of the FP12 format is 1 bit, the exponent bit is 8 bits, and the mantissa bit is 3 bits.
23. The method according to claim 21, wherein the sign bit of the FP12 format is 1 bit, the exponent bits are 5 bits, and the mantissa bits are 6 bits.
24. The method according to claim 21, wherein the sign bit of the FP12 format is 1 bit, the exponent bits are 4 bits, and the mantissa bits are 7 bits.
25. The method according to claim 18, further comprising: The bandwidth and storage space of data transmission in the first computing device and the second computing device are configured according to the exponent bit width and the mantissa bit width of the operand.
26. The method according to claim 25, further comprising: The number of data multiplexing times is configured according to the total data volume of the operands and the bandwidth so that the configured number of data multiplexing times satisfies the total data volume used by at least one of the first exponential operation circuits and / or the first mantissa operation circuit in one operation cycle, and the first computing device and the second computing device are used to control data reading and / or operation according to the configured number of data multiplexing times.
27. The method according to claim 26, comprising: The minimum value of the data multiplexing times is determined according to the total data volume of the operands and the bandwidth, and the data multiplexing times is configured according to the minimum value so that the configured data multiplexing times is greater than or equal to the minimum value.
28. The method according to claim 27, wherein the number of data multiplexing times includes the number of spatial multiplexing times; the number of spatial multiplexing times is the number of mantissa operation circuits processing the operands; the method comprises: Get the default time reuse times; Calculating the minimum value of the spatial multiplexing number according to the minimum value of the data multiplexing number and the default time multiplexing number; The spatial multiplexing number is configured according to the minimum value of the spatial multiplexing number so that the configured spatial multiplexing number is greater than or equal to the minimum value of the spatial multiplexing number.
29. The method according to claim 27, wherein the data multiplexing times include time multiplexing times, and the time multiplexing times are the number of operation cycles used by the first computing device and the second computing device to calculate the operands; the method comprising: Obtain a default spatial multiplexing number, and calculate a minimum value of the time multiplexing number according to the minimum value of the data multiplexing number and the default spatial multiplexing number; The time multiplexing number is configured according to the minimum value of the time multiplexing number, so that the configured time multiplexing number is greater than or equal to the minimum value of the time multiplexing number.
30. The method according to claims 28 and 29, wherein the data multiplexing times include spatial multiplexing times and temporal multiplexing times, and the method comprises: The spatial multiplexing times and the time multiplexing times are configured according to the minimum value of the data multiplexing times, so that the product of the configured time multiplexing times and spatial multiplexing times is greater than or equal to the minimum value of the data multiplexing times.
31. A computer-readable storage medium having stored thereon a computer program code of a floating-point number calculation method, wherein when the computer program code is executed by a processing device, the method according to any one of claims 17 to 30 is executed.
32. A computer program product, comprising a computer program of a floating point number calculation method, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 17 to 30 are implemented.
33. A computer device comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 17 to 30.
