Operating system and method

Abstract

The invention discloses an operation system and method, which can be included in a combined processing device, and the combined processing device can also comprise a universal interconnection interface and other processing devices. And the computing device interacts with other processing devices to jointly complete the computing operation specified by the user. The combined processing device can further comprise a storage device, and the storage device is connected with the equipment and the other processing devices and used for storing data of the equipment and the other processing devices.

Description

Technical Field

[0001] The present disclosure generally relates to the field of data storage and more specifically to a data computing system and method. Background Art

[0002] Before the rise of deep learning hardware, most scientific computing was based on IEEE Float32 and IEEE Float64. With the development of deep learning and the evolution of artificial intelligence (AI) processors, research has found that during training and reasoning, the operating speed, power consumption, and chip area can be effectively improved by reducing the precision of floating-point operations.

[0003] For example, after algorithm research, in Leopard's AI-NR network, most network layer applications of FP16 can meet accuracy and performance requirements. However, some network layer applications of FP16 data type accuracy is insufficient, and TF32 or FP32 is required to meet accuracy requirements.

[0004] However, although FP32 has higher precision, its computing efficiency is lower, which is a problem that needs to be solved for some applications that require higher computing speeds. Summary of the invention

[0005] In order to at least partially solve one or more technical problems mentioned in the background technology, the present disclosure provides a computing system and method, which can implicitly convert data formats when necessary to achieve a required computing speed.

[0006] According to a first aspect of the present disclosure, a computing system is provided, which includes an instruction decoding unit, a front-end conversion unit and a computing unit, wherein the instruction decoding unit is configured to: receive a control instruction; decode a first data type of data to be operated and a type of operation for the data to be operated from the control instruction; if the first data type and the type of operation meet a preset condition, instruct the data to be operated to be input into the front-end conversion unit; the front-end conversion unit is configured to convert the data to be operated having the first data type into intermediate data having a second data type, wherein a first efficiency of operating on the data to be operated of the first data type is lower than a second efficiency of operating on the intermediate data of the second data type; and the computing unit is configured to receive the intermediate data from the front-end conversion unit and operate on the intermediate data.

[0007] According to a second aspect of the present disclosure, there is provided a computing method, comprising: receiving a control instruction; decoding a first data type of data to be operated and a computing type for the data to be operated from the control instruction; if the first data type and the computing type meet a preset condition, instructing to perform a pre-transformation on the data to be operated; the pre-transformation comprises: converting the data to be operated having the first data type into intermediate data having a second data type, wherein a first efficiency of operating on the data to be operated having the first data type is lower than a second efficiency of operating on the intermediate data of the second data type; and operating on the intermediate data.

[0008] According to a third aspect of the present disclosure, a chip is provided, comprising the computing system as described above.

[0009] According to a fourth aspect of the present disclosure, a board is provided, comprising the chip as described above.

[0010] According to one embodiment of the present disclosure, when it is detected that a specific condition is met, the system automatically converts the data into a required format, thereby implicitly improving the computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0012] Figure 1 A structural diagram of a board according to an embodiment of the present disclosure is shown;

[0013] Figure 2 A structural diagram showing a combined processing device according to an embodiment of the present disclosure;

[0014] Figure 3 A schematic diagram showing the internal structure of a computing device according to an embodiment of the present disclosure;

[0015] Figure 4a , 4b and 4c show schematic formats of several data types;

[0016] Figure 5 An exemplary flow chart showing a data storage method according to an embodiment of the present disclosure;

[0017] Figure 6 An exemplary flow chart showing a data access method according to an embodiment of the present disclosure;

[0018] Figure 7 An exemplary flow chart of a data operation method according to an embodiment of the present disclosure is shown;

[0019] Figure 8 A schematic diagram of space allocation according to an embodiment of the present disclosure is shown;

[0020] Fig. 9 A schematic block diagram of a computing system 900 according to an embodiment of the present disclosure is shown;

[0021] Fig.10 A schematic block diagram showing a computing system according to another embodiment of the present disclosure; and

[0022] Fig.11 An operation method according to one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0024] It should be understood that the terms "first", "second", "third", "fourth", etc. that may appear in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0025] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0026] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

[0027] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0028] Figure 1FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1 As shown, the board 10 includes a chip 101, which is a system-on-chip (SoC), or system on chip, integrated with one or more combined processing devices, which are artificial intelligence computing units that support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 10 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage, and powerful computing power.

[0029] The chip 101 is connected to an external device 103 via an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 103 to the chip 101 via the external interface device 102. The calculation result of the chip 101 can be transmitted back to the external device 103 via the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface.

[0030] The board 10 also includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected to the control device 106 and the chip 101 through a bus and transmits data. The control device 106 in the board 10 is configured to control the state of the chip 101. To this end, in an application scenario, the control device 106 may include a microcontroller (MCU).

[0031] Figure 2 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a storage device 204 .

[0032] The computing device 201 is configured to execute user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0033] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it into the storage device on the computing device 201 chip. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them into the control cache on the computing device 201 chip. Alternatively or optionally, the interface device 202 can also read data in the storage device of the computing device 201 and transmit it to the processing device 203.

[0034] The processing device 203, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 201 of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0035] The storage device 204 is used to store data to be processed, which may be DRAM or DDR memory, and is usually 16G or larger in size, and is used to store data of the computing device 201 and / or the processing device 203 .

[0036] Figure 3 The internal structure diagram of the computing device 201 is shown. The computing device 301 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 301 includes three modules: a control module 31 (also called a controller), an operation module 32 (also called an operator), and a storage module 33 (also called a memory).

[0037] The control module 31 is used to coordinate and control the operation of the operation module 32 and the storage module 33 to complete the deep learning task, and includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 is used to obtain instructions from the processing device 203, and the instruction decode unit 312 decodes the obtained instructions and sends the decoding results to the operation module 32 and the storage module 33 as control information.

[0038] The operation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 322 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.

[0039] The storage module 33 is used to store or transfer relevant data, including a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access module (DMA) 333. NRAM 331 is used to store input neurons, output neurons, and intermediate results after calculation; WRAM 332 is used to store the convolution kernel of the deep learning network, that is, the weight; DMA 333 is connected to DRAM 204 through bus 34, and is responsible for data transfer between the computing device 301 and DRAM 204.

[0040] Floating-point numbers are digital representations of numbers that belong to a specific subset of rational numbers. They are used in computers to approximate any real number, and the decimal point can "float". The IEEE floating-point standard represents a number in the form of V = (-1) ∧ sign*mantissa*2∧E. Among them, sign is the sign bit, 0 represents a positive number, and 1 represents a negative number; E represents the exponent, and the floating-point number is weighted, and the weight is 2 to the E power (may be a negative power); mantissa represents the number of bits, and mantissa is a binary decimal whose range is 1 to 2-ε, or 0-ε. The representation of floating-point numbers in computers is divided into three fields, and these fields are encoded separately:

[0041] (1) A single sign bit s directly encodes the sign s.

[0042] (2) The k-bit exponent field encodes the exponent, exp = e(k-1)...e(1)e(0).

[0043] (3) An n-bit decimal field mantissa, encoding the mantissa.

[0044] Figure 4a Schematic formats of several data types are shown.

[0045] As shown in the figure, FP32 has 32 bits, including a 1-bit sign bit s, an 8-bit exponent field E representing the numerical range, and a 23-bit mantissa B representing the precision.

[0046] FP16 has 16 bits, including 1-bit sign bit s, 5-bit exponent field E representing the numerical range, and 10-bit mantissa B representing the precision.

[0047] TF32 has 19 bits, including 1 bit for the sign bit s, 8 bits for the exponent field representing the numerical range, and 10 bits for the mantissa representing the precision.

[0048] It can be seen that TF32 uses the same 10-bit mantissa B as half-precision (FP16) math and the same 8-bit exponent E as FP32, so it can support the same numerical range.

[0049] Figure 4b shows a further comparison of FP32, FP16, and TF32 formats, and Figure 4c An example calculation process for FP32 is shown.

[0050] like Figure 4b As shown in the figure, the sign bits of FP32, TF32 and FP16 are all 1 bit; the exponent bits of FP32 and TF32 are 8 bits, and the sign bit of FP32 is 5 bits; and for the mantissa bits, FP32 is 23 bits, and TF32 and FP16 are both 10 bits. When performing multiplication in a 12-bit multiplier, for data in TF32 and FP16 formats, one operation cycle can complete the operation of one data; for the FP32 data format, the 23-bit mantissa bits need to be divided into two data, that is, data Ina is 12 bits, and data Inb is 11 bits (but it also needs to occupy 12 bits of space), as shown in Figure 4c When performing a multiplication operation, two operation cycles are required to complete the calculation: the first operation cycle implements 12-bit multiplication, the second time cycle processes 11-bit multiplication, and finally the multiplication results of the two time cycles are added according to the appropriate exponent bits. This makes the area overhead of the multiplier much larger than that of the adder.

[0051] To this end, multiple embodiments of the present disclosure solve at least one of the above-mentioned problems.

[0052] Fig. 9A schematic block diagram of a computing system 900 according to an embodiment of the present disclosure is shown. The system 900 may include an instruction decoding unit (IDU) 910, a pre-rotation unit 920, and a computing unit 930, wherein the instruction decoding unit 910 is configured to: receive a control instruction; decode a first data type of data to be operated and a computing type for the data to be operated from the control instruction; if the first data type and the computing type meet a preset condition, instruct the data to be operated to be input to the pre-rotation unit 920; the pre-rotation unit 920 is configured to convert the data to be operated having the first data type into intermediate data having the second data type, wherein a first efficiency of computing the data to be operated having the first data type is lower than a second efficiency of computing the intermediate data having the second data type; and the computing unit 930 is configured to receive the intermediate data from the pre-rotation unit 920 and to operate on the intermediate data.

[0053] like Fig. 9 As shown in FIG. 1 , the instruction signal is represented by a dotted line, and the data signal is represented by a solid line. The instruction decoding unit 910 may be as follows: Figure 3 The instruction decoding unit 312 shown can determine the type of data to be operated and the operation to be performed on these data according to the received control instructions, and comprehensively determine whether to convert the data to be operated based on this information. If conversion is required, the data to be operated from the RAM is instructed to be input into the front-end conversion unit 920. If conversion is not required, the data to be operated from the RAM is instructed to be directly input into the operation unit 930 for operation.

[0054] The front-end conversion unit 920 converts the data to be operated of one data type from the RAM when necessary to generate intermediate data of another data type. The first efficiency of operating the data to be operated of the first data type is lower than the second efficiency of operating the intermediate data of the second data type. It should be understood that the first data type and the second data type here do not refer to a specific type, but may refer to any data type that may currently affect the operation efficiency. For example, under the same hardware conditions, the operation efficiency of FP32 type data may be higher than that of FP64 type data, and the operation efficiency of TF32 type data may be higher than that of FP32 type data. Therefore, the first data type may be FP64, and the second data type may be FP32; the first data type may be FP32, and the second data type may be TF32 or FP16, etc. It should also be understood that the term "efficiency" can also cover different aspects. For example, efficiency can be improved operation speed, reduced storage space requirements, or reduced hardware requirements, all of which belong to the category of efficiency improvement.

[0055] According to the above embodiments of the present disclosure, the instruction decoding unit 910 determines whether the data to be operated and the operation type meet specific conditions, and when the specific conditions are met, the data to be operated can be implicitly converted. This conversion does not require manual control and courage by the user, so that the user can experience the improvement of operation efficiency without perception.

[0056] The above-mentioned "predetermined condition" may be multiple. For example, according to one embodiment of the present disclosure, the preset condition is: a first efficiency of performing operations on the to-be-operated data of the first data type is lower than a predetermined efficiency.

[0057] In this embodiment, a predetermined efficiency can be set as a threshold value. If the operation efficiency of the data to be operated is lower than the threshold value (for example, the storage space occupied by the data to be operated is higher than the preset storage space, or the amount of data involved in the operation per unit time is lower than the preset amount of data, etc.), then the conversion operation of the data to be operated can be triggered, and the data to be operated is converted into a data type that can meet the predetermined efficiency. Such a threshold value depends on which efficiency improvement the user is more concerned about, and thus the corresponding conversion is performed according to the efficiency concerned.

[0058] For example, as mentioned above Figure 4c As shown in the figure, for the FP32 data format, the 23-bit mantissa needs to be divided into 12-bit data Ina and 11-bit data Inb. When performing operations, two operation cycles are required to complete the calculation, that is, the first operation cycle realizes 12-bit multiplication, the second time cycle processes 11-bit multiplication, and finally the multiplication results of the two time cycles are added according to the appropriate exponent bits. Obviously, in Figure 4c In the illustrated implementation, the operation efficiency of FP32 is lower than that of TF32 or FP16, for example, so FP32 needs to be converted. However, on the other hand, if the number of bits of the hardware circuit or memory reaches 24 bits or higher, so that the FP32 data does not need to be split, and the operation of one FP32 data can be completed within one time cycle, then there is no need to convert the data. Therefore, the level of "efficiency" does not absolutely depend on the data type itself, but also depends on the processing capability of the current hardware structure relative to the data.

[0059] According to a more specific implementation of the present disclosure, the preset condition is: the first data type is an FP32 data type, and the operation type is a multiplication operation.

[0060] Table 1 below shows the ratio of the computational efficiency of FP32, TF32, and FP16.

[0061]

[0062] Table 1

[0063] As shown in Table 1, it is assumed that the bandwidth of the operator is 128B. For the FP32 data type, the bit width of a single element is 4B, and 32 numbers can be calculated in 2 operation cycles, so the bandwidth of the data supply path is required to be 64B / cycle; for the TF32 data type, the bit width of a single element is 4B, and 64 numbers can be calculated in 1 cycle, so the bandwidth of the data supply path is required to be 256B / cycle; and for the FP16 data type, the bit width of a single element is 2B, and 64 numbers can be calculated in 1 cycle, so the bandwidth of the data supply path is required to be 128B / cycle. It can be seen that if the bandwidth of the data supply path is not considered, the computing efficiency of TF32 type data is four times that of PF32 type, and the computing efficiency of FP16 type data is twice that of PF32 type. If the traditional 128B bandwidth of the data supply path is maintained, the computing efficiency of TF32 type data is twice that of PF32 type, and the computing efficiency of FP16 type data is four times that of PF32 type.

[0064] Therefore, according to actual needs, such as the bandwidth of the operator and / or the bandwidth of the data supply path, the second data type can be FP16 or TF32, that is, when FP32 type data needs to be multiplied, it can be converted into FP16 or TF32 data type.

[0065] It should be noted that the above-mentioned "data supply channel" refers to Fig. 9 The path between the RAM and the front-end data transfer unit 920. The required data path bandwidth can be provided according to the actual needs described above.

[0066] An example of a specific instruction is given below

[0067] Dst_addr: the first address of the output data

[0068] Src0_addr: the first address of input data 0

[0069] Src1_addr: the first address of input data 1

[0070] Size: The amount of data to be calculated

[0071] Data_type: data type

[0072] Op_cpde: Operation to be calculated

[0073] When the IDU detects that the “data type” in “Data_type” is FP32 and the “operation to be calculated” in Op_cpde is multiplication, the front-end conversion unit 920 may be instructed to convert the FP32 data, for example, to TF32 data.

[0074] After the data to be calculated is converted into intermediate data by the pre-transfer unit 920, a storage space is required to temporarily store the intermediate data for subsequent calculation operations. When converting PF32 to TF32, the required storage space can be further reduced in a specific manner.

[0075] Considering storage space (e.g. Figure 3 NRAM) is usually addressed in bytes, so even though a data type such as TF32 has only 19 bits, 32 bits are usually allocated in the memory for storage to facilitate byte-aligned access. Obviously, this memory access method wastes memory space. For example, each TF32 type of data wastes 13 bits of storage space. When data is retrieved and transmitted on the data path, it also occupies the transmission bandwidth of the data path. This is described in detail above.

[0076] In view of this, the embodiments of the present disclosure propose a memory access solution for a special data type such as TF32, which can effectively utilize storage space without occupying additional storage resources by splitting the number of bits of the data into an integer part and a remainder part, and storing them in a suitable storage area or memory respectively. Accordingly, no additional bandwidth needs to be occupied when transmitting on the data path.

[0077] Figure 5 An exemplary flow chart of a data storage method according to an embodiment of the present disclosure is shown.

[0078] As shown in the figure, in step 510, the number of bits of the data may be divided into an integer part and a remainder part, wherein the number of bits of the integer part is aligned to the memory alignment value, and the number of bits of the remainder part is smaller than the memory alignment value.

[0079] Here, the memory alignment value refers to the addressing alignment value of the conventional memory. For example, most memories currently address and access memory in bytes, so the memory alignment value is 1 byte, that is, 8 bits. "Aligned to the memory alignment value" means that it can be divided by the memory alignment value. Taking the above memory alignment value of 8 bits as an example, the data types such as int8, int16, FP16, and FP32 are all aligned to 8 bits. When they are stored in these memories, they are also stored according to byte alignment, so that addressing and memory access can be convenient and fast.

[0080] When the number of bits of data is not aligned to the memory alignment value, such as the TF32 data type, it is either stored as 32 bits as mentioned above so that it can be addressed and accessed by byte alignment, or it is padded with 5 bits to align to 8 bits. Both methods will take up extra storage space to store meaningless values.

[0081] Therefore, in the embodiment of the present disclosure, the number of bits of data is divided into an integer part and a remainder part according to the memory alignment value. For example, for TF32 with 19 bits, assuming that the memory alignment value is 8 bits, it can be divided into: a 16-bit integer part and a 3-bit remainder part.

[0082] The specific format division of the integer part and the remainder part can be arbitrary. In some embodiments, it is preferred to divide the 1-bit sign bit s into the integer part, so that when the integer part is read first, a judgment can be made in advance based on the sign bit, which is conducive to the decision-making of subsequent processing. For example, when some operations are performed only when the data is positive, a judgment can be made based on the extracted sign bit s, so that it can be skipped when the data is judged to be negative, saving subsequent operations such as reading the remainder part and calculation, and improving processing efficiency. In one implementation, 3 bits of the exponent bit of TF32 can be divided into the remainder part, such as the 15th-17th bits (see the shaded part of TF32 in Figure 4); and the remaining 16 bits are divided into the integer part, that is, 10 bits of mantissa bits (0th-9th bits), 5 bits of exponent bits (10th-14th bits) and 1 bit of sign bit (18th bit).

[0083] Next, in step 520, the integer part is stored in the first address of the first storage area. Since the integer part is aligned to the memory access alignment value, in some embodiments, the first storage area may be a memory that performs addressing and memory access according to the memory access alignment value.

[0084] Furthermore, in some embodiments, the first storage area may be located in a memory that is already configured in the computing device and performs addressing and memory access according to the memory access alignment value. That is, the first storage area may reuse the existing memory, thereby eliminating the need to configure a new storage circuit.

[0085] Then, in step 530, the remainder part is stored in the second address of the second storage area, wherein the second address has a one-to-one mapping relationship with the first address. Since the remainder part is not aligned to the memory access alignment value and is smaller than the memory access alignment value, the remainder part should not be stored in the first storage area where the integer part is located, but needs to be configured with another memory, that is, the second storage area and the first storage area belong to different memories. Preferably, the second storage area can be addressed and accessed according to the alignment of the remainder part. For example, taking TF32 as an example, the second storage area can be addressed and accessed according to 3-bit alignment, so that at least the remainder part of 1 TF32 data can be accessed each time.

[0086] In order to associate the integer part and the remainder part of the same data, there needs to be a one-to-one mapping relationship between the first address storing the integer part and the second address storing the remainder part.

[0087] In some embodiments, the mapping relationship between the first address of the first storage area and the second address of the second storage area is preset. For example, there is a fixed address mapping relationship between the first storage area and the second storage area. Specifically, taking TF32 data as an example, the first 16-bit address of the first storage area is fixedly associated with the first 3-bit address of the second storage area, and the second 16-bit address of the first storage area is fixedly associated with the second 3-bit address of the second storage area, and so on. It can be understood that when the address of the first storage area is not fixed due to the reuse of the configured memory, the fixed association between the above-mentioned first address and the second address is relatively fixed. For example, the first address of the first storage area and / or the second storage area may change, but the mapping relationship between the first address offset by a fixed amount relative to the first address of the first storage area and the second address offset by a fixed amount relative to the first address of the second storage area is fixed. It can also be understood that since this mapping relationship is fixed, the second address can be directly determined according to the offset of the first address relative to the first address, and vice versa.

[0088] In other embodiments, the mapping relationship between the first address and the second address can be established in real time when storing data, and maintained through an address table. For example, the integer part of data A is stored in the first address, and the remainder part of data A is stored in the second address, and then recorded in the address correspondence table: first address <——> second address, and the double-headed arrow indicates that bidirectional query is possible. Therefore, in subsequent access, the second address can be determined based on the first address according to the address table, and vice versa.

[0089] In some embodiments, the second storage area may also be located in a memory configured in the computing device, that is, the second storage area may reuse an existing memory as long as the memory at least supports addressing and memory access according to the remainder part.

[0090] Thus, the data storage method of the embodiment of the present disclosure is described above in conjunction with the accompanying drawings. Although the storage of the integer part and the remainder part is described in chronological order above, those skilled in the art can understand that the embodiment of the present disclosure is not limited in this regard, and the storage order of the two can be reversed or performed simultaneously (as long as the hardware supports simultaneous memory access). Accordingly, the embodiment of the present disclosure also provides a corresponding data access method.

[0091] Figure 6 An exemplary flow chart of a data access method according to an embodiment of the present disclosure is shown.

[0092] As shown in the figure, in step 610, the integer part of the number of bits of data is read from the first address of the first storage area, wherein the number of bits of the integer part is aligned to the memory access alignment value.

[0093] In step 620, a second address of a second storage area having a one-to-one mapping relationship with the first address is determined based on the first address. As mentioned above, the mapping relationship between the first address and the second address can be expressed in various forms, such as a fixed relative position, which can be calculated and determined using a formula; or a searchable address table, which can be determined by searching.

[0094] Next, in step 630, a remainder portion of the number of bits of the data is read from the second address of the second storage area, wherein the number of bits of the remainder portion is smaller than the above-mentioned memory access alignment value.

[0095] Finally, in step 640, the read integer part and the remainder part are concatenated to form the data.

[0096] Understandably, the above reference Figure 5 The various technical features of the various steps described in the data storage method can be similarly applied to Figure 6 Similarly, although the reading of the integer part and the remainder part is described in chronological order above, those skilled in the art will understand that the embodiments of the present disclosure are not limited in this regard, and the reading order of the two can be reversed or performed simultaneously (as long as the hardware supports simultaneous memory access).

[0097] As mentioned in the background, the use of data types such as TF32 may appear in the operations of some network layers of the neural network. For example, in Leopard's AI-NR network, the storage of TF32 data types appears in the AI-ISP core, mainly involving post-processing (post-fusion, operations such as addition, subtraction, multiplication, etc., such as bias operations) of the 1×1 convolutional layer of the AI-NR network, and is stored in, for example, Figure 3 The memory access solution for special data types such as TF32 provided by the embodiment of the present disclosure can reduce storage requirements and effectively utilize computing bandwidth.

[0098] For example, in some scenarios, for regular data, such as FP16, the bandwidth of the convolution operation is 64B, and the convolution operation result can be post-fused, and the operation bandwidth of the post-fusion is 128B. For TF32 data, its operation bandwidth is 152B. Therefore, there are three operation bandwidths available for selection in different modes to maximize the operation efficiency.

[0099] If the convolution operation precision is in accordance with the FP16 data type, 64B and 128B operation bandwidths can be used. Specifically, if a 64B operation bandwidth is used, 32 numbers can be calculated at the same time; if a 128B operation bandwidth is used, 64 numbers can be calculated at the same time. If the FP32 data type is used, the 64B operation bandwidth can calculate 16 numbers, and the 128B operation bandwidth can calculate 32 numbers. Using the TF32 data type, the 152B operation bandwidth can calculate 64 numbers. It can be seen that the amount of data of TF32 is the largest among the 152B operation bandwidths, and its hardware overhead is only an additional 24B on the basis of 128B. This additional 24B of overhead can increase the amount of computational data to a maximum of 64 numbers, while the accuracy is higher than FP16.

[0100] Figure 7 An exemplary flow chart of a data operation method according to an embodiment of the present disclosure is shown.

[0101] As shown in the figure, in step 710, the operation is performed on the data to be operated to obtain an intermediate operation result. According to the configuration of the hardware, the data to be operated can be a specified data type, such as int8, int16, etc. Usually, the intermediate operation result is output according to the desired accuracy, for example, it is output according to the FP32 type. This operation can be, for example, a convolution operation in a neural network.

[0102] Next, in step 720, the intermediate operation results are converted into data of a specified type. The specified type of data is, for example, a special type of data such as TF32 targeted by the embodiments of the present disclosure. As mentioned above, the output intermediate operation results may require some post-fusion operation processing, and the accuracy requirements of these post-fusion operations may not be so high, but the operation bandwidth can be appropriately increased, for example, 152B. Therefore, converting the intermediate operation results into TF32 type can effectively utilize the operation bandwidth.

[0103] Then, in step 730, as required, the data of the specified type may be stored in the format described in the previous reference. Figure 5 When the data needs to be used for post-fusion processing, it can be temporarily stored and read again during the operation.

[0104] Optionally or additionally, in step 740, as required, according to the above reference Figure 6 Describes a data access method that reads data of the specified type.

[0105] Finally, in step 750, further operations are performed using the data of the specified type.

[0106] It is understandable that after the above calculation, data type conversion may be performed. For example, when the accuracy does not need to be maintained so high, it may be converted into a conventional int8 data type for final output.

[0107] It is also understandable that the above reference Figure 5 and Figure 6 The various technical features of the various steps described in the data storage method and data access method can be similarly applied to Figure 7 The data calculation method will not be described here.

[0108] An exemplary application scenario of the data access method of an embodiment of the present disclosure is described above. The data access method provided in this article can also be applied in other scenarios. For example, in the operation of a convolutional neural network, the convolution kernel sizes of different convolutional layers may be different, such as including 3×3 convolution kernels, and may also include 1×1 convolution kernels. When the convolution kernel is larger, such as 3×3, the amount of computation is also large. At this time, the output bandwidth is not the bottleneck of the entire operation process. When outputting, the highest precision FP32 is used for output, which is acceptable even if the speed is slower. When the convolution kernel is small, such as 1×1, the calculation is very fast. At this time, the output bandwidth can easily become a bottleneck in the entire operation process. To output faster, you can use the TF32 type for output.

[0109] Although the above describes dividing the number of bits of data into an integer part and a remainder part for separate storage, in order to facilitate the description of this data type, the above division and separate storage can be set to be invisible to the programmer, thereby simplifying the programmer's processing. For example, for TF32, it can be agreed that when the programmer allocates space for this data type, the data size of each TF32 is the same as the data size of FP32, that is, 32 bits (4B). In this way, assuming that 64 TF32 data need to be stored / transmitted, a space size of 64*4B=256B will be occupied.

[0110] Figure 8 A schematic diagram of space allocation according to an embodiment of the present disclosure is shown. In the illustrated embodiment, the first storage area for storing the integer part reuses a configured memory, such as NRAM.

[0111] The left side of the figure shows the logical space of NRAM. The size of the ordinary neuron data storage space 810 is, for example, 1280KB, the address range is 1M (NRAM offset of uniform addressing) ~ 1M + 1280KB, and access is performed according to 8-bit alignment. This logical address space can be called CT_RAM0.

[0112] The storage space of TF32 data type is special. It is within the address range of 1M+1.5M~1M+1.5M+480KB and is part of NRAM. In this article, it is uniformly defined as compressed space (area 812 in the figure). This logical address space can be called TF_RAM. In other words, the address space allocated by software for TF32 data type can only be defined between 2.5M~2.5M+480KB, and the space size is 480KB. Once the allocated address range is not within this range, the hardware will trigger an interrupt error.

[0113] As can be seen from the figure, there is an invalid space 811 between the ordinary neuron data storage space 810 and the TF32 storage space 812. That is, 1M+1280KB~1.5MKB is not allowed to be accessed.

[0114] If the hardware allocates the actual physical storage space according to the software's address space, the resource overhead for the hardware is very large. In addition, a special data type such as TF32 is usually not used all the time, but only when there is a precision requirement, or there are scenarios where TF32 is not used at all. Therefore, in some embodiments, although the address space of TF32 is separated from the address space of ordinary neuron data in the diagram in logical space, the physical space of the TF32 data type actually reuses the physical space of ordinary neuron data in NRAM, that is, reuses part of the 1M~1M+1280KB physical interval of NRAM. In other words, there is a mapping relationship in the physical implementation, that is, the logical address of 2.5M~2.5M+480KB (TF_RAM) will be mapped to a part of the 1M~1M+1280KB physical interval. This reuse can effectively save hardware overhead, and it is also convenient to use a set of storage hardware to support multiple data types.

[0115] In order to describe this mapping relationship, logical addresses L_ADDR_CT_RAM0 and L_ADDR_TF_RAM may be defined first, where 1M<=L_ADDR_CT_RAM0<1M+1280KB, 2.5M<=L_ADDR_TF_RAM<2.5M+480KB; and physical addresses P_ADDR_CT_RAM0 and P_ADDR_TF_RAM.

[0116] For 480KB of storage space, when it is used to store data of TF32 data type, since the software still processes it in the same way as FP32 size, 480KB / 32bit=120K TF32 data can be stored. According to the storage scheme of TF32 in the embodiment of the present disclosure, the actual storage space required for these 120K TF32 data is: the integer part requires 120K*16bit=240KB, and the remainder part requires 120K*3bit=45KB.

[0117] The right side of the figure shows the physical space mapped to NRAM through the address. The integer part reuses part of the 1M~1M+1280KB physical area 820 of NRAM, as shown in the area TF_L_RAM 822 in the figure; the remainder is stored in an additional small memory, as shown in the area TF_H_RAM 823. That is, the logical space 812 will be mapped to the physical area 822+physical area 823; and the logical space 810 of the ordinary neuron data will be mapped to the area of ​​the physical area 820 except 822.

[0118] The specific mapping relationship between the logical address space and the physical address space is as follows:

[0119] (1) The total physical space size of NRAM is CT_RAM0+TF_H_RAM, i.e. 1280KB+45KB;

[0120] (2) The L_ADDR_CT_RAM0 address and the P_ADDR_TF_RAM0 address mapping are one-to-one, as indicated by the solid arrows;

[0121] (3) The logical address of TF_RAM (480KB in size) is mapped to TF_L_RAM (240KB) + TF_H_RAM (45KB) in CT_RAM0, as indicated by the dotted arrow. The mapping relationship satisfies {L_ADDR_TF_RAM (1M + 1536KB)}

[0122] ÷2+CFG_BASE_ADDR=P_ADDR_TF_RAM, where CFG_BASE_ADDR is configurable;

[0123] (4)CFG_BASE_ADDR is aligned to 512B.

[0124] From the mapping relationship described above, it can be seen that the TF_L_RAM and CT_RAM0 after TF_RAM mapping can overlap. This part requires software to ensure "consistency", that is, when the software uses TF_RAM, it needs to ensure that its mapped TF_L_RAM in CT_RAM0 is only used by TF32 data type.

[0125] As can be seen from the figure, the integer part area 822 is actually shared with the normal neuron data area 820. That is, the 240KB required to store the integer part belongs to the 1280KB of normal neuron data. The specific position of area 822 in area 820 can be configured, for example, by the above parameter CFG_BASE_ADDR. In other words, area 822 can slide arbitrarily in area 820.

[0126] Since the entire NRAM is usually designed to be stored and accessed according to 8-bit alignment for design convenience or to reduce hardware overhead. The integer part (16 bits) of TF32 is also aligned to 8 bits, so the storage area 822 of the integer part can be reused in NRAM and accessed according to 8-bit alignment. The remainder part (3 bits) of TF32 is aligned according to 3 bits, which is different from the access mode of the integer part and will have more overhead, so the storage area 823 required for the remainder part needs to be separated.

[0127] from Figure 8As can be seen, what the programmer can see is the logical space on the left, such as the 480KB space dedicated to storing TF32 data and the 1280KB space for ordinary neuron data storage. The hardware will find the corresponding storage location through address mapping. The storage area 823 of the remainder is hidden and invisible to the programmer. This design is very friendly to programmers and can simplify their programming. There is no need to consider the segmentation and underlying storage of data such as TF32, and it can still be processed in the FP32 way.

[0128] Those skilled in the art will appreciate that the 1280KB, 480KB, etc. mentioned in the above examples are merely exemplary. Different space sizes may be configured according to actual computing needs, and the space sizes required for the integer part and the remainder part may be calculated according to the principles disclosed in the present disclosure. The embodiments of the present disclosure are not limited in this regard. For a data type such as TF32, the storage space size of the integer part and the storage space size of the remainder part may be maintained in a ratio of 16:3.

[0129] from Figure 8 It can be seen that the embodiment of the present disclosure also provides a data storage device, including a first storage area (e.g., TF_L_RAM), configured to store the integer part of the number of bits of data, wherein the number of bits of the integer part is aligned to the memory access alignment value; a second storage area (e.g., TF_H_RAM), configured to store the remainder part of the number of bits of data, wherein the number of bits of the remainder part is less than the memory access alignment value. There is a one-to-one mapping relationship between the first address in the first storage area corresponding to the integer part of the same data and the second address in the second storage area corresponding to the remainder part.

[0130] Fig.10 FIG. 2 shows a schematic block diagram of a computing system according to another embodiment of the present disclosure. Fig.10 As shown, the computing system further includes a post-conversion unit 940 configured to convert the post-computation intermediate data having the second data type into the first data type.

[0131] like Fig.10 As shown, the post-transfer unit 940 can convert, for example, TF32 data into FP32 data, thereby restoring the original data type for further processing of the data. It should be understood that the post-transfer unit 940 can be started and shut down according to the instructions of the IDU to selectively. For example, if the next level of calculation or storage in the calculation pipeline requires the use of FP32 type data, the IDU can instruct the post-transfer unit to convert the TF32 data into FP32 data.

[0132] It should also be understood that the front rotation unit 920 and the rear rotation unit 940 can be in the same pipeline, so the rotation operations of the two will not cause additional performance overhead, so that the rotation can be completed without the user's perception.

[0133] The present disclosure also provides a computing device, which may include the data storage device of any of the aforementioned embodiments. The present disclosure also provides a chip, which may include the aforementioned computing device. Further, the present disclosure also provides a board, which may include the aforementioned chip.

[0134] Fig.11 An operation method according to an embodiment of the present disclosure is shown, the method comprising, at operation S1110, receiving a control instruction; at operation S1120, decoding a first data type of data to be operated and an operation type for the data to be operated from the control instruction; at operation S1130, if the first data type and the operation type meet a preset condition, indicating to perform a pre-transformation on the data to be operated; wherein the pre-transformation comprises: converting the data to be operated having the first data type into intermediate data having the second data type, wherein a first efficiency of operating on the data to be operated having the first data type is lower than a second efficiency of operating on the intermediate data of the second data type; and, at operation S1140, operating on the intermediate data.

[0135] According to an embodiment of the present disclosure, the preset condition is that a first efficiency of performing operations on the to-be-operated data of the first data type is lower than a predetermined efficiency.

[0136] According to one embodiment of the present disclosure, the preset condition is: the first data type is the FP32 data type, and the operation type is a multiplication operation.

[0137] According to one embodiment of the present disclosure, the second data type is FP16.

[0138] According to one embodiment of the present disclosure, the second data type is TF32.

[0139] According to one embodiment of the present disclosure, the method further includes: storing the intermediate data in the following manner: dividing the number of bits of the intermediate data into an integer part and a remainder part, wherein the number of bits of the integer part is aligned to a memory access alignment value, and the number of bits of the remainder part is smaller than the memory access alignment value; storing the integer part in a first address of a first storage area; and storing the remainder part in a second address of a second storage area, wherein the second address has a one-to-one mapping relationship with the first address.

[0140] According to one embodiment of the present disclosure, the intermediate data is read in the following manner: an integer part of the number of bits of the intermediate data is read from a first address of a first storage area; a second address of a second storage area having a one-to-one mapping relationship therewith is determined based on the first address; a remainder part of the number of bits of the intermediate data is read from the second address of the second storage area; and the integer part and the remainder part are concatenated to form the intermediate data.

[0141] According to one embodiment of the present disclosure, the mapping relationship between the first address and the second address is preset.

[0142] According to an embodiment of the present disclosure, the mapping relationship between the first address and the second address is established in real time when the intermediate data is stored, and is maintained through an address table.

[0143] According to an embodiment of the present disclosure, the first storage area is addressed and accessed according to the memory access alignment value.

[0144] According to an embodiment of the present disclosure, the first storage area is located in a memory that is configured to perform addressing and memory access according to the memory access alignment value.

[0145] According to an embodiment of the present disclosure, the second storage area is addressed and accessed according to the alignment of the remainder part.

[0146] According to one embodiment of the present disclosure, the memory access alignment value is 8 bits, the integer part includes 16 bits, and the remainder part includes 3 bits.

[0147] According to one embodiment of the present disclosure, the sign bit of the intermediate data is included in the integer part.

[0148] According to an embodiment of the present disclosure, the method further comprises: converting the intermediate data after operation having the second data type into the first data type.

[0149] The system and method disclosed herein can choose to implicitly convert data in FP32 format according to specific operation conditions, thereby improving operation efficiency without user perception. Specifically, when FP32 type data is multiplied, it can be implicitly converted to TF32 format or FP16 format, thereby improving efficiency; more specifically, when FP32 type data is multiplied, it can be implicitly converted to TF32 format, thereby providing higher precision than FP16 while improving operation efficiency.

[0150] Furthermore, when converting PF32 type data into TF32, the system and method of the present disclosure can access the TF32 type data in a specific manner, reducing the storage space requirements and the required transmission bandwidth, thereby further reducing the demand for storage resources and transmission resources and reducing hardware costs.

[0151] According to different application scenarios, the electronic equipment or device disclosed in the present invention may include a server, a cloud server, a server cluster, a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a PC device, an Internet of Things terminal, a mobile terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a visual terminal, an automatic driving terminal, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship, and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric light, a gas stove, and a range hood; the medical device includes a nuclear magnetic resonance instrument, a B-ultrasound instrument, and / or an electrocardiograph. The electronic equipment or device disclosed in the present invention can also be applied to the Internet, the Internet of Things, a data center, energy, transportation, public administration, manufacturing, education, power grid, telecommunications, finance, retail, construction sites, medical treatment and other fields. Further, the electronic equipment or device disclosed in the present invention can also be used in the cloud, edge end, terminal and other application scenarios related to artificial intelligence, big data and / or cloud computing. In one or more embodiments, electronic devices or devices with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or devices with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0152] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will appreciate that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teaching of the present disclosure, those skilled in the art will appreciate that some of the steps therein may be performed in other orders or simultaneously. Further, those skilled in the art will appreciate that the embodiments described in the present disclosure may be regarded as optional embodiments, i.e., the actions or modules involved therein may not necessarily be necessary for the implementation of one or some of the schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different focuses on the description of some embodiments. In view of this, those skilled in the art will appreciate that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0153] In terms of specific implementation, based on the disclosure and teaching of the present disclosure, those skilled in the art can understand that several embodiments disclosed in the present disclosure can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article splits them on the basis of considering the logical functions, and there may be other splitting methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. In terms of the connection relationship between different units or components, the connection discussed in the above text in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0154] In the present disclosure, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located at the same location or distributed to multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0155] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs, etc. Further, the aforementioned storage unit or storage device may be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which may be, for example, a variable resistive memory (Resistive Random Access Memory, RRAM), a dynamic random access memory (Dynamic Random Access Memory, DRAM), a static random access memory (Static Random Access Memory, SRAM), an enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, EDRAM), a high bandwidth memory (High Bandwidth Memory, HBM), a hybrid memory cube (Hybrid Memory Cube, HMC), ROM and RAM, etc.

[0156] The embodiments of the present disclosure are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present disclosure, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A computing system, comprising an instruction decoding unit, a pre-processing unit and a computing unit, wherein: The instruction decoding unit is configured as follows: Receive control instructions; Decoding a first data type of the data to be operated and a type of operation for the data to be operated from the control instruction; If the first data type and operation type meet the preset conditions, instruct the data to be operated to be input into the front-end rotation unit; The front conversion unit is configured to convert the to-be-operated data of the first data type into the intermediate data of the second data type, wherein a first efficiency of operating on the to-be-operated data of the first data type is lower than a second efficiency of operating on the intermediate data of the second data type; and The operation unit is configured to receive the intermediate data from the front-end rotation unit and perform operation on the intermediate data.

2. The computing system according to claim 1, wherein: The preset condition is that a first efficiency of performing operations on the to-be-operated data of the first data type is lower than a predetermined efficiency.

3. The computing system according to claim 1 or 2, wherein: The preset condition is: the first data type is the FP32 data type, and the operation type is a multiplication operation.

4. The computing system according to claim 3, wherein: The second data type is FP16.

5. The computing system according to claim 3, wherein: The second data type is TF32.

6. The computing system according to claim 5, further comprising a storage device for storing intermediate data having a second data type, the storage device comprising: A first storage area is configured to store an integer part of the number of bits of the intermediate data, wherein the number of bits of the integer part is aligned to a memory access alignment value; A second storage area is configured to store a remainder part of the number of bits of the intermediate data, wherein the number of bits of the remainder part is smaller than the memory access alignment value; There is a one-to-one mapping relationship between the first address in the first storage area corresponding to the integer part of the same intermediate data and the second address in the second storage area corresponding to the remainder part. 7 . The operating system according to claim 6 , wherein the mapping relationship between the first address and the second address is preset.

8. The computing system according to claim 6, wherein the mapping relationship between the first address and the second address is established in real time when the intermediate data is stored, and is maintained through an address table.

9. The operating system according to any one of claims 6 to 8, wherein the first storage area is addressed and accessed according to the memory alignment value.

10. The operating system according to claim 9, wherein the first storage area is located in a memory that is configured to perform addressing and memory access according to the memory alignment value.

11. The computing system according to any one of claims 6 to 10, wherein the second storage area is addressed and accessed according to the alignment of the remainder part.

12. According to the computing system of any one of claims 6 to 11, the memory access alignment value is 8 bits, the integer part includes 16 bits, and the remainder part includes 3 bits.

13. An operating system according to clause 12, wherein a sign bit of the intermediate data is included in the integer part.

14. The computing system according to any one of claims 1 to 13, further comprising a post-conversion unit configured to convert the post-computation intermediate data having the second data type into the first data type.

15. A computing method, comprising: Receive control instructions; Decoding a first data type of the data to be operated and a type of operation for the data to be operated from the control instruction; If the first data type and operation type meet the preset conditions, instructing to perform pre-processing on the data to be operated; The pre-processing comprises: converting data to be operated of a first data type into intermediate data of a second data type, wherein a first efficiency of operating the data to be operated of the first data type is lower than a second efficiency of operating the intermediate data of the second data type; and An operation is performed on the intermediate data.

16. The operation method according to claim 15, wherein: The preset condition is that a first efficiency of performing operations on the to-be-operated data of the first data type is lower than a predetermined efficiency.

17. The calculation method according to claim 15 or 16, wherein: The preset condition is: the first data type is the FP32 data type, and the operation type is a multiplication operation.

18. The operation method according to claim 17, wherein: The second data type is FP16.

19. The operation method according to claim 17, wherein: The second data type is TF32.

20. The operation method according to claim 19, further comprising: The intermediate data is stored in the following manner: Divide the number of bits of the intermediate data into an integer part and a remainder part, wherein the number of bits of the integer part is aligned to the memory access alignment value, and the number of bits of the remainder part is smaller than the memory access alignment value; storing the integer part in a first address of a first storage area; as well as The remainder part is stored in a second address of a second storage area, wherein the second address has a one-to-one mapping relationship with the first address.

21. The operation method according to claim 20, wherein: The intermediate data is read in the following manner: Reading an integer part of the number of bits of the intermediate data from a first address of the first storage area; Determine, based on the first address, a second address of a second storage area having a one-to-one corresponding mapping relationship therewith; Reading a remainder of the number of bits of the intermediate data from a second address of the second storage area; as well as The integer part and the remainder part are concatenated to form the intermediate data.

22. The computing method according to claim 20 or 21, wherein the mapping relationship between the first address and the second address is preset.

23. The computing method according to claim 20 or 21, wherein the mapping relationship between the first address and the second address is established in real time when the intermediate data is stored, and is maintained through an address table.

24. The computing method according to any one of claims 20 to 23, wherein the first storage area is addressed and accessed according to the memory alignment value.

25. The method according to claim 24, wherein the first storage area is located in a memory that is configured to perform addressing and memory access according to the memory access alignment value.

26. The computing method according to any one of claims 20 to 25, wherein the second storage area is addressed and accessed according to the alignment of the remainder part.

27. The computing method according to any one of claims 20 to 26, wherein: The memory access alignment value is 8 bits, the integer part includes 16 bits, and the remainder part includes 3 bits.

28. The method of claim 27, wherein a sign bit of the intermediate data is included in the integer portion.

29. The computing method according to any one of claims 15 to 28, further comprising: Convert the calculated intermediate data of the second data type to the first data type.

30. A chip comprising the computing system according to any one of claims 1-14.

31. A board comprising the chip according to claim 30.

Images