Processor, floating point arithmetic unit and arithmetic method

Abstract

The embodiment of the invention belongs to the technical field of calculation, and particularly relates to a processor, a floating point arithmetic unit and an arithmetic method. The processor comprises a data carrying unit and a floating point arithmetic unit, and the data carrying unit further comprises a data decompression module besides a data carrying module for realizing a data carrying function. Wherein the data decompression module can perform decompression processing on first weight data subjected to compression processing in the AI model to obtain second weight data represented by a floating-point number. The data carrying unit can carry the first activation data and the second weight data to the floating point operation unit for floating point operation. By adopting the method and the device, the compressed first weight data and the uncompressed first activation data can be subjected to floating-point operation, so that only the weight data can be subjected to compression processing when the AI model is subjected to quantitative compression, and the influence of quantitative compression on the precision of the AI model can be reduced.

Description

[0001] This application is a divisional application. The application number of the original application is 202311086354.8, and the original application date is August 24, 2023. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computing technology, and in particular to a processor, a floating-point operation unit and an operation method. Background Art

[0003] Quantization and compression of artificial intelligence (AI) models refers to a method of quantizing the weight data and activation data in the AI ​​model from floating point numbers to integers, thereby improving the running speed of the AI ​​model.

[0004] Although quantization compression can improve the running speed of AI models, it will also reduce the accuracy of AI models. Among them, because the value range of activation data is larger and there is a certain proportion of outliers, the impact of activation data quantization on AI model accuracy is greater than that of weight data quantization on AI model accuracy. Summary of the invention

[0005] The embodiments of the present application provide a processor, a floating-point operation unit, and an operation method, which can improve the calculation efficiency of activation data and weight data and reduce the impact of quantization compression on the accuracy of AI models. The corresponding technical solutions are as follows:

[0006] In a first aspect, a processor is provided, the processor comprising a data handling unit and a floating-point operation unit, the data handling unit realizing a data handling function for the processor, comprising a data handling module and a data decompression module, wherein:

[0007] The data handling module is used to obtain the first weight data and the first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model. The data decompression module is used to decompress the first weight data to obtain the second weight data represented by a floating point number. The data handling module is also used to input the first activation data and the second weight data into the floating point operation unit for floating point operation.

[0008] In the scheme shown in the present application, in the process of the data handling module carrying the first weight data and the first activation data to the floating-point operation unit, the data decompression module added in the data handling unit can decompress the compressed first weight data to obtain the second weight data represented by a floating-point number, and then the data handling module can input the second weight data and the uncompressed first activation data into the floating-point operation unit to realize floating-point operations of the weight data and the activation data.

[0009] In this way, on the one hand, only the weight data in the AI ​​model is compressed, which can avoid the impact of the quantization of the activation data on the accuracy of the AI ​​model. On the other hand, adding the data decompression module to the data handling unit avoids the data handling unit from carrying the first weight data to other computing units (such as vector computing cores) for decompression, which can improve the efficiency of decompressing the first weight data, and thus improve the efficiency of the processor in performing floating-point operations on the first activation data and the second weight data.

[0010] In an achievable manner, the compression process includes at least one of coding compression and quantization compression. When the compression process includes coding compression, the decompression process includes decoding. When the compression process includes quantization compression, the decompression process includes numerical conversion.

[0011] In the scheme shown in the present application, the compression process can be to encode and compress the original weight data in the AI ​​model. For example, the original weight data of FP16 can be encoded into a block floating point format to obtain the first weight data, so that the storage space occupied by the weight data in the AI ​​model can be reduced. Correspondingly, when the compression process is encoding compression, the corresponding decompression process is decoding processing, that is, decoding the encoded first weight data into the second weight data represented by a floating point number.

[0012] In the scheme shown in the present application, the compression process can also be to quantize and compress the original weight data in the AI ​​model. For example, the original weight data of FP16 can be quantized into the first weight data of INT8, so as to reduce the storage space occupied by the weight data in the AI ​​model. Correspondingly, when the compression process is quantization compression, the corresponding decompression process is numerical conversion, that is, converting the first weight data into the second weight data represented by a floating point number.

[0013] In one achievable manner, the data handling module includes a data storage device and a handling controller. The handling controller is used to store the first weight data in the GM into the data storage device, and then input the data storage device into the data decompression module. After the data decompression module completes the decompression process, the second weight data is input into the floating point operation unit.

[0014] In the scheme shown in the present application, the transport controller may input the first weight data stored in the data storage device into the data decompression module before inputting the first weight data into the floating-point operation unit. After the data decoding module decompresses the first weight data into the second weight data, the transport controller may input the second weight data into the floating-point operation unit. In this way, the decompression processing of the first weight data can be implemented in the data transport unit, and there is no need to transport the first weight data to other operation units for decompression processing, which can improve the efficiency of decompression processing of the first weight data.

[0015] In one achievable manner, the data handling module includes a data storage device and a handling controller. The handling controller is used to input the first weight data in the GM into the data decompression module. After the data decompression module completes the decompression process, the second weight data is stored in the data storage device, and then the second weight data stored in the data storage device is input into the floating point operation unit.

[0016] In the scheme shown in the present application, before the transport controller transports the first weight data stored in the GM to the data storage device, it can first input the first weight data into the data decompression module, and the data decoding module decompresses the first weight data into the second weight data. The transport controller can store the second weight data in the data storage device, and then input the second weight data in the data storage device into the floating-point operation unit. In this way, the decompression processing of the first weight data can be implemented in the data transport unit, and there is no need to transport the first weight data to other operation units for decompression processing, which can improve the efficiency of decompression processing of the first weight data.

[0017] In an implementable manner, the data transport unit is a direct memory access DMA or a tensor memory access TMA.

[0018] In one implementable manner, the floating point operation unit is a matrix operation core.

[0019] In a second aspect, a floating-point operation unit is provided, which includes a data cache module, a data decompression module and a data calculation module, wherein: the data cache module is used to cache the first weight data and the first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model. The data decompression module is used to decompress the first weight data to obtain the second weight data represented by a floating point number. The data calculation module is used to perform floating-point operations on the first activation data and the second weight data to obtain the operation results.

[0020] In the solution shown in the present application, the floating-point operation unit can be a unit in the processor that performs floating-point operations on the weight data and activation data of the AI ​​model. A data decompression module is added to the floating-point operation unit, so that the compressed first weight data can be decompressed in the floating-point operation unit to obtain the second weight data represented by a floating-point number, and then the floating-point operation unit can perform floating-point operations on the first activation data and the second weight data.

[0021] In this way, on the one hand, only the weight data in the AI ​​model is compressed, which can avoid the impact of the quantization of the activation data on the accuracy of the AI ​​model. On the other hand, adding the data decompression module to the floating-point operation unit avoids the need to move the first weight data to other operation units (such as vector operation cores) for decompression, which can improve the efficiency of decompression of the first weight data, and thus improve the efficiency of floating-point operations on the first activation data and the second weight data.

[0022] In an achievable manner, the compression process includes at least one of coding compression and quantization compression. When the compression process includes coding compression, the decompression process includes decoding. When the compression process includes quantization compression, the decompression process includes numerical conversion.

[0023] In the scheme shown in the present application, the compression process can be to encode and compress the original weight data in the AI ​​model. For example, the original weight data of FP16 can be encoded into a block floating point format to obtain the first weight data, so that the storage space occupied by the weight data in the AI ​​model can be reduced. Correspondingly, when the compression process is encoding compression, the corresponding decompression process is decoding processing, that is, decoding the encoded first weight data into the second weight data represented by a floating point number.

[0024] In the scheme shown in the present application, the compression process can also be to quantize and compress the original weight data in the AI ​​model. For example, the original weight data of FP16 can be quantized into the first weight data of INT8, so as to reduce the storage space occupied by the weight data in the AI ​​model. Correspondingly, when the compression process is quantization compression, the corresponding decompression process is numerical conversion, that is, converting the first weight data into the second weight data represented by a floating point number.

[0025] In one implementable manner, the data cache module includes a core cache, an activation matrix register, and a weight matrix register. The core cache is used to cache the first weight data and the first activation data, input the first weight data into the weight matrix register, and input the first activation data into the activation matrix register. The weight matrix register is used to input the first weight data into the data decompression module. The activation matrix register is used to input the first activation data into the data calculation module.

[0026] In the scheme shown in the present application, the first weight data and the first activation data can form a matrix to perform matrix operations of floating-point numbers. The data cache module can first cache the first weight data and the first activation data, and then the first weight data can be input into the weight matrix register, and the first activation data can be input into the activation matrix register. Before the first weight data in the weight matrix register is input into the data calculation module, the first weight data can be input into the data decompression module for decompression processing to obtain the second weight data. In this way, the data calculation module can perform matrix operations on the second weight data and the first activation data. In this way, the data decompression module is added to the floating-point operation unit to avoid the first weight data being transported to other operation units (such as vector operation cores) for decompression processing, which can improve the efficiency of decompression processing of the first weight data, thereby improving the efficiency of floating-point operations of the first activation data and the second weight data.

[0027] In one implementable manner, the floating-point operation unit includes a plurality of data decompression modules.

[0028] In the solution shown in the present application, since the first weight data is over-compressed, the data cache module can store more first weight data relative to the first activation data. By setting multiple data decompression modules to perform over-compression processing on the first weight data in the data cache module in parallel, the efficiency of decompressing the first weight data can be improved.

[0029] In one implementable manner, the floating point operation unit is a matrix operation core.

[0030] In a third aspect, a calculation method is provided, which is executed by a processor, the processor includes a data handling unit and a floating-point operation unit, the data handling unit implements a data handling function for the processor, and includes a data handling module and a data decompression module. The calculation method includes:

[0031] The data handling module obtains the first weight data and the first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model. The data decompression module decompresses the first weight data to obtain the second weight data represented by a floating point number. The data handling module inputs the first activation data and the second weight data into the floating point operation unit for floating point operation.

[0032] In an achievable manner, the compression process includes at least one of coding compression and quantization compression. When the compression process includes coding compression, the decompression process includes decoding. When the compression process includes quantization compression, the decompression process includes numerical conversion.

[0033] In an achievable manner, the compression process includes encoding the original weight data into first weight data represented by a block floating point number, and the decoding process includes decoding the first weight data into second weight data represented by a floating point number;

[0034] The quantization compression includes quantizing the original weight data into first weight data represented by an integer number, and the numerical conversion includes converting the first weight data into second weight data represented by a floating point number.

[0035] In one achievable manner, the data handling module includes a data storage device and a handling controller. The data handling module obtains the first weight data and the first activation data of the artificial intelligence AI model, including: the handling controller stores the first weight data in the GM into the data storage device, and then inputs the data storage device into the data decompression module.

[0036] The data transport module inputs the first activation data and the second weight data into the floating point operation unit, including: the transport controller inputs the second weight data into the floating point operation unit after the data decompression module completes the decompression processing.

[0037] In one feasible manner, the data transport module includes a data storage device and a transport controller; the above-mentioned data transport module obtains the first weight data and the first activation data of the artificial intelligence AI model, including: the transport controller inputs the first weight data in the GM into the data decompression module.

[0038] The above-mentioned data handling module inputs the first activation data and the second weight data into the floating-point operation unit, including: after the data decompression module completes the decompression processing, the handling controller stores the second weight data into the data storage device, and inputs the second weight data stored in the data storage device into the floating-point operation unit.

[0039] In an implementable manner, the data transport unit is a direct memory access DMA or a tensor memory access TMA.

[0040] In one implementable manner, the floating point operation unit is a matrix operation core.

[0041] In a fourth aspect, an operation method is provided, which is executed by a floating-point operation unit, and the floating-point operation unit includes a data cache module, a data decompression module and a data calculation module. The operation method includes: the data cache module caches the first weight data and the first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model. The data decompression module decompresses the first weight data to obtain the second weight data represented by a floating point number. The data calculation module performs floating-point operations on the first activation data and the second weight data to obtain an operation result.

[0042] In an achievable manner, the compression process includes at least one of coding compression and quantization compression. When the compression process includes coding compression, the decompression process includes decoding process, and when the compression process includes quantization compression, the decompression process includes numerical conversion.

[0043] In one achievable manner, the compression process includes encoding original weight data into first weight data represented by block floating point numbers, and the decoding process includes decoding the first weight data into second weight data represented by floating point numbers.

[0044] The quantization compression includes quantizing the original weight data into first weight data represented by an integer number, and the numerical conversion includes converting the first weight data into second weight data represented by a floating point number.

[0045] In one achievable manner, the data cache module includes a core cache, an activation matrix register, and a weight matrix register. The above-mentioned data cache module caches the first weight data and the first activation data of the artificial intelligence AI model, including: a core cache, for caching the first weight data and the first activation data, inputting the first weight data into the weight matrix register, and inputting the first activation data into the activation matrix register. The weight matrix register inputs the first weight data into the data decompression module. The activation matrix register is used to input the first activation data into the data calculation module.

[0046] In one implementable manner, a floating-point operation unit includes a plurality of data decompression modules, and the floating-point operation unit is a matrix operation core.

[0047] In a fifth aspect, a computing device is provided, which includes a memory and a processor as described in the first aspect above, and the processor can be used to implement the calculation method described in the third aspect above.

[0048] In a sixth aspect, a computing device is provided, the computing device comprising a memory and a floating-point operation unit as described in the second aspect above, and the floating-point operation unit can be used to implement the operation method described in the fourth aspect above.

[0049] In the seventh aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by a computing device, the computing device executes the operation method described in the third aspect or the fourth aspect above.

[0050] In an eighth aspect, a computer program product comprising instructions is provided, and when the computer program product is run on a computing device, the computing device is enabled to execute the operation method described in the third aspect or the fourth aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flowchart of the operation of weight data and activation data in the related art;

[0052] Figure 2 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0053] Figure 3 is a schematic diagram of a processor structure provided in an embodiment of the present application;

[0054] Figure 4 is a flow chart of a calculation method provided in an embodiment of the present application;

[0055] Figure 5 It is a structural schematic diagram of a data handling unit provided in an embodiment of the present application;

[0056] Figure 6 It is a structural schematic diagram of a data handling unit provided in an embodiment of the present application;

[0057] Figure 7 is a schematic diagram of a method for executing a calculation on a computing device provided by an embodiment of the present application;

[0058] Figure 8 It is a schematic diagram of the structure of a floating-point operation unit provided in an embodiment of the present application;

[0059] Fig. 9 is a flow chart of a calculation method provided in an embodiment of the present application;

[0060] Fig.10 It is a schematic diagram of a method for executing a calculation on a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0062] At the beginning of the design of the Artificial Intelligence (AI) model, in order to improve the accuracy of the AI ​​model, the weight data and activation data will be set to higher-precision floating-point numbers, such as FP32 and FP16. Floating-point numbers occupy more bits, such as FP32 floating-point numbers occupy 32 bits, and FP16 floating-point numbers occupy 16 bits. Therefore, floating-point numbers require more storage and computing resources when stored and calculated. Especially in some large models, such as the Large Language Model (LLM), the amount of data corresponding to the weight data is huge, and the storage of the weight data alone requires hundreds of GB of storage space. Among them:

[0063] The AI ​​model can be any trained AI model, such as a large language model, an image recognition model, etc.

[0064] Weight data includes the weights of the AI ​​model after training. In the actual operation of the AI ​​model, weight data is generally expressed in the form of vectors or matrices.

[0065] Activation data can be the output data of the middle layer of the AI ​​model, which is related to the sample data input into the AI ​​model. In the actual operation of the AI ​​model, the activation data can also be expressed in the form of vectors or matrices, and often performs vector or matrix operations with weight data.

[0066] In order to improve the universality of AI models, quantization compression will be used to reduce the storage space required for storing models and improve the running speed of AI models. Quantization compression of AI models refers to a method of quantizing the weight data and activation data in the AI ​​model from floating point numbers to integers, thereby improving the running speed of the AI ​​model. For example, the original FP16 weight data and activation data in the AI ​​model can be quantized to INT8. In this way, the storage of weight data is reduced from 16 bits to 8 bits.

[0067] However, when converting floating-point numbers to integers, the precision of the values ​​will decrease. The larger the range of floating-point numbers, the more serious the precision loss when converting floating-point numbers to integers. Therefore, quantization compression will have a greater impact on the precision of AI models.

[0068] Because in the AI ​​model, the numerical range of activation data is generally larger than the numerical range of weight data, and there is a certain proportion of outliers in the activation data, which leads to a larger error in the quantization of activation data, and the impact on the accuracy of the AI ​​model is greater than the quantization of weight data. In most AI models, the amount of weight data is much larger than the amount of activation data. Therefore, quantizing the weight data can bring better quantization effect to the AI ​​model, and the impact on the accuracy of the AI ​​model is less than the quantization of activation data. Therefore, in order to ensure the accuracy of the AI ​​model in the relevant technology, asymmetric quantization can be used, that is, only the weight data is quantized, and the activation data is not quantized.

[0069] Figure 1 It is a flowchart of the processor operating on the quantized weight data and the unquantized activation data when asymmetric quantization is used in the related technology. The processor includes a matrix operation core, a vector operation core and a data handling unit. The data handling unit can be a direct memory access (DMA) or a tensor memory access (TMA). Figure 1As shown, the data handling unit is used to implement the data handling function for the processor, and the operation process includes:

[0070] S1. The data handling unit may write the activation data stored in the global memory (GM) into the local buffer of the matrix operation core. The activation data is represented by floating point numbers, such as FP16.

[0071] S2. The data handling unit may write the weight data stored in the GM into the local buffer of the vector operation core. The weight data may be the weight data represented by an integer obtained by quantizing the original weight data in the AI ​​model, such as the weight data represented by INT4.

[0072] S3. The vector calculator (vector calculation unit) in the vector operation core may convert the weight data in the first cache from integers to floating-point numbers, and write the converted floating-point numbers back to the local buffer of the vector operation core.

[0073] S4. The data transfer unit can transfer the weight data that has completed the numerical conversion in the local buffer of the vector operation core, that is, the weight data represented by floating point numbers, to the local buffer of the vector operation core (the specific transfer process is not shown in the figure).

[0074] S5. The local buffer of the matrix operation core transfers the activation data and weight data represented by floating point numbers to two matrix registers respectively.

[0075] S6. Input the activation data and weight data cached in the matrix register into the matrix calculator (matrix calculation unit), perform corresponding matrix operations, and write the obtained operation results into the result register.

[0076] S7. The data handling unit writes the operation result stored in the result register into GM to complete the operation.

[0077] Depend on Figure 1 It can be seen that to realize the operation of floating-point activation data and integer weight data, the vector operation core needs to convert the integer weight data into floating-point weight data, and the data needs to be transported across cores through the data transport unit, that is, the floating-point weight data is transported from the vector operation core to the matrix operation core. This occupies a large amount of computing resources and bandwidth resources, resulting in low computing efficiency.

[0078] The computing method provided in the embodiment of the present application can avoid converting the weight data of integer numbers into the weight data of floating-point numbers through the vector computing core, and further avoid cross-core data transfer through the data transfer unit, which can reduce the occupation of computing resources and bandwidth resources and improve computing efficiency. The computing method provided in the embodiment of the present application is implemented in a computing device. Figure 2 Schematic diagram of a computing device provided in an embodiment of the present application. Figure 2 As shown, the computing device 200 may include: a bus 202, a processor 204, a memory 206, and the optional computing device 200 may also include a communication interface 208. The processor 204, the memory 206 and the communication interface 208 communicate through the bus 202. The computing device 200 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 200. The computing device 200 may be a device for running a model, a terminal or a server. When the computing device 200 is a terminal, the computing device 200 includes but is not limited to a desktop computer, a mobile phone, a notebook, a tablet computer, etc. When the computing device 200 is a server, the computing device 200 may be a separate server, or a server cluster composed of multiple servers, or may be a physical machine, or may be a virtual machine, a container, etc. virtualized by virtual technology.

[0079] The bus 202 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 The bus 202 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 202 may include a path for transmitting information between various components of the computing device 200 (eg, the memory 206, the processor 204, and the communication interface 208).

[0080] The processor 204 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP). The processor 204 may further include a floating point operation unit and a data handling unit, and the floating point operation unit may be a matrix operation core.

[0081] The memory 206 may include a volatile memory, such as a random access memory (RAM). The memory 206 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD). The above memories may all be referred to as global memories.

[0082] The communication interface 208 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 200 and other devices or a communication network.

[0083] Figure 3 is a schematic diagram of a processor structure provided in an embodiment of the present application, and the processor may be the above-mentioned Figure 2 The processor 204 in the computing device, such as Figure 3 As shown, the processor includes a data handling unit and a floating point operation unit, and the data handling unit includes a data handling module and a data decompression module. The processor can execute the operation method provided in the embodiment of the present application. Figure 4 is a flowchart of another calculation method provided in an embodiment of the present application, see Figure 4 , the operation method executed by the processor includes:

[0084] Step 401: The data handling module obtains first weight data and first activation data of the AI ​​model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model.

[0085] In implementation, the data handling module can read the first weight data and the first activation data of the AI ​​model from the GM, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model, and the first activation data is a floating point number, which can be the activation data in the AI ​​model that has not been compressed.

[0086] In an embodiment of the present application, the compression process may include coding compression and quantization compression. Coding compression refers to encoding the original weight data into a specified coding format to reduce the storage space required to store the weight data of the AI ​​model. The coding format may be a block floating point format, or may be an 8-bit floating point (8bitEfloat) coding format. Quantization compression refers to quantizing the original weight data from a floating point number to an integer. For example, the original weight data of FP16 can be quantized into the first weight data of INT4. In an achievable manner, the first activation data may also be the weight data obtained by quantizing and compressing the original weight data and then encoding and compressing it. The compression process for the first weight data belongs to the prior art and can be performed before running the AI ​​model, and will not be described in detail in the embodiment of the present application.

[0087] Step 402: The data decompression module decompresses the first weight data to obtain second weight data represented by floating point numbers.

[0088] During implementation, after the data handling module reads the first weight data and the first activation data of the AI ​​model from the GM, the data handling module can input the first weight data into the data decompression module for decompression processing to obtain the second weight data represented by a floating point number.

[0089] Corresponding to the compression process in the above step 401 including coding compression and quantization compression, the decompression process in step 402 includes decoding processing and numerical conversion processing. Wherein, when the compression process includes coding compression, the decompression process includes decoding processing, that is, decoding the encoded first weight data into a floating point number. When the compression process includes quantization compression, the decompression process includes numerical conversion, that is, converting the quantized compressed first weight data into a floating point number.

[0090] In one example, when the compression process is coding compression, the data decompression module is used to decode the first weight data, for example, decoding the first weight data encoded in block floating point format into floating point numbers in FP16 format to obtain second weight data. When the compression process is quantization compression, the data decompression module is used to perform numerical conversion on the first weight data, for example, converting the first weight data represented by INT4 into the second weight data represented by FP16.

[0091] In another example, the compression process is to first quantize and compress the original weight data, and then encode and compress the quantized and compressed weight data. For example, the compression process may be to first quantize the original weight data through FP16 into weight data represented by INT8, and then encode and compress the weight data represented by INT8. In this case, the data decompression module may include a data decoding submodule and a numerical conversion submodule. Among them, the data decoding submodule may first decode the first weight data obtained by the data handling module, and then the numerical conversion submodule may perform numerical conversion on the decoded weight data to obtain the second weight data represented by FP16.

[0092] Among them, the data decompression module is a logic circuit added to the data handling unit. Depending on the different decoding processes, the technician can add different logic circuits in the data handling unit to realize the decompression process of the first weight data. For example, adding a logic circuit to realize the conversion of data from INT4 to FP16, or adding a logic circuit to realize the decoding of data in block floating point format or 8bit Efloat encoding format into data represented by FP16, etc. In addition, during the execution of the embodiment of the present application, the software interface can be configured to indicate whether the data decoding module in the data handling unit needs to perform data decoding on the first weight data.

[0093] Step 403: The data handling module inputs the first activation data and the second weight data into a floating point operation unit to perform floating point operation.

[0094] In implementation, after the data decompression module decompresses the first weight data to obtain the second weight data, the data handling unit can store the second weight data and the first activation data into the registers in the floating-point operation unit, so that the floating-point operation unit performs floating-point operations on the second weight data and the first activation data.

[0095] A direct path may be provided between the data decompression module and the data handling module, so as to realize data transmission between the data decompression module and the data handling module. Figure 5 and Figure 6 Schematic diagrams of the structures of two data handling units provided in the embodiments of the present application. Figure 5 or Figure 6 As shown, the data handling module can be further divided into a handling controller and a data storage device. The handling controller is used to control the handling of data between the memory and / or the register, and the data storage device is used to store the handling data. For example, the process of handling data in the memory to the register may include the handling controller reading the data in the memory and writing the read data into the data storage device, and then the handling controller reads the data in the data storage device again, and then writes the read data into the register in the processor.

[0096] like Figure 5 As shown, the decompression process of the data handling module inputting the first weight data to the data decompression module in the above step 402 may include: the handling controller stores the first weight data stored in the GM into the data memory, and then inputs the data memory into the data decompression module. After the data decompression module completes the decompression process, the handling controller inputs the second weight data into the floating point operation unit.

[0097] In implementation, after the transport controller reads the first weight data and the first activation data to be calculated from the GM, the first weight data and the first activation data can be stored in the data memory, and then the first weight data stored in the data memory is input into the data decompression module, and the data decompression module decompresses the first weight data to obtain the second weight data represented by a floating point number. After the data decompression module outputs the second weight data, the transport controller can store the second weight data and the first activation data in the register in the floating-point operation unit, so that the floating-point operation unit performs floating-point operations on the second weight data and the first activation data.

[0098] like Figure 6 As shown, the decompression processing of the data transport module inputting the first weight data into the data decompression module in the above step 402 may also include: the transport controller first inputs the first weight data stored in the GM into the data decompression module, and after the data decompression module completes the decompression processing, the second weight data can be stored in the data storage device, and then the transport controller can input the second weight data stored in the data storage device into the floating-point operation unit.

[0099] In implementation, after the transport controller reads the first weight data and the first activation data to be calculated from the GM, the first weight data can be input into the data decompression module, and the first activation data can be stored in the data memory. After the data decompression module decompresses the first weight data to obtain the second weight data represented by a floating point number, the transport controller can store the second weight data and the first activation data in the register in the floating point operation unit, so that the floating point operation unit performs floating point operations on the second weight data and the first activation data.

[0100] exist Figure 5 and Figure 6 In the data transport unit shown, there is a direct path between the data storage device and the data decompression module, and the transmission rate of the first weight data between the data storage device and the data decompression module is much greater than the transmission rate of the first weight data between the matrix operation core and the vector operation core.

[0101] In an embodiment of the present application, a data decoding module is added to the data handling unit. In this way, the data handling unit can decompress the first weight data in the process of carrying the first weight data to obtain the second weight data represented by a floating point number. In this way, on the one hand, only the weight data in the AI ​​model is compressed, and the influence of quantization compression on the accuracy of the AI ​​model can be reduced while the AI ​​model is quantized and compressed. On the other hand, adding the data decompression module to the data handling unit avoids the data handling unit from carrying the first weight data to the vector operation core for decompression, which can improve the efficiency of decompression of the first weight data, and thus improve the efficiency of floating-point operations of the first activation data and the second weight data. On the other hand, relative to the related art, in addition to reducing the storage space of weight data by quantization compression in the embodiment of the present application, the storage space of weight data can also be reduced by encoding and compressing the weight data.

[0102] Figure 7 is a schematic diagram of a method for performing a calculation on a computing device provided by an embodiment of the present application. Figure 7 The computing device shown includes a global memory and a processor, which may be the above-mentioned Figure 3 In the processor shown in FIG. 1 , the floating point operation unit in the processor may be a matrix operation core. Figure 7 As shown, the matrix operation core includes at least a core buffer, a matrix register, a result buffer and a matrix calculator. Among them, the core buffer is used to cache the first activation data and the second weight data written by the data handling unit, and the matrix register is used to store the first activation data and the second weight data in the form of a matrix, respectively, as an input matrix of the matrix calculator. The matrix calculator is capable of realizing a matrix outer product or a matrix inner product, and is used to perform a floating-point matrix operation on the first activation data and the second weight data. The result buffer is used to cache the result data obtained after the floating-point matrix operation on the first activation data and the second weight data is performed.

[0103] like Figure 7 As shown, the data transport unit obtains the first activation data and the first weight data from the GM, and decodes the first weight data into the second weight data, and then the first activation data and the second weight data can be transported to the matrix operation core buffer.

[0104] In the matrix operation core, the matrix operation core can store the first activation data in the form of a vector to a matrix register, which can be called an activation matrix register or a left matrix register, and store the second weight data in the form of a vector to another matrix register, which can be called a weight matrix register or a right matrix register, according to a pre-set Tiling algorithm for performing matrix operations on the first activation data and the second weight data.

[0105] The first activation data and the second weight data stored in the activation matrix register and the weight matrix register can be input into the matrix calculator to perform matrix operations of floating point numbers. For example, the matrix calculator can perform matrix inner product operations on the first activation data and the second weight data. After the matrix calculator obtains the matrix operation result, it can store the matrix operation result in the result register. The data handling unit can then carry the matrix operation result in the result register to the GM.

[0106] It can be seen that the use of the calculation method provided in the embodiment of the present application can avoid the data transfer unit from transferring the first weight data to the vector operation core for decompression processing, avoid the cross-core transmission of the decompressed second weight data, and improve the efficiency of floating-point operations of the first activation data and the second weight data.

[0107] Figure 8 is a schematic diagram of the structure of a floating-point operation unit provided in an embodiment of the present application, and the floating-point operation unit may be the above-mentioned Figure 2 The processor 204 in the computing device shown in FIG. Figure 8 As shown, the floating-point operation unit includes a data cache module, a data decompression module and a data calculation module, wherein the data cache module can be a register in the floating-point operation unit, used to cache the first activation data and the first weight data to be calculated in the AI ​​model. The data decompression module is used to decompress the first weight data to obtain the second weight data. The data calculation module is used to perform a floating-point matrix multiplication operation on the first activation data and the second weight data. The floating-point operation unit can implement another operation method provided in an embodiment of the present application.

[0108] Fig. 9 is a flowchart of a calculation method provided in an embodiment of the present application, see Fig. 9 , the operation method performed by the floating-point operation unit includes:

[0109] Step 901: The data cache module caches the first weight data and the first activation data of the AI ​​model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model.

[0110] In implementation, the data handling module can read the first weight data and the first activation data of the AI ​​model from the GM, and then input the read first weight data and the first activation data into the data cache module in the floating point operation unit. The first weight data is obtained by compressing the original weight data in the AI ​​model, and the first activation data is a floating point number, which can be the activation data in the AI ​​model that has not been compressed.

[0111] In an embodiment of the present application, the compression process may include coding compression and quantization compression. Coding compression refers to encoding the original weight data into a specified coding format to reduce the storage space required to store the weight data of the AI ​​model. The coding format may be a block floating point format, or may be an 8bit Efloat coding format. Quantization compression refers to quantizing the original weight data from a floating point number to an integer. For example, the original weight data of FP16 can be quantized into the first weight data of INT4. In an achievable manner, the first activation data may also be the weight data obtained by quantizing and compressing the original weight data and then encoding and compressing it again.

[0112] Step 902: The data decompression module decompresses the first weight data to obtain second weight data represented by floating-point numbers.

[0113] In implementation, the data handling module may input the cached first weight data into the data decompression module, and the data decompression module decompresses the first weight data to obtain the second weight data represented by floating point numbers.

[0114] Corresponding to the compression process in the above step 901 including coding compression and quantization compression, the decompression process in step 902 includes decoding processing and numerical conversion processing. Wherein, when the compression process includes coding compression, the decompression process includes decoding processing, that is, decoding the encoded first weight data into a floating point number. When the compression process includes quantization compression, the decompression process includes numerical conversion, that is, converting the quantized compressed first weight data into a floating point number.

[0115] Step 903: The data calculation module performs floating-point operations on the first activation data and the second weight data to obtain a calculation result.

[0116] In implementation, the data cache module performs floating-point operations on the first activation data in the data cache module and the second weight data obtained after decompression processing by the data decompression module to obtain an operation result.

[0117] In an embodiment of the present application, a data decompression module is added to a floating-point operation unit, so that the compressed first weight data can be directly decompressed in the floating-point operation unit to obtain the second weight data represented by a floating-point number, and then the floating-point operation unit can perform floating-point operations on the first activation data and the second weight data. In this way, on the one hand, only the weight data in the AI ​​model is compressed, which can avoid the impact of the quantization of the activation data on the accuracy of the AI ​​model. On the other hand, adding a data decompression module to the floating-point operation unit avoids the need to transport the first weight data to the vector operation core for decompression, which can improve the efficiency of decompression of the first weight data, and thus improve the efficiency of floating-point operations on the first activation data and the second weight data. On the other hand, relative to the related art, in addition to reducing the storage space of weight data through quantization compression in the embodiment of the present application, the storage space of weight data can also be reduced by encoding and compressing the weight data.

[0118] Fig.10 is a schematic diagram of a method for performing a calculation on a computing device provided by an embodiment of the present application. Fig.10 The computing device shown includes a global memory and a processor, and the processor may include the above-mentioned Figure 8 The floating point operation unit shown in FIG. 1 may be a matrix operation core. Fig.10 As shown, the data cache module in the matrix operation core includes at least a core cache, an activation matrix register, a weight matrix register, a result register and a data decompression module. Among them, the core cache is used to cache the first activation data and the second weight data written by the data handling unit, and the matrix register is used to store the first activation data and the second weight data in the form of a matrix, respectively, as an input matrix of the matrix calculator. The result cache is used to cache the result data obtained after the first activation data and the second weight data are subjected to the matrix operation of floating point numbers. The data decompression module is used to decompress the compressed weight data.

[0119] like Fig.10 As shown, after the data handling unit reads the first activation data and the first weight data from the GM, the first activation data and the first weight data can be written into the core buffer in the matrix operation core. Then the processor can store the first activation data in the matrix operation core in the form of a vector to a matrix register according to a preset Tiling algorithm for performing matrix operations on the first activation data and the first weight data. The matrix register can be called an activation matrix register or a left matrix register, and the first weight data can be stored in the form of a vector to another matrix register, which can be called a weight matrix register or a right matrix register.

[0120] The first activation data stored in the activation matrix register can be input into the data calculation module in the matrix operation core, that is, the matrix calculator. The first weight data stored in the weight matrix register can be input into the data decompression module located between the weight matrix register and the matrix calculator for decompression processing. The second weight data obtained by the decompression processing of the data decompression module can be input into the matrix calculator. Since there is a direct connection between the data decompression module, the weight matrix register and the matrix calculator, the transmission rate of the first weight data between the data decompression module, the weight matrix register and the matrix calculator is much greater than the transmission rate of the first weight data in the matrix operation core and the vector operation core.

[0121] Among them, the data decompression module is a logic circuit added between the weight matrix register and the matrix calculator. Depending on the decoding process, the technician can add different logic circuits in the data handling unit to realize the decompression process of the first weight data. For example, adding a logic circuit to realize the conversion of data from INT4 to FP16, or adding a logic circuit to realize the decoding of data in block floating point format or 8bit Efloat encoding format into data represented by FP16, etc. In addition, during the execution of the embodiment of the present application, it can be indicated by configuring the software interface whether the data decoding module in the floating point operation unit needs to perform data decoding on the first weight data.

[0122] In addition, since the matrix calculator is generally designed symmetrically, that is, the data bit widths corresponding to the second weight data and the first activation data are required to be the same, the first weight data can be padded with 0s before being input into the data decoding module, so that the data bit width of the first weight data is the same as the first activation data. In this way, the first weight data can be decompressed into second weight data having the same data bit width as the first activation data through the data decoding module.

[0123] The matrix calculator can perform floating-point matrix operations on the input first activation data and the second weight data. For example, a matrix inner product operation is performed on the first activation data and the second weight data. The matrix calculator can store the operation result in the result register. The data transfer unit can then transfer the operation result in the result register to the GM.

[0124] It can be seen that the use of the calculation method provided in the embodiment of the present application can avoid the data transfer unit from transferring the first weight data to the vector operation core for decompression processing, and avoid the cross-core transmission of the decompressed second weight data, which can improve the efficiency of floating-point operations on the first activation data and the second weight data.

[0125] In one example, since the first weight data stored in the core buffer and the weight matrix register is compressed weight data, the first weight data occupies less storage space, for example, the first weight data is quantized into 4-bit INT4 data. In this way, in the same storage space, the core buffer and the weight matrix register can store more first weight data. Therefore, multiple data decoding modules can be set in parallel between the weight matrix register and the matrix calculator. In implementation, the first weight data stored in the weight matrix register can be input into multiple data decoding modules for decoding processing, which can further improve the decoding efficiency of the first weight data, thereby improving the computing efficiency of the second weight data and the first activation data.

[0126] The present application embodiment provides a processor, which may be the above-mentioned Figure 3 The processor shown in the figure includes a data handling unit and a floating point operation unit, wherein the data handling unit implements the data handling function in the processor, including a data handling module and a data decompression module. The processor can be used to implement the above Figure 4 The operation method shown, where:

[0127] The data handling module is used to obtain the first weight data and the first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model.

[0128] The data decompression module is used to decompress the first weight data to obtain second weight data represented by floating point numbers.

[0129] The data handling module is used to input the first activation data and the second weight data into the floating point operation unit to perform floating point operation.

[0130] In one possible implementation, the compression process includes at least one of coding compression and quantization compression; when the compression process includes coding compression, the decompression process includes decoding processing; when the compression process includes quantization compression, the decompression process includes numerical conversion.

[0131] In one implementable embodiment, the compression processing includes encoding the original weight data into first weight data represented by block floating point numbers, and the decoding processing includes decoding the first weight data into second weight data represented by floating point numbers; the quantization compression includes quantizing the original weight data into first weight data represented by integer numbers, and the numerical conversion includes converting the first weight data into second weight data represented by floating point numbers.

[0132] In one implementable method, the data transport module includes a data storage device and a transport controller; the transport controller is used to store the first weight data in the GM to the data storage device, and then input the data storage device to the data decompression module; the transport controller is used to input the second weight data to the floating-point operation unit after the data decompression module completes the decompression processing.

[0133] In one implementable method, the data transport module includes a data storage device and a transport controller; the transport controller is used to input the first weight data in the GM into the data decompression module; the transport controller is also used to store the second weight data into the data storage device after the data decompression module completes the decompression processing; the second weight data stored in the data storage device is input into the floating-point operation unit.

[0134] In an implementable manner, the data handling unit is a direct memory access DMA or a tensor memory access TMA. The floating point operation unit is a matrix operation core.

[0135] The processor provided in the embodiment of the present application implements the processing of the first weight data and the first activation data operation and the above embodiment Figure 4 The contents of the corresponding operation methods belong to the same inventive concept, and their specific implementation processes are detailed in the above-mentioned method embodiments, which will not be repeated here. Using the processor provided in the embodiment of the present application, when the data handling unit is carrying the first weight data, the data decoding module in the data handling unit can decompress the first weight data to obtain the second weight data represented by a floating point number. In this way, on the one hand, only the weight data in the AI ​​model is compressed, which can reduce the impact of quantization compression on the accuracy of the AI ​​model while quantizing and compressing the AI ​​model. On the other hand, adding a data decompression module to the data handling unit avoids the data handling unit from carrying the first weight data to the vector operation core for decompression, which can improve the efficiency of decompression of the first weight data, thereby improving the efficiency of floating-point operations on the first activation data and the second weight data.

[0136] The present application embodiment provides a floating-point operation unit, which may be an operation core for performing floating-point operations in a processor, such as the above-mentioned Figure 8 The floating point operation unit shown includes a data cache module, a data decompression module and a data calculation module. The floating point operation unit can be used to implement the above Fig. 9 The operation method shown, where:

[0137] A data cache module is used to cache first weight data and first activation data of an artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model.

[0138] The data decompression module is used to decompress the first weight data to obtain second weight data represented by floating point numbers.

[0139] The data calculation module is used to perform floating-point operations on the first activation data and the second weight data to obtain operation results.

[0140] In one possible implementation, the compression process includes at least one of coding compression and quantization compression; when the compression process includes coding compression, the decompression process includes decoding processing, and when the compression process includes quantization compression, the decompression process includes numerical conversion.

[0141] In one implementable embodiment, the compression processing includes encoding the original weight data into first weight data represented by block floating point numbers, and the decoding processing includes decoding the first weight data into second weight data represented by floating point numbers; the quantization compression includes quantizing the original weight data into first weight data represented by integer numbers, and the numerical conversion includes converting the first weight data into second weight data represented by floating point numbers.

[0142] In one implementable manner, the data cache module includes a core cache, an activation matrix register and a weight matrix register; the core cache is used to cache first weight data and first activation data, input the first weight data into the weight matrix register, and input the first activation data into the activation matrix register; the weight matrix register is used to input the first weight data into the data decompression module; and the activation matrix register is used to input the first activation data into the data calculation module.

[0143] In one possible implementation, the floating point operation unit is a matrix operation core.

[0144] The floating point operation unit provided in the embodiment of the present application implements the processing of the first weight data and the first activation data operation and the above embodiment Fig. 9 The contents of the corresponding operation methods belong to the same inventive concept, and their specific implementation processes are detailed in the above-mentioned method embodiments, which will not be repeated here. Using the floating-point operation unit provided in the embodiment of the present application, the first weight data can be decompressed by the data decoding module before the first weight data is input into the data calculation module to obtain the second weight data represented by a floating point number. In this way, on the one hand, only the weight data in the AI ​​model is compressed, which can avoid the influence of the quantization of the activation data on the accuracy of the AI ​​model. On the other hand, adding the data decompression module to the floating-point operation unit avoids the need to transport the first weight data to the vector operation core for decompression, which can improve the efficiency of the decompression of the first weight data, thereby improving the efficiency of floating-point operations on the first activation data and the second weight data.

[0145] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the computing method provided in the present application.

[0146] The present application embodiment also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the computing method provided in the present application embodiment.

[0147] In this application, the words such as term "first", "second" are used to distinguish the same items or similar items with basically the same effect and function. It should be understood that there is no logical or temporal dependency between "first" and "second", and the quantity and execution order are not limited. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish an element from another element. For example, without departing from the scope of various examples, the first weight data can be referred to as the second weight data, and similarly, the second weight data can be referred to as the first weight data. The first weight data and the second weight data can both be collectively referred to as weight data, and in some cases, can be separate and different weight data.

[0148] The term "at least one" in this application means one or more, and the term "plurality" in this application means two or more.

[0149] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A processor, characterized in that: The processor includes a data handling unit and a floating point operation unit, and the data handling unit includes a data handling module and a data decompression module, wherein: The data handling module is used to obtain first weight data and first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model; The data decompression module is used to decompress the first weight data to obtain second weight data represented by floating point numbers; The data handling module is used to input the first activation data and the second weight data into the floating point operation unit; The floating-point operation unit is used to perform floating-point operation on the first activation data and the second weight data to obtain an operation result.

2. The processor according to claim 1, characterized in that The compression process includes at least one of coding compression and quantization compression; When the compression process includes the encoding compression, the decompression process includes the decoding process; When the compression process includes the quantization compression, the decompression process includes numerical conversion.

3. The processor according to claim 2, characterized in that The compression process includes encoding the original weight data into first weight data represented by block floating point numbers, and the decoding process includes decoding the first weight data into second weight data represented by floating point numbers; The quantization compression includes quantizing the original weight data into first weight data represented by an integer number, and the numerical conversion includes converting the first weight data into second weight data represented by a floating point number.

4. The processor according to any one of claims 1 to 3, characterized in that: The data handling module includes a data storage device and a handling controller; The transport controller is used to store the acquired first weight data in the data storage device, and then input the data storage device into the data decompression module; The transport controller is used to input the second weight data into the floating-point operation unit after the data decompression module completes the decompression process.

5. The processor according to any one of claims 1 to 3, characterized in that: The data handling module includes a data storage device and a handling controller; The transport controller is used to input the acquired first weight data into the data decompression module; The transport controller is further used to store the second weight data in the data storage device after the data decompression module completes the decompression process; and input the second weight data stored in the data storage device into the floating-point operation unit.

6. The processor according to any one of claims 1 to 5, characterized in that: The data handling unit is a direct memory access DMA or a tensor memory access TMA.

7. The processor according to any one of claims 1 to 6, characterized in that: The floating point operation unit is a matrix operation core.

8. A floating point arithmetic unit, characterized in that: The floating point operation unit includes a data cache module, a data decompression module and a data calculation module, wherein: The data cache module is used to cache first weight data and first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model; The data decompression module is used to decompress the first weight data to obtain second weight data represented by floating point numbers; The data calculation module is used to perform floating-point operations on the first activation data and the second weight data to obtain a calculation result.

9. The floating point unit according to claim 8, characterized in that: The compression process includes at least one of coding compression and quantization compression; When the compression process includes the encoding compression, the decompression process includes the decoding process; when the compression process includes the quantization compression, the decompression process includes numerical conversion.

10. The floating point operation unit according to claim 9, characterized in that: The compression process includes encoding the original weight data into first weight data represented by block floating point numbers, and the decoding process includes decoding the first weight data into second weight data represented by floating point numbers; The quantization compression includes quantizing the original weight data into first weight data represented by an integer number, and the numerical conversion includes converting the first weight data into second weight data represented by a floating point number.

11. The floating point operation unit according to any one of claims 8 to 10, characterized in that: The data cache module includes a core cache, an activation matrix register and a weight matrix register; The core buffer is used to cache the first weight data and the first activation data, input the first weight data into the weight matrix register, and input the first activation data into the activation matrix register; The weight matrix register is used to input the first weight data into the data decompression module; The activation matrix register is used to input the first activation data into the data calculation module.

12. The floating point operation unit according to any one of claims 8 to 11, characterized in that: The floating point operation unit includes a plurality of data decompression modules.

13. The floating point operation unit according to any one of claims 8 to 12, characterized in that: The floating point operation unit is a matrix operation core.

14. A computing method, characterized in that: The method is executed by a processor, the processor includes a data handling unit and a floating point operation unit, including a data handling module and a data decompression module, and the method includes: The data handling module obtains first weight data and first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model; The data decompression module decompresses the first weight data to obtain second weight data represented by floating point numbers; The data handling module inputs the first activation data and the second weight data into the floating point operation unit; The floating-point operation unit is used to perform floating-point operation on the first activation data and the second weight data to obtain an operation result.

15. A computing method, characterized in that: The method is performed by a floating-point operation unit, wherein the floating-point operation unit includes a data cache module, a data decompression module, and a data calculation module. The method includes: The data cache module caches first weight data and first activation data of the artificial intelligence AI model, wherein the first weight data is obtained by compressing the original weight data in the AI ​​model; The data decompression module decompresses the first weight data to obtain second weight data represented by floating point numbers; The data calculation module performs floating-point operations on the first activation data and the second weight data to obtain a calculation result.

16. A computing device, characterized in that The computing device comprises a memory and a processor as claimed in any one of claims 1 to 7.

17. A computing device, characterized in that The computing device comprises a memory and a floating point operation unit as claimed in any one of claims 8 to 13.

Images