Data operation method, equipment and medium

Abstract

The invention provides a data operation method and device and a medium, and the method comprises the steps: determining first data under a first type, reading the first data in the form of a second type to obtain second data, the first type being outside the type range supported by calculation hardware, the second type being within the type range, and the second type being within the type range; the first memory space occupied by the first data is smaller than or equal to the second memory space occupied by the second data; data occupying the size of the first memory space in the second data is expanded into third data in a third type, data operation is carried out based on the third data, an operation result is obtained, the third type is within the type range, and the precision of the third type is higher than that of the first type. According to the method, the equipment and the medium provided by the invention, the first data is read by using the second type supported by the computing hardware, so that the compatibility of the computing hardware is improved. And during operation, the second data is expanded into a third type with higher precision, so that the accuracy of an operation result is ensured.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data computing method, device and medium. Background Art

[0002] As the scale of deep learning models continues to expand, the complexity of calculations is also increasing. In order to improve computing efficiency and reduce computing resource consumption, a method of using low-precision operations is proposed. This method can speed up calculations and reduce storage requirements by reducing the precision of data, while maintaining a good balance between performance and accuracy. In recent years, low-precision computing technology has been widely used in various deep learning frameworks and hardware devices, significantly improving the efficiency of model training and reasoning, and becoming one of the key means to optimize the performance of artificial intelligence algorithms.

[0003] However, low-precision computing technology has certain limitations in practical applications. Some hardware architectures do not support some types of low-precision data operations, which makes it impossible to directly use low-precision technology to accelerate computing on these hardware. Summary of the invention

[0004] The present invention provides a data operation method, device and medium, which are used to solve the defect that the computing hardware in the related art does not support low-precision operations.

[0005] The present invention provides a data computing method, comprising the following steps: Determine first data of a first type, and read the first data in a second type to obtain second data, wherein the first type is outside a type range supported by computing hardware, the second type is within the type range, and a first memory space occupied by the first data is less than or equal to a second memory space occupied by the second data; The data in the second data that occupies the size of the first memory space is expanded into third data of a third type, and data calculation is performed based on the third data to obtain a calculation result, wherein the third type is within the type range and the precision of the third type is higher than the precision of the first type.

[0006] According to a data operation method provided by the present invention, the second memory space is a positive integer multiple of the first memory space.

[0007] According to a data operation method provided by the present invention, the step of expanding the data in the second data that occupies the size of the first memory space into third data of a third type includes: When the positive integer is 1, the second data is expanded to third data of the third type; When the positive integer is greater than 1, the positive integer number of data in the second data occupying the first memory space are respectively expanded into third data of the third type.

[0008] According to a data operation method provided by the present invention, performing data operation based on the third data to obtain an operation result includes: In the case where the third type is outside the data calculation type range supported by the calculation module of the computing hardware, converting the third data into fourth data of a fourth type, performing data calculation based on the fourth data, and obtaining the calculation result; The fourth type is a data type within the range of data calculation types supported by the operation module, and the precision of the fourth type is less than or equal to the precision of the third type and greater than the precision of the first type.

[0009] According to a data operation method provided by the present invention, converting the third data into fourth data of a fourth type includes: In a case where the operation module is a tensor processor of the computing hardware, transmitting the third data from a computing unit of the computing hardware to the tensor processor; The tensor processor is controlled to convert the third data into fourth data of a fourth type.

[0010] A data operation method provided by the present invention further includes: Converting the operation result into fifth data of the first type; The fifth data is written into the memory in the form of the second type.

[0011] According to a data computing method provided by the present invention, the computing hardware is an artificial intelligence chip.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned data operation methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data operation method described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned data operation methods.

[0015] The data computing method, device and medium provided by the present invention improve the compatibility of computing hardware and expand the application scenarios of computing hardware by using the second type supported by computing hardware to read the first data. In the computing stage, the second data has a higher precision after being expanded to the third type, and can be used for more complex computing, reducing computing errors and ensuring the accuracy of computing results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of the data calculation method provided by the present invention.

[0018] Figure 2 It is a structural schematic diagram of the data computing device provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] With the rapid development of artificial intelligence and deep learning technologies, the scale and complexity of models are increasing, and the demand for computing resources is also rising sharply. In order to improve computing efficiency, reduce energy consumption and hardware costs, low-precision data operations have gradually become one of the mainstream methods for optimizing the performance of deep learning models. Low-precision operations can reduce the memory space and bandwidth required for calculations, while speeding up calculations, so they have been widely used in model training and reasoning. Many hardware platforms and deep learning frameworks already support low-precision operations, such as using data types with specific precision to replace traditional high-precision data types to improve computing efficiency. However, with the popularization of low-precision computing technology, the compatibility of hardware architecture with data types of different precisions has also been challenged.

[0022] The traditional data precision conversion process usually involves direct numerical conversion operations. Taking the conversion from FP32 to FP16 as an example, 1.0 of FP32 is represented as 0x3f800000 in hexadecimal, and after conversion to FP16, 1.0 is represented as 0x3c00 in hexadecimal. Although the numerical value is the same, the storage and representation methods of the data are completely different under different precisions. This conversion is based on the IEEE 754 floating-point standard and follows a specific bit width, exponent, and mantissa format. Therefore, the conversion between the two precisions is not just a change in the number of bits, but also involves complex operations such as rounding and truncation.

[0023] In actual applications, different hardware architectures have different support for various precision types. For example, some artificial intelligence chips do not support FP16 type data operations, especially in deep learning models that require high precision. These hardware platforms find it difficult to directly perform low-precision calculations. On chips that do not support FP16, you may choose to use BF16 for calculations. Although this conversion can ensure that the calculation is completed smoothly, it may cause the accuracy of the calculation results to decrease, affecting the training and reasoning effects of the model.

[0024] In order to solve these problems, there is an urgent need for a method that can efficiently perform low-precision data operations on hardware platforms that do not support specific low-precision data types. This will not only expand the application scope of low-precision operations, but also further improve the performance of deep learning models while ensuring the accuracy of the operation results.

[0025] In view of the above problems, an embodiment of the present invention provides a data operation method. It should be noted that the data operation method provided by at least one embodiment of the present invention can implement a low-precision analog conversion method in different application fields to meet the needs of different scenarios. For example: the precision conversion method provided by at least one embodiment of the present invention can be applied in the fields of speech processing, image processing, text processing, and video processing.

[0026] For example, in the field of speech processing, the data calculation method provided by the present invention can be applied to tasks such as speech noise suppression, speech separation, and feature extraction, such as noise suppression methods based on Fourier transform, speech separation models based on deep learning, etc.

[0027] For example, in the field of image processing, the data calculation method provided by the present invention can be applied to various image analysis tasks such as image edge detection, image segmentation, feature extraction, etc., such as edge detection algorithms based on Sobel operator, Canny operator, etc.

[0028] For example, in the field of text processing, the data calculation method provided by the present invention can be applied to tasks such as text analysis, semantic classification, and sentiment recognition, such as word vector representation and feature extraction parameters.

[0029] For example, in the field of video processing, the data calculation method provided by the present invention can be applied to tasks such as object tracking, inter-frame motion estimation, scene recognition, such as scene recognition based on convolutional neural networks, etc.

[0030] Figure 1 It is a flow chart of the data operation method provided by the present invention, such as Figure 1 As shown, the method can be applied to computing hardware, where the computing hardware can be an artificial intelligence chip, such as TPU (Tensor Processing Unit), NPU (Neural network Processing Unit), DPU (Deep learning Processing Unit), APU (Accelerated Processing Unit), GPU (Graphics Processing Unit) and GPGPU (General-Purpose computing on Graphics Processing Unit), or other computing hardware that can be used for data computing.

[0031] The method includes: Step 110, determine first data under a first type, and read the first data in the form of a second type to obtain second data, the first type is outside the type range supported by the computing hardware, the second type is within the type range, and a first memory space occupied by the first data is less than or equal to a second memory space occupied by the second data.

[0032] Specifically, since the data type of the first data exceeds the type range supported by the computing hardware, the operation cannot be performed directly on the computing hardware, so it is necessary to indirectly process the first data of the first type by using the second type supported by the computing hardware. The specific operation method is to read the first data of the first type in the form of the second type. In other words, the first type to which the first data itself belongs is ignored, and the first data is regarded as data of the second type for data reading.

[0033] It is worth noting that when reading the first data in the form of the second type, it is particularly important to ensure that the first memory space occupied by the first data is less than or equal to the second memory space of the second data. The first memory space here is the memory space occupied by the first data, and the second memory space is the memory space occupied by the second data. The first memory space and the second memory space only indicate the size of the memory space occupied by the first data and the second data, and do not mean that the two are in different memory spaces. This design can effectively avoid data overflow. If the memory space occupied by the first data exceeds the memory space occupied by the second data, the data may not be fully retained during the conversion process, resulting in partial data loss and loss of precision. On the contrary, if the memory space occupied by the first data is less than or equal to the memory space occupied by the second data, the computing hardware can directly read the first data in the form of the second type without complex compression or segmentation operations. This margin of memory space can improve reading efficiency and ensure that the data is read with as little precision loss as possible. For example, the first type is FP16, the sign bit width of FP16 is 1, the exponent bit width is 5, and the base bit width is 10, that is, the memory space occupied by FP16 is 16 bits. The sign bit width of BF16 is 1, the exponent bit width is 8, and the base bit width is 7, that is, the memory space occupied by BF16 is also 16 bits. The memory space occupied by FP16 is the same as that occupied by BF16, so BF16 can be selected as the second type.

[0034] In the specific operation, first determine the first data to be processed, that is, the data of the first data type in the data input to the computing hardware. The first data can be determined by the preprocessing module according to the characteristics or storage location of the first data or by directly analyzing the data characteristics of the input data. The embodiment of the present invention does not limit this. Since this type of data cannot be directly processed by the computing hardware, it is necessary to read these data in the form of the second type. The first data is loaded into the memory or register of the computing hardware through the read operation to ensure that these data can be used for subsequent calculations. For ease of explanation, in the embodiment of the present invention, the first data read in the form of the second type is recorded as the second data.

[0035] It is understandable that the data types of the first data and the second data have changed in form, but the values ​​of the first data and the second data have not changed, and there is no substantial type conversion. For example, when the first data is of FP16 data type and the second data type is of BF16 data type, when the first data is 1.0, it is represented as 0x3c00 in FP16 hexadecimal. The memory or register of the computing hardware will read 0x3c00 in the second data type, that is, BF16 data type, even though 0x3c00 may not even be a legal expression under the BF16 data type.

[0036] The advantage of reading the first data in the form of the second type is that it allows the processing of these data to be indirectly realized by reading the second type of data that is equal to or larger than the first memory space occupied by the first type when the computing hardware does not directly support the first type of data. Here, since the memory space of the second type is greater than or equal to the first type, when reading in the form of the second type, each bit of the first data in the memory space can be completely retained, thereby avoiding the numerical change of the first data. Moreover, since the second type itself is a type supported by the computing hardware, reading in the form of the second type can also ensure the feasibility of the computing hardware when reading data, thereby expanding the application scope of the computing hardware. In addition, by ensuring that the memory space of the second type is greater than or equal to the first type, data overflow and precision loss can be effectively avoided.

[0037] Step 120: Expand the data in the second data that occupies the size of the first memory space into third data of a third type, and perform data operations based on the third data to obtain operation results. The third type is within the type range, and the precision of the third type is higher than the precision of the first type.

[0038] Specifically, after reading the first data to obtain the second data, in order to achieve high-precision data operations, it is necessary to expand the data occupying the first memory space in the second data to the third data under the third type. It can be understood that the data occupying the size of the first memory space in the second data is the first data. For example, an input data of the FP16 data type occupies 16 bits of memory space, but the computing hardware does not support this type. Therefore, when reading the data, it is regarded as the second type supported by the hardware-for example, the FP32 format is read. In this way, the read data "second data" will occupy 32 bits of the second memory space, although the actual data content contained therein only corresponds to the size of the first memory space (ie, 16 bits). It should be noted that the first data is not read directly with the third data type because the third data type has higher precision and occupies a larger memory space, which will cause a waste of storage resources.

[0039] Here, the third type is within the range of types supported by the computing hardware and has higher precision than the first type. Since the second data retains the original information of the first data, and the third data has higher precision than the first data, for example, if the first type is FP16, the second data needs to be expanded to a third type such as FP32 or BF24 with higher precision. The expanded third data can enable the data to have more precise expression and operation in subsequent calculations.

[0040] During the expansion process, each data unit of the second data that occupies the first memory space (such as a floating point number or a single element in a matrix) is mapped to the third type, and the same value is represented by the larger bit width and precision of the third type. This process does not change the actual content of the data, but redefines it within the representation range of the third type to better support high-precision operations. This expansion uses the specifications of the third type to expand the data from a lower-precision representation to a higher precision range to ensure that subsequent operations can be more accurate. For example, assume that the second type is BF16 type and the third type is FP32 type. The second type may have a smaller bit width (7-bit mantissa and 8-bit exponent of BF16), while the third type has a wider bit width (23-bit mantissa and 8-bit exponent of FP32). During the expansion process, each data unit of the second data is re-represented in the third type, which not only retains the original value, but also provides higher precision and a wider representation range through the third type, thereby reducing precision loss in subsequent calculations.

[0041] After expanding to the third type of data, the computing modules of the computing hardware can perform complex calculations based on these high-precision data. The accuracy of the third type of data is much higher than that of the first type, which ensures that the loss of precision is reduced during the calculation process and can handle complex mathematical operations more accurately. For computing tasks that require high precision, such as scientific computing or complex model operations, using the third type of data can significantly improve the accuracy of the results and avoid errors caused by insufficient precision.

[0042] Under this mechanism of data precision expansion, the entire process does not damage the data information of the first data. It only expands the originally limited information to a larger representation range by using a higher-precision data type, thereby ensuring the accuracy and reliability of the data in subsequent operations. This processing method is particularly suitable for scenarios that require high-precision calculations, ensuring that the integrity and accuracy of the data will not be affected even in more complex operations.

[0043] The data calculation method provided by the embodiment of the present invention improves the compatibility of the computing hardware and expands the application scenarios of the computing hardware by reading the first data using the second type supported by the computing hardware. In the calculation stage, the second data has a higher precision after being expanded to the third type, and can be used for more complex calculations, reducing calculation errors and ensuring the accuracy of the calculation results.

[0044] Based on the above embodiment, the second memory space is a positive integer multiple of the first memory space.

[0045] Specifically, the positive integer multiple relationship between the second memory space and the first memory space ensures that the space occupied by the second type in the memory is large enough to accommodate and accurately represent the data of the first type, so that each data unit of the second type can accurately correspond to the data unit of the first type, thereby avoiding data truncation or inability to represent.

[0046] The positive integer multiple design also greatly improves the processing efficiency of the hardware. First, computing hardware usually requires that data be stored in an aligned manner in memory, that is, the starting address of the data in the memory needs to meet a certain multiple relationship (for example, the first type of data occupies 4 bytes, the second type of data occupies 8 bytes, and the memory space of the second type of data is twice the memory space of the first type of data). When the second memory space is a positive integer multiple of the first memory space, the data naturally meets this alignment requirement. In addition, aligned memory access can significantly improve the access speed of the hardware because the processor can read and write aligned data in a faster manner, thereby avoiding additional operations and delays caused by data spanning multiple memory blocks or address misalignment.

[0047] More importantly, the allocation of memory space in positive integer multiples enables full use of existing memory resources when data is expanded to a third type with higher precision during subsequent data operations, without the need for complex reallocation or data compression operations. This multiple relationship provides sufficient memory space for high-precision and complex operations, ensuring that while the calculation accuracy is improved, the performance and processing efficiency of the hardware will not be affected, thus optimizing the entire calculation process.

[0048] Based on the above embodiment, in step 120, the step of expanding the data in the second data that occupies the size of the first memory space into third data of the third type includes: When the positive integer is 1, the second data is expanded to third data of the third type; When the positive integer is greater than 1, the positive integer number of data in the second data occupying the first memory space are respectively expanded into third data of the third type.

[0049] Specifically, the process of expanding the data occupying the first memory space in the second data into third data of the third type can be divided into two cases: when the positive integer is 1, the second memory space is the same size as the first memory space; and when the positive integer is greater than 1, the second memory space is a multiple of the first memory space.

[0050] In the first case, that is, when the positive integer is 1, the data of the second type has completely represented the data of the first type. Under this condition, the expansion operation is relatively simple. It only needs to re-express the data according to the standard of the third type, that is, to increase the precision and storage bit width of the data to the level required by the third type. This expansion process does not substantially change the data content, but represents it in a higher precision format, ensuring that it can be calculated with higher precision in subsequent operations.

[0051] In the second case, that is, when the positive integer is greater than 1, it is necessary to expand multiple data units in the second data separately. Each data unit occupying the first memory space will be independently expanded to the third data under the third type. In this case, the expansion operation is more complicated because it is necessary to ensure that each data unit is accurately expanded to a data unit of the third type with higher precision. However, due to the relationship of positive integer multiples, multiple data units can be efficiently expanded independently and ensure that the expansion result of each data unit meets the precision requirements of the third type. For example, in the case where the first type occupies 4 bytes, the second type occupies 8 bytes, and the second memory space is twice the size of the first memory space, the first 4 bytes in the second data can be expanded to one third data, and the last 4 bytes in the second data can be expanded to another third data.

[0052] Through this flexible expansion method, no matter whether the size of the second memory space is the same as or multiple of the first memory space, the expansion strategy can be reasonably selected according to the size of the memory space to ensure that the data remains consistent and accurate when it is expanded to the third type. This design enhances the flexibility of data processing and ensures that when the data is expanded to the third type, whether in simple or complex scenarios, the consistency and high accuracy of the data before and after the conversion can be maintained.

[0053] Based on the above embodiment, in step 120, performing data operation based on the third data to obtain the operation result includes: In the case where the third type is outside the data calculation type range supported by the calculation module of the computing hardware, converting the third data into fourth data of a fourth type, performing data calculation based on the fourth data, and obtaining the calculation result; The fourth type is a data type within the range of data calculation types supported by the operation module, and the precision of the fourth type is less than or equal to the precision of the third type and greater than the precision of the first type.

[0054] Here, the computing module refers to a module in computing hardware used for performing data operations, for example, it may be an EU (Execution Unit), or a TCORE (Tensor Core), or a VCORE (Vector Core).

[0055] Specifically, although the computing hardware can handle the storage and transmission of the third data, there may be a situation where the operation module itself cannot directly calculate the third type of data. That is, the third type is outside the data calculation type range supported by the operation module of the computing hardware. The data calculation type range here is a subset of the type range supported by the computing hardware, and only calculates whether the operation module of the computing hardware supports the data type; and the support range of the computing hardware includes the data types that the computing hardware supports reading, storage, calculation and other processing operations. In this case, data type conversion must be performed to ensure that the data can be processed by the operation module.

[0056] In this process, the third data is converted into fourth data of the fourth type supported by the operation module. The fourth type of data is adapted to the operation module of the hardware, and its accuracy is lower than or equal to that of the third type of data, but higher than that of the first type of data. This conversion ensures that when the compatibility of the operation module is limited, the data can still participate in the operation with higher accuracy, avoiding the loss of accuracy due to hardware limitations.

[0057] The selection of the fourth type must not only comply with the support range of the operation module, but also retain the accuracy information of the third data as much as possible. The process of converting the third data to the fourth data effectively avoids calculation errors or data processing problems caused by the operation module not supporting the third type. The fourth type of data is fully compatible with the operation module to ensure that the operation can be performed smoothly. Although the accuracy of the third type cannot be fully retained after conversion, the fourth type can still ensure the accuracy and reliability of the operation results. Through this conversion, the data can be smoothly calculated through the operation module. Even if the operation module cannot directly process the third type, the converted fourth data still provides a high enough accuracy to meet the calculation requirements.

[0058] The process of data type conversion may involve a variety of methods, which are not limited by the embodiments of the present invention. Common methods include truncation, which directly discards the low-bit precision part to meet the requirements of the fourth type, or uses rounding to reduce precision loss by rounding. In addition, scaling and normalization can also be used to adjust the data range to adapt it to the fourth type of representation, and then restore the ratio after calculation, which is particularly effective for a large range of values. If it is necessary to keep some high-precision information, the segmented representation method splits the data to adapt to the hardware operation limitations. For floating-point numbers, the commonly used sign and exponent splitting can retain the main numerical information and reduce the precision of the mantissa. At the same time, the conversion from double precision to single precision can convert 64-bit floating-point numbers into 32-bit floating-point numbers to adapt to the capabilities of the computing hardware operation module. Through these flexible conversion methods, data can be effectively processed under the limitations of computing hardware, ensuring the calculation accuracy while improving the calculation efficiency, and ultimately achieving reliable calculation results.

[0059] Based on the above embodiment, converting the third data into fourth data of the fourth type includes: In a case where the operation module is a tensor processor of the computing hardware, transmitting the third data from a computing unit of the computing hardware to the tensor processor; The tensor processor is controlled to convert the third data into fourth data of a fourth type.

[0060] A tensor processor is a hardware module specifically designed to process tensor operations and is widely used in fields such as artificial intelligence, deep learning, and high-performance computing. Tensors are an extension of multidimensional arrays and are suitable for processing complex data structures such as vectors, matrices, and high-dimensional arrays. The design of tensor processors focuses on large-scale parallel computing and can efficiently process complex operations such as matrix multiplication and vector addition and subtraction. Compared with traditional processors, tensor processors have higher efficiency and speed in processing multidimensional data operations, especially in neural network training and reasoning tasks in deep learning.

[0061] Specifically, when complex tensor calculation tasks need to be performed, data operations occur in the tensor processor. After receiving the third data, the tensor processor converts the data into the fourth type according to its internal calculation requirements. The fourth type is the calculation type supported by the tensor processor, which may be lower than the third type, but still higher than the first type. This conversion process ensures that the data can be directly processed by the tensor processor while maintaining relatively high accuracy. The converted fourth data meets the calculation requirements of the tensor processor, so it can smoothly participate in subsequent calculations.

[0062] During the data transmission and conversion process, the powerful parallel processing capabilities of the tensor processor can ensure that this process is completed efficiently and avoid performance bottlenecks. The tensor processor uses an internally optimized computing architecture to quickly convert the third data into the fourth type supported by the hardware, allowing the operation to proceed smoothly. Since the tensor processor is specially optimized for multi-dimensional data operations, it can not only handle these type conversions, but also ensure that subsequent operations remain efficient and accurate.

[0063] This conversion mechanism ensures that when the tensor processor of the computing hardware does not directly support the third type of data, the type conversion and subsequent calculations can still be performed with the help of the tensor processor, avoiding calculation errors or data processing problems caused by the tensor processor not supporting the third type. In this way, the tensor processor can convert the third data into the fourth type and perform calculations based on it. Even if the tensor processor does not support the third type of data, it can still provide sufficient precision after data conversion to ensure the accuracy and reliability of the calculation.

[0064] Based on the above embodiment, the data calculation method further includes: Converting the operation result into fifth data of the first type; The fifth data is written into the memory in the form of the second type.

[0065] After completing the high-precision data operation, the operation result is in the third or fourth type format with higher precision or larger storage space, so it needs to be converted back to the first type supported by the memory for data storage. It can be understood that the memory referred to here is the memory where the first data is stored in step 110. The purpose of this conversion step is to match the accuracy of the operation result with the original first data, ensuring that the data format and storage size of the first data can be used in subsequent operations. By converting the operation result to the first type, storage space can be effectively saved and the accuracy requirements of the computing task can be met.

[0066] Due to the hardware characteristics of the memory and the limited data types supported, the fifth data needs to be stored in the form of the second type. The second type is supported in the hardware, which can ensure the compatibility of the data when it is written to the memory while retaining the highest possible accuracy. In this way, the fifth data is stored in the form of the second type, which not only maintains the accessibility of the data, but also maintains a high accuracy within the range supported by the hardware.

[0067] Through this process, the calculation results are converted from high precision (third or fourth type) to lower precision (first type), and then stored in the second type supported by hardware, ensuring the accuracy of the data and the effectiveness of storage, facilitating the subsequent use of the data.

[0068] Based on the above embodiment, the computing hardware is an artificial intelligence chip.

[0069] Artificial intelligence chips are hardware specially designed to handle complex computing tasks, especially in the fields of deep learning, neural networks, computer vision, etc. Compared with traditional general-purpose processors, artificial intelligence chips are specially optimized for parallel computing and large-scale data processing, which can effectively accelerate tasks such as matrix operations and convolution operations, and significantly improve computing efficiency and performance.

[0070] Taking a specific AI application scenario as an example, suppose a deep learning model relies on the Pytorch framework to run, and the model needs to use the FP16 data type for calculation, that is, the FP16 data type is the first type. Then, as shown in the following table, some hardware architectures in the model do not support FP16 type data operations, but support other types of data:

[0071] The data types involved and the information of each data type under the IEEE754 standard description are shown in the following table:

[0072] Therefore, it is necessary to process FP16 data into data types supported by each hardware through simulation. First, the data in FP16 format is loaded into the video memory and read into the registers in the computing unit EU. FP16 format data can be written to the video memory in the form of pseudo code description: fp16_tensor = torch.tensor(shape, data_type=”fp16”) bf16_usharp = usharp(shape, data_type=”bf16”) bind(bf16_usharp, fp16_tensor) … TLR = load(bf16_usharp, coordinate) In this case, FP16 data is treated as BF16 type, that is, BF16 type is the second type, and the data is loaded from the memory to the EU register through hardware instructions. Next, the computing unit EU uses bit operations or precision extension instructions to expand the data from BF16 to a higher precision FP32 type, that is, FP32 is the third type. Since FP32 has higher precision, the accuracy of the data is improved, and no information loss occurs during the expansion process.

[0073] The computing unit EU is not only responsible for loading data from the memory, but also can perform some basic matrix calculations. In this case, after the data is expanded from BF16 to FP32, the EU can perform basic matrix operations such as matrix addition and small-scale matrix multiplication in FP32 format. The advantage of EU is that it can handle various types of operations, so when the matrix size is small, EU can perform these operations directly, making full use of its versatility and flexibility.

[0074] However, when it comes to large-scale or higher-parallel matrix calculations, the EU will transfer the expanded FP32 data to the tensor processor, which will handle more complex matrix operation tasks. Since the tensor processor does not directly support FP32 operations, the FP32 data will be truncated and converted to BF24 or TF32 precision supported by the hardware for processing, that is, BF24 and TF32 are the fourth type.

[0075] In the tensor processor, FP32 data is converted to BF24 or TF32 precision through truncation operations. This conversion method reduces the bit width of the mantissa by retaining the high bits of the data, thereby reducing the computational overhead. BF24 precision provides higher representation capabilities than FP16, while reducing computational complexity compared to FP32. As a floating-point format optimized for parallel computing, TF32 can retain precision close to FP32 in operations, but with greatly improved computational efficiency. After this truncation process, the tensor processor can achieve higher computing performance with lower hardware resource consumption when performing complex operations such as matrix multiplication.

[0076] When the tensor processor completes these matrix operations, the results are sent back to the compute unit EU, which may need further processing, such as converting the results from FP32, BF24, or TF32 precision back to FP16 and reading them into the video memory as BF16 data types to save storage space and keep them consistent with the original data format.

[0077] The data operation device provided by the present invention is described below. The data operation device described below and the data operation method described above can be referenced to each other.

[0078] Figure 2 Schematic diagram of the structure of the data operation device provided by the present invention. Figure 2 As shown, the device comprises: The data type conversion unit 210 is used to determine first data under a first type, and read the first data in the form of a second type to obtain second data, wherein the first type is outside the type range supported by the computing hardware, the second type is within the type range, and the first memory space occupied by the first data is less than or equal to the second memory space occupied by the second data.

[0079] The data expansion and operation unit 220 is used to expand the data in the second data that occupies the size of the first memory space into third data of a third type, and perform data operations based on the third data to obtain operation results. The third type is within the type range, and the precision of the third type is higher than the precision of the first type.

[0080] The data computing device provided by the present invention improves the compatibility of computing hardware and expands the application scenarios of computing hardware by using the second type supported by computing hardware to read the first data. In the computing stage, the second data has a higher precision after being expanded to the third type, and can be used for more complex computing, reducing computing errors and ensuring the accuracy of computing results.

[0081] Based on any of the above embodiments, the data type conversion unit is specifically used for: The second memory space is a positive integer multiple of the first memory space.

[0082] Based on any of the above embodiments, the data expansion and operation unit is specifically used for: When the positive integer is 1, the second data is expanded to third data of the third type; When the positive integer is greater than 1, the positive integer number of data in the second data occupying the first memory space are respectively expanded into third data of the third type.

[0083] In the case where the third type is outside the data calculation type range supported by the calculation module of the computing hardware, converting the third data into fourth data of a fourth type, performing data calculation based on the fourth data, and obtaining the calculation result; The fourth type is a data type within the range of data calculation types supported by the operation module, and the precision of the fourth type is less than or equal to the precision of the third type and greater than the precision of the first type.

[0084] In a case where the operation module is a tensor processor of the computing hardware, transmitting the third data from a computing unit of the computing hardware to the tensor processor; The tensor processor is controlled to convert the third data into fourth data of a fourth type.

[0085] Converting the operation result into fifth data of the first type; The fifth data is written into the memory in the form of the second type.

[0086] The computing hardware is an artificial intelligence chip.

[0087] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the data operation method, which includes: Determine first data of a first type, and read the first data in a second type to obtain second data, wherein the first type is outside a type range supported by computing hardware, the second type is within the type range, and a first memory space occupied by the first data is less than or equal to a second memory space occupied by the second data; The data in the second data that occupies the size of the first memory space is expanded into third data of a third type, and data calculation is performed based on the third data to obtain a calculation result, wherein the third type is within the type range and the precision of the third type is higher than the precision of the first type.

[0088] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0089] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the data operation method provided by the above methods, the method comprising: Determine first data of a first type, and read the first data in a second type to obtain second data, wherein the first type is outside a type range supported by computing hardware, the second type is within the type range, and a first memory space occupied by the first data is less than or equal to a second memory space occupied by the second data; The data in the second data that occupies the size of the first memory space is expanded into third data of a third type, and data calculation is performed based on the third data to obtain a calculation result, wherein the third type is within the type range and the precision of the third type is higher than the precision of the first type.

[0090] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the data operation method provided by the above methods is implemented, and the method includes: Determine first data of a first type, and read the first data in a second type to obtain second data, wherein the first type is outside a type range supported by computing hardware, the second type is within the type range, and a first memory space occupied by the first data is less than or equal to a second memory space occupied by the second data; The data in the second data that occupies the size of the first memory space is expanded into third data of a third type, and data calculation is performed based on the third data to obtain a calculation result, wherein the third type is within the type range and the precision of the third type is higher than the precision of the first type.

[0091] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0092] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data computing method, characterized in that: include: Determine first data of a first type, and read the first data in a second type to obtain second data, wherein the first type is outside a type range supported by computing hardware, the second type is within the type range, and a first memory space occupied by the first data is less than or equal to a second memory space occupied by the second data; The data in the second data that occupies the size of the first memory space is expanded into third data of a third type, and data calculation is performed based on the third data to obtain a calculation result, wherein the third type is within the type range and the precision of the third type is higher than the precision of the first type.

2. The data calculation method according to claim 1, characterized in that: The second memory space is a positive integer multiple of the first memory space.

3. The data calculation method according to claim 2, characterized in that: The step of expanding the data in the second data that occupies the size of the first memory space into third data of a third type includes: When the positive integer is 1, the second data is expanded to third data of the third type; When the positive integer is greater than 1, the positive integer number of data in the second data occupying the first memory space are respectively expanded into third data of the third type.

4. The data operation method according to claim 1, characterized in that: The performing data operation based on the third data to obtain the operation result includes: In the case where the third type is outside the data calculation type range supported by the calculation module of the computing hardware, converting the third data into fourth data of a fourth type, performing data calculation based on the fourth data, and obtaining the calculation result; The fourth type is a data type within the range of data calculation types supported by the operation module, and the precision of the fourth type is less than or equal to the precision of the third type and greater than the precision of the first type.

5. The data calculation method according to claim 4, characterized in that: The converting the third data into fourth data of a fourth type includes: In a case where the operation module is a tensor processor of the computing hardware, transmitting the third data from a computing unit of the computing hardware to the tensor processor; The tensor processor is controlled to convert the third data into fourth data of a fourth type.

6. The data calculation method according to claim 1, characterized in that: Also includes: Converting the operation result into fifth data of the first type; The fifth data is written into the memory in the form of the second type.

7. The data calculation method according to any one of claims 1 to 6, characterized in that: The computing hardware is an artificial intelligence chip.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data operation method according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data operation method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data operation method according to any one of claims 1 to 7 is implemented.

Images