A spmv mixed precision optimization method decoupled from storage and computation format

Abstract

This invention relates to a SpMV mixed-precision optimization method that decouples storage and computation format, comprising: given a set of floating-point numbers, dividing the set into an exponent part, a sign bit, and a mantissa part, wherein all floating-point numbers in the set have the same exponent; storing the common exponent of the set of floating-point numbers, and continuously storing the sign bit and mantissa of all floating-point numbers in the set to meet different precision requirements of different algorithms; performing SpMV computation based on the partitioned and stored sparse matrix, reading the sparse matrix into memory, concatenating the exponent and mantissa parts to form a standard floating-point format, and finally performing computation according to the standard floating-point format. This invention proposes a new floating-point format for storing sparse matrices, decoupling storage and computation format, and solving the limitation problem of SpMV memory constraints.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a SpMV hybrid precision optimization method that decouples storage and computation formats. Background Technology

[0002] In digital processing, the cost of communication, memory access, and arithmetic operations is often closely related to the floating-point representation format used; that is, the more bits a floating-point representation format has, the higher the cost of reading and writing data to memory, communication between the processor, or performing operations. In numerical linear algebra, the accuracy of the result is usually related to the conditions of the problem and the precision format used to represent floating-point numbers. Using a "lower precision format" may lead to a decrease in the precision of the solution due to numerical effects (such as rounding errors). For many applications in scientific computing and data analysis, the IEEE 754 double-precision floating-point format (FP64) is used by default. Nevertheless, in order to reduce resource consumption while maintaining the accuracy of algorithm output, some research has combined double-precision and low-precision formats for mixed-precision computation.

[0003] Because the growth rate of hardware computing power far outpaces memory bandwidth, memory bandwidth constraints in many linear algebra-related programs have driven research into decoupling arithmetic formats from storage formats. If an algorithm can accept lower-precision read and write values, data can be accessed faster in a lower-precision format. As long as the algorithm remains within memory bandwidth constraints, arithmetic operations can still be performed with high precision without affecting performance. Meanwhile, it has been found that the conversion overhead of various floating-point formats in the IEEE 754 standard is significant in applications. Therefore, this invention proposes a new floating-point format for storing sparse matrices based on the numerical characteristics of non-zero elements in sparse matrices, and implements a SpMV kernel with a hybrid precision representation of the new floating-point format and double-precision floating-point format. Summary of the Invention

[0004] The purpose of this invention is to provide a SpMV hybrid precision optimization method that decouples storage and computation formats, in order to solve the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A SpMV hybrid precision optimization method with decoupled storage and computation formats includes:

[0007] Given a set of floating-point numbers, the set of floating-point numbers is divided into an exponent part, a sign part, and a mantissa part, wherein all floating-point numbers in the set of floating-point numbers have the same exponent;

[0008] The common exponent of the floating-point number set is stored, and the sign bit and mantissa of all floating-point numbers in the floating-point number set are stored consecutively to meet the different precision requirements of different algorithms.

[0009] Based on the partitioned and stored sparse matrix, SpMV calculation is performed. The sparse matrix is ​​read into memory, and the exponent and mantissa are concatenated to form a standard floating-point format. Finally, the calculation is performed according to the standard floating-point format.

[0010] Preferably, the exponent portion is obtained from the common exponent of all floating-point numbers in the floating-point number set, and the floating-point number portion is the sign bit and mantissa portion of each floating-point number.

[0011] Preferably, storing all floating-point numbers in the set of floating-point numbers consecutively includes:

[0012] The common exponent of the floating-point number set is stored in the first address, and then the sign bit and mantissa of the floating-point number are stored sequentially after the common exponent to improve the data layout and obtain floating-point numbers of arbitrary precision.

[0013] Preferably, obtaining the arbitrary precision floating-point number includes:

[0014] Determine the type of floating-point conversion. If it is a conversion from high-precision floating-point to low-precision floating-point, simply truncate the mantissa to the corresponding bit width requirement. If it is a conversion from low-precision floating-point to high-precision floating-point, then expand the mantissa.

[0015] Preferably, the concatenation to form the standard floating-point format includes: inputting the double-precision floating-point number to be converted, then converting the double-precision address to a new address through forced type conversion, performing bit operations on the data in the new address to extract the sign bit and mantissa, concatenating the data, and finally returning the data in compressed floating-point format; wherein the new address is an int32 type address.

[0016] Preferably, the forced type conversion includes: performing the conversion of all data types based on the GPU; first, inputting a compressed set of floating-point numbers and their corresponding common exponents; then concatenating the floating-point numbers using bit operations; and finally returning the concatenated double-precision floating-point numbers.

[0017] The beneficial effects of this invention are as follows:

[0018] This invention proposes a SpMV hybrid precision optimization method that decouples storage and computation formats. By storing sparse matrices in a novel floating-point format, the storage and computation formats are decoupled, separating data storage precision from computation precision. The floating-point representation format of this invention allows for efficient use of storage bits to represent data. When the storage bit width is fixed, it can store higher precision data than the IEEE 754 standard, improving the effective utilization of storage space and providing greater optimization space for low-precision computations. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the method of the present invention;

[0021] Figure 2 This is a schematic diagram of the new floating-point format according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the new floating-point format memory layout according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the new floating-point format mantissa partitioning and memory layout according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram illustrating the equivalent conversion between the IEEE 754 floating-point format and the new floating-point format according to an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram illustrating the format conversion in an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of sparse matrix partitioning based on non-zero element exponents in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] SpMV computation is a memory-bandwidth-constrained program, and the intuitive optimization strategy is to improve memory access and data transfer efficiency. In the IEEE 754 standard, the exponent widths for single-precision and double-precision are 8 and 11 bits respectively, accounting for 25% and 17% of the total floating-point format bit widths. This invention proposes a new floating-point format in which the exponent of the floating-point number is stored separately, and the extra exponent bits are used to store more mantissa. This allows for the representation of higher precision with a lower bit width, such as using 32 bits to represent double precision. However, this approach relies on the assumption that non-zero elements in a sparse matrix have the same exponent value; otherwise, storing the exponents of different non-zero elements in a large-scale sparse matrix would incur a very high storage overhead.

[0030] Based on the analysis of real matrices in the SuiteSparse matrix set, a new floating-point format can be designed by leveraging the fact that most non-zero elements in sparse matrices have the same exponent, thus compressing data storage. The floating-point format designed in this embodiment is suitable for applications where a large number of floating-point numbers have the same exponent; the exponent field is no longer retained in the floating-point format. Because the exponent values ​​are identical, only one copy of the exponent needs to be stored, avoiding redundant storage and increased memory overhead.

[0031] This invention proposes a SpMV hybrid precision optimization method that decouples storage and computation formats, such as... Figure 1 The binary floating-point number set proposed in this embodiment consists of two parts: exponent and sign value. Given a floating-point number set F, in which all floating-point numbers have the same exponent, the set is represented in the floating-point format proposed in the text as follows: Figure 2 As shown. Figure 2 The left side of the diagram represents the common exponent of all floating-point numbers in the floating-point set, while the right side represents the sign part of each floating-point number. In the sign part, 's' indicates the sign of the floating-point number, and 'm' is the mantissa.

[0032] The specific forms of the designed floating-point format are shown in Table 1, which provides the binary representations of 16-bit, 32-bit, and 64-bit floating-point formats. To facilitate address alignment in memory, for each set of floating-point numbers, the common exponent width is 16 when represented by a 16-bit floating-point number, 32 when represented by a 32-bit floating-point number, and 64 when represented by a 64-bit floating-point format. Each floating-point number in the set has a binary representation containing one sign bit and u-1 mantissa bits, where u is the floating-point format width. The 64-bit floating-point format proposed in this embodiment has higher representation precision than the IEEE 754 format's 64-bit double-precision floating-point format.

[0033] Table 1

[0034]

[0035] On a reconfigurable system like an FPGA, the bit width of the exponent and mantissa can be customized.

[0036] The memory layout of the floating-point format proposed in this invention is as follows: Figure 3 As shown. In Figure 3 The first address stores the common exponent of the floating-point set, followed by consecutive storage of all floating-point numbers in the set, i.e., f1, f2, f3 to f4 after the exponent e. n By improving the data layout, floating-point numbers of arbitrary precision can be obtained. Given the bit width of the exponent field, the precision of a floating-point number depends on the bit width of the mantissa; the more bits in the mantissa, the higher the precision. By segmenting the mantissa of a floating-point number, different precision requirements of various algorithms can be met. Figure 4 The floating-point number is divided into two parts – “head” and “tail”. When the application requires low precision, only the “head” needs to be read. When the application requires higher precision, the “head” and “tail” can be read and simply concatenated. Figure 4 This demonstrates dividing the mantissa of a floating-point number into two parts and the memory layout of the resulting data. Figure 4 The "head" and "tail" are stored separately, while the exponent and "head" are stored consecutively. In practical applications, it can be divided into multiple parts according to specific accuracy requirements.

[0037] The conversion between 16-bit, 32-bit, and 64-bit floating-point formats in the embodiments is very convenient. When converting from a high-precision floating-point number to a low-precision floating-point number, simply truncate the mantissa to the corresponding bit width requirement. For example, truncating the lower 32 bits of a 64-bit floating-point number converts it to a 32-bit floating-point number. IEEE 754, however, requires truncating the mantissa and recalculating the exponent. Conversion from low precision to high precision only requires expanding the mantissa.

[0038] Figure 5 This demonstrates the equivalent conversion between floating-point formats and IEEE 754 when a large number of floating-point numbers have the same exponent. Assume... Figure 5 The floating-point numbers on the left are represented in double-precision floating-point format (1 sign bit, 11 exponent bits, 52 mantissa bits). These floating-point numbers may have different signs and mantissas, but the exponents are the same. The common exponent of this set of floating-point numbers is stored separately, such as... Figure 5As shown in box e on the right, the sign and mantissa of each floating-point number (1 + 52 = 53 bits) are then stored consecutively, where the dashed box represents the free space after removing the exponent field. Assuming the set contains n floating-point numbers, representing them in double precision would require 64 × n bits of memory, while the floating-point format proposed in this invention only requires (53 × n + 11) bits. When n is very large, the new format will save approximately 11 × n bits of memory. Typically, for easier data access, byte alignment can be used to reduce or increase the mantissa bits. For example, the free space after removing the exponent field can be used for more mantissa representation, improving the precision of the floating-point representation.

[0039] Sparse matrix partitioning based on non-zero element exponents

[0040] Currently, common computing devices such as CPUs and GPUs support the IEEE 754 standard. For the floating-point format proposed in this invention, only its storage advantages can be utilized; computation still requires conversion to the IEEE 754 floating-point format. On CPUs and GPUs, the smallest unit of memory is a byte (8 bits). Furthermore, the starting address for computer memory access is typically a multiple of 4 or 8, i.e., memory address alignment. Figure 5 The example shown compresses a 64-bit floating-point number to 53 bits. Since 53 is not a multiple of 8, this results in wasted storage space and increased memory access overhead. To address these issues, the floating-point format designed in this invention is verified on a GPU. This invention uses an exponent-based floating-point format to compress double-precision floating-point numbers to 32 bits (1 sign bit + 31 mantissa bits). Directly truncating the first 31 bits of the 52-bit mantissa of a double-precision floating-point number inevitably leads to precision loss. Therefore, to compress double-precision floating-point numbers to 32 bits without loss, the lower 21 bits of the mantissa are checked. If they are equal to 0, they can be compressed into the new 32-bit floating-point format, such as... Figure 6 As shown.

[0041] Since a sparse matrix may have multiple different exponents and some non-zero elements cannot be losslessly partitioned into the new 32-bit format, this embodiment divides the sparse matrix into two sub-matrices: one part is represented in double precision; the other part is represented in 32-bit floating-point format. The computation still uses double precision. Performing SpMV computation in this non-zero element exponent-driven storage format and computational precision format separation mode ensures that the overall precision is consistent with the double-precision computation result and has additional performance advantages. One performance advantage is the overall reduction in data transfer, because the memory overhead of non-zero elements stored in 32-bit format is reduced by half after partitioning. Secondly, there are advantages in memory access, with improved memory access efficiency. Figure 7The partitioning of sparse matrices based on non-zero element indices is visualized. bp_600, rajat17, and para-4 are all matrices in the SuiteSparse matrix set. The proportions of non-zero elements in the 32-bit submatrices are 74.59%, 91.72%, and 93.75%, respectively.

[0042] A sparse matrix of size M×N has nnz non-zero elements. Table 2 compares the memory requirements of the matrix under double-precision and mixed-precision representations. It is clear from Table 2 that if a large proportion of non-zero elements can be represented using 32 bits, the amount of data transfer between the CPU and GPU will be reduced by 4nnx-4(M+1)-4nnz. 64 byte. Nnz 32 nnz represents the number of non-zero elements in the new floating-point format submatrix. 64 This represents the number of non-zero elements in a double-precision submatrix. As long as formula (1) is satisfied, the memory overhead of mixed-precision representation is smaller.

[0043] nnz 32 -M-1>0 (1)

[0044] Table 2

[0045]

[0046] Ideally, the number of non-zero elements under each index should be recorded, sorted according to the number of non-zero elements, and finally the index corresponding to the case with the largest number of non-zero elements should be selected as the basis for division.

[0047] SpMV Design and Implementation:

[0048] Since GPUs only support standard floating-point format, this invention reads the sparse matrix into memory in double precision and then converts it to compressed floating-point format. Functions modified by the `device` prefix are device functions called by GPU threads; these functions are embedded inline into the binary code. Corresponding to `global__`, this prefix modifies GPU kernel functions, which call multiple execution units for parallel computation on the GPU.

[0049] First, input the double-precision floating-point number val to be converted; then, convert the double-precision address to an int32t type address through type casting. In the array, halfval[0] corresponds to the lower 32 bits of val, and halfval[1] corresponds to the higher 32 bits of val; then perform bit operations on the data in the address - extract the sign bit, concatenate the 20-bit mantissa in halfval[1] and the 11-bit mantissa in halfval[0], and the data in the register ret is the floating-point representation proposed in this invention; finally, return the data ret in the new floating-point format.

[0050] Hardware devices typically only support IEEE 754 standard format calculations. To calculate the compressed sparse matrix on a GPU, non-zero elements need to be reorganized to obtain an arithmetic precision format, i.e., double precision. On a GPU, this implementation completes all conversions in registers, avoiding the use of low-performance storage layers such as shared memory and global memory. First, input the floating-point representation data val and its corresponding exponent proposed in this paper. Second, define an array halfval of length 2 to store the high 32 bits and low 32 bits of the double-precision floating-point number respectively. Then, the floating-point number is concatenated in bit operations: the third line takes out the first 11 bits from left to right of the low 32 bits of the double-precision floating-point format and fills them into halfval[0], and the bits after the 11th bit are all 0; the fourth line obtains the sign bit of the double-precision floating-point number through an AND operation; the fifth line adds the offset value to the exponent; the sixth line obtains the 20-bit mantissa of the high 32 bits of the double-precision floating-point number. Finally, the concatenated double-precision floating-point number is returned. There is also a type casting here—the int32t type address is forcibly converted to a double-precision address.

[0051] The SpMV kernel is implemented based on CSR-Coop, where a group of cooperating threads process the same row of a sparse matrix. The input is the number of rows M of the sparse matrix, the exponents corresponding to the non-zero elements in the compressed floating-point format submatrix, the double-precision vector xD, and the row offset arrays (ApS and ApD), list arrays (AjS and AjD), and value arrays (AxS and AxD) of the new floating-point format submatrix and the double-precision submatrix.

[0052] The thread with the new thread number 0 retrieves the number of non-zero elements in the row processed by its cooperating thread group and stores it in a register (ptrs). Synchronization is performed within a warp thread. The SpMV kernel contains two local SpMV computations, performing multiplication of the new floating-point format submatrices and double-precision submatrices with vectors, respectively. A for loop merges the computation results of the cooperating threads. The thread with threadLane=0 writes the computation results back to global memory.

[0053] This invention proposes a novel floating-point format for storing sparse matrices, decoupling storage and computation formats to separate data storage precision from computational precision. Using this invention's floating-point representation format allows for efficient use of storage bits to represent data. With a fixed storage width, it can store higher precision data than the IEEE 754 standard, improving the effective utilization of storage space and providing greater optimization space for low-precision computations.

[0054] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A SpMV hybrid precision optimization method with decoupling of storage and computation formats, characterized in that, include: Given a set of floating-point numbers, the set of floating-point numbers is divided into an exponent part, a sign part, and a mantissa part, wherein all floating-point numbers in the set of floating-point numbers have the same exponent; The common exponent of the floating-point number set is stored, and the sign bit and mantissa of all floating-point numbers in the floating-point number set are stored consecutively to meet the different precision requirements of different algorithms. Based on the partitioned storage of the sparse matrix, SpMV calculation is performed. The sparse matrix is ​​read into memory, and the exponent and mantissa are concatenated to form a standard floating-point format. Finally, the calculation is performed according to the standard floating-point format. Continuously store all floating-point numbers in the set of floating-point numbers, including: The common exponent of the floating-point number set is stored in the first address, and then the sign bit and mantissa of the floating-point number are stored sequentially after the common exponent to improve the data layout and obtain floating-point numbers of arbitrary precision. Obtaining the arbitrary precision floating-point number includes: Determine the type of floating-point conversion. If it is a conversion from high-precision floating-point to low-precision floating-point, simply truncate the mantissa to the corresponding bit width requirement. If it is a conversion from low-precision floating-point to high-precision floating-point, then expand the mantissa. The process of concatenating the data into the standard floating-point format includes: inputting the double-precision floating-point number to be converted, then converting the double-precision address to a new address through type casting, performing bit operations on the data in the new address to extract the sign bit and mantissa, concatenating the data, and finally returning the data in the compressed floating-point format; wherein the new address is an address of type int32. The type casting process includes: performing all data type conversions based on the GPU. First, input a compressed set of floating-point numbers and their corresponding common exponents; then, concatenate the floating-point numbers using bitwise operations; and finally, return the concatenated double-precision floating-point numbers.

2. The SpMV hybrid precision optimization method with decoupled storage and computation formats according to claim 1, characterized in that, The exponent portion is obtained from the common exponent of all floating-point numbers in the set of floating-point numbers, and the floating-point portion consists of the sign bit and mantissa of each floating-point number.

Images