Discrete Fourier transform implementation method and device based on matrix operation unit

Abstract

The invention relates to the technical field of computing power optimization, in particular to a discrete Fourier transform implementation method and device based on a matrix operation unit, and the method comprises the steps: carrying out transformation arrangement on a DFT matrix to generate a transformed and arranged DFT matrix, and copying the transformed and arranged DFT matrix and an input data matrix to a global memory of the device at a host end; starting a computing kernel, reading an input data matrix and a DFT (Discrete Fourier Transform) matrix in transformation arrangement from a global memory to an on-chip cache, and calling a matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix to obtain an operation result of discrete Fourier transform; and the host end copies result data from the global memory of the equipment to a host memory, and releases computing resources. According to the method, direct matrix multiplication is adopted to calculate the DFT, additional complex number virtual-real separation and combination operation is not needed, the number of times of global memory access can be reduced, the execution efficiency of the whole complex number matrix multiplication is improved, and the DFT calculation efficiency of equipment is improved.

Description

Technical Field

[0001] The present invention relates to the fields of high-performance computing and computing power optimization, and in particular to a discrete Fourier transform implementation method and device based on a matrix operation unit. Background Art

[0002] Discrete Fourier Transform (DFT) is a method for converting signals from the time domain to the frequency domain. DFT has a wide range of applications in the field of scientific computing, including molecular dynamics calculations, electronic structure calculations, digital signal processing, etc. In deep learning, DFT also has important application value. Compared with traditional convolution algorithms, convolution algorithms based on DFT have better results in network training. Due to the wide range of application scenarios of DFT, parallel optimization of DFT algorithms has become a research hotspot in the field of high-performance computing.

[0003] GPU has become the mainstream computing hardware in the field of artificial intelligence and scientific computing due to its powerful parallel computing capabilities. In view of the characteristics of existing GPU hardware, it is of great significance to optimize and accelerate the DFT algorithm. In order to accelerate the commonly used matrix multiplication operations in artificial intelligence computing, NVIDIA designed the matrix operation core Tensor Core in its GPU to accelerate matrix calculations from the hardware level. AMD later designed the corresponding Matrix Core.

[0004] The computing power of the hardware needs to be called through software. If the software is not optimized and adapted properly, it will waste the computing power and computing resources of the hardware and increase the computing cost. In order to give full play to the computing power of the GPU and improve the computing efficiency of DFT on the GPU, it is necessary to optimize the design of the traditional DFT algorithm in combination with the characteristics of the new hardware.

[0005] There are many mature DFT computing libraries on GPUs, such as NVIDIA's cuFFT, AMD's rocFFT, and the cross-platform Vulkan FFT. However, these computing libraries do not fully utilize the computing power of modern GPUs, and there is still a lot of room for optimization of GPU-based DFT algorithms. If the matrix operation unit in modern GPUs is used to accelerate DFT calculations, the computing efficiency of DFT can be greatly improved, thus providing underlying computing power support for scientific computing and deep learning.

[0006] The complex matrix multiplication used in DFT can actually be implemented through four real matrix multiplications and two real matrix additions. However, considering that the matrix operation unit and other units on the device need to exchange data through the on-chip cache, if the complex matrix multiplication steps are mechanically decomposed into four independent matrix multiplication processes and two independent matrix addition processes, it will lead to a large number of additional operations, which will lead to performance degradation. This is because the matrix operation unit is responsible for matrix multiplication operations, while other units are responsible for element-by-element addition operations of the matrix. The data exchange between the two must be carried, which increases the burden of data transmission. Therefore, in view of the characteristics of the matrix operation unit being independent and having high computing power, the optimization strategy should avoid overly dispersed operation division of complex matrix multiplication, but seek a method that can effectively integrate real matrix multiplication and addition processes to minimize the number of global memory accesses, thereby improving the execution efficiency of the entire complex matrix multiplication operation. Summary of the invention

[0007] In order to solve the technical problems existing in the prior art, the present invention provides a discrete Fourier transform implementation method and device based on a matrix operation unit, performs a specific transformation arrangement on the DFT matrix, utilizes the high-performance matrix operation unit on the existing system equipment to accelerate the calculation, and improves the DFT calculation efficiency of the equipment. A method for effectively integrating the real matrix multiplication and addition process is used to minimize the number of global memory accesses, thereby improving the execution efficiency of the entire complex matrix multiplication operation.

[0008] The first object of the present invention is to provide a discrete Fourier transform implementation method based on a matrix operation unit.

[0009] A second object of the present invention is to provide a computer device.

[0010] The first object of the present invention can be achieved by adopting the following technical solutions:

[0011] A method for implementing discrete Fourier transform based on a matrix operation unit, characterized in that it includes the following steps

[0012] S1. Apply for computing resources, transform and arrange the DFT matrix to generate a transformed DFT matrix, and copy the transformed DFT matrix and the input data matrix to the device global memory on the host side;

[0013] S2, start the calculation kernel, read the input data matrix and the DFT matrix of the transformation arrangement from the global memory to the on-chip cache, call the matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix, and obtain the operation result of the discrete Fourier transform;

[0014] S3. The host will copy the result data from the device global memory to the host memory to release computing resources.

[0015] Specifically, the step S1 includes:

[0016] Transform and arrange the elements of the DFT matrix on the host side, transform each element of the DFT matrix from a complex number into a real number sub-matrix, and obtain a transformed and arranged DFT matrix;

[0017] Treat the batch input sequence vector as the input data matrix in the batch direction;

[0018] The transformed DFT matrix is ​​copied from the host memory to the device global memory, and the input data matrix is ​​copied to the device global memory.

[0019] Specifically, transforming each element of the DFT matrix from a complex number into a real number sub-matrix includes: transforming each element of the DFT matrix from a complex number into a 2*2 real number sub-matrix.

[0020] Specifically, the DFT matrix of the transformation arrangement is expressed as:

[0021]

[0022] Among them, F N is the DFT matrix of the transform arrangement, express The real part of express The imaginary part of .

[0023] Specifically, the step S2 includes:

[0024] S21, reading the input data matrix and the transformed DFT matrix from the global memory to the on-chip cache, and adopting a heuristic strategy to optimize the data reading amount;

[0025] S22, calling the matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix, and outputting the calculation result data of the discrete Fourier transform. The calculation result data can be expressed as:

[0026]

[0027] Among them, X is the vector of input sequence after Fourier transformation, N is the length of input sequence, and B is the sequence batch size.

[0028] Specifically, the input data matrix and the transformed DFT matrix are read from the global memory to the on-chip cache, and a heuristic strategy is adopted to optimize the data reading amount, including:

[0029] Read the input data matrix from the global memory to the on-chip cache. When the input sequence length N is less than or equal to the preset threshold, the data is read from the on-chip cache to the register in blocks according to the direction of increasing batch input sequence numbers, and is arranged in column direction first;

[0030] When the input sequence length N is greater than a preset threshold, the read from the on-chip cache to the register is performed in blocks according to the direction of a single input sequence.

[0031] The second object of the present invention can be achieved by adopting the following technical solutions:

[0032] A computer device comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned discrete Fourier transform implementation method based on a matrix operation unit is implemented.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] The present invention provides a method and device for realizing discrete Fourier transform based on matrix operation unit, by transforming and arranging the elements of DFT matrix on the host side, transforming each element of DFT matrix from complex number to a real number sub-matrix, obtaining the transformed and arranged DFT matrix, using direct matrix multiplication operation to calculate DFT, without the need for additional complex number virtual and real separation and combination operation, using high-performance matrix operation unit on existing system equipment to accelerate calculation, and improving the DFT calculation efficiency of the equipment. It takes advantage of the high computing power of matrix operation core, reduces the interaction between global memory and on-chip cache, has the characteristics of fast calculation speed, low calculation delay, easy batching, etc., and has good practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0036] Figure 1 is a flow chart of a method for implementing a discrete Fourier transform based on a matrix operation unit in an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of matrix multiplication operation in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is obvious that the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and the implementation of the present invention is not limited to this. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] Embodiment 1:

[0040] The present invention proposes a method for accelerating discrete Fourier transform (DFT) calculation using a high-performance matrix operation unit on an existing GPU, which takes advantage of the high computing power of the matrix operation core and reduces the interaction between the global memory and the on-chip cache. It has the characteristics of fast computing speed, low computing latency, and easy batching, and has good practical value. It can give full play to the computing power of the GPU, accelerate DFT calculations, and provide computing power support for fields such as deep learning and scientific computing.

[0041] The matrix operation of discrete Fourier transform (DFT) can generally be implemented by using a specific matrix multiplication. The matrix representation of discrete Fourier transform involves the multiplication of two matrices, that is, the vector of the input sequence is multiplied by a complex exponential function matrix called the DFT matrix. The result of this matrix multiplication operation is a complex spectrum, which contains the frequency component information of the input signal.

[0042] like Figure 1 As shown, the discrete Fourier transform implementation method based on the matrix operation unit of the present invention comprises the following steps:

[0043] S1. Apply for computing resources, transform and arrange the DFT matrix to generate a transformed DFT matrix, and copy the transformed DFT matrix and input data to the device global memory on the host side, and store them in the system persistent storage space.

[0044] S11. Transform and arrange the elements of the DFT matrix on the host side, transform each element of the DFT matrix from a complex number into a real number sub-matrix, and generate a transformed and arranged DFT matrix on the host memory.

[0045] Suppose the input sequence vector composed of the input sequence of length N is x, and the vector composed of the sequence after Fourier transform is X, then:

[0046]

[0047] k 0,1,…,n 1;

[0048] Where X[k] is the kth frequency component of the frequency domain signal, x[n] is the nth time point of the time domain signal, and i is an imaginary unit. is a complex exponential function, and the vector X is expressed in matrix form as follows:

[0049] X=F N x;

[0050] Among them, F N is the DFT matrix, which can be expressed as follows:

[0051]

[0052] in, The DFT matrix F N Each element of is a complex number, and each element is transformed into a 2*2 real number sub-matrix. The DFT matrix F of the transformed arrangement N It can be expressed as:

[0053]

[0054] in, express The real part of express The imaginary part of .

[0055] By transforming each element of the discrete Fourier transform matrix from a complex number into a 2*2 real number sub-matrix, the complex matrix multiplication used in the DFT process is directly converted into a real number matrix multiplication. This can avoid the redundant complex virtual and real separation and combination process (the data exchange between the two must pass through the global memory, which will increase the burden of data transmission) to minimize the number of global memory accesses, thereby accelerating the DFT calculation and improving the execution efficiency of the entire DFT process.

[0056] The DFT matrix is ​​transformed and arranged according to the length of the original input sequence N. Therefore, in the scenario of limited N selection, the transformed and arranged DFT matrix can be generated in advance, or even stored in the local persistent storage space, and can be read before calculation. It should be noted that the size of the DFT matrix is ​​N*N. In the case of a larger input size of N, it is more appropriate to use a matrix-based idea similar to FFT.

[0057] S12. Consider the batch input sequence vector as the input data matrix in the batch direction. The input data matrix x can be expressed as:

[0058]

[0059] Among them, x 0 [0 r represents the real part of the 0th element of the input sequence 0, x 0 [0 iIt represents the imaginary part of the 0th element of the input sequence 0, and so on. When the input sequence vector is transformed into the input data matrix, it is arranged in column priority. Each column is a sequence, and B is the sequence batch size.

[0060] S13, copying the transformed DFT matrix from the host memory to the device global memory, and copying the input data matrix to the device global memory.

[0061] Assuming the calculation batch is 256, the length of the DFT is 64, the calculated data is copied to the device global memory, and each block takes the corresponding data from it for calculation. For each DFT of length 64. Since the 64×64 DFT matrix is ​​the same for all sequences, it is generated in advance on the host and transformed and arranged, and then copied to the device global memory for direct reading by each core.

[0062] S2. Start the calculation kernel, read the input data matrix and the DFT matrix of the transformation arrangement from the global memory to the on-chip cache, call the matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix, and obtain the calculation result data of the discrete Fourier transform.

[0063] S21, read the input data matrix and the transformed DFT matrix from the global memory to the on-chip cache, and adopt a heuristic strategy to optimize the data reading amount.

[0064] The device code is called on the host side, the calculation kernel is started, and the kernel program starts running. Before performing matrix multiplication, the input data matrix and DFT matrix need to be loaded into the corresponding on-chip cache, and then the matrix operation units such as CubeUnit, Tensor Core or Matrix Core in the NPU / GPU are used to complete the heuristic strategy selection matrix multiplication operation of the input data matrix and the DFT matrix. Since the calculation data is already in the global memory through the previous step, the calculation data (input data matrix) and DFT matrix are first read into the device register or shared memory to speed up data reading efficiency and reduce memory access time. Since each core calculates a sub-result matrix of size 128*128, each core loads its own 128*256 DFT matrix elements and 256*128 input data into the on-chip cache, and the loading process is parallel loading.

[0065] Specifically, the heuristic strategy for optimizing the amount of data read includes: reading the input data matrix to read data from the global memory to the on-chip cache, when the input sequence length N is less than or equal to the preset threshold value Nh, when reading from the on-chip cache to the register, the batch input sequence number (ie, the sequence batch size B) is read in blocks in the direction of growth, and the columns are prioritized. When the input sequence length N is greater than the preset threshold value Nh, the block reading is performed in the direction of a single input sequence (ie, N) when reading from the on-chip cache to the register. In this embodiment, the DFT matrix multiplication is fixed to a matrix multiplication form of 2*64×2*64×2*128. Here, when the sequence length N is greater than the threshold value 45, the block reading is performed according to the arrangement of the input sequence when reading from the on-chip cache to the register.

[0066] The threshold Nh is determined by the size of the on-chip cache. Here, we take the base radix, the sequence length N, and the batch size batch as an example: the matrix multiplication (m, k, n) = (radix, radix, N*batch / radix), the matrix is ​​kerneled in the k and n directions, and the n direction needs to use twice the space for double buffering. When the amount of calculation is sufficient, the number of allocated cores is the same as the number of physical cores. Blocks of different colors are allocated to different cores. The kernels in the k direction need to be atomically added, and several cores need to be atomically added to the same position. Another matrix multiplication is used when radix is ​​small, such as setting it to radix <= 45. In this case, the zero-level cache can completely accommodate the DFT matrix. Load the DFT matrix into the zero-level cache before the loop. When the matrix multiplication (batch, M, K, N) DFT matrix is ​​the left matrix, then the left matrix is ​​fixed, and the right matrix is ​​the matrix multiplication of the input data. When N is large, read two matrix blocks for double buffering. When N is small, read two different batches for double buffering.

[0067] like Figure 2 As shown in the figure, the matrix multiplication operation diagram is completed: X = F N x, the mode of DFT matrix multiplication of the transformed arrangement is different from the mode of usual matrix multiplication, which is fixed to the matrix multiplication form of 2N*2N*2B, where N is the length of the original input sequence and B is the sequence batch size.

[0068] S22, calling the matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix, and outputting the calculation result data of the discrete Fourier transform. The calculation result data can be expressed as:

[0069]

[0070] Where X is the vector of the input sequence after Fourier transformation, N is the length of the input sequence, B is the batch size of the sequence, and X 0 [0r , X 0 [0 i They represent the real and imaginary parts of element 0 of sequence 0, and so on.

[0071] The matrix operation unit on the GPU has extremely high matrix calculation performance, and matrix operations account for a considerable part of the calculation ratio in DFT, so the use of the matrix operation unit can improve the overall calculation efficiency of DFT to a certain extent. The present invention analyzes the transformation arrangement of matrix multiplication in DFT, and designs the process accordingly for the transformation arrangement of matrix multiplication to reduce the amount of data read, further accelerating the DFT process.

[0072] S3. The host copies the calculation result data of the discrete Fourier transform from the device global memory to the host memory to release computing resources.

[0073] Finally, the DFT calculation results are stored in the device global memory. The host needs to copy the calculation results from the global memory to the host memory for output or other use. Because a lot of device global memory space and host memory space are opened before calculating DFT, they need to be released after the calculation is completed to prevent memory leaks.

[0074] In summary, the present invention first applies for computing resources, generates pre-data with transformation arrangement required for calculation, then starts the computing kernel, completes matrix element multiplication in parallel, calls a high-speed matrix operation unit to complete matrix multiplication, and then copies the result data to the host memory to complete the calculation and release computing resources. Since the present invention uses a high-speed matrix operation unit on an existing GPU to complete the DFT calculation, and uses a transformation arrangement, the complex matrix multiplication used in the DFT process can be directly converted into real matrix multiplication, avoiding redundant complex virtual and real separation and combination processes, greatly improving the calculation efficiency of DFT, and having obvious advantages over traditional DFT calculation methods.

[0075] Embodiment 2:

[0076] This embodiment provides a computer device, which may be a server, a computer, etc., and includes a processor, a memory, an input device, a display, and a network interface connected through a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor executes the computer program stored in the memory, a discrete Fourier transform implementation method based on a matrix operation unit in the above-mentioned embodiment 1 is implemented as follows:

[0077] S1. Apply for computing resources, transform and arrange the DFT matrix to generate a transformed DFT matrix, and copy the transformed DFT matrix and the input data matrix to the device global memory on the host side;

[0078] S2, start the calculation kernel, read the input data matrix and the DFT matrix of the transformation arrangement from the global memory to the on-chip cache, call the matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix, and obtain the operation result of the discrete Fourier transform;

[0079] S3. The host will copy the result data from the device global memory to the host memory to release computing resources.

[0080] Specifically, the step S1 includes:

[0081] Transform and arrange the elements of the DFT matrix on the host side, transform each element of the DFT matrix from a complex number into a real number sub-matrix, and obtain a transformed and arranged DFT matrix;

[0082] Transform the input sequence vector into an input data matrix;

[0083] The transformed DFT matrix is ​​copied from the host memory to the device global memory, and the input data matrix is ​​copied to the device global memory.

[0084] Specifically, transforming each element of the DFT matrix from a complex number into a real number sub-matrix includes: transforming each element of the DFT matrix from a complex number into a 2*2 real number sub-matrix.

[0085] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement methods and shall be included in the protection scope of the present invention.

Claims

1. A method for implementing discrete Fourier transform based on a matrix operation unit, characterized in that: The following steps are included S1. Apply for computing resources, transform and arrange the DFT matrix to generate a transformed DFT matrix, and copy the transformed DFT matrix and the input data matrix to the device global memory on the host side; S2, start the calculation kernel, read the input data matrix and the DFT matrix of the transformation arrangement from the global memory to the on-chip cache, call the matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix, and obtain the operation result of the discrete Fourier transform; S3. The host will copy the result data from the device global memory to the host memory to release computing resources.

2. A method for implementing discrete Fourier transform based on a matrix operation unit according to claim 1, characterized in that: The step S1 comprises: Transform and arrange the elements of the DFT matrix on the host side, transform each element of the DFT matrix from a complex number into a real number sub-matrix, and obtain a transformed and arranged DFT matrix; Treat the batch input sequence vector as the input data matrix in the batch direction; The transformed DFT matrix is ​​copied from the host memory to the device global memory, and the input data matrix is ​​copied to the device global memory.

3. A method for implementing discrete Fourier transform based on a matrix operation unit according to claim 2, characterized in that: The step of transforming each element of the DFT matrix from a complex number into a real number sub-matrix includes: transforming each element of the DFT matrix from a complex number into a 2*2 real number sub-matrix.

4. The method for implementing discrete Fourier transform based on a matrix operation unit according to claim 3, characterized in that: The DFT matrix of the transform arrangement is expressed as: Among them, F N is the DFT matrix of the transform arrangement, express The real part of express The imaginary part of .

5. A method for implementing discrete Fourier transform based on a matrix operation unit according to claim 4, characterized in that: The step S2 comprises: S21, reading the input data matrix and the transformed DFT matrix from the global memory to the on-chip cache, and adopting a heuristic strategy to optimize the data reading amount; S22, calling the matrix operation unit to perform matrix multiplication on the input data matrix and the DFT matrix, and outputting the calculation result data of the discrete Fourier transform. The calculation result data can be expressed as: Where X is the vector of the input sequence after Fourier transformation, N is the length of the input sequence, B is the sequence batch size, and X0[0] r 、X0[0] i Represent the real and imaginary parts of element 0 of sequence 0 respectively.

6. A method for implementing discrete Fourier transform based on a matrix operation unit according to claim 5, characterized in that: The input data matrix and the transformed DFT matrix are read from the global memory to the on-chip cache, and a heuristic strategy is adopted to optimize the data reading amount, including: Read the input data matrix from the global memory to the on-chip cache. When the input sequence length N is less than or equal to the preset threshold, the data is read from the on-chip cache to the register in blocks according to the direction of increasing batch input sequence numbers, and is arranged in column direction first; When the input sequence length N is greater than a preset threshold, the read from the on-chip cache to the register is performed in blocks according to the direction of a single input sequence.

7. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, it implements the discrete Fourier transform implementation method based on the matrix operation unit described in any one of claims 1 to 6.

Images