An Image Processing Method of Haar Wavelet Transform Using SIMT-Structured GPU

Abstract

The present invention relates to the field of image processing technologies, and particularly to an image processing method for Haar wavelet transform using a GPU with a SIMT architecture, which includes a GPU with a SIMT architecture; constructing a virtual data address using virtual addressing, changing the data synchronization method, partitioning the data, and adjusting the logical layout; corresponding the matrix data position to the data position during actual storage; and obtaining a data processing matrix according to the Haar wavelet transform formula and the obtained matrix mapping coordinates, and simultaneously obtaining data storage matrices in two directions of row and column. The present invention is based on the Haar wavelet to calculate the parallel formula for wavelet transform, and the parallel formula can be directly applied at the pseudocode level to complete the writing of pseudocode faster; and for the first time, it is proposed to partition the data, change the synchronization strategy for data parallel processing, and further improve the processing speed of wavelet transform for large-resolution images by changing the addressing method, synchronization method, and logical layout of the GPU.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an image processing method for Haar wavelet transform using a SIMT-structured GPU. Background Art

[0002] Wavelet transform can separate the basic information, variation information, and edge information of an image through low-pass and high-pass filters. And due to the property of wavelet transform having compact support, that is, being 0 outside a certain interval and being able to reflect the transform of local information, it is often used in the field of images.

[0003] Wavelet transform first performs a matrix transformation on the image matrix in the row direction, and then performs a matrix transformation in the column direction. After extracting the feature information to be extracted or adding some feature information, the image is reconstructed through the inverse transformation in the row and column directions. This process is rather cumbersome. Therefore, although the effect of feature extraction is relatively good, it takes a long time, especially for images with high resolution, the processing speed is significantly reduced. This makes some tasks that require real-time processing abandon the use of wavelet transform.

[0004] In the prior art, there are also adding a sliding window mechanism at the pseudo-code level to process large data streams and improve the speed of processing high-resolution images, and some improve the processing speed of SIMD-structured GPUs by rewriting the register file structure of the GPU.

[0005] However, these methods have obvious deficiencies: Existing SIMD-structured GPUs are based on wavelet transform, but different types of wavelet transforms use different wavelet bases. Summary of the Invention

[0006] In view of the deficiencies of the existing algorithms, the present invention is based on a SIMT-structured GPU, and calculates the parallel formula of wavelet transform based on the Haar wavelet. The parallel formula can be directly applied at the pseudo-code level to complete the writing of the pseudo-code faster; and for the first time, it is proposed to divide the data, change the synchronization strategy of data parallel processing, and further improve the processing speed of wavelet transform for high-resolution images by changing the addressing method, synchronization method, and logical layout of the GPU.

[0007] The technical solution adopted by the present invention is: An image processing method for Haar wavelet transform using a SIMT-structured GPU includes the following steps:

[0008] Step 1: Use a SIMT-structured GPU;

[0009] Put the data to be processed into different threads, the threads are combined into thread blocks, and the thread blocks are combined into grid blocks. Divide the high-resolution image into grid block dimensions and thread block dimensions, construct a basic logical layout, and the high resolution is more than 1 million pixels;

[0010] Step 2: Use virtual addressing to construct data virtual addresses, change the data synchronization method, divide the data, and adjust the logical layout;

[0011] Change the addressing method of the data. Adopt the unified addressing method of virtual memory to make the virtual addresses of the GPU virtual memory and the CPU virtual memory unified. Allocate virtual addresses to the GPU and the CPU, so that the GPU can directly read data from the physical memory of the CPU;

[0012] Furthermore, divide the data into n parts. After completing the parallel row calculation of the first part of the data, while performing the parallel column calculation on the first part of the data, start the parallel row calculation of the second part of the data; after the first part of the data completes the parallel column calculation, put the data into the virtual address; while the CPU takes out the first part of the data from the virtual address for image reconstruction operations, after the second part of the data completes the parallel row calculation, start the parallel column calculation, and repeat the execution to complete the calculation of n parts of the data;

[0013] In the prior art, the data is not divided, that is, there is only 1 part of the data. First, perform the processing row calculation on 1 part of the data, then the column calculation, and finally perform image reconstruction on the data.

[0014] Furthermore, according to the characteristics of the SIMT - structured GPU and the constructed logical layout, the divided data is a multiple of 2. For large - resolution images, it is divided into 8 parts.

[0015] Furthermore, the logical layout is thread block 64 * 1, grid block 16 * 1024.

[0016] Step 3: Put the divided image data matrix into the logical layout, correspond the matrix data position to the data position during actual storage, obtain the correspondence between the matrix data coordinates and the data coordinates in the logical layout, and obtain the matrix mapping coordinates;

[0017] Furthermore, the data is actually stored in threads. The threads are combined into a thread - block matrix, and the thread blocks are combined into a grid - block matrix; among them, the thread - block matrix is represented by U, and the element is u i,j , the grid - block matrix is represented by V, the size of matrix V is m * n, and the element is v i,j , the output image matrix is represented by A, and the matrix mapping coordinates a i,j ;

[0018] Among them, the matrix mapping coordinates a i,j are:

[0019]

[0020] Among them, a i is the matrix mapping row coordinate, a jLet \(a\) be the column coordinate of the matrix mapping, \(m\) be the length of the input image data matrix, and \(n\) be the width of the input image data matrix;

[0021] Step 4: According to the Haar wavelet transform formula and the obtained matrix mapping coordinates, obtain the parallel mode of wavelet transform, and obtain the data processing matrix. The data processing matrix is divided into data processing matrices in two directions, namely the row and column directions, and at the same time, data storage matrices in the row and column directions are obtained;

[0022] Furthermore, it specifically includes:

[0023] S41: Obtain the data processing matrix in the row direction. The relationship between the elements in the data processing matrix in the row direction and the matrix mapping coordinates is:

[0024]

[0025] where \(L\) i is the row coordinate of the data processing matrix in the row direction, \(L\) j is the column coordinate of the data processing matrix in the row direction, \(a\) i is the row coordinate of the matrix mapping, \(a\) j is the column coordinate of the matrix mapping, and \(m\) is the length of the image data matrix;

[0026] S42: Obtain the data processing matrix in the column direction. The relationship between the elements in the data processing matrix in the column direction and the matrix mapping coordinates is:

[0027]

[0028] where \(L'\) i is the row coordinate of the data processing matrix in the column direction, \(L'\) j is the column coordinate of the data processing matrix in the column direction, \(a\) i is the row coordinate of the matrix mapping, \(a\) j is the column coordinate of the matrix mapping, and \(m\) is the length of the image data matrix;

[0029] S43: Obtain the data storage matrix in the row direction. The relationship between the elements in the data storage matrix in the row direction and the matrix mapping coordinates is:

[0030]

[0031] where \(K\) i is the row coordinate of the data storage matrix in the row direction, \(K\) j is the column coordinate of the data storage matrix in the row direction, \(a\) i is the row coordinate of the matrix mapping, \(a\) j is the column coordinate of the matrix mapping, and \(m\) is the length of the image data matrix;

[0032] S44. Obtain the column-direction data storage matrix. The relationship between the elements in the column-direction data storage matrix and the matrix mapping coordinates is as follows:

[0033]

[0034] Among them, K' i is the row coordinate of the column-direction data storage matrix, K' j is the column coordinate of the column-direction data storage matrix, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix.

[0035] Advantages of the present invention:

[0036] 1. By deriving the parallel calculation formula of Haar wavelet transform on the SIMT architecture GPU, using the data virtual addressing method, splitting the data, changing the data synchronization method, and finding the most suitable logical layout, the operation efficiency is greatly improved; in the image processing scenarios where wavelet transform is required, the method of the present invention greatly improves the processing speed and achieves the purpose of real-time processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is the flowchart of the image processing method for Haar wavelet transform using the SIMT architecture GPU of the present invention;

[0038] Figure 2 is the comparative experimental graph of the number of divided parts of large-resolution image data of the present invention;

[0039] Figure 3 is the comparative experimental graph of the data processing time under different logical layouts of the present invention;

[0040] Figure 4 is the flowchart of data splitting, synchronization, and processing of the present invention;

[0041] Figure 5 is the schematic diagram of one of the 8 parts of data after row transformation of the present invention;

[0042] Figure 6 is the schematic diagram of one of the 8 parts of data after row transformation of the present invention;

[0043] Figure 7 is the schematic diagram of one of the 8 parts of data after row and column transformation of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The present invention will be further described below in conjunction with the drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, so it only shows the components related to the present invention.

[0045] The hardware of the present invention is based on a Dell server, and the main hardware configurations are: the CPU chip model is AMD Ryzen 7 5800H, the graphics card model is NVIDIA GeForce RTX 3060, and the memory is 16G; the software environment is based on Visual Studio.

[0046] As Figure 1 shown, an image processing method for Haar wavelet transform using a SIMT-structured GPU includes the following steps:

[0047] Step 1: Employ a GPU with a SIMT structure;

[0048] Clearly use a GPU with a SIMT structure as the basis for accelerated computing. For the SIMT-structured GPU, each piece of image data is placed into different threads, the threads are combined into thread blocks, and the thread blocks are combined into grid blocks to process data. According to common large-resolution images, divide appropriate grid block dimensions and thread block dimensions, and construct a basic logical layout;

[0049] As Figure 2 , Step 2: Use virtual addressing to construct a data virtual address, change the data synchronization method, divide the data, and adjust the logical layout, change the data synchronization method, divide the data, and adjust to the optimal logical layout to obtain the maximum operating efficiency;

[0050] Change the addressing method of the data. Adopt the unified addressing method of virtual memory, so that the virtual addresses of the GPU virtual memory and the CPU virtual memory are unified, allocate virtual addresses to the GPU and the CPU, and enable the GPU to directly read data from the physical memory of the CPU;

[0051] Furthermore, divide the data into 8 parts. After completing the parallel row calculation of the first part of the data, while performing the parallel column calculation on the first part of the data, start the parallel row calculation of the second part of the data; after the first part of the data completes the parallel column calculation, put the data into the virtual address; while the CPU takes out the first part of the data from the virtual address for image reconstruction operations, after the second part of the data completes the parallel row calculation, start the parallel column calculation, and repeat the execution to complete the calculation of 8 parts of the data;

[0052] As Figure 3 , from the comparative experiment, for large-resolution images, dividing into 8 parts is the optimal solution. In the figure, the data is divided into 1 - 10 parts, and the processing time consumption is the lowest when the data is in 8 parts; in addition, the slope of the processing time of data 2048x2048 is greater than that of 1024*1024, indicating that the larger the image resolution, the more obvious the segmentation processing effect.

[0053] As Figure 4As shown in the figure, the logical layout is adjusted to: the logical layout of thread block 64*1 and grid block 16*1024 has the highest efficiency;

[0054] Step 3: Place the divided image data matrix into the logical layout, correspond the matrix data position to the data position during actual storage, deduce the corresponding relationship between the matrix data coordinates and the data coordinates in the logical layout, and obtain the matrix mapping coordinates;

[0055] Furthermore, the data is actually stored in threads, the threads are combined into a thread block matrix, and the thread blocks are combined into a grid block matrix; among them, the thread block matrix is represented by U, and the element is u i,j , the grid block matrix is represented by V, the size of matrix V is m*n, and the element is v i,j , the output image matrix is represented by A, and the matrix mapping coordinate is a i,j ;

[0056] Among them, the matrix mapping coordinate a i,j is:

[0057]

[0058] Among them, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, m is the length of the input image data matrix, and n is the width of the input image data matrix;

[0059] Step 4: According to the Haar wavelet transform formula and the obtained matrix mapping coordinates, deduce the parallel mode of wavelet transform to obtain the data processing matrix. The data processing matrix is divided into data processing matrices in the row and column directions, and at the same time, data storage matrices in the row and column directions are obtained;

[0060] Furthermore, it specifically includes:

[0061] S41: Obtain the data processing matrix in the row direction. The relationship between the elements in the data processing matrix in the row direction and the matrix mapping coordinates is:

[0062]

[0063] Among them, L i is the row coordinate of the data processing matrix in the row direction, L j is the column coordinate of the data processing matrix in the row direction, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix;

[0064] S42: Obtain the data processing matrix in the column direction. The relationship between the elements in the data processing matrix in the column direction and the matrix mapping coordinates is:

[0065]

[0066] Among them, L' i is the row coordinate of the column-direction data processing matrix, and L' j is the column coordinate of the column-direction data processing matrix, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix;

[0067] S43. Obtain the row-direction data storage matrix. The relationship between the elements in the row-direction data storage matrix and the matrix mapping coordinates is as follows:

[0068]

[0069] Among them, K i is the row coordinate of the row-direction data storage matrix, K j is the column coordinate of the row-direction data storage matrix, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix;

[0070] Such as Figure 5 is the schematic diagram after the row transformation;

[0071] S44. Obtain the column-direction data storage matrix. The relationship between the elements in the data storage matrix (column direction) and the matrix mapping coordinates is as follows:

[0072]

[0073] Among them, K' i is the row coordinate of the column-direction data storage matrix, K' j is the column coordinate of the column-direction data storage matrix, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix;

[0074] Such as Figure 6 、 7 are the schematic diagrams after the column transformation and after the row and column transformations are completed;

[0075] The implementation pseudo-code is as follows:

[0076]

[0077]

[0078] The present invention uses a SIMT architecture GPU for data acceleration calculation; the parallel calculation formula of wavelet transform on a SIMT architecture GPU is derived, and by using data virtual addressing, changing the data synchronization method, and finding the optimal logical layout, the operation efficiency is greatly improved; in the image processing scenarios where wavelet transform is required, applying the method of the present invention greatly improves the processing speed.

[0079] Inspired by the ideal embodiments of the present invention described above, through the above description, relevant staff can make various changes and modifications completely within the scope without departing from the technical idea of the present invention. The technical scope of the present invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. An image processing method for Haar wavelet transform using a SIMT architecture GPU, characterized in that, It includes the following steps: Step 1: Adopt a GPU with SIMT architecture; Step 2: Use virtual addressing to construct data virtual addresses, change the data synchronization method, partition the data, and adjust the logical layout; Step 3: Put the partitioned data matrix into the logical layout, make the matrix data positions correspond to the data positions during actual storage, and use the correspondence between the matrix data coordinates and the data coordinates in the logical layout to obtain the matrix mapping coordinates; Step 3 includes: Data is stored in threads, and threads are combined into a thread block matrix, and thread blocks are combined into a grid block matrix; among them, the thread block matrix is represented by U, and the element is u i,j , the grid block matrix is represented by V, the size of matrix V is m*n, and the element is v i,j , the output data matrix is represented by A, and the matrix mapping coordinate is a i,j ; Among them, the matrix mapping coordinate a i,j is as follows: Among them, a i is the row coordinate of the matrix mapping, a j is the column coordinate of the matrix mapping, m is the length of the input data matrix, and n is the width of the input data matrix; Step 4: According to the Haar wavelet transform formula and the matrix mapping coordinates, obtain the parallel mode of wavelet transform, obtain the data processing matrix. The data processing matrix is divided into data processing matrices in the row and column directions, and at the same time, data storage matrices in the row and column directions are obtained; Step 4 includes: S41: Obtain the data processing matrix in the row direction. The relationship between the elements in the data processing matrix in the row direction and the matrix mapping coordinates is: Among them, L i is the row coordinate of the row-direction data processing matrix, L j is the column coordinate of the row-direction data processing matrix, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix; S42: Obtain the data processing matrix in the column direction. The relationship between the elements in the data processing matrix in the column direction and the matrix mapping coordinates is: Among them, L' i is the row coordinate of the column-direction data processing matrix, L' j is the column coordinate of the column-direction data processing matrix, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix; S43: Obtain the data storage matrix in the row direction. The relationship between the elements in the data storage matrix in the row direction and the matrix mapping coordinates is: Among them, K i is the row coordinate of the row-direction data storage matrix, K j is the column coordinate of the row-direction data storage matrix, a i is the matrix mapping row coordinate, a j is the matrix mapping column coordinate, and m is the length of the image data matrix; S44: Obtain the data storage matrix in the column direction. The relationship between the elements in the data storage matrix in the column direction and the matrix mapping coordinates is: Among them, K' i is the row coordinate of the column-direction data storage matrix, and K' j is the column coordinate of the column-direction data storage matrix. a i is the matrix mapping row coordinate, and a j is the matrix mapping column coordinate. m is the length of the image data matrix.

2. The image processing method of Haar wavelet transform using a SIMT architecture GPU according to claim 1, wherein, Partitioning the data includes: Partition the data into n parts. After completing the parallel row calculation of the first part of the data, perform the parallel row and column calculation on the first part of the data and at the same time perform the parallel row calculation of the second part of the data; after the first part of the data completes the parallel row and column calculation, put the first part of the data into the virtual address, and the CPU takes out the first part of the data from the virtual address for image reconstruction operations. At the same time, after the second part of the data completes the parallel row calculation, start the parallel row and column calculation, and repeat the execution to complete the calculation of n parts of the data.

3. The image processing method of Haar wavelet transform using a SIMT architecture GPU according to claim 2, characterized in that: The n parts of data are obtained by dividing an image with a resolution exceeding 1 million pixels into 8 parts.

4. The image processing method of Haar wavelet transform using a SIMT architecture GPU according to claim 1, characterized in that: The logical layout is thread block 64*1, grid block 16*1024.

Images