Quick and accurate JPEG picture decoding method based on CPU

Abstract

The invention relates to the field of high-performance computing of deep learning computer vision, in particular to a quick and accurate JPEG (Joint Photographic Experts Group) picture decoding method based on a CPU (Central Processing Unit). The method comprises the following steps: S1, loading JPEG picture data by using an AVX2 instruction, and decoding the data, including inverse quantization and color space conversion; s2, performing parallel computing on inverse quantization and inverse Fourier transform by using an AVX2 instruction set; s3, converting the decoded YCbCr data into an RGB format, carrying out parallel calculation by using an AVX2 instruction set, and rapidly completing conversion of a color space through a conversion formula; and S4, outputting and storing the converted RGB data according to a CHW format for subsequent face recognition processing.

Description

Technical Field

[0001] The present invention relates to the field of high performance computing of deep learning computer vision, and specifically to a fast and accurate JPEG picture decoding method based on CPU. Background Art

[0002] Jpeg format images can be compressed through methods such as quantization and Huffman coding, which greatly reduces the amount of data, reduces the storage space occupied and the efficiency of data transmission. Therefore, Jpeg images are one of the most commonly used data formats in computer vision (compute vision, CV) processing. In the visual deep learning model training and model inference scenarios, the RGB image format needs to be directly read, so the Jpeg format image needs to be decoded into RGB data format. The algorithm decoding process is as follows, mainly including: Huffman decoding, inverse quantization, inverse discrete cosine transform and color space conversion process, among which inverse quantization, inverse discrete cosine transform (IDCT) and color space conversion can realize computational parallelism, so some algorithm modules can be accelerated using parallel instructions.

[0003] Existing image decoding has corresponding implementation methods in hardware devices such as CPU / GPU and FPGA. The existing JPEG image decoding methods of CPU mainly include libJpeg and turboJpeg. Among them, libjpeg is a standard decoding library of C++, which is widely called in CV processing libraries such as opencv library and pillow library. Turbojepg uses avx2 and sse instruction rewriting algorithm for parallel acceleration. However, TurboJpeg has disadvantages: (1) First, the inverse cosine discrete block of turbojpeg is fixed to 8x8, and it cannot adopt multiple block methods such as 8x8, 16x8, etc. The algorithm efficiency is low and it is not aligned with jpeg decoding; (2) Secondly, in the calculation of IDCT inverse quantization and color space conversion, the turbojpeg method converts the data into int16, and uses SSE and AVX2 to use _mm_madd_epi16 and _mm256_madd_epi16 instructions for calculation respectively. Theoretically, one avx2 instruction can support 16 int16 for one multiplication and addition calculation, that is, one avx2 instruction can complete 32 calculations and one SSE instruction can complete 16 calculations. However, the data in the jpeg decoding process is int32 calculation. The turboJpeg decoding method converts the data into int16 calculation, which has a loss of precision. Therefore, a fast and accurate JPEG image decoding method based on CPU is provided. Summary of the invention

[0004] The purpose of the present invention is to provide a fast and accurate JPEG picture decoding method based on CPU to solve the shortcomings of TurboJpeg mentioned in the above background technology: (1) First, the inverse cosine discrete block of turbojpeg is fixed to 8x8, and multiple block methods such as 8x8 and 16x8 cannot be used. The algorithm efficiency is low and is not aligned with jpeg decoding; (2) Secondly, in the calculation of IDCT inverse quantization calculation and color space conversion calculation, the turbojpeg method converts the data into int16, and uses SSE and AVX2 to use _mm_madd_epi16 and _mm256_madd_epi16 instructions for calculation respectively. In theory, one avx2 instruction can support 16 int16 for one multiplication and addition calculation, that is, one avx2 instruction can complete 32 calculations, and SSE instruction completes 16 calculations. However, the data in the jpeg decoding process is int32 calculation, and the turboJpeg decoding method converts the data into int16 calculation, resulting in the problem of precision loss.

[0005] To achieve the above object, the present invention aims to provide a fast and accurate JPEG picture decoding method based on CPU, comprising the following steps: S1. Use AVX2 instructions to load JPEG image data and decode the data, including dequantization and color space conversion; S2, use AVX2 instruction set to perform parallel calculation of inverse quantization and inverse Fourier transform (IDCT); S3, converting the decoded YCbCr data into RGB format, using the AVX2 instruction set for parallel computing, and quickly completing the color space conversion through the conversion formula; S4. Output and store the converted RGB data in CHW format for subsequent face recognition processing.

[0006] As a further improvement of the technical solution, in S1, the AVX2 instruction is used to load the JPEG picture data, and the specific steps of decoding the data are as follows: S2.1, use Intel-SIMD vector instruction avx2 to rewrite the upsampling of the image; S2.2. Data rearrangement and instruction data parallel acceleration for color space conversion and inverse Fourier transform are used to improve computational parallelism.

[0007] As a further improvement of the technical solution, in S2, the specific steps of performing parallel calculation of inverse quantization and inverse Fourier transform (IDCT) are: S3.1, use Loeffler's algorithm to calculate, and divide it into one-dimensional processing and two-dimensional processing; S3.2, one-dimensional processing use function; S3.3, two-dimensional processing uses two-dimensional discrete quantization; As a further improvement of the technical solution, in S3.3, the specific steps of two-dimensional discrete quantization are: S3.31, load the row data directly into the 256-bit register, and directly perform scale and idct_id parallel calculations; S3.32. Read each column from the Y matrix into a 256-bit register, perform point calculations between rows, then transpose, and finally store in a 256-bit register for AVX parallel calculations.

[0008] As a further improvement of the technical solution, in S3.31, the detailed steps are as follows: Read the 8x8 data block of the input IDCT matrix and multiply it with the parameters of the inverse quantization table (composed of cosine functions), and then perform scale transformation and data rearrangement; Load the input pixel and each row of the quantization table into a 256-bit register, process it using idct_1d, and store it in a buffer variable.

[0009] Specifically, read the 8x8 data block of the input IDCT matrix and multiply it with the parameters of the inverse quantization table (composed of cosine functions), then through scale transformation and data rearrangement, so that the input pixel and each row of the quantization table can be loaded into a 256-bit register, and then processed using idct_1d and stored in the buffer variable. Because idct one-dimensional is a point operation calculation, 1x8 data, that is, one row, can be stored in the register, then calculated in parallel, and finally stored in the buffer Y variable; As a further improvement of the technical solution, in S3.32, the transposition is specifically: The 8x8 matrix of int32 can be transposed through the __mm256_unpacklo_epi32 and __mm256_permute2x128_si256 instructions.

[0010] As a further improvement of the technical solution, in S3, the specific steps of conversion are: S4.1. Use libJpeg algorithm to load YCbCr from the input image. The entire pixel arrangement is as follows: |YYYY……|CbCbCb……|CrCrCr……|->|RRR……|GGG……|BBB……|; S4.2. Convert uint8 data to int32 and calculate RGB pixel points using the conversion formula.

[0011] As a further improvement of the technical solution, in S4.2, the conversion formula is specifically: ; ; ; Among them, R represents the red channel; G represents the green channel; B represents the blue channel; Y represents the brightness component; Cr represents the blue chrominance component; Cb represents the red chrominance component.

[0012] As a further improvement of the technical solution, in S4, the specific steps of outputting and storing in CHW format are: S5.1. Use AVX2 instruction parallel acceleration algorithm to divide HxW pixels into 1x32 rows of data; S5.2. Use the _mm256_loadu_si256 instruction to load the uint8 YCbCr data into a 256-bit register. S5.3. Use the _mm256_unpacklo_epi8 instruction to convert the 32xuint8 pixels into two groups of 16xint16 pixels, and then use the _mm256_unpacklo_epi16 instruction to convert them into four groups of 4xint32 formats; S5.4, use the conversion formula to calculate the YCbCr to RGB process through the __mm256_sub_epi32 and __mm256_add_epi32 point calculation instructions; S5.5. After YCbCr data conversion, the RGB data format is , that is, |RRRR RRRR|GGGG GGGG|BBBB BBBB|…, and then save it as |RRR…|GGG…|BBB…| format, that is, the final output format is CHW.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. In this CPU-based fast and accurate JPEG image decoding method, the AVX2 instruction set is used to parallelize the inverse quantization and inverse Fourier transform (IDCT) to parallelize data and instructions, thereby improving the parallelism of algorithm calculation; 2. In this CPU-based fast and accurate JPEG image decoding method, the AVX2 instruction set is used for parallel calculation and conversion formula to quickly complete the color space conversion, thereby improving the output efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of the overall method of the present invention; Figure 2 Quantize specific codes for two-dimensional discrete; Figure 3 Specific code for parallel calculation of scale and idct_id; Figure 4 For the specific transposition logic. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Example: See also Figure 1-4 As shown, this embodiment provides a fast and accurate JPEG picture decoding method based on CPU, comprising the following steps: S1. Use AVX2 instructions to load JPEG image data and decode the data, including dequantization and color space conversion; In this example, AVX2 instructions are used to load JPEG image data and decode the data. The specific steps are as follows: S2.1, use Intel-SIMD vector instruction avx2 to rewrite the upsampling of the image; S2.2. Data rearrangement and instruction data parallel acceleration for color space conversion and inverse Fourier transform are used to improve computational parallelism.

[0017] S2, use AVX2 instruction set to perform parallel calculation of inverse quantization and inverse Fourier transform (IDCT); In this example, the specific steps of parallel calculation of inverse quantization and inverse Fourier transform (IDCT) are: S3.1, use Loeffler's algorithm to calculate, and divide it into one-dimensional processing and two-dimensional processing; S3.2, one-dimensional processing use function; S3.3, two-dimensional processing uses two-dimensional discrete quantization; In this example, the specific steps of two-dimensional discrete quantization are: S3.31, load the row data directly into the 256-bit register, and directly perform scale and idct_id parallel calculations; Among them, the specific code of two-dimensional discrete quantization is as follows Figure 2 As shown in the figure, the formula can directly parallelize data and instructions. Because the columns are processed and the rows are continuous, the row data is directly loaded into the 256-bit register, and the scale and idct_id are directly calculated in parallel. The specific code is as follows: Figure 3As shown; In this example, in S3.31, the detailed steps are: Read the 8x8 data block of the input IDCT matrix and multiply it with the parameters of the inverse quantization table (composed of cosine functions), and then perform scale transformation and data rearrangement; Load the input pixel and each row of the quantization table into a 256-bit register, process it using idct_1d, and store it in a buffer variable.

[0018] Specifically, read the 8x8 data block of the input IDCT matrix and multiply it with the parameters of the inverse quantization table (composed of cosine functions), then through scale transformation and data rearrangement, so that the input pixel and each row of the quantization table can be loaded into a 256-bit register, and then processed using idct_1d and stored in the buffer variable. Because idct one-dimensional is a point operation calculation, 1x8 data, that is, one row, can be stored in the register, then calculated in parallel, and finally stored in the buffer Y variable; S3.32, read each column from the Y matrix into a 256-bit register, perform point calculations between rows, then transpose, and finally store in a 256-bit register for avx parallel calculations. The specific logic is as follows Figure 4 As shown; In this example, the transposition is: The 8x8 matrix of int32 can be transposed through the __mm256_unpacklo_epi32 and __mm256_permute2x128_si256 instructions.

[0019] S3, converting the decoded YCbCr data into RGB format, using the AVX2 instruction set for parallel computing, and quickly completing the color space conversion through the conversion formula; In this example, the specific steps of conversion are: S4.1. Use libJpeg algorithm to load YCbCr from the input image. The entire pixel arrangement is as follows: |YYYY……|CbCbCb……|CrCrCr……|->|RRR……|GGG……|BBB……|; S4.2, convert uint8 data into int32, and calculate RGB pixel points through the conversion formula; In this example, the conversion formula is: ; ; ; Among them, R represents the red channel; G represents the green channel; B represents the blue channel; Y represents the brightness component; Cr represents the blue chrominance component; Cb represents the red chrominance component.

[0020] S4. Output and store the converted RGB data in CHW format for subsequent face recognition processing.

[0021] In this example, the specific steps for outputting and storing in CHW format are as follows: S5.1. Use AVX2 instruction parallel acceleration algorithm to divide HxW pixels into 1x32 rows of data; S5.2. Use the _mm256_loadu_si256 instruction to load the uint8 YCbCr data into a 256-bit register. S5.3. Use the _mm256_unpacklo_epi8 instruction to convert the 32xuint8 pixels into two groups of 16xint16 pixels, and then use the _mm256_unpacklo_epi16 instruction to convert them into four groups of 4xint32 formats; S5.4, use the conversion formula to calculate the YCbCr to RGB process through the __mm256_sub_epi32 and __mm256_add_epi32 point calculation instructions; S5.5. After YCbCr data conversion, the RGB data format is , that is, |RRRR RRRR|GGGG GGGG|BBBB BBBB|…, and then save it as |RRR…|GGG…|BBB…| format, that is, the final output format is CHW.

[0022] By using Intel-SIMD vector instructions AVX2 to rewrite the upsampling of the image, color space conversion, and data rearrangement and instruction data parallel acceleration of the inverse Fourier transform, the computational parallelism can be improved. The patent of this invention solves the problem of time-consuming decoding in image preprocessing during the training and reasoning of visual algorithms, ensures the precise alignment of decoding accuracy and libjpeg methods, greatly improves computing efficiency, and reduces actual hardware costs.

[0023] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A fast and accurate JPEG picture decoding method based on CPU, characterized by: The following steps are involved: S1. Use AVX2 instructions to load JPEG image data and decode the data, including dequantization and color space conversion; S2, use AVX2 instruction set to perform parallel calculation of inverse quantization and inverse Fourier transform; S3, converting the decoded YCbCr data into RGB format, using the AVX2 instruction set for parallel computing, and quickly completing the color space conversion through the conversion formula; S4. Output and store the converted RGB data in CHW format for subsequent face recognition processing.

2. The CPU-based fast and accurate JPEG picture decoding method according to claim 1, characterized in that: In S1, AVX2 instructions are used to load JPEG image data, and the specific steps for decoding the data are as follows: S2.1, use Intel-SIMD vector instruction avx2 to rewrite the upsampling of the image; S2.

2. Data rearrangement and instruction data parallel acceleration for color space conversion and inverse Fourier transform are used to improve computational parallelism.

3. The CPU-based fast and accurate JPEG picture decoding method according to claim 1, characterized in that: In S2, the specific steps of performing parallel calculation of inverse quantization and inverse Fourier transform are: S3.1, use Loeffler's algorithm to calculate, and divide it into one-dimensional processing and two-dimensional processing; S3.2, one-dimensional processing use function; S3.

3. Two-dimensional processing uses two-dimensional discrete quantization.

4. The CPU-based fast and accurate JPEG picture decoding method according to claim 3, characterized in that: In S3.3, the specific steps of two-dimensional discrete quantization are: S3.31, load the row data directly into the 256-bit register, and directly perform scale and idct_id parallel calculations; S3.

32. Read each column from the Y matrix into a 256-bit register, perform point calculations between rows, then transpose, and finally store in a 256-bit register for AVX parallel calculations.

5. The CPU-based fast and accurate JPEG picture decoding method according to claim 4, characterized in that: In S3.31, the detailed steps are: Read the 8x8 data block of the input IDCT matrix and multiply it with the parameters of the inverse quantization table, and then perform scale transformation and data rearrangement; Load the input pixel and each row of the quantization table into a 256-bit register, process it using idct_1d, and store it in a buffer variable.

6. The CPU-based fast and accurate JPEG picture decoding method according to claim 4, characterized in that: In S3.32, transposition is specifically: The 8x8 matrix of int32 can be transposed through the __mm256_unpacklo_epi32 and __mm256_permute2x128_si256 instructions.

7. The CPU-based fast and accurate JPEG picture decoding method according to claim 1, characterized in that: In S3, the specific steps of conversion are: S4.

1. Use libJpeg algorithm to load YCbCr from the input image. The entire pixel arrangement is as follows: |YYYY……|CbCbCb……|CrCrCr……|->|RRR……|GGG……|BBB……|; S4.

2. Convert uint8 data to int32 and calculate RGB pixel points using the conversion formula.

8. The CPU-based fast and accurate JPEG picture decoding method according to claim 7, characterized in that: In S4.2, the conversion formula is specifically: ； ； ； Among them, R represents the red channel; G represents the green channel; B represents the blue channel; Y represents the brightness component; Cr represents the blue chrominance component; Cb represents the red chrominance component.

9. The CPU-based fast and accurate JPEG picture decoding method according to claim 1, characterized in that: In S4, the specific steps of outputting and storing in CHW format are as follows: S5.

1. Use AVX2 instruction parallel acceleration algorithm to divide HxW pixels into 1x32 rows of data; S5.

2. Use the _mm256_loadu_si256 instruction to load the uint8 YCbCr data into a 256-bit register. S5.

3. Use the _mm256_unpacklo_epi8 instruction to convert the 32xuint8 pixels into two groups of 16xint16 pixels, and then use the _mm256_unpacklo_epi16 instruction to convert them into four groups of 4xint32 formats; S5.4, use the conversion formula to calculate the YCbCr to RGB process through the __mm256_sub_epi32 and __mm256_add_epi32 point calculation instructions; S5.

5. After YCbCr data conversion, the RGB data format is , that is, |RRRR RRRR|GGGG GGGG|BBBBBBBB|…, and then save it as |RRR…|GGG…|BBB…| format, that is, the final output format is CHW.

Images