A fast and accurate CPU-based JPEG picture decoding method

Abstract

This invention relates to the field of high-performance computing in deep learning computer vision, specifically to a CPU-based fast and accurate JPEG image decoding method. It includes the following steps: S1, loading JPEG image data using AVX2 instructions and decoding the data, including inverse quantization and color space conversion; S2, performing parallel computation on inverse quantization and inverse Fourier transform using the AVX2 instruction set; S3, converting the decoded YCbCr data to RGB format, performing parallel computation using the AVX2 instruction set, and quickly completing the color space conversion using a conversion formula; S4, outputting and storing the converted RGB data in CHW format for subsequent face recognition processing.

Description

Technical Field

[0001] This invention relates to the field of high-performance computing in deep learning computer vision, specifically to a CPU-based method for fast and accurate JPEG image decoding. Background Technology

[0002] JPEG images can be compressed using methods such as quantization and Huffman coding, significantly reducing data size, storage space, and data transmission efficiency. Therefore, JPEG images are one of the most commonly used data formats in computer vision (CV) processing. In visual deep learning model training and inference scenarios, RGB image format is required for direct reading, necessitating decoding JPEG images into RGB data format. The decoding algorithm mainly includes Huffman decoding, inverse quantization, inverse discrete cosine transform (IDCT), and color space conversion. Inverse quantization, ICT, and color space conversion can be parallelized, thus accelerating some algorithm modules using parallel instructions.

[0003] Existing image decoding methods are implemented in hardware devices such as CPUs, GPUs, and FPGAs. The main existing JPEG image decoding methods for CPUs include libjpeg and turbojpeg. libjpeg is a standard C++ decoding library widely used in CV processing libraries such as OpenCV and Pillow, while turbojpeg uses AVX2 and SSE instruction rewrite algorithms for parallel acceleration. However, TurboJpeg has the following drawbacks: (1) Firstly, the inverse cosine discrete block of turbojpeg is fixed at 8x8, and cannot use multiple block methods such as 8x8 and 16x8. The algorithm is inefficient and not aligned with JPEG decoding. (2) Secondly, in the calculation of IDCT inverse quantization and color space conversion, the turbojpeg method converts the data into int16 and uses SSE and AVX2 to calculate using _mm_madd_epi16 and _mm256_madd_epi16 instructions respectively. Theoretically, one AVX2 instruction can support 16 int16 to perform one multiplication and addition calculation, that is, one AVX2 instruction can complete 32 calculations, and the SSE instruction completes 16 calculations. However, the data in the JPEG decoding process is calculated using int32. The turboJpeg decoding method converts the data into int16 for calculation, which results in a loss of precision. Therefore, a fast and accurate JPEG image decoding method based on CPU is provided. Summary of the Invention

[0004] The purpose of this invention is to provide a fast and accurate JPEG image decoding method based on CPU, in order to solve the shortcomings of TurboJpeg mentioned in the background technology: (1) Firstly, the inverse cosine discrete block of turbojpeg is fixed at 8x8, and cannot use multiple block methods such as 8x8 and 16x8, resulting in low algorithm efficiency and misalignment with JPEG decoding; (2) Secondly, in the calculation of IDCT inverse quantization and color space conversion, the turbojpeg method converts the data into int16 and uses SSE and AVX2 to calculate using _mm_madd_epi16 and _mm256_madd_epi16 instructions respectively. Theoretically, one AVX2 instruction can support 16 int16 to perform one multiplication and addition calculation, that is, one AVX2 instruction can complete 32 calculations, and the SSE instruction completes 16 calculations. However, the data in the JPEG decoding process is calculated using int32, while the turboJpeg decoding method converts the data into int16 for calculation, which leads to the problem of precision loss.

[0005] To achieve the above objectives, the present invention aims to provide a CPU-based fast and accurate JPEG image decoding method, comprising the following steps:

[0006] S1. Load JPEG image data using AVX2 instructions and decode the data, including inverse quantization and color space conversion;

[0007] S2. Parallel computation of inverse quantization and inverse Fourier transform (IDCT) is performed using the AVX2 instruction set;

[0008] S3. Convert the decoded YCbCr data into RGB format, use the AVX2 instruction set for parallel computation, and quickly complete the color space conversion through the conversion formula;

[0009] S4. Output and store the converted RGB data in CHW format for subsequent face recognition processing.

[0010] As a further improvement to this technical solution, in step S1, the specific steps for loading JPEG image data using AVX2 instructions and decoding the data are as follows:

[0011] S2.1. Rewrite the image upsampling using the Intel-SIMD vector instruction avx2;

[0012] S2.2, color space conversion, and data rearrangement and instruction data parallel acceleration for inverse Fourier transform, are used to improve computational parallelism.

[0013] As a further improvement to this technical solution, the specific steps for parallel computation of inverse quantization and inverse Fourier transform (IDCT) in step S2 are as follows:

[0014] S3.1. Use Loeffler's algorithm to calculate, and divide the process into one-dimensional and two-dimensional processing;

[0015] S3.2, One-dimensional processing function;

[0016] S3.3 Two-dimensional processing uses two-dimensional discrete quantization;

[0017] As a further improvement to this technical solution, the specific steps of two-dimensional discrete quantization in S3.3 are as follows:

[0018] S3.31. Load the row data directly into the 256-bit register and perform parallel calculations of scale and idct_id.

[0019] S3.32. Read each column from the Y matrix into a 256-bit register, perform point calculations between rows, transpose the matrix, and finally store the data in the 256-bit register for parallel AVX calculation.

[0020] As a further improvement to this technical solution, the detailed steps in S3.31 are as follows:

[0021] The input IDCT matrix is ​​read as an 8x8 data block and multiplied with the parameters of the inverse quantization table (composed of cosine functions), and then scaled and rearranged.

[0022] The input pixels and each row of the quantization table are loaded into a 256-bit register, processed using idct_1d, and stored in a buffer variable.

[0023] Specifically, the 8x8 data block of the input IDCT matrix is ​​read and multiplied with the parameters of the inverse quantization table (composed of cosine functions). Then, through scale transformation and data rearrangement, the input pixels and each row of the quantization table can be loaded into a 256-bit register. Then, idct_1d is used for processing, and the result is stored in a buffer variable. Because idct is a one-dimensional point operation calculation, 1x8 data, i.e., one row, can be stored in a register, then processed in parallel, and finally stored in the buffer variable Y.

[0024] As a further improvement to this technical solution, in S3.32, the transpose is specifically as follows:

[0025] The __mm256_unpacklo_epi32 and __mm256_permute2x128_si256 instructions can be used to transpose an 8x8 int32 matrix.

[0026] As a further improvement to this technical solution, the specific conversion steps in S3 are as follows:

[0027] S4.1. Use the libJpeg algorithm to load YCbCr from the input image. The entire pixel arrangement is as follows:

[0028] |YYYY……|CbCbCb……|CrCrCr……|->|RRR……|GGG……|BBB……|;

[0029] S4.2 Convert uint8 data to int32, and calculate the RGB pixel points using the conversion formula.

[0030] As a further improvement to this technical solution, the conversion formula in S4.2 is specifically as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] Where R represents the red channel; G represents the green channel; B represents the blue channel; Y represents the luminance component; Cr represents the blue chrominance component; and Cb represents the red chrominance component.

[0035] As a further improvement to this technical solution, the specific steps for outputting and storing in CHW format in step S4 are as follows:

[0036] S5.1. Use the AVX2 instruction parallel acceleration algorithm to divide the HxW pixels into rows of 1x32 data.

[0037] S5.2. Use the _mm256_loadu_si256 instruction to load uint8 YCbCr data into a 256-bit register;

[0038] S5.3. Use the _mm256_unpacklo_epi8 command to first convert 32 xint8 pixels into two groups of 16 xint16 pixels, and then use the _mm256_unpacklo_epi16 command to convert it into a format of 4 groups of 4 xint32 pixels.

[0039] S5.4. Using the __mm256_sub_epi32 and __mm256_add_epi32 point calculation commands, the conversion formula is used to calculate the YCbCr to RGB conversion process;

[0040] S5.5 After conversion of YCbCr data, the RGB data format is as follows: That is, |RRRR RRRR|GGGG GGGG|BBBB BBBB|……, then save it as |RRR……|GGG……|BBB……|, that is, the final output format is CHW.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. In this CPU-based fast and accurate JPEG image decoding method, the parallel computation of inverse quantization and inverse Fourier transform (IDCT) is achieved through the AVX2 instruction set, thereby parallelizing data and instructions and improving the computational parallelism of the algorithm;

[0043] 2. In this CPU-based fast and accurate JPEG image decoding method, the color space conversion is completed quickly by using the AVX2 instruction set for parallel calculation and conversion formula, thereby improving output efficiency. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the overall method of the present invention; Figure 2 This is the specific code for two-dimensional discrete quantization; Figure 3 The specific code for parallel computation of scale and idct_id; Figure 4 This refers to the specific transpose logic. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example:

[0047] Please see Figure 1-4 As shown, this embodiment provides a fast and accurate JPEG image decoding method based on CPU, including the following steps:

[0048] S1. Load JPEG image data using AVX2 instructions and decode the data, including inverse quantization and color space conversion;

[0049] In this example, the specific steps for loading JPEG image data using AVX2 instructions and decoding the data are as follows:

[0050] S2.1. Rewrite the image upsampling using the Intel-SIMD vector instruction avx2;

[0051] S2.2, color space conversion, and data rearrangement and instruction data parallel acceleration for inverse Fourier transform, are used to improve computational parallelism.

[0052] S2. Parallel computation of inverse quantization and inverse Fourier transform (IDCT) is performed using the AVX2 instruction set;

[0053] In this example, the specific steps for parallel computation of inverse quantization and inverse Fourier transform (IDCT) are as follows:

[0054] S3.1. Use Loeffler's algorithm to calculate, and divide the process into one-dimensional and two-dimensional processing;

[0055] S3.2, One-dimensional processing function;

[0056] S3.3 Two-dimensional processing uses two-dimensional discrete quantization;

[0057] In this example, the specific steps of two-dimensional discrete quantization are as follows:

[0058] S3.31. Load the row data directly into the 256-bit register and perform parallel calculations of scale and idct_id.

[0059] The specific code for two-dimensional discrete quantization is as follows: Figure 2 As shown; the formula can be directly parallelized in terms of data and instructions. Because of the inter-column processing and the continuous rows, the row data is directly loaded into a 256-bit register, and the scale and idct_id are calculated in parallel. The specific code is as follows. Figure 3 As shown;

[0060] In this example, the detailed steps in S3.31 are as follows:

[0061] The input IDCT matrix is ​​read as an 8x8 data block and multiplied with the parameters of the inverse quantization table (composed of cosine functions), and then scaled and rearranged.

[0062] The input pixels and each row of the quantization table are loaded into a 256-bit register, processed using idct_1d, and stored in a buffer variable.

[0063] Specifically, the 8x8 data block of the input IDCT matrix is ​​read and multiplied with the parameters of the inverse quantization table (composed of cosine functions). Then, through scale transformation and data rearrangement, the input pixels and each row of the quantization table can be loaded into a 256-bit register. Then, idct_1d is used for processing, and the result is stored in a buffer variable. Because idct is a one-dimensional point operation calculation, 1x8 data, i.e., one row, can be stored in a register, then processed in parallel, and finally stored in the buffer variable Y.

[0064] S3.32. Read each column from the Y matrix into a 256-bit register, perform point calculations between rows, transpose the matrix, and finally store the result in a 256-bit register for parallel AVX calculation. The specific logic is as follows: Figure 4 As shown;

[0065] In this example, transpose is specifically:

[0066] The __mm256_unpacklo_epi32 and __mm256_permute2x128_si256 instructions can be used to transpose an 8x8 int32 matrix.

[0067] S3. Convert the decoded YCbCr data into RGB format, use the AVX2 instruction set for parallel computation, and quickly complete the color space conversion through the conversion formula;

[0068] In this example, the specific conversion steps are as follows:

[0069] S4.1. Use the libJpeg algorithm to load YCbCr from the input image. The entire pixel arrangement is as follows:

[0070] |YYYY……|CbCbCb……|CrCrCr……|->|RRR……|GGG……|BBB……|;

[0071] S4.2 Convert uint8 data to int32, and calculate the RGB pixel points using the conversion formula;

[0072] In this example, the conversion formula is as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] Where R represents the red channel; G represents the green channel; B represents the blue channel; Y represents the luminance component; Cr represents the blue chrominance component; and Cb represents the red chrominance component.

[0077] S4. Output and store the converted RGB data in CHW format for subsequent face recognition processing.

[0078] In this example, the specific steps for outputting and storing in CHW format are as follows:

[0079] S5.1. Use the AVX2 instruction parallel acceleration algorithm to divide the HxW pixels into rows of 1x32 data.

[0080] S5.2. Use the _mm256_loadu_si256 instruction to load uint8 YCbCr data into a 256-bit register;

[0081] S5.3. Use the _mm256_unpacklo_epi8 command to first convert 32 xint8 pixels into two groups of 16 xint16 pixels, and then use the _mm256_unpacklo_epi16 command to convert it into a format of 4 groups of 4 xint32 pixels.

[0082] S5.4. Using the __mm256_sub_epi32 and __mm256_add_epi32 point calculation commands, the conversion formula is used to calculate the YCbCr to RGB conversion process;

[0083] S5.5 After conversion of YCbCr data, the RGB data format is as follows: That is, |RRRR RRRR|GGGG GGGG|BBBB BBBB|……, then save it as |RRR……|GGG……|BBB……|, that is, the final output format is CHW.

[0084] By rewriting image upsampling, color space conversion, and inverse Fourier transform data rearrangement and instruction-data parallel acceleration using the Intel-SIMD vector instruction AVX2, computational parallelism can be improved. This invention patent solves the problem of time-consuming decoding in image preprocessing during the training and inference of visual algorithms, ensuring decoding accuracy and precise alignment with the libjpeg method, greatly improving computational efficiency and reducing actual hardware costs.

[0085] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A CPU-based method for fast and accurate JPEG image decoding, characterized in that: Includes the following steps: S1. Load JPEG image data using AVX2 instructions and decode the data, including inverse quantization and color space conversion; S2. Use the AVX2 instruction set to perform parallel computation of inverse quantization and inverse Fourier transform; S3. Convert the decoded YCbCr data into RGB format, use the AVX2 instruction set for parallel computation, and quickly complete the color space conversion through the conversion formula; S4. Output and store the converted RGB data in CHW format for subsequent face recognition processing; In step S4, the specific steps for outputting and storing data in CHW format are as follows: S5.

2. Use the _mm256_loadu_si256 instruction to load uint8 YCbCr data into a 256-bit register; S5.

3. Use the _mm256_unpacklo_epi 8 command to first convert 32 xut8 pixels into two groups of 16 xint16 pixels, and then use the _mm256_unpacklo_epi 16 command to convert it into a format of 4 groups of 4 xint32 pixels. S5.

4. The YCbCr to RGB conversion process is calculated using the conversion formula through the __mm256_sub_epi 32 and __mm256_add_epi 32 point calculation instructions.

2. The CPU-based fast and accurate JPEG image decoding method according to claim 1, characterized in that: In step S1, the specific steps for loading JPEG image data using AVX2 instructions and decoding the data are as follows: S2.

1. Re-write the image upsampling using the Intel-SIMD vector instruction avx2; S2.2, color space conversion, and data rearrangement and instruction data parallel acceleration for inverse Fourier transform, are used to improve computational parallelism.

3. The CPU-based fast and accurate JPEG image decoding method according to claim 1, characterized in that: In step S2, the specific steps for parallel computation of inverse quantization and inverse Fourier transform are as follows: S3.1 Calculate using Loeffler's algorithm, and divide the process into one-dimensional and two-dimensional processing; S3.2, One-dimensional processing uses the void idct_1d(int &Y[64], ints) function; S3.3 Two-dimensional processing uses two-dimensional discrete quantization.

4. The CPU-based fast and accurate JPEG image decoding method according to claim 3, characterized in that: In S3.3, the specific steps of two-dimensional discrete quantization are as follows: S3.

31. Load the row data directly into the 256-bit register and perform parallel calculations of scale and idct_id. S3.

32. Read each column from the Y matrix into a 256-bit register, perform point calculations between rows, transpose the matrix, and finally store the data in the 256-bit register for parallel AVX calculation.

5. The CPU-based fast and accurate JPEG image decoding method according to claim 4, characterized in that: The detailed steps in S3.31 are as follows: Read the 8x8 data block of the input IDCT matrix and multiply it with the parameters of the inverse quantization table, then perform scale transformation and data rearrangement; The input pixels and each row of the quantization table are loaded into a 256-bit register, processed using idct_1d, and stored in a buffer variable.

6. The CPU-based fast and accurate JPEG image decoding method according to claim 4, characterized in that: In S3.32, the transpose is specifically as follows: The __mm256_unpacklo_epi32 and __mm256_permute2x128_si256 instructions are used to transpose an 8x8 int32 matrix.

7. The CPU-based fast and accurate JPEG image decoding method according to claim 1, characterized in that: In step S3, the specific conversion steps are as follows: S4.

1. Use the libJpeg algorithm to load YCbCr from the input image. The entire pixel arrangement is as follows: |YYYY……|CbCbCb……|CrCrCr……|->|RRR……|GGG……|BBB……|; S4.2 Convert uint8 data to int32, and calculate the RGB pixel points using the conversion formula.

8. The CPU-based fast and accurate JPEG image decoding method according to claim 7, characterized in that: In S4.2, the conversion formula is as follows: R = Y + 1.402 * Cr; G=Y-0.344136286*Cb-0.714136286*Cr; B = Y + 1.772 * Cb; Where R represents the red channel; G represents the green channel; B represents the blue channel; Y represents the luminance component; Cr represents the blue chrominance component; and Cb represents the red chrominance component.

9. The CPU-based fast and accurate JPEG image decoding method according to claim 1, characterized in that: Prior to S5.2, it also included: S5.

1. Use the AVX2 instruction parallel acceleration algorithm to divide the HxW pixels into rows of 1x32 data. Following S5.4, it also includes: S5.5 After conversion using YCbCr data, the RGB data format is R8G8B8, i.e., |RRRR RRRR|GGGG GGGG|BBBBBBBB|……, which is then saved as |RRR……|GGG……|BBB……| format, i.e., the final output format is CHW.

Images