Image processing method, electronic device and readable storage medium

Abstract

The present application relates to the technical field of image processing, and particularly relates to an image processing method, an electronic device and a readable storage medium, the image processing method comprises the following steps: fusing and blocking a current frame and a reference frame to obtain a plurality of to-be-encoded spatial image blocks, and combining the to-be-encoded spatial image blocks into a plurality of macroblocks, and configuring a corresponding encoding quantization table and a decoding quantization table for each macroblock; performing discrete cosine transform by using a first integer DCT transform kernel; performing quantization by using the encoding quantization table; performing entropy encoding and storing into an off-chip memory; reading a compressed code stream from the off-chip memory, performing entropy decoding, and performing inverse quantization by using the decoding quantization table; performing inverse discrete cosine transform by using a second integer DCT transform kernel; performing block recombination to obtain a reconstructed reference frame, and taking the reconstructed reference frame as a new reference frame, taking a next frame input as a new current frame, and recycling. The present application can effectively solve the technical problem of distortion accumulation in the multi-round iteration encoding and decoding process based on JPEG.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, an electronic device, and a readable storage medium. Background Technology

[0002] In a typical video processing pipeline, the temporal enhancement module needs to rely on the previous frame as a reference frame to achieve multi-frame fusion motion compensation, noise reduction, sharpening, etc. In order to reduce the overhead of the chip's internal static random access memory (SRAM) or the bandwidth occupation of the external dynamic random access memory (DRAM), the industry generally adopts a compression-storage-decompression architecture: that is, the reference frame is compressed and written to the memory, and when the next frame arrives, it is read out and decompressed for weighted synthesis processing with the current frame.

[0003] In this architecture, the reconstructed quality sequence formed by the repeated compression and decompression of the reference frame constitutes a first-order Markov process—that is, the reconstruction quality of the current frame is independent of earlier history and only related to the reconstruction quality of the previous frame. Specifically, the reference frame at time t and the input frame at time t are weighted and synthesized, compressed and stored in Double Data Rate (DDR) memory, retrieved from DDR and decompressed at time t+1, and weighted and synthesized with the input frame at time t+1. The synthesized result is then used as the reference frame in the next cycle. This recursive structure makes the quality evolution of the reconstructed frame follow a first-order Markov process. Especially in static scenes, the same frame image may be repeatedly encoded and decoded multiple times, forming a deep iterative error feedback loop. If the distortion introduced by the compression algorithm cannot converge in this loop, an error accumulation effect may occur: the distortion may be enhanced frame by frame, leading to an avalanche-like decline in image quality, or the negative feedback mechanism may mistakenly treat the reconstruction distortion as object motion, noise, boundary texture, etc., affecting the weighted synthesis effect of the temporal fusion module and the final visual experience.

[0004] In existing technologies, hardware compression schemes for the Joint Photographic Experts Group (JEP) JPEG standard mostly focus on the coding efficiency of a single frame, hardware area optimization, or power consumption control. For example, the ZR36050 image compression processor implements complete JPEG baseline compression and decompression functions, supports programmable quantization tables and Huffman coding, but its design goal is to control the bitrate and accelerate image preview in a single compression, without considering the distortion propagation problem of the reference frame in multiple iterations. Similarly, although the Discrete Cosine Transform (DCT) and quantization techniques proposed by academia have significantly reduced the power consumption and area of ​​JPEG hardware (e.g., through approximate division by shift operators and cyclic punching), their optimization goal is the trade-off between energy consumption and peak signal-to-noise ratio (PSNR) in a single encoding, without involving error stability analysis under temporal recursive structures. Although the Nuvoton N9H26 series JPEG encoder supports dynamic adjustment of the quantization table to achieve bitrate control, its application scenarios are still limited to independent image encoding or thumbnail generation, and it cannot solve the distortion accumulation problem in closed-loop recursive compression.

[0005] In closed-loop video processing systems, existing JPEG compression schemes have the following common shortcomings: First, traditional quantization table design is based on the frequency sensitivity of the human visual system (HVS), aiming to obtain the subjectively optimal reconstruction quality after a single decoding. However, in multi-generation coding scenarios, repeated quantization loss of high-frequency coefficients may lead to the amplification of errors generation by generation. Second, the precision control of DCT transform is usually based on the hardware overhead of a single operation, lacking modeling and suppression of the error transfer function in the Markov process. Summary of the Invention

[0006] The purpose of this invention is to provide an image processing method, an electronic device, and a readable storage medium that can effectively solve the technical problem of distortion accumulation during multi-round iterative encoding and decoding based on JPEG.

[0007] To solve the above technical problems, the present invention provides an image processing method, comprising: step S100, fusing the current frame and a reference frame to obtain an enhanced image; step S200, dividing the enhanced image into blocks to obtain multiple spatial image blocks to be encoded, and combining the multiple spatial image blocks to be encoded into multiple macroblocks, and configuring a corresponding encoding quantization table and decoding quantization table for each macroblock, wherein the quantization step size of the corresponding high-frequency coefficients in the decoding quantization table of a macroblock with a scaling factor less than a preset scaling factor threshold is less than the corresponding quantization step size in the encoding quantization table; step S300, performing a discrete cosine positive transform on each spatial image block to be encoded using a first integer DCT transform kernel to obtain the corresponding original frequency domain coefficient matrix; step S400, for each spatial image block to be encoded, quantizing the original frequency domain coefficient matrix corresponding to the spatial image block to be encoded using the encoding quantization table of the corresponding macroblock to obtain the corresponding quantization index sequence; step S500, targeting... For each spatial image block to be encoded, entropy encoding is performed on the quantization index sequence corresponding to the spatial image block to generate a corresponding compressed bitstream, and the compressed bitstream is stored in off-chip memory; Step S600: Each compressed bitstream is read sequentially from the off-chip memory, and for each compressed bitstream, entropy decoding is performed on the compressed bitstream to obtain a corresponding decoded quantization index sequence, and the decoded quantization index sequence corresponding to the compressed bitstream is dequantized using the decoded quantization table of the corresponding macroblock to obtain a corresponding reconstructed frequency domain coefficient matrix; Step S700: The inverse discrete cosine transform is performed on the reconstructed frequency domain coefficient matrix corresponding to each compressed bitstream using a second integer DCT transform kernel to obtain a corresponding reconstructed spatial image block; Step S800: All the reconstructed spatial image blocks are reassembled to obtain a corresponding reconstructed reference frame, and the reconstructed reference frame is used as a new reference frame, the next input frame is used as the new current frame, and the process returns to step S100.

[0008] Optionally, the product of the coefficient matrix of the first integer DCT transform kernel and the coefficient matrix of the second integer DCT transform kernel is an integer multiple of the identity matrix to reduce fixed-point rounding error.

[0009] Optionally, the coefficient matrices of both the first integer DCT transform kernel and the second integer DCT transform kernel are:

[0010] K={a,a,a,a,a,a,a}, \

[0011] {d,e,f,g,-g,-f,-e,-d}, \

[0012] {b,c,-c,-b,-b,-c,c,b}, \

[0013] {e,-g,-d,-f,f,d,g,-e}, \

[0014] {a,-a,-a,a,a,-a,-a,a}, \

[0015] {f,-d,g,e,-e,-g,d,-f}, \

[0016] {c,-b,b,-c,-c,b,-b,c}, \

[0017] {g,-f,e,-d,d,-e,f,-g} \

[0018] Where K is the coefficient matrix, a=181, b=236, c=99, d=251, e=213, f=142, g=50.

[0019] Optionally, the step of dividing the enhanced image into blocks includes: dividing the Y component, U component, and V component of the enhanced image into blocks respectively to obtain multiple Y component spatial image blocks to be encoded, multiple U component spatial image blocks to be encoded, and multiple V component spatial image blocks to be encoded; the step of combining the multiple spatial image blocks to be encoded into multiple macroblocks includes: combining the multiple Y component spatial image blocks to be encoded, the multiple U component spatial image blocks to be encoded, and the multiple V component spatial image blocks to be encoded into multiple macroblocks according to corresponding combination rules based on the YUV image format of the enhanced image.

[0020] Optionally, the quantization table includes a luminance quantization table and a chrominance quantization table. For each macroblock, the luminance quantization table corresponding to that macroblock satisfies the following relationship with the standard luminance quantization table:

[0021] ,

[0022] in, For the first The first macroblock in the luminance coding quantization table is the... Quantization step size The first in the standard luminance quantization table Quantization step size For the first The scaling factor corresponding to each macroblock , ∈[0,1,2,……255].

[0023] Optionally, the decoding quantization table includes a luminance decoding quantization table and a chrominance decoding quantization table. For each macroblock, the luminance decoding quantization table corresponding to the macroblock satisfies the following relationship with the standard luminance quantization table:

[0024]

[0025] in, For the first The first macroblock in the luminance decoding quantization table is the... Quantization step size To preset the scaling factor threshold, The preset standard brightness quantization step size threshold.

[0026] Optionally, for each macroblock, the chroma-coded quantization table corresponding to that macroblock and the standard chroma-coded quantization table satisfy the following relationship:

[0027] ,

[0028] in, For the first The first macroblock in the chroma coding quantization table is the... Quantization step size For the first in the standard colorimetric table Each quantization step size.

[0029] Optionally, the decoding quantization table includes a luminance decoding quantization table and a chrominance decoding quantization table. For each macroblock, the chrominance decoding quantization table corresponding to that macroblock satisfies the following relationship with the standard chrominance quantization table:

[0030] in, For the first The first macroblock in the chroma decoding quantization table is the... Quantization step size To preset the scaling factor threshold, The preset standard colorimetric step size threshold is set.

[0031] To address the aforementioned technical problems, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the image processing method described above.

[0032] To address the aforementioned technical problems, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the image processing method described above.

[0033] Compared with the prior art, the image processing method, electronic device, and readable storage medium provided by the present invention have the following beneficial effects:

[0034] The image processing method provided by this invention, during decoding, for macroblocks (MacroCodingUnits, MCUs) with scaling factors less than a preset scaling factor threshold, adopts a decoding quantization table with a quantization step size smaller than the corresponding quantization step size in the encoding quantization table. This ensures that the absolute value of the high-frequency coefficients after decoding is less than the original high-frequency coefficients before encoding quantization, thereby causing the high-frequency components to gradually attenuate and preventing them from accumulating and becoming larger, resulting in a smoother image effect and reducing the high-frequency noise introduced during multi-round iterative compression.

[0035] Furthermore, the image processing method provided by this invention, by employing a newly designed first integer DCT transform kernel and a second integer DCT transform kernel, can avoid the problem of multi-round iterative encoding and decoding error divergence caused by the inability of integer DCT matrices to satisfy orthogonality.

[0036] Since the electronic device and readable storage medium provided by this invention belong to the same inventive concept as the image processing method provided by this invention, the electronic device and readable storage medium provided by this invention have at least all the beneficial effects of the image processing method provided by this invention. For details, please refer to the relevant description above. Therefore, the beneficial effects of the electronic device and readable storage medium provided by this invention will not be elaborated here. Attached Figure Description

[0037] Figure 1 A flowchart of an image processing method provided according to an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram illustrating the working principle of an image processing method provided in one embodiment of the present invention.

[0039] Figure 3 This is a diagram showing the cumulative effect of multi-round iterative encoding and decoding errors.

[0040] Figure 4 The image shows the result of multi-round iterative encoding and decoding using the image processing method provided by this invention.

[0041] Figure 5 This is a schematic diagram illustrating the appearance of a black border around a long white LED strip when using existing technology for multi-round iterative encoding and decoding.

[0042] Figures 6a to 6c To use existing technology for 8 8. Simulation results of black borders generated by multiple iterations under microquantization.

[0043] Figures 7a to 7c To use existing technology for 8 8. Simulation results of black borders generated by multiple iterations under unquantized conditions.

[0044] Figures 8a to 8c To verify 8 using existing DCT transform 8. Simulation results after multiple rounds of iteration.

[0045] Figures 9a to 9c To verify 8 using the DCT transform provided by this invention 8. Simulation results after multiple rounds of iteration.

[0046] Figure 10 This is a block diagram illustrating the principle of an image processing method according to an embodiment of the present invention. Detailed Implementation

[0047] The image processing method, electronic device, and readable storage medium proposed in this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of this invention will become clearer from the following description. Please refer to the accompanying drawings for the objectives, features, and advantages of this invention to make them more apparent and understandable.

[0048] The core idea of ​​this invention is to provide an image processing method, electronic device, and readable storage medium that can effectively solve the technical problem of distortion accumulation during multi-round iterative encoding and decoding based on JPEG.

[0049] To achieve the above-mentioned goals, this invention provides an image processing method, please refer to... Figure 1 ,like Figure 1As shown, the image processing method provided by the present invention includes: step S100, fusing the current frame and the reference frame to obtain an enhanced image; step S200, dividing the enhanced image into blocks to obtain multiple spatial image blocks to be encoded, and combining the multiple spatial image blocks to be encoded into multiple macroblocks, and configuring a corresponding encoding quantization table and decoding quantization table for each macroblock, wherein the quantization step size of the corresponding high-frequency coefficients in the decoding quantization table of the macroblock with a scaling factor less than a preset scaling factor threshold is less than the corresponding quantization step size in the encoding quantization table; step S300, performing a discrete cosine positive transform on each spatial image block to be encoded using a first integer DCT transform kernel to obtain the corresponding original frequency domain coefficient matrix; step S400, for each spatial image block to be encoded, using the encoding quantization table of the corresponding macroblock to quantize the original frequency domain coefficient matrix corresponding to the spatial image block to be encoded to obtain the corresponding quantization index sequence; step S500, for each spatial image block to be encoded, quantizing the original frequency domain coefficient matrix corresponding to the spatial image block to be encoded using the encoding quantization table of the corresponding macroblock to obtain the corresponding quantization index sequence; Encode a spatial image block, perform entropy encoding on the quantization index sequence corresponding to the spatial image block to be encoded to generate a corresponding compressed bitstream, and store the compressed bitstream in off-chip memory; Step S600: Read each compressed bitstream sequentially from the off-chip memory, and for each compressed bitstream, perform entropy decoding on the compressed bitstream to obtain a corresponding decoded quantization index sequence, and use the decoded quantization table of the corresponding macroblock to dequantize the decoded quantization index sequence corresponding to the compressed bitstream to obtain a corresponding reconstructed frequency domain coefficient matrix; Step S700: Use a second integer DCT transform kernel to perform inverse discrete cosine transform on the reconstructed frequency domain coefficient matrix corresponding to each compressed bitstream to obtain a corresponding reconstructed spatial image block; Step S800: Reassemble all the reconstructed spatial image blocks to obtain a corresponding reconstructed reference frame, use the reconstructed reference frame as a new reference frame, use the input next frame as the new current frame, and return to step S100.

[0050] Therefore, the image processing method provided by the present invention, during decoding, adopts a decoding quantization table with a quantization step size smaller than the corresponding quantization step size in the encoding quantization table for macroblocks (MacroCodingUnit, MCU) whose scaling factor is less than a preset scaling factor threshold. This ensures that the absolute value of the high-frequency coefficients after decoding is less than the original high-frequency coefficients before encoding quantization, thereby causing the high-frequency components to gradually attenuate and avoiding the high-frequency components from accumulating and becoming larger, resulting in a smoother image effect and reducing the high-frequency noise introduced during multi-round iterative compression.

[0051] Furthermore, the image processing method provided by this invention, by employing a newly designed first integer DCT transform kernel and a second integer DCT transform kernel, can avoid the problem of multi-round iterative encoding and decoding error divergence caused by the inability of integer DCT matrices to satisfy orthogonality.

[0052] For further details, please refer to... Figure 2 ,like Figure 2 As shown, the current frame and reference frame with H*W resolution are enhanced by multi-frame fusion image enhancement, and then successively processed by block segmentation, Discrete Cosine Transform (DCT), quantization and entropy encoding before being stored in the chip memory DDR. When the next frame arrives, it is read from the chip memory DDR and processed by entropy decoding, inverse quantization and inverse Discrete Cosine Transform (IDCT) to obtain the reconstructed reference frame, which is then fused and enhanced with the next frame image.

[0053] It should be noted that multi-frame fusion image enhancement includes aligning the inter-frame pixel positions of the current frame and the reference frame through motion estimation and compensation, and then weighting and synthesizing the aligned current frame and the reference frame based on a temporal filtering algorithm to achieve multi-frame noise reduction, motion compensation and temporal sharpening to obtain an enhanced image, which is then subjected to subsequent block operations.

[0054] Furthermore, in the JPEG chip implementation process, each rounding step—including rounding the DCT transform kernel, rounding the DCT left multiplication result, rounding the DCT right multiplication result, quantization, and rounding the IDCT (Inverse Discrete Cosine Transform) transform kernel, rounding the IDCT left multiplication result, and rounding the IDCT right multiplication result—leads to a loss of precision in the frequency domain. This manifests as a smoothing out of small amounts of low-amplitude noise in flat regions, but a significant amount of high-amplitude noise in flat regions will propagate into stronger noise, and strong boundaries will also amplify noise, such as... Figure 3 As shown.

[0055] Therefore, during decoding, for macroblocks (MacroCodingUnits, MCUs) with scaling factors less than a preset scaling factor threshold, a decoded quantization table is used where the quantization step size of the corresponding high-frequency coefficients is smaller than the corresponding quantization step size in the encoded quantization table. This ensures that the absolute value of the decoded high-frequency coefficients is smaller than the original high-frequency coefficients before encoded quantization, allowing the high-frequency components to gradually attenuate and preventing them from accumulating and becoming increasingly larger, resulting in a smoother image effect. Figure 4 As shown.

[0056] Furthermore, the product of the coefficient matrix of the first integer DCT transform kernel and the coefficient matrix of the second integer DCT transform kernel is an integer multiple of the identity matrix to reduce fixed-point rounding error.

[0057] Therefore, this setting can ensure the accuracy of data reconstruction during the positive and inverse discrete cosine transform processes, reduce drift errors during encoding and decoding, and thus significantly improve the accuracy and quality of image reconstruction.

[0058] Specifically, both the first integer DCT transform kernel and the second integer DCT transform kernel are integer DCT transform kernels. The elements of their coefficient matrices are adjusted by a scaling factor so that the product of their coefficient matrices is approximately an integer multiple of the identity matrix, thereby reducing fixed-point rounding errors.

[0059] Optionally, the coefficient matrices of both the first integer DCT transform kernel and the second integer DCT transform kernel are:

[0060] K={a,a,a,a,a,a,a}, \

[0061] {d,e,f,g,-g,-f,-e,-d}, \

[0062] {b,c,-c,-b,-b,-c,c,b}, \

[0063] {e,-g,-d,-f,f,d,g,-e}, \

[0064] {a,-a,-a,a,a,-a,-a,a}, \

[0065] {f,-d,g,e,-e,-g,d,-f}, \

[0066] {c,-b,b,-c,-c,b,-b,c}, \

[0067] {g,-f,e,-d,d,-e,f,-g} \

[0068] Where K is the coefficient matrix, a=181, b=236, c=99, d=251, e=213, f=142, g=50.

[0069] Therefore, by employing an optimized integer discrete cosine transform kernel, the frame-by-frame accumulation of fixed-point rounding error in the recursive compression closed loop can be effectively suppressed, ensuring the long-term stability of the image quality of the reference frame.

[0070] For details, please refer to Figure 5 ,like Figure 5 As shown, multi-round iterative encoding and decoding based on JPEG will produce black borders around the long light strip. This is because the chip needs to use an integer DCT transform kernel, and the integerization of the DCT transform kernel will destroy the orthogonality of its coefficient matrix. The 7-parameter DCT transform kernel is defined as follows:

[0071] K={a,a,a,a,a,a,a}, \

[0072] {d,e,f,g,-g,-f,-e,-d}, \

[0073] {b,c,-c,-b,-b,-c,c,b}, \

[0074] {e,-g,-d,-f,f,d,g,-e}, \

[0075] {a,-a,-a,a,a,-a,-a,a}, \

[0076] {f,-d,g,e,-e,-g,d,-f}, \

[0077] {c,-b,b,-c,-c,b,-b,c}, \

[0078] {g,-f,e,-d,d,-e,f,-g} \

[0079] The orthogonality requirement of the DCT transform kernel is that the product of the coefficient matrix of the DCT transform kernel and its transpose is equal to the product of the transpose matrix of the DCT transform kernel and its coefficient matrix is ​​equal to the identity matrix.

[0080]

[0081] That is, requirements

[0082]

[0083]

[0084] Currently, it is difficult to find integer solutions that satisfy the orthogonality equation mathematically; only approximate solutions are available. After multiplying the floating-point DCT by 64 and rounding, a=64, b=84, c=35, d=89, e=75, f=50, g=18.

[0085] At this point, the orthogonality error is relatively large.

[0086]

[0087]

[0088] The necessary modifications are needed: change b to 83 and c to 36 to reduce the error.

[0089]

[0090]

[0091]

[0092] This is the DCT / IDCT transform core used in many chips, including the 265HEVC. However, using iterative encoding and decoding in multiple rounds will produce black borders. Please refer to Figures 6 and 7. In each block of Figures 6 and 7, the numbers represent the values ​​of the luminance components, ranging from 0 to 255. The smaller the value, the darker the color; the larger the value, the brighter the color. =0 indicates no quantization. =1 indicates the existence of a tiny quantization. Indicates to (Input) Number of times to encode and decode; Figure 6a The input is a brightness simulation image. Figure 6b for When =1, existing technology is used to Figure 6a shown in 8 A simulation image of brightness after 100 iterations of encoding and decoding using 8 inputs. Figure 6c for When =1, existing technology is used to Figure 6a shown in 8 A simulated brightness image after 1000 iterations of encoding and decoding with 8 inputs. Comparison shows that... Figure 6b and Figure 6c The values ​​in column 7 are relative to Figure 6a The median value is significantly smaller, indicating that black borders will appear after multiple iterations. Figure 7a The input is a brightness simulation image. Figure 7b for When =0, existing technology is used to Figure 7a shown in 8 A simulation image of brightness after 100 iterations of encoding and decoding using 8 inputs. Figure 7c for When =0, existing technology is used to Figure 7a shown in 8 A simulated brightness image after 1000 iterations of encoding and decoding with 8 inputs. Comparison shows that... Figure 7b and Figure 7c The values ​​in column 7 are relative to Figure 7a The median value is significantly smaller, indicating that black borders will appear after multiple iterations.

[0093] Floating-point DCT multiplied by 64 After rounding, and Both are positive numbers. If the absolute value of the error decreases after adjusting b / c, the other must increase.

[0094] After multiplying the floating-point DCT by 128 and rounding it, and Although one is positive and the other is negative, the difference is quite large. Modifying b / c will also cause one absolute value of error to decrease while the other increases.

[0095] Floating-point DCT multiplied by 128 After rounding, the first and second equations above have one positive and one negative value, so the kernel coefficients of b / c are modified as follows:

[0096]

[0097] Then the error can be reduced.

[0098] Floating-point DCT multiplied by 256, 256 Rounding to 512 results in a smaller error, but the bit width is larger when implemented within the chip.

[0099] Therefore, the values ​​of a, b, c, d, e, f, and g in the coefficient matrix of the first integer DCT transform kernel and the second integer DCT transform kernel in the image processing method provided by this invention are as follows:

[0100]

[0101] Please refer to Figures 8a to 8c ,in, Figure 8a The input is a brightness simulation image. Figure 8b for When =1, the existing DCT transform is used for verification. Figure 8a shown in 8 A simulation image of brightness after 50 iterations of encoding and decoding with 8 inputs. Figure 8c for When =1, the existing DCT transform is used for verification. Figure 8a shown in 8 A simulated brightness image after 1000 iterations of encoding and decoding with 8 inputs. Comparison shows that... Figure 8b and Figure 8c The value in the 7th row is relative to Figure 8a The median value is significantly smaller, indicating that the DCT transform kernel used by 265HEVC will produce black borders after multiple rounds of iterative encoding and decoding.

[0102] Please refer to Figure 9, where, Figure 9a The input is a brightness simulation image. Figure 9b for When =1, the DCT transform verification provided by this invention is used. Figure 9a shown in 8 A simulation image of brightness after 50 iterations of encoding and decoding with 8 inputs. Figure 9c for When =1, the DCT transform verification provided by this invention is used. Figure 9a shown in 8 A simulated brightness image after 1000 iterations of encoding and decoding with 8 inputs. Comparison shows that... Figure 9b and Figure 9c Each row of values ​​in the data is relative to Figure 9a The similarity of the median values ​​indicates that the DCT transform kernel used in this invention will not produce black borders during multi-round iterative encoding and decoding.

[0103] Furthermore, the step of dividing the enhanced image into blocks includes: dividing the Y component, U component, and V component of the enhanced image into blocks respectively, to obtain multiple Y component spatial image blocks to be encoded, multiple U component spatial image blocks to be encoded, and multiple V component spatial image blocks to be encoded; the step of combining the multiple spatial image blocks to be encoded into multiple macroblocks includes: according to the YUV image format of the enhanced image, combining the multiple Y component spatial image blocks to be encoded, the multiple U component spatial image blocks to be encoded, and the multiple V component spatial image blocks to be encoded into multiple macroblocks according to corresponding combination rules.

[0104] Therefore, by dividing the Y, U, and V components into blocks, independent preprocessing of luminance and chrominance data can be achieved, facilitating subsequent differentiated encoding based on the characteristics of different components. Furthermore, by combining the image blocks of each component into macroblocks according to the YUV image format, multiple sampling formats can be flexibly adapted, ensuring that the macroblock structure corresponds to the image sampling structure.

[0105] Specifically, the block segmentation operation in the image processing method provided by this invention conforms to the JPEG standard; please refer to [link / reference]. Figure 10 ,like Figure 10 As shown, the segmentation includes dividing the input image into three components (Y, U, and V) into 8... Divide into multiple units of 8 8 spatial domain image blocks to be encoded, each 8 8. The spatial image block to be encoded contains 64 pixels. For YUV420 format images, the basic coding units (macroblocks) are combined in the order of Y0, Y1, Y2, Y3, U0, V0. For YUV422 format images, the macroblocks are combined in the order of Y0, Y1, U0, V0. For YUV420 format images, the macroblocks are combined in the order of Y0, U0, V0.

[0106] Furthermore, the quantization table includes a luminance quantization table and a chrominance quantization table. For each macroblock, the luminance quantization table corresponding to that macroblock satisfies the following relationship with the standard luminance quantization table:

[0107] ,

[0108] in, For the first The first macroblock in the luminance coding quantization table is the... Quantization step size The first in the standard luminance quantization table Quantization step size For the first The scaling factor corresponding to each macroblock , ∈[0,1,2,……255].

[0109] Therefore, based on the reference luminance quantization table recommended by the JPEG standard, a corresponding luminance coding quantization table for encoding can be generated by independently setting a scaling factor for each macroblock and performing fixed-point operations.

[0110] Specifically, the quantization step size at each frequency position in the standard luminance quantization table is multiplied by the scaling factor of the current macroblock and then normalized and rounded to obtain the actual quantization step size used by the macroblock during encoding. When both the luminance encoding quantization table and the luminance decoding quantization table of the macroblock use the reference luminance quantization table, the implementation scheme degenerates into the standard JPEG quantization scheme to ensure compatibility with the standard decoding process.

[0111] Preferred, The offset is 16. (Displacement) is 5, standard brightness quantization table As shown in Table 1 below.

[0112] Table 1 Standard Brightness Quantification Table

[0113] 16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87 80 62 18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 72 92 95 98 112 100 103 99

[0114] Optionally, the decoding quantization table includes a luminance decoding quantization table and a chrominance decoding quantization table. For each macroblock, the luminance decoding quantization table corresponding to the macroblock satisfies the following relationship with the standard luminance quantization table:

[0115]

[0116] in, For the first The first macroblock in the luminance decoding quantization table is the... Quantization step size To preset the scaling factor threshold, The preset standard brightness quantization step size threshold.

[0117] Therefore, at the decoding end, based on the comparison result of the scaling factor of the current macroblock with the preset scaling factor threshold, and the comparison result of the quantization step size corresponding to the current processing frequency position in the standard brightness quantization table with the preset standard brightness quantization step size threshold, the inverse quantization scheme can be flexibly selected.

[0118] Specifically, when the scaling factor is greater than or equal to the preset scaling factor threshold, or when the corresponding quantization step size in the standard brightness quantization table is less than or equal to the preset standard brightness quantization step size threshold, the standard quantization step size is used for inverse quantization; when the scaling factor is less than the preset scaling factor threshold and the corresponding quantization step size in the standard brightness quantization table is greater than the preset standard brightness quantization step size threshold, the reduced quantization step size calculated based on the quantization scaling factor minus one is used for inverse quantization, so that the absolute value of the recovered high-frequency coefficients is systematically less than the absolute value of the original high-frequency coefficients before encoding quantization.

[0119] Furthermore, for each macroblock, the chroma-coded quantization table corresponding to that macroblock and the standard chroma-coded quantization table satisfy the following relationship:

[0120] ,

[0121] in, For the first The first macroblock in the chroma coding quantization table is the... Quantization step size For the first in the standard colorimetric table Each quantization step size.

[0122] Therefore, based on the reference chromaticity quantization table recommended by the JPEG standard, a chromaticity coding quantization table for encoding can be generated by independently setting a scaling factor for each macroblock and performing fixed-point operations.

[0123] Specifically, the quantization step size at each frequency position in the standard chroma quantization table is multiplied by the scaling factor of the current macroblock and then normalized and rounded to obtain the actual quantization step size used by the macroblock during encoding. When both the chroma encoding quantization table and the chroma decoding quantization table of the macroblock use the reference chroma quantization table, the implementation scheme degenerates into the standard JPEG quantization scheme to ensure compatibility with the standard decoding process.

[0124] Preferred, standard colorimetric table As shown in Table 2 below.

[0125] Table 2 Standard Colorimetric Table

[0126] 17 18 24 47 99 99 99 99 18 21 26 66 99 99 99 99 24 26 56 99 99 99 99 99 47 66 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99

[0127] Furthermore, the decoding quantization table includes a luminance decoding quantization table and a chrominance decoding quantization table. For each macroblock, the chrominance decoding quantization table corresponding to that macroblock satisfies the following relationship with the standard chrominance quantization table:

[0128] in, For the first The first macroblock in the chroma decoding quantization table is the... Quantization step size To preset the scaling factor threshold, The preset standard colorimetric step size threshold is set.

[0129] Therefore, at the decoding end, based on the comparison result between the scaling factor of the current macroblock and the preset scaling factor threshold, and the comparison result between the quantization step size corresponding to the current processing frequency position in the standard chromaticity quantization table and the preset standard chromaticity quantization step size threshold, it is possible to flexibly select the inverse quantization scheme.

[0130] Specifically, when the scaling factor is greater than or equal to the preset scaling factor threshold, or when the corresponding quantization step size in the standard color quantization table is less than or equal to the preset standard color quantization step size threshold, the standard quantization step size is used for dequantization. When the scaling factor is less than the preset scaling factor threshold and the corresponding quantization step size in the standard color quantization table is greater than the preset standard color quantization step size threshold, the reduced quantization step size calculated based on the quantization scaling factor minus one is used for dequantization, so that the absolute value of the recovered high-frequency coefficients is systematically less than the absolute value of the original high-frequency coefficients before encoding quantization.

[0131] Based on the same inventive concept, the present invention also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the image processing method described above.

[0132] Therefore, the electronic device provided by the present invention, during decoding, adopts a decoding quantization table with a quantization step size smaller than the corresponding quantization step size in the encoding quantization table for macroblocks (MacroCodingUnits, MCUs) whose scaling factor is less than a preset scaling factor threshold. This ensures that the absolute value of the high-frequency coefficients after decoding is less than the original high-frequency coefficients before encoding quantization, thereby causing the high-frequency components to gradually attenuate and avoiding the high-frequency components from accumulating and becoming larger, resulting in a smoother image effect and reducing the high-frequency noise introduced during multi-round iterative compression.

[0133] Based on the same inventive concept, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the image processing method described above.

[0134] Therefore, the readable storage medium provided by the present invention, during decoding, adopts a decoding quantization table with a quantization step size smaller than the corresponding quantization step size in the encoding quantization table for macroblocks (MacroCodingUnits, MCUs) whose scaling factor is less than a preset scaling factor threshold. This ensures that the absolute value of the high-frequency coefficients after decoding is less than the original high-frequency coefficients before encoding quantization, thereby causing the high-frequency components to gradually attenuate and avoiding the high-frequency components from accumulating and becoming larger, resulting in a smoother image effect and reducing the high-frequency noise introduced during multi-round iterative compression.

[0135] Compared with the prior art, the image processing method, electronic device, and readable storage medium provided by the present invention have the following beneficial effects:

[0136] This invention employs a decoding quantization table with a quantization step size smaller than the corresponding quantization step size in the encoding quantization table for macroblocks (MacroCodingUnits, MCUs) whose scaling factor is less than a preset scaling factor threshold during decoding. This ensures that the absolute value of the high-frequency coefficients after decoding is less than the original high-frequency coefficients before encoding quantization, thereby gradually attenuating the high-frequency components and preventing them from accumulating and becoming larger, resulting in a smoother image effect and reducing the high-frequency noise introduced during multi-round iterative compression.

[0137] Furthermore, by employing a newly designed first integer DCT transform kernel and a second integer DCT transform kernel, this invention can avoid the problem of multi-round iterative encoding and decoding error divergence caused by the inability of integer DCT matrices to satisfy orthogonality.

[0138] The above description is only a description of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, include: Step S100: Fuse the current frame and the reference frame to obtain an enhanced image; Step S200: Divide the enhanced image into blocks to obtain multiple spatial image blocks to be encoded, and combine the multiple spatial image blocks to be encoded into multiple macroblocks. Configure a corresponding encoding quantization table and decoding quantization table for each macroblock. The quantization step size of the corresponding high-frequency coefficient in the decoding quantization table of the macroblock whose scaling factor is less than a preset scaling factor threshold is less than the corresponding quantization step size in the encoding quantization table. Step S300: Use the first integer DCT transform kernel to perform discrete cosine transform on each of the spatial domain image blocks to be encoded to obtain the corresponding original frequency domain coefficient matrix; Step S400: For each spatial image block to be encoded, the original frequency domain coefficient matrix corresponding to the spatial image block to be encoded is quantized using the coding quantization table of the corresponding macroblock to obtain the corresponding quantization index sequence. Step S500: For each spatial image block to be encoded, entropy encoding is performed on the quantization index sequence corresponding to the spatial image block to be encoded to generate a corresponding compressed bitstream, and the compressed bitstream is stored in off-chip memory. Step S600: Read each of the compressed bitstreams sequentially from the off-chip memory, and for each compressed bitstream, perform entropy decoding on the compressed bitstream to obtain the corresponding decoding quantization index sequence, and use the decoding quantization table of the corresponding macroblock to dequantize the decoding quantization index sequence corresponding to the compressed bitstream to obtain the corresponding reconstructed frequency domain coefficient matrix. Step S700: Use the second integer DCT transform kernel to perform inverse discrete cosine transform on the reconstructed frequency domain coefficient matrix corresponding to each compressed bitstream to obtain the corresponding reconstructed spatial domain image block; Step S800: Perform block reorganization on all the reconstructed spatial image blocks to obtain the corresponding reconstructed reference frame, and use the reconstructed reference frame as the new reference frame, use the input next frame as the new current frame, and return to step S100.

2. The image processing method as described in claim 1, characterized in that, The product of the coefficient matrix of the first integer DCT transform kernel and the coefficient matrix of the second integer DCT transform kernel is an integer multiple of the identity matrix to reduce fixed-point rounding error.

3. The image processing method as described in claim 1, characterized in that, The coefficient matrices of both the first integer DCT transform kernel and the second integer DCT transform kernel are: K={a,a,a,a,a,a,a}, \ {d,e,f,g,-g,-f,-e,-d}, \ {b,c,-c,-b,-b,-c,c,b}, \ {e,-g,-d,-f,f,d,g,-e}, \ {a,-a,-a,a,a,-a,-a,a}, \ {f,-d,g,e,-e,-g,d,-f}, \ {c,-b,b,-c,-c,b,-b,c}, \ {g,-f,e,-d,d,-e,f,-g} \ Where K is the coefficient matrix, a=181, b=236, c=99, d=251, e=213, f=142, g=50.

4. The image processing method as described in claim 1, characterized in that, The step of dividing the enhanced image into blocks includes: The Y, U, and V components of the enhanced image are divided into blocks to obtain multiple Y component spatial image blocks to be encoded, multiple U component spatial image blocks to be encoded, and multiple V component spatial image blocks to be encoded, respectively. The step of combining the plurality of spatial image blocks to be encoded into a plurality of macroblocks includes: Based on the YUV image format of the enhanced image, the plurality of Y component spatial image blocks to be encoded, the plurality of U component spatial image blocks to be encoded, and the plurality of V component spatial image blocks to be encoded are combined into a plurality of macroblocks according to the corresponding combination rules.

5. The image processing method as described in claim 1, characterized in that, The quantization table includes a luminance quantization table and a chrominance quantization table. For each macroblock, the luminance quantization table corresponding to that macroblock satisfies the following relationship with the standard luminance quantization table: ， in, For the first The first macroblock in the luminance coding quantization table is the... Quantization step size The first in the standard luminance quantization table Quantization step size For the first The scaling factor corresponding to each macroblock , ∈[0,1,2,……255].

6. The image processing method as described in claim 5, characterized in that, The decoding quantization table includes a luminance decoding quantization table and a chrominance decoding quantization table. For each macroblock, the luminance decoding quantization table corresponding to that macroblock satisfies the following relationship with the standard luminance quantization table: in, For the first The first macroblock in the luminance decoding quantization table is the... Quantization step size To preset the scaling factor threshold, The preset standard brightness quantization step size threshold.

7. The image processing method as described in claim 5, characterized in that, For each macroblock, the chroma-coded quantization table corresponding to that macroblock and the standard chroma-coded quantization table satisfy the following relationship: ， in, For the first The first macroblock in the chroma coding quantization table is the... Quantization step size For the first in the standard colorimetric table Each quantization step size.

8. The image processing method as described in claim 7, characterized in that, The decoding quantization table includes a luminance decoding quantization table and a chrominance decoding quantization table. For each macroblock, the chrominance decoding quantization table corresponding to that macroblock satisfies the following relationship with the standard chrominance quantization table: in, For the first The first macroblock in the chroma decoding quantization table is the... Quantization step size To preset the scaling factor threshold, The preset standard colorimetric step size threshold is set.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the image processing method according to any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the image processing method according to any one of claims 1 to 8.

Images