A method and hardware system for realizing pixel value prediction

Abstract

The present invention discloses a method and hardware system for implementing pixel value prediction, which includes the following steps: determining that the image block to be predicted is a rectangular block; respectively obtaining the first adjacent image block and the second adjacent image block of the image block to be predicted; filtering some pixel values ​​in the first adjacent image block to obtain filtered pixel values; using the obtained quantization parameters, performing quantization parameter adaptive mixing processing on some pixel values ​​in the first adjacent image block and the filtered pixel values ​​to obtain pixel mixing values; using only the pixel mixing values ​​and the reconstructed values ​​of the pixels in the second adjacent image block to obtain the predicted values ​​of the pixels in the image block to be predicted. The present invention speeds up the calculation speed of the median adaptive prediction algorithm, breaks the correlation of the intermediate data when calculating the pixel prediction value, does not rely on the dequantization residual value of the previous pixel, and can still maintain a good compression ratio and visual lossless effect.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and in particular to a method and a hardware system for realizing pixel value prediction. Background Art

[0002] Display Stream Compression (DSC) is a visually lossless compression method proposed by the Video Electronics Standards Association (VESA) to address the issue of display link bandwidth not meeting the rapidly increasing display resolution. The DSC algorithm performs a series of operations on the original pixel values, including prediction, reconstruction, and flatness detection, to produce a compressed bitstream. At the decoding end, the reverse of the encoding operations is performed on the compressed bitstream to produce visually lossless image data. The DSC algorithm performs encoding and decoding only within a single frame, and the encoding and decoding of a single frame is independent of other frames. Furthermore, the algorithm can set different compression rates based on user needs, achieving visual losslessness while maintaining good compression efficiency. Furthermore, codecs based on DSC are low-cost, low-latency, simple, efficient, and easy to implement in hardware, making them ideal for encoding and decoding images and videos on display links.

[0003] The display stream compression algorithm first caches the input raw image data, then uses the improved median adaptive prediction algorithm, midpoint prediction algorithm, and color history index prediction algorithm to calculate the pixel prediction value. The pixel prediction value is then used to calculate the pixel reconstruction value and the pixel quantization residual value. Finally, the quantization residual value is encoded into the compressed code stream and sent to the decoder. The decoder performs the opposite operation on the compressed code stream to obtain a visually lossless image. However, the improved median adaptive prediction algorithm in the display stream compression algorithm involves more mathematical operations, and it calculates the prediction value based on three pixels as a group, such as Figure 2As shown in the figure, P0, P1, and P2 are a group of predicted pixel values, and P3, P4, and P5 are another group of predicted pixel values. When calculating the predicted value of pixel P1, the inverse quantized residual value of pixel P0 is needed, and the inverse quantized residual value of pixel P0 is calculated from the predicted value of pixel P0, that is, the predicted value of pixel P0 is needed to calculate the predicted value of P1; similarly, when calculating the predicted value of pixel P2, the predicted values ​​of pixels P0 and P1 are needed, that is, there is data correlation between the predicted values ​​of the three pixels P0, P1, and P2, and the correlation between these data seriously slows down the execution speed of the algorithm and increases the clock cycle required to calculate the predicted pixel value, especially on a platform with parallel execution statements such as FPGA. The current optimization method mainly accelerates the median adaptive prediction algorithm by increasing the system's operating clock frequency. However, the acceleration effect achieved by increasing the system's operating frequency has great limitations. For 2K and 4K video images, the system's own clock frequency is already high enough, and it is no longer possible to accelerate the median adaptive prediction algorithm or even the entire display stream compression algorithm by increasing the system's operating frequency. Summary of the Invention

[0004] In view of this, the present invention provides a method and hardware system for realizing pixel value prediction, which optimizes the calculation speed of the median adaptive prediction algorithm in the display stream compression algorithm, breaks the correlation of intermediate data when calculating a group of three pixel prediction values, does not rely on the dequantized residual value of the previous pixel, and can calculate the pixel prediction value only by using the reconstructed pixel value and the mixed pixel value. At the same time, the improved algorithm can still maintain a good compression ratio and visual lossless effect.

[0005] The present invention discloses a method for realizing pixel value prediction, comprising the following steps:

[0006] Determining that the image block to be predicted is a rectangular block; respectively obtaining a first adjacent image block and a second adjacent image block of the image block to be predicted;

[0007] performing filtering processing on some pixel values ​​in the first adjacent image block to obtain filtered pixel values;

[0008] Using the obtained quantization parameter, perform quantization parameter adaptive mixing processing on part of the pixel values ​​in the first adjacent image block and the filtered pixel value to obtain a pixel mixing value;

[0009] The predicted values ​​of the pixels in the image block to be predicted can be obtained by using only the pixel mixing values ​​and the pixel reconstruction values ​​in the second adjacent image block.

[0010] Furthermore, the short side of the image block to be predicted is M, the long side is N, M and N are both positive integers and N is greater than M;

[0011] The length of the side between the first adjacent image block and the image block to be predicted is greater than N, and the length of the side between the second adjacent image block and the image block to be predicted is equal to M.

[0012] Furthermore, the first adjacent image block is located above the image block to be predicted, and the second adjacent image block is located on the left side of the image block to be predicted;

[0013] The values ​​in the first adjacent image block and the second adjacent image block are respectively reconstructed values ​​of the corresponding pixels.

[0014] Furthermore, when the size of the first adjacent image block is 1*6 and the size of the image block to be predicted is 1*N, the calculation formula used for performing horizontal low-pass filtering on the pixel values ​​in the second adjacent image block is:

[0015] filtB=(5*a+6*b+5*c+52)>>4;

[0016] filtC=(5*b+6*c+5*d+52)>>4;

[0017] filtD=(5*c+6*d+5*e+52)>>4;

[0018] filtE=(5*d+6*e+5*f+52)>>4;

[0019] Among them, a, b, c, d, e, and f are six consecutive reconstructed values ​​in the first adjacent image block, and filtB, filtC, filtD, and filtE are filtered pixel values ​​respectively.

[0020] Furthermore, the low-pass filter is [5 6 5]; the method of calculating filtB, filtC, filtD, and filtE is parallel calculation, that is, low-pass filtering is performed on b, c, d, and e at the same time to obtain filtered pixel values.

[0021] Furthermore, the calculation formula used in the quantization parameter adaptive hybrid processing is:

[0022] blendB=b+CLAMP(filtB-b,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0023] blendC=c+CLAMP(filtC-c,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0024] blendD=d+CLAMP(filtD-d,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0025] blendE=e+CLAMP(filtE-e,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0026] Among them, blendB, blendC, blendD, and blendE are the pixel blending values ​​corresponding to b, c, d, and e respectively; qLevel is the quantization level value obtained by mapping the obtained quantization parameter value; CLAMP is the limit function, CLAMP(filtE-e,-(1<<qLevel)> >1,(1<<qLevel)> >1) means limiting the value of filtE-e to -(1<<qLevel)> >1 and (1<<qLevel)> >1.

[0027] Furthermore, when the size of the second adjacent image block is 1*1 and the size of the image block to be predicted is 1*3, the calculation formula used to obtain the predicted value of the pixel in the image block to be predicted is:

[0028] P0=CLAMP(g+blendC–blendB,MIN(g,blendC),MAX(g,blendC));

[0029] P1=CLAMP(g+blendD–blendB+blendD–blendC,MIN(g,blendC,blendD),MAX(g,blendC,blendD));

[0030] P2=CLAMP(g+blendE–blendB+blendE–blendD+blendE–blendC,MIN(g,blendC,blendD,blendE),MAX(g,blendC,blendD,blendE));

[0031] Wherein, P0, P1, and P2 are respectively the predicted values ​​of the pixels in the image block to be predicted, g is the reconstructed value of the pixels in the second adjacent image block, MIN is the function of taking the minimum value, and MAX is the function of taking the maximum value.

[0032] Furthermore, the reconstruction values ​​and quantization parameter values ​​in the first adjacent image block and the second adjacent image block are acquired through a RAM bus.

[0033] The present invention also discloses a hardware system for realizing pixel value prediction, comprising:

[0034] An input data buffer unit, configured to buffer original pixel values ​​to be predicted in the image block to be predicted, reconstructed values ​​of pixels in the first adjacent image block, reconstructed values ​​of pixels in the second adjacent image block, and quantization parameter data;

[0035] a horizontal low-pass filtering calculation unit, configured to perform horizontal low-pass filtering on the reconstructed values ​​of the pixels in the first adjacent image block;

[0036] a quantization parameter adaptive hybrid processing unit, comprising an error calculation unit and a quantization parameter processing unit, for calculating pixel errors between horizontal low-pass filter values ​​and corresponding pixel reconstruction values, as well as hybrid pixel values;

[0037] The prediction value calculation unit is used to calculate the final median adaptive prediction pixel value and output the calculated pixel prediction value to the subsequent module.

[0038] Furthermore, the hardware system is composed of a three-stage pipeline, wherein the first stage pipeline is used for performing horizontal low-pass filtering processing, the second stage pipeline is used for performing quantization parameter adaptive mixing processing, and the third stage pipeline is used for calculating the final predicted pixel value.

[0039] Due to the adoption of the above-mentioned technical solution, the present invention has the following advantages: In terms of image processing effects, the present invention reduces the impact of deleting the data correlation between predicted pixel values ​​on image effects by changing the filter value calculation method of the horizontal low-pass filter, thereby improving image quality to a certain extent. In terms of image data processing speed, the present invention deletes the data correlation between predicted pixel values. Under the premise of maintaining the visual lossless effect and good compression rate of display stream compression, the display stream compression algorithm can simultaneously calculate the predicted value of the pixel within one clock cycle during the hardware implementation process, giving full play to the advantages of parallel computing of the hardware platform and greatly improving the frequency of system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments described in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] Figure 1 A flow chart of an improved median adaptive prediction algorithm method for display stream compression provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of pixel grouping involved in a specific implementation manner provided by an embodiment of the present invention;

[0043] Figure 3A schematic diagram of the locations of predicted pixels involved in a specific implementation of an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of a design of a horizontal low-pass filtering unit involved in a specific implementation manner provided by an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of a pipeline for median adaptive prediction before algorithm modification in a specific implementation manner provided by an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of a pipeline for median adaptive prediction after algorithm modification in a specific implementation manner provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described with reference to the accompanying drawings and embodiments. Obviously, the embodiments described are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those skilled in the art should fall within the scope of protection of the embodiments of the present invention.

[0048] In the process of seeking solutions to the problems in the background, the present invention fully investigates the principles of the median adaptive prediction algorithm in the display stream compression algorithm, conducts a detailed analysis of the computational complexity therein, and proposes an acceleration method and system based on digital circuits (hardware structure characteristics), which optimizes the digital image processing calculation speed of the median adaptive prediction algorithm and the filtering effect of the horizontal filter.

[0049] To improve the computational speed of the display stream compression algorithm's digital image processing, the time-consuming median adaptive prediction algorithm was accelerated. Furthermore, to reduce the impact of data correlation on image quality, the calculation method for the horizontal low-pass filter was modified, optimizing overall image quality.

[0050] As a specific embodiment of the present invention, Figure 1 As shown, the acceleration method for digital image processing of the display stream compression algorithm provided by the present invention includes the following steps:

[0051] The first step is to obtain the six reconstructed pixel values ​​in the previous row corresponding to the group of pixels to be predicted and the first reconstructed pixel value on the left side of the group of pixels to be predicted, and obtain the quantization parameter value required to calculate the three predicted pixel values ​​of this group;

[0052] In the second step, the six reconstructed pixel values ​​in the previous row obtained in the first step are subjected to horizontal low-pass filtering to obtain corresponding horizontal low-pass filtered pixel values;

[0053] The third step is to perform quantization parameter adaptive mixing processing on the horizontal low-pass filtered pixel value obtained in the second step and the reconstructed pixel value obtained in the first step to obtain a pixel mixing value;

[0054] In the fourth step, the final pixel prediction value is calculated based on the pixel mixed value obtained in the third step and the pixel reconstruction value obtained in the first step.

[0055] The technical solutions of the embodiments of the present invention are further described below with reference to the accompanying drawings.

[0056] The median adaptive prediction (MMAP) method in the display stream compression algorithm calculates the prediction value in groups, and each group contains 3 pixels, such as Figure 3 As shown, P0, P1, and P2 are the three pixels to be predicted in the current row, a, b, c, d, e, and f are the six reconstructed pixel values ​​in the previous row, and g is the first reconstructed pixel value on the left side of the current prediction group.

[0057] Before calculating the median adaptive algorithm prediction value, it is necessary to first obtain the six reconstructed pixel values ​​in the previous row and the first reconstructed pixel value on the left side of the current prediction group, as well as the quantization parameter required to calculate the current group's prediction value. The input data buffer unit is responsible for receiving the reconstructed pixel values ​​and quantization parameter values ​​input by other external modules via the RAM bus.

[0058] like Figure 4 As shown in Figure 1, the reconstructed values ​​obtained from the input data buffer unit are input into the horizontal low-pass filter to calculate the pixel filter values ​​filtB, filtC, filtD, and filtE corresponding to the reconstructed pixels b, c, d, and e in the previous row. The horizontal low-pass filter used to calculate the pixel filter values ​​is [5 6 5], and the pixel filter values ​​filtB, filtC, filtD, and filtE are calculated in parallel. The calculation formula is expressed as follows:

[0059] filtB=(5*a+6*b+5*c+52)>>4;

[0060] filtC=(5*b+6*c+5*d+52)>>4;

[0061] filtD=(5*c+6*d+5*e+52)>>4;

[0062] filtE=(5*d+6*e+5*f+52)>>4;

[0063] After obtaining the pixel filtering value, the pixel mixing value is obtained by the quantization parameter adaptive mixing process. The formula of the quantization parameter adaptive mixing process is expressed as follows:

[0064] blendB=b+CLAMP(filtB-b,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0065] blendC=c+CLAMP(filtC-c,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0066] blendD=d+CLAMP(filtD-d,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0067] blendE=e+CLAMP(filtE-e,-(1<<qLevel)> >1,(1<<qLevel)> >1);

[0068] Where b, c, d, and e are the reconstructed pixel values ​​of the previous row, blendB, blendC, blendD, and blendE are the pixel mixed values ​​corresponding to the reconstructed pixels b, c, d, and e of the previous row; filtB, filtC, filtD, and filtE are the horizontal low-pass filter values ​​corresponding to the reconstructed pixels b, c, d, and e of the previous row; qLevel is the quantization level value obtained by mapping the obtained quantization parameter value; CLAMP is a limit function, for example, CLAMP(a, b, c), which limits the value of a to between b and c.

[0069] Finally, the first reconstructed pixel value g on the left side of the current prediction group and the pixel blending values ​​blendB, blendC, blendD, and blendE are used to calculate the median adaptive prediction value of the three pixels in the current group. The calculation formula is:

[0070] P0=CLAMP(g+blendC–blendB,MIN(g,blendC),MAX(g,blendC));

[0071] P1=CLAMP(g+blendD–blendB+blendD–blendC,MIN(g,blendC,blendD),MAX(g,blendC,blendD));

[0072] P2=CLAMP(g+blendE–blendB+blendE–blendD+blendE–blendC,MIN(g,blendC,blendD,blendE),MAX(g,blendC,blendD,blendE));

[0073] Among them, P0, P1, and P2 are the current set of three consecutive pixel prediction values ​​required to be obtained, g is the reconstructed value of the first pixel on the left of the group where the current predicted pixel is located, blendB, blendC, blendD, and blendE are the pixel mixing values ​​corresponding to the reconstructed pixels b, c, d, and e in the previous row; CLAMP is a limit function, for example, CLAMP(a, b, c), which limits the value of a to between b and c; MIN is a function that takes the minimum value; and MAX is a function that takes the maximum value.

[0074] like Figure 5 As shown in , when the median adaptive prediction algorithm is not changed, due to the data correlation between the predicted values ​​of a group of 3 pixels, a 5-stage pipeline is required to calculate a group of 3 pixels. Figure 6 As shown, after deleting the data correlation between the predicted values ​​of a group of 3 pixels, the embodiment of the present invention only requires a 3-stage pipeline to calculate the predicted pixel value, which is of great help to improve the clock rate of the subsequent modules.

[0075] In terms of image processing effectiveness, this embodiment reduces the impact of removing the data correlation between a group of three predicted pixel values ​​by modifying the filter value calculation method of the horizontal low-pass filter, thereby improving image quality to a certain extent. In terms of image data processing speed, this embodiment removes the data correlation between a group of three predicted pixel values. While maintaining the visual losslessness and good compression rate of the display stream compression, this allows the display stream compression algorithm to simultaneously calculate the predicted values ​​for a group of three pixels within a single clock cycle during hardware implementation. This fully utilizes the parallel computing advantages of the hardware platform and greatly increases the frequency of system operation.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A method for realizing pixel value prediction, characterized in that: The following steps are involved: Determining that the image block to be predicted is a rectangular block; respectively obtaining a first adjacent image block and a second adjacent image block of the image block to be predicted; performing filtering processing on some pixel values ​​in the first adjacent image block to obtain filtered pixel values; Using the obtained quantization parameter, perform quantization parameter adaptive mixing processing on part of the pixel values ​​in the first adjacent image block and the filtered pixel value to obtain a pixel mixing value; The predicted values ​​of the pixels in the to-be-predicted image block can be obtained by using only the pixel mixing values ​​and the reconstructed values ​​of the pixels in the second adjacent image block.

2. The method according to claim 1, characterized in that The short side of the image block to be predicted is M, the long side is N, M and N are both positive integers and N is greater than M; The length of the side between the first adjacent image block and the image block to be predicted is greater than N, and the length of the side between the second adjacent image block and the image block to be predicted is equal to M.

3. The method according to claim 2, characterized in that The first adjacent image block is located above the image block to be predicted, and the second adjacent image block is located on the left side of the image block to be predicted; The values ​​in the first adjacent image block and the second adjacent image block are respectively reconstructed values ​​of the corresponding pixels.

4. The method according to claim 2, characterized in that When the size of the first adjacent image block is 1 6. The size of the image block to be predicted is 1 When N, the calculation formula used for performing horizontal low-pass filtering on the pixel values ​​in the second adjacent image block is: filtB = ( 5 a + 6 b + 5 c + 52) >> 4 ; filtC = ( 5 b + 6 c + 5 d + 52) >> 4 ; filtD = ( 5 c + 6 d + 5 e + 52) >> 4 ; filterE = ( 5 d + 6 it + 5 f + 52) >> 4 ; Among them, a, b, c, d, e, and f are six consecutive reconstructed values ​​in the first adjacent image block, and filtB, filtC, filtD, and filtE are filtered pixel values ​​respectively.

5. The method according to claim 4, characterized in that The low-pass filter is [5 6 5]; the method of calculating filtB, filtC, filtD, and filtE is parallel calculation, that is, low-pass filtering is performed on b, c, d, and e at the same time to obtain filtered pixel values.

6. The method according to claim 4, characterized in that The calculation formula used in the quantization parameter adaptive hybrid processing is: blendB = b + CLAMP(filtB - b, - (1 << qLevel) >> 1, (1 << qLevel) >> 1); blendC = c + CLAMP(filtC - c, - (1 << qLevel) >> 1, (1 << qLevel) >> 1); blendD = d + CLAMP(filtD - d, - (1 << qLevel) >> 1, (1 << qLevel) >> 1); blendE = e + CLAMP(filtE - e, - (1 << qLevel) >> 1, (1 << qLevel) >> 1); Wherein, blendB, blendC, blendD, and blendE are the pixel blending values ​​corresponding to b, c, d, and e respectively; qLevel is the quantization level value obtained by mapping the obtained quantization parameter value; CLAMP is the limit function, and CLAMP(filtE - e, -(1 << qLevel) >> 1, (1 << qLevel) >> 1) means limiting the value of filtE - e to - (1 <<qLevel) > > 1 and (1 << qLevel) >> 1.

7. The method according to claim 6, characterized in that When the size of the second adjacent image block is 1 1, the size of the image block to be predicted is 1 3, the calculation formula used to obtain the predicted value of the pixel in the image block to be predicted is: P0 = CLAMP(g + blendC – blendB,MIN (g,blendC),MAX(g,blendC)); P1 = CLAMP(g + blendD – blendB + blendD – blendC , MIN(g,blendC,blendD) ,MAX(g,blendC,blendD)); P2 = CLAMP(g + blendE – blendB + blendE – blendD + blendE – blendC , MIN(g,blendC,blendD,blendE) , MAX(g,blendC,blendD,blendE)); Wherein, P0, P1, and P2 are respectively the predicted values ​​of the pixels in the image block to be predicted, g is the reconstructed value of the pixels in the second adjacent image block, MIN is the function of taking the minimum value, and MAX is the function of taking the maximum value.

8. The method according to claim 1, characterized in that The reconstruction values ​​and quantization parameter values ​​in the first adjacent image block and the second adjacent image block are acquired through a RAM bus.

9. A hardware system for implementing pixel value prediction based on the method according to any one of claims 1 to 8, characterized in that: include: An input data buffer unit, configured to buffer original pixel values ​​to be predicted in the image block to be predicted, reconstructed values ​​of pixels in the first adjacent image block, reconstructed values ​​of pixels in the second adjacent image block, and quantization parameter data; a horizontal low-pass filtering calculation unit, configured to perform horizontal low-pass filtering on the reconstructed values ​​of the pixels in the first adjacent image block; a quantization parameter adaptive hybrid processing unit, comprising an error calculation unit and a quantization parameter processing unit, for calculating pixel errors between horizontal low-pass filter values ​​and corresponding pixel reconstruction values, as well as hybrid pixel values; The prediction value calculation unit is used to calculate the final median adaptive prediction pixel value and output the calculated pixel prediction value to the subsequent module.

10. The hardware system according to claim 9, characterized in that: The hardware system is composed of three pipelines. The first pipeline is used for performing horizontal low-pass filtering processing, the second pipeline is used for performing quantization parameter adaptive mixing processing, and the third pipeline is used for calculating the final predicted pixel value.

Images