A method and system for two-dimensional recursive filtering to reduce color steps in a digital image
By employing a two-dimensional recursive filtering method, utilizing dynamic recursive coefficients and a memory buffer, the problem of excessive memory consumption in high-resolution image processing is solved, achieving efficient image processing and pseudo-contour reduction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- V SILICON SEMICON (HANGZHOU) CO LTD
- Filing Date
- 2022-06-06
- Publication Date
- 2026-06-16
AI Technical Summary
Existing color level filters consume excessive memory when processing high-resolution images, leading to increased costs.
A two-dimensional recursive filtering method is adopted, which combines low-pass filtering and detail detection with dynamically changing recursive coefficients and memory buffers to reduce memory usage and improve image processing efficiency.
It effectively reduces memory usage, improves the cost-effectiveness of image processing, and reduces false contours in high-resolution images, ensuring image quality.
Smart Images
Figure CN116563176B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a recursive filtering method, and more particularly to a recursive filtering method for image data. Background Technology
[0002] De-leveling filters can eliminate or reduce artifacts caused when quantizing images using low bit width sampling. In order to still obtain high-resolution images efficiently (e.g., increasing image resolution from HD to UHD 4K, or to UHD 8K), such filters tend to become increasingly large, especially in terms of memory footprint. Summary of the Invention
[0003] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a recursive filtering method for image data to solve the problem of memory consumption by the de-leveling filter in the prior art when processing high-resolution images.
[0004] A first aspect of the present invention provides a recursive filtering method for image data, comprising:
[0005] Receive several raw pixel values sequentially;
[0006] Multiply the original pixel value of the current pixel among the plurality of pixel values by a dynamically changing recursive coefficient;
[0007] The recursive filtered pixel values from the left and right adjacent pixels of the current pixel are added together, and the recursive filtered pixel values of the left and right adjacent pixels of the current pixel are retrieved from a memory buffer storing the filtered pixel data of the previous image row.
[0008] The sum of the recursive filtered pixel values multiplied by 1 is the difference between the summed values and the dynamically changing recursive coefficients.
[0009] Add the two values obtained by multiplying them together to get the filtered pixel value of the current pixel;
[0010] Write the filtered pixel value of the current pixel back to the memory buffer, and
[0011] The filtered pixel values are displayed on the monitor. Attached Figure Description
[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 This is a de-gradation system according to one embodiment of the present invention.
[0014] Figure 2 This is a de-gradation system according to another embodiment of the present invention.
[0015] Figure 3 This is a recursive filtering system with detail bias in an embodiment of the present invention.
[0016] Figure 4 This is a nonlinear detail separation filter in an embodiment of the present invention.
[0017] Figure 5 This is a recursive method for removing color levels in an embodiment of the present invention. Detailed Implementation
[0018] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification.
[0019] Please see Figures 1 to 5 It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are merely for illustrative purposes to aid those skilled in the art and are not intended to limit the scope of the invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to size, without affecting the effectiveness and purpose of the invention, should still fall within the scope of the disclosed technical content. Furthermore, the terms such as "upper," "lower," "left," "right," "middle," and "one" used in this specification are merely for clarity and are not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.
[0020] Some pseudo-contours or tonal gradations seen in low-gradient regions of an image are simply the result of insufficient quantization of image data in previous operations. Typically, quantization is applied in the early stages of a chain unit at a length of 8 bits, especially in video encoders, for example. Simultaneously, some noise, jitter, and error dispersion are eliminated, thus creating noticeable tonal transition bands in areas that were originally smoother. Therefore, without specific measures, this quantization level will remain in the image representation in the later stages of the chain unit. The flat slope of an 8-bit image will show clearly visible pseudo-contours, thus giving different levels of tonal gradations. These are undesirable phenomena, especially when clearly displayed on large displays with significant light output, such as existing television screens. The embodiments of the present invention strictly limit the total memory footprint of the required pixel data. This significantly improves the cost advantage of high-pixel images. The method of the present invention implements a recursive automatic adaptive low-pass filter operating in a two-dimensional manner.
[0021] Please see Figure 1 , Figure 1 This is a de-leveling system 100 according to an embodiment of the present invention. The system 100 can be represented by the following mathematical formula:
[0022] To obtain a filter at a single pixel location, the recursion is performed as follows:
[0023] Out = k C + (1-k) (ULrec + URrec), where,
[0024] • k is a dynamically changing recursive coefficient, and 0 <k≤1;
[0025] • C is the original pixel value at the current pixel position located at pixel coordinates (y, x);
[0026] •ULrec is the recursive filtering pixel level (for micro-low-pass processing) located at the coordinates (y-1, xn) of the top-left adjacent pixel;
[0027] •URrec is the recursive filtering pixel level (for micro-low-pass processing) located at the coordinates (y-1, x+n) of the upper right adjacent pixel;
[0028] •Out is the filtered pixel value at the current pixel position, where:
[0029] • n is a design-optional parameter, such as 2;
[0030] • 'Micro-low-pass filtering' is a design choice, for example, filtering by [1 2 1] / 4 (thus averaging the value of the center pixel with the average of its two neighboring pixels). This 'micro-low-pass filtering' is required when the positions of the top-left and top-right pixels are more than one pixel away from the x-coordinate of the current pixel. This prevents all filtered odd pixels from being filtered only from the original odd pixels, and vice versa for all even pixels, which could result in a checkerboard-like pattern.
[0031] This allows us to obtain data from the same row to the left of the C position coordinate, such as coordinates (y, x-1), instead of using the ULrec pixel data from the previous row.
[0032] The k-factor represents the strength of the recursive filter, and it needs to at least match the local information at the input. By setting the k-factor to a very low value (on the order of 0.01 to 0.001), the recursive filter can keep up with the input information very slowly, even when the tonal contours are tens to hundreds of pixels and / or separated by rows, thus smoothly navigating tonal transition regions. Considering memory usage, efficient filtering is achieved at the cost of a single-row buffer approximately equal to the width of the image. Because the k-factor is set to a very low order of magnitude, the recursion (multiplication and row delay) is performed with high precision, resulting in an actual output bit width that is larger than needed (e.g., 8 bits wider than required). This increased precision in the loop prevents the filtered output from remaining at its previous order of magnitude when there is a consecutive positive or negative difference between the previous signal and the output.
[0033] In terms of cost, the necessary latency is achieved using approximately one row of memory. To apply the solution to higher image resolutions, the self-adaptation is implemented as follows:
[0034] • Make the k-factor closer to (but not to) 0;
[0035] • Improved accuracy in loops;
[0036] • Implement row memory based on the horizontal dimension of the image resolution.
[0037] The above method can be foreseen to have a maximum applicable design image resolution in the early stages, and it is also applicable to processing lower resolution images, provided that the range of the k-factor is appropriately adjusted. When the recursive input shows detail, the k-factor will rapidly increase to 1.0, but when the detail ends, the k-factor is only allowed to slowly decrease to the required low level. This ensures that the content of the filter is not related to any particular pixel input value (resulting in an unexpected tail value), but conforms to the average value of a certain region, thus maintaining a smooth output result. The k-factor is controlled by a detail sensor (or color gradient sensor). When there is no image detail judged to be important in the surrounding image region, a low k-factor value (close to but not reaching 0) is obtained, while when the surrounding image detail is judged to be important, the k-factor value will gradually approach and eventually reach 1. The detail sensing of the region around the current pixel position can also internally apply recursive filtering and a large number of secondary samples, so the required memory footprint is also very small, only a small fraction, such as 1 / 8 of the row memory. The bit precision at the output of the recursive filter can be much higher than the bit depth at the final output (equal to the input of any subsequent processing stage). To maintain higher precision, a form of dithering rounding or error dispersion rounding can be used.
[0038] Figure 2 This is shown as a de-leveling system 200 in one embodiment of the present invention. The de-leveling system 200 reduces the drift between intermediate recursive outputs and their inputs. Figure 2 As shown, the drift signal is generated by low-pass filtering the difference between the input signal and the intermediate filtered signal. This low-pass filtering is also implemented using a recursive filter. A small fraction (e.g., 0.02) of the low-pass form of the drift difference signal is added to the loop to reduce it to zero. Given the very low frequency of this drift signal, the memory footprint of this drift calculation recursive loop can be reduced by applying a large number of secondary samples (e.g., 8 times the amount). Careful consideration is needed when choosing the values of the aforementioned fractions to maintain the stability of the loop structure within the loop and to achieve a significant "drift" limiting effect.
[0039] Figure 3 This is shown as a recursive filtering system 300 with detail bias in an embodiment of the present invention. The system 300 is capable of separating small details and bypassing the aforementioned recursive filtering process (e.g., Figure 1 or Figure 2 (In the middle). The size of the small details to be separated here is between one pixel and tens of pixels in width and line height. The recursive filtering system 300 that can separate details can be a linear frequency band separation filter, whose pre-filtered signal is low-frequency content, while the residual signal is equal to the remaining high-frequency content.
[0040] Figure 4 This is illustrated as a nonlinear detail separation filter 400 in an embodiment of the present invention. The filter 400 can be based on some form of median filtering. In the case of nonlinear filtering, distortion may exist in both the pre-filtered signal and the residual signal, but these distortions compensate for each other when added back together. This enhancement can preserve spatially small details at the desired size (determined by the design of the pre-filter) without requiring a recursive smoothing filter to fit the original details (instead of only preserving the remainder in the pre-filtered signal), thus maintaining the effectiveness of low-pass filtering in the region surrounding the details. Details can be fully preserved when the "k_residue" factor applied in the split path is the same as the "k" of the recursive smoothing filter. Optionally, when details are found to have passed through the recursive filter, the level of "k_residue" is also increased to 1.0. In this case, the details will be partially distorted and reproduced.
[0041] Figure 5 This is illustrated as a recursive de-grading method 500 according to an embodiment of the present invention. The method 500 includes: receiving a plurality of original pixel values sequentially in step 502; multiplying the original pixel value of the current pixel from the plurality of pixel values by a dynamically changing recursive coefficient in step 504; adding the recursively filtered pixel values from the left and right adjacent pixels of the current pixel, wherein the recursively filtered pixel values of the left and right adjacent pixels of the current pixel are retrieved from a memory buffer storing filtered pixel data of the previous image row in step 506; multiplying the added recursively filtered pixel value by the difference between 1 and the dynamically changing recursive coefficient in step 508; adding the two multiplied values to obtain a filtered pixel value for the current pixel in step 510; writing the filtered pixel value of the current pixel back to the memory buffer in step 512; and displaying the filtered pixel value on a display in step 514. In step 510, adding the two values obtained by multiplication means adding the product value obtained in step 504 to the product value obtained in step 508.
[0042] The following examples illustrate various embodiments of the methods and systems (e.g., machines, devices or other apparatuses) provided in this application.
[0043] 1. A recursive filtering method for image data, comprising:
[0044] Receive several raw pixel values sequentially;
[0045] Multiply the original pixel value of the current pixel among the plurality of pixel values by a dynamically changing recursive coefficient;
[0046] The recursive filtered pixel values from the left and right adjacent pixels of the current pixel are added together, and the recursive filtered pixel values of the left and right adjacent pixels of the current pixel are retrieved from a memory buffer storing the filtered pixel data of the previous image row.
[0047] The sum of the recursive filtered pixel values multiplied by 1 is the difference between the summed values and the dynamically changing recursive coefficients.
[0048] Add the two values obtained by multiplying them together to get the filtered pixel value of the current pixel;
[0049] Write the filtered pixel value of the current pixel back to the memory buffer, and
[0050] The filtered pixel values are displayed on the monitor.
[0051] 2. In one embodiment of the present invention, the left and right adjacent pixels are the two pixels to the left and right of the current pixel, respectively.
[0052] 3. In one embodiment of the present invention, the recursive filtering method for image data further includes low-pass filtering of the recursive filtered pixel values of the left and right adjacent pixels of the current pixel.
[0053] 4. In one embodiment of the present invention, the recursive filtering method for the image data further includes increasing the recursive coefficient of the dynamic change when details are detected, and decreasing the recursive coefficient of the dynamic change when details are not detected.
[0054] 5. In one embodiment of the present invention, the rate of decrease of the recursive coefficient of the dynamic change is less than the rate of increase of the recursive coefficient of the dynamic change.
[0055] 6. In one embodiment of the present invention, the recursive filtering method for the image data further includes dithered rounding of the filtered pixel value of the current pixel before display.
[0056] 7. In one embodiment of the present invention, the recursive filtering method for the image data further includes generating a drift signal, performing low-pass filtering on the generated drift signal, and applying the low-pass filtered drift signal to the plurality of pixel values.
[0057] 8. In one embodiment of the present invention, the recursive filtering method for the image data further includes detecting details containing detail pixels and splitting the recursive filtering of the detail pixels.
[0058] 9. In one embodiment of the invention, the detection of the detail including the detail pixels is performed by a linear band separation filter.
[0059] 10. In one embodiment of the present invention, the detection of the detail including the detail pixel is accomplished by median filtering.
[0060] 11. A second aspect of the present invention also provides a system for recursive filtering of image data, comprising:
[0061] Memory buffer;
[0062] A multiplier is used to multiply the original pixel value of the current pixel from a set of pixel values by a dynamically changing recursive coefficient.
[0063] An adder is used to add the recursively filtered pixel values of the left and right adjacent pixels of the current pixel, wherein the recursively filtered pixel values of the left and right adjacent pixels of the current pixel are retrieved from a memory buffer storing the filtered pixel data of the previous image row;
[0064] The second multiplier is used to multiply the summed recursive filtered pixel value by 1 and the difference between the dynamically changing recursive coefficients.
[0065] The second adder is used to add the two values obtained by multiplication to obtain the filtered pixel value of the current pixel;
[0066] The memory buffer is used to store the filtered pixel value of the current pixel, and
[0067] The output port is used to output the filtered pixel values for display on the monitor.
[0068] 12. In one embodiment of the present invention, the left and right adjacent pixels are the two pixels to the left and right of the current pixel, respectively.
[0069] 13. In one embodiment of the present invention, the system for recursive filtering of image data further includes a low-pass filter for low-pass filtering the recursive filtered pixel values from the left and right adjacent pixels of the current pixel.
[0070] 14. In one embodiment of the present invention, the system for recursive filtering of image data further includes a detail sensor for increasing the recursive coefficient of the dynamic change when detail is detected, and decreasing the recursive coefficient of the dynamic change when no detail is detected.
[0071] 15. In one embodiment of the present invention, the rate at which the recursive coefficient of the dynamic change is reduced is less than the rate at which the recursive coefficient of the dynamic change is increased.
[0072] 16. In one embodiment of the present invention, the system for recursive filtering of image data further includes a dithered rounding unit for dithered rounding of the filtered pixel value of the current pixel before display.
[0073] 17. In one embodiment of the present invention, the system for recursive filtering of image data further includes a recursive filter for generating a drift signal.
[0074] 18. In one embodiment of the invention, the system for recursive filtering of image data further includes a detail sensor for detecting details containing detail pixels and bypassing the recursive filtering of the detail pixels.
[0075] 19. In one embodiment of the invention, the detail sensor includes a linear band separation filter.
[0076] 20. In one embodiment of the invention, the detail sensor includes a median filter.
[0077] Although examples have been described with reference to specific exemplary embodiments or methods, any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the invention. Therefore, the specification and drawings are merely illustrative of the principles and effects of the invention and are not intended to limit the invention. Specific embodiments in which the invention can be implemented are illustrated in the drawings by way of illustration rather than limitation. The invention provides a detailed description of these embodiments to enable those skilled in the art to implement the technical solutions disclosed herein. Other embodiments can be derived from other embodiments, thereby allowing for structural and logical substitutions and changes without departing from the scope of the invention. Therefore, this detailed description should not be construed as limiting the invention, and the scope of the various embodiments is defined only by the claims and the full scope of their equivalents.
[0078] Embodiments of the present invention may be referred to herein, individually and / or collectively, by the term "invention." If such reference is made more than once, it is for convenience only and does not voluntarily limit the scope of this application to any single invention or inventive concept. Therefore, although specific embodiments have been described and illustrated herein, it should be understood that all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed herein should be covered by the claims of the present invention. Combinations of the above embodiments, as well as other embodiments not specifically described herein, will be apparent to those skilled in the art upon reading the above description.
Claims
1. A recursive filtering method for image data, comprising: Receive several raw pixel values sequentially; Multiply the original pixel value of the current pixel from a set of pixel values by a dynamically changing recursive coefficient; The recursive filtered pixel values from the left and right adjacent pixels of the current pixel are added together, and the recursive filtered pixel values of the left and right adjacent pixels of the current pixel are retrieved from a memory buffer storing the filtered pixel data of the previous image row. The sum of the recursive filtered pixel values multiplied by 1 is the difference between the summed values and the dynamically changing recursive coefficients. Add the two values obtained by multiplying them together to get the filtered pixel value of the current pixel; Write the filtered pixel value of the current pixel back to the memory buffer; The filtered pixel value of the current pixel is dithered and rounded before display; as well as The filtered pixel values are displayed on the monitor.
2. The recursive filtering method for image data according to claim 1, characterized in that: The left and right adjacent pixels are the two pixels to the left and right of the current pixel, respectively.
3. The recursive filtering method for image data according to claim 1, characterized in that: It also includes low-pass filtering of the recursively filtered pixel values from the left and right adjacent pixels of the current pixel.
4. The recursive filtering method for image data according to claim 1, characterized in that: It also includes increasing the recursive coefficient of the dynamic change when details are detected, and decreasing the recursive coefficient of the dynamic change when details are not detected.
5. The recursive filtering method for image data according to claim 4, characterized in that: The rate at which the recursive coefficient of the dynamic change decreases is less than the rate at which the recursive coefficient of the dynamic change increases.
6. The recursive filtering method for image data according to claim 1, characterized in that: It also includes generating a drift signal, performing low-pass filtering on the generated drift signal, and applying the low-pass filtered drift signal to the plurality of pixel values.
7. The recursive filtering method for image data according to claim 1, characterized in that: It also includes detecting details containing detail pixels and bypassing the recursive filtering of those detail pixels.
8. The recursive filtering method for image data according to claim 7, characterized in that, include: The detection of the details, including the detailed pixels, is performed by a linear frequency band separation filter.
9. The recursive filtering method for image data according to claim 7, characterized in that: The detection of the details, including the detailed pixels, is accomplished through median filtering.
10. A system for recursive filtering of image data, comprising: Memory buffer; A multiplier is used to multiply the original pixel value of the current pixel from a set of pixel values by a dynamically changing recursive coefficient. An adder is used to add the recursively filtered pixel values of the left and right adjacent pixels of the current pixel, wherein the recursively filtered pixel values of the left and right adjacent pixels of the current pixel are retrieved from a memory buffer storing the filtered pixel data of the previous image row; The second multiplier is used to multiply the summed recursive filtered pixel value by 1 and the difference between the dynamically changing recursive coefficients. The second adder is used to add the two values obtained by multiplication to obtain the filtered pixel value of the current pixel; The memory buffer is used to store the filtered pixel value of the current pixel; A dithering rounder is used to dither round the filtered pixel value of the current pixel before display. as well as The output port is used to output the filtered pixel values for display on the monitor.
11. The system for recursive filtering of image data according to claim 10, characterized in that: The left and right adjacent pixels are the two pixels to the left and right of the current pixel, respectively.
12. The system for recursive filtering of image data according to claim 10, characterized in that: It also includes a low-pass filter for low-pass filtering the recursively filtered pixel values from the left and right adjacent pixels of the current pixel.
13. The system for recursive filtering of image data according to claim 10, characterized in that: It also includes a detail sensor for increasing the recursive coefficient of the dynamic change when detail is detected, and decreasing the recursive coefficient of the dynamic change when no detail is detected.
14. The system for recursive filtering of image data according to claim 13, characterized in that: The rate at which the recursive coefficient of the dynamic change decreases is less than the rate at which the recursive coefficient of the dynamic change increases.
15. The system for recursive filtering of image data according to claim 10, characterized in that: It also includes a recursive filter for generating drift signals.
16. The system for recursive filtering of image data according to claim 10, characterized in that: It also includes a detail sensor for detecting details containing detail pixels and bypassing the recursive filtering of the detail pixels.
17. The system for recursive filtering of image data according to claim 16, characterized in that: The detailed sensor includes a linear frequency band separation filter.
18. The system for recursive filtering of image data according to claim 16, characterized in that: The detail sensor includes a median filter.