Image enhancement method and system based on guided filtering

Abstract

The invention provides an image enhancement method and system based on guided filtering, and the method comprises the steps: carrying out the line caching of an input image p in each filtering window, and forming the convolution of nn; calculating a mean value in the convolution kernel and a mean value of squares, and calculating a corresponding variance; calculating a filter coefficient sum based on the variance and a preset weighting coefficient; respectively carrying out average filtering processing on the sums of the plurality of filtering windows to obtain sums; and performing guided filtering processing on the input image p according to the filtering coefficient mean value and the filtering coefficient mean value. According to the invention, guided filtering enhancement processing is carried out on the input image on the SOC chip based on the line cache, and the problem of large noise in image display and image preprocessing in the image processing field is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to an image enhancement method and system based on guided filtering. Background Technology

[0002] A System-on-a-Chip (SoC) is a fully programmable chip that integrates a dual-core ARM Cortex-A9 processor with a Xilinx-7 series FPGA based on a 28nm process. This chip combines the rich system resources of ARM with the flexibility, real-time performance, and scalability of FPGAs.

[0003] Developing image processing algorithms on a SOC can further improve the computation speed of the algorithms. Combined with the excellent real-time performance of correlation filtering algorithms, developing algorithms using a SOC can further enhance the performance of the algorithms.

[0004] Guided filtering has a wide range of applications in image processing. Its core advantage lies in its ability to remove image noise while effectively preserving image details and edge information. It has significant technical value in areas such as image smoothing, image dehazing, and depth map analysis. Summary of the Invention

[0005] This invention addresses the technical problems existing in the prior art by providing an image enhancement method and system based on guided filtering, thus solving the problem of high noise in image display and image preprocessing in the field of image processing.

[0006] According to a first aspect of the present invention, a guided filtering-based image enhancement method is provided, comprising: In each filter window Internally, the input image p is buffered on the SOC chip to form n. A convolution of n, where n is the size of the convolution kernel; Calculate the mean value within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ; According to the mean and the mean of the squares Based on the variance formula Calculate the corresponding variance ; Based on the variance and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients ; Filter coefficients for multiple filter windows and filter coefficients The mean of the filter coefficients is obtained by performing mean filtering on each sample. and the mean of the filter coefficients ; Based on the mean of the filter coefficients and the mean of the filter coefficients Guided filtering is applied to the input image p: Where q is the output image after guided filtering, and I is the guided image. .

[0007] Based on the above technical solution, the present invention can also be improved as follows.

[0008] Optionally, the n n convolution kernel is 3 3 convolution kernels, as described in each filtering window Internally, the input image p is buffered on the SOC chip. include: Configure a two-level FIFO buffer on the SOC chip, and use the current row of the input image p as 3. The first row of the 3-kernel convolution takes the current row of the input image p after passing it through the first-level FIFO buffer as the 3-kernel convolution. The second row of the 3-kernel convolution takes the current row of the input image p, passes it through the first-level FIFO buffer, and then through the second-level FIFO buffer, as the result of the 3-kernel convolution. The third row of 3 convolution kernels yields 3 Nine numbers from a 3x3 convolution.

[0009] Optionally, the mean value within the convolution kernel is calculated using the mean filtering algorithm. and the mean of the squares ,include: The nine numbers output from the line buffer are used as a sliding window for convolution, and the mean value within the convolution kernel is calculated. and the mean of the squares .

[0010] Optionally, based on the variance and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients ,include:

[0011]

[0012]

[0013] in, For filtering window The guide image inside, To guide the image and input image covariance, To guide the image variance To preset the weighting coefficients, For filtering window Inside The mean, For filtering window Guide image inside The mean.

[0014] According to a second aspect of the present invention, a guided filtering-based image enhancement system is provided, comprising: The first processing module is used in each filtering window. Internally, the input image p is buffered on the SOC chip to form n. A convolution of n, where n is the size of the convolution kernel.

[0015] The first calculation module is used to calculate the mean within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ; and according to the mean and the mean of the squares Based on the variance formula Calculate the corresponding variance ; The second calculation module is used to calculate based on the variance. and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients ; The second processing module is used to process the filtering coefficients of multiple filtering windows. and filter coefficients The mean of the filter coefficients is obtained by performing mean filtering on each sample. and the mean of the filter coefficients ; The guided filtering module is used to filter based on the mean of the filter coefficients. and the mean of the filter coefficients Guided filtering is applied to the input image p: Where q is the output image after guided filtering, and I is the guided image. .

[0016] Optionally, the first processing module is specifically used for: The current row of the input image p is used as 3. The first row of the 3-kernel convolution takes the current row of the input image p after passing it through the first-level FIFO buffer as the 3-kernel convolution. The second row of the 3-kernel convolution takes the current row of the input image p, passes it through the first-level FIFO buffer, and then through the second-level FIFO buffer, as the result of the 3-kernel convolution. The third row of 3 convolution kernels yields 3 Nine numbers from a 3x3 convolution; The SOC chip is equipped with a two-level FIFO cache.

[0017] Optionally, the first calculation module is used to calculate the mean within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ,include: The nine numbers output from the line buffer are used as a sliding window for convolution, and the mean value within the convolution kernel is calculated. and the mean of the squares .

[0018] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, the processor being configured to implement the steps of a guided filtering-based image enhancement method when executing a computer program stored in the memory.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer management program, when executed by a processor, implements the steps of a guided filtering-based image enhancement method.

[0020] This invention provides an image enhancement method and system based on guided filtering, which performs guided filtering enhancement processing on the input image based on the row buffer on the SOC chip, solving the problem of high noise in image display and image preprocessing in the field of image processing. Attached Figure Description

[0021] Figure 1 A flowchart of an image enhancement method based on guided filtering is provided in one embodiment of the present invention; Figure 2 This is a schematic diagram of row caching of an image using a two-level FIFO buffer; Figure 3 A structural block diagram of an image enhancement system based on guided filtering is provided in one embodiment of the present invention; Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention; Figure 5This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0023] Before describing the guided filtering-based image enhancement method provided in the embodiments of the present invention, guided filtering image enhancement will be introduced first.

[0024] Guided filtering requires a guide image I and an input image p to obtain an output image q. The guide image needs to be set according to the specific situation. In the scenario required by the project, the guide image I is set to be the same as the input image p.

[0025] Guided filtering is based on a fundamental assumption: the filtered output is a local linear transformation of the guided image. Therefore, for any position k in the image, the corresponding filter window is... ,have: , (1).

[0026] In edge-preserving smoothing algorithms, the gradient of the filtered output image is guaranteed to be as consistent as possible with the guiding image I. Given an input image p, to ensure that the output q is locally approximately the same as p, the optimization objective is: (2); To avoid ill-conditioned phenomena during calculation, a regularization term is added to the above formula, thus: (3).

[0027] The effectiveness of guided filtering depends on the guided filtering algorithm. From equation (1), we know that the effectiveness of guided filtering is related to the filter coefficients. and Therefore, based on formula (3), the least squares method is used to solve the problem. Specifically, taking the partial derivative of formula (3) yields: (4); (5).

[0028] Separately =0, =0, solve for the problem, and simplify to get: (6); (7).

[0029] Figure 1 A flowchart of an image enhancement method based on guided filtering is provided as an embodiment of the present invention, as follows: Figure 1 As shown, the method includes: Step 1, in each filtering window Internally, the input image p is buffered on the SOC chip to form n. A convolution of n, where n is the size of the convolution kernel.

[0030] Understandably, the input image p is buffered in rows to form n. In this embodiment of the invention, a 3-order convolution with n is used. 3 convolutions.

[0031] In this embodiment of the invention, the image row buffer is implemented using FIFO, such as... Figure 2 As shown, this embodiment of the invention configures a two-level FIFO on the SOC, using the current row of the input image p as the 3rd row input. The first row of the 3-kernel convolution takes the current row of the input image p after passing it through the first-level FIFO buffer as the 3-kernel convolution. The second row of the 3-kernel convolution takes the current row of the input image p, passes it through the first-level FIFO buffer, and then through the second-level FIFO buffer, as the result of the 3-kernel convolution. The third row of 3 convolution kernels yields 3 Nine numbers from a 3x3 convolution.

[0032] Step 2: Calculate the mean within the convolution kernel based on the mean filtering algorithm. and the mean of the squares .

[0033] Understandably, after performing line buffering on the input image p, we obtain 9 numbers. We then use these 9 numbers from the line buffer as a sliding window to perform convolution and calculate the mean. mean of the sum of squares .

[0034] Step 3, based on the mean and the mean of the squares Based on the variance formula Calculate the corresponding variance .

[0035] Step 4, based on the variance and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients .

[0036] Among them, the mean calculated in step 2 mean of squares and variance The filter coefficients are calculated based on the aforementioned formulas (6) and (7). Calculate the filter coefficients .

[0037] In this embodiment of the invention, the guiding image I is the same as the input image p, therefore: (6); (7).

[0038] Step 5: Filter coefficients for multiple filter windows. and filter coefficients The mean of the filter coefficients is obtained by performing mean filtering on each sample. and the mean of the filter coefficients .

[0039] Understandably, for multiple filtering windows The filter coefficients calculated from multiple filter windows and filter coefficients The mean of the filter coefficients is obtained by performing mean filtering on each sample. and the mean of the filter coefficients , where is the filtering coefficient of the guided filtering algorithm.

[0040] Step 6, based on the mean of the filter coefficients and the mean of the filter coefficients Guided filtering is applied to the input image p: Where q is the output image after guided filtering, and I is the guided image. .

[0041] Understandably, the mean of the filter coefficients calculated in step 5... and the mean of the filter coefficients Based on the aforementioned formula (1), the input image p is subjected to guided filtering to obtain the output image after guided filtering: ,in, .

[0042] See Figure 3 This invention provides an image enhancement system based on guided filtering, comprising: The first processing module 301 is used to process each filtering window. Internally, the input image p is buffered on the SOC chip to form n. A convolution of n, where n is the size of the convolution kernel.

[0043] The first calculation module 302 is used to calculate the mean value within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ; and according to the mean and the mean of the squares Based on the variance formula Calculate the corresponding variance ; The second calculation module 303 is used to calculate based on the variance. and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients ; The second processing module 304 is used to process the filtering coefficients of multiple filtering windows. and filter coefficients The mean of the filter coefficients is obtained by performing mean filtering on each sample. and the mean of the filter coefficients ; Guided filtering module 305 is used to filter based on the mean of the filtering coefficients. and the mean of the filter coefficients Guided filtering is applied to the input image p: Where q is the output image after guided filtering, and I is the guided image. .

[0044] The first processing module 301 is specifically used for: The current row of the input image p is used as 3. The first row of the 3-kernel convolution takes the current row of the input image p after passing it through the first-level FIFO buffer as the 3-kernel convolution. The second row of the 3-kernel convolution takes the current row of the input image p, passes it through the first-level FIFO buffer, and then through the second-level FIFO buffer, as the result of the 3-kernel convolution. The third row of 3 convolution kernels yields 3 Nine numbers from a 3x3 convolution; the SOC chip is equipped with a two-level FIFO cache.

[0045] The first calculation module 302 is used to calculate the mean within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ,include: The nine numbers output from the line buffer are used as a sliding window for convolution, and the mean value within the convolution kernel is calculated. and the mean of the squares .

[0046] It is understood that the image enhancement system based on guided filtering provided by the present invention corresponds to the image enhancement method based on guided filtering provided in the foregoing embodiments. The relevant technical features of the image enhancement system based on guided filtering can be referred to the relevant technical features of the image enhancement method based on guided filtering, and will not be repeated here.

[0047] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, it implements the steps of an image enhancement method based on guided filtering.

[0048] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 5 As shown, this embodiment provides a computer-readable storage medium 500 on which a computer program 511 is stored. When the computer program 511 is executed by a processor, it implements the steps of an image enhancement method based on guided filtering.

[0049] This invention provides an image enhancement method and system based on guided filtering. It uses a two-level FIFO to implement row buffer on a SOC chip and performs guided filtering enhancement processing on the input image based on the row buffer. This solves the problem of high noise in image display and image preprocessing in the field of image processing. It realizes the full pipelined processing of the algorithm and the maximum processing clock can support a single frame image processing latency of 350M microseconds.

[0050] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0051] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0052] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0053] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0054] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0055] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0056] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An image enhancement method based on guided filtering, characterized in that, include: In each filter window Internally, the input image p is buffered on the SOC chip to form n. A convolution of n, where n is the size of the convolution kernel; Calculate the mean value within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ; According to the mean and the mean of the squares Based on the variance formula Calculate the corresponding variance ; Based on the variance and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients ; Filtering coefficients for multiple filtering windows and filter coefficients The mean of the filter coefficients is obtained by performing mean filtering on each sample. and the mean of the filter coefficients ; Based on the mean of the filter coefficients and the mean of the filter coefficients Guided filtering is applied to the input image p: Where q is the output image after guided filtering, and I is the guided image. .

2. The image enhancement method based on guided filtering according to claim 1, characterized in that, The n n convolution kernel is 3 3 convolution kernels, as described in each filtering window Internally, the input image p is buffered on the SOC chip, including: Configure a two-level FIFO buffer on the SOC chip, and use the current row of the input image p as 3. The first row of the 3-kernel convolution takes the current row of the input image p after passing it through the first-level FIFO buffer as the 3-kernel convolution. The second row of the 3-kernel convolution takes the current row of the input image p, passes it through the first-level FIFO buffer, and then through the second-level FIFO buffer, as the result of the 3-kernel convolution. The third row of 3 convolution kernels yields 3 Nine numbers from a 3x3 convolution.

3. The image enhancement method based on guided filtering according to claim 2, characterized in that, The mean value within the convolution kernel is calculated using the mean filtering algorithm. and the mean of the squares ,include: The nine numbers output from the line buffer are used as a sliding window for convolution, and the mean value within the convolution kernel is calculated. and the mean of the squares .

4. The image enhancement method based on guided filtering according to claim 1, characterized in that, Based on the variance and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients ,include: in, For filtering window The guide image inside, To guide the image and input image covariance, To guide the image variance To preset the weighting coefficients, For filtering window Inside The mean, For filtering window Guide image inside The mean.

5. An image enhancement system based on guided filtering, characterized in that, include: The first processing module is used in each filtering window. Internally, the input image p is buffered on the SOC chip to form n. A convolution of n, where n is the size of the convolution kernel; The first calculation module is used to calculate the mean within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ; and according to the mean and the mean of the squares Based on the variance formula Calculate the corresponding variance ; The second calculation module is used to calculate based on the variance. and preset weighting coefficients Calculate the filter coefficients and based on the mean and filter coefficients Calculate the filter coefficients ; The second processing module is used to process the filtering coefficients of multiple filtering windows. and filter coefficients The mean of the filter coefficients is obtained by performing mean filtering on each sample. and the mean of the filter coefficients ; The guided filtering module is used to filter based on the mean of the filter coefficients. and the mean of the filter coefficients Guided filtering is applied to the input image p: Where q is the output image after guided filtering, and I is the guided image. .

6. The image enhancement system based on guided filtering according to claim 5, characterized in that, The first processing module is specifically used for: The current row of the input image p is used as 3. The first row of the 3-kernel convolution takes the current row of the input image p after passing it through the first-level FIFO buffer as the 3-kernel convolution. The second row of the 3-kernel convolution takes the current row of the input image p, passes it through the first-level FIFO buffer, and then through the second-level FIFO buffer, as the result of the 3-kernel convolution. The third row of 3 convolution kernels yields 3 Nine numbers from a 3x3 convolution; The SOC chip is equipped with a two-level FIFO cache.

7. The image enhancement system based on guided filtering according to claim 6, characterized in that, The first calculation module is used to calculate the mean within the convolution kernel based on the mean filtering algorithm. and the mean of the squares ,include: The nine numbers output from the line buffer are used as a sliding window for convolution, and the mean value within the convolution kernel is calculated. and the mean of the squares .

8. An electronic device, characterized in that, It includes a memory and a processor, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the guided filtering-based image enhancement method according to any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the guided filtering-based image enhancement method according to any one of claims 1-4.

Images