High-speed Image Smoothing Method and System with Controllable Intention

Abstract

The present invention relates to an intentionally controllable high-speed image smoothing method and system, the method comprising: step S1: according to the user's intention, using the image content measurement method or the manually specified method, the pixels in the image are divided into pixels to be smoothed q s and the pixel q to be retained r Step S2: Calculate the weight of the pixels in the image according to the user's wishes, that is, assign a very small weight ω(q s ); and for the pixels to be retained, a very high weight ω(q r ); Step S3: construct an objective function of the improved global optimization image smoothing method based on the weights, which is used to perform image smoothing calculations to obtain a smoothed image that meets the user's intentions. The method provided by the present invention can efficiently filter out strong noise with high gradients in the image and effectively maintain weak structures with low gradients at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and particularly relates to a high-speed image smoothing method and system with controllable intention. Background Art

[0002] Image smoothing refers to smoothing the noise content of an image to eliminate its appearance, thereby enhancing the presentation of the main content and facilitating a more concise expression of the image content. Existing image smoothing methods can be divided into image smoothing methods based on local weighting, image smoothing methods based on global optimization, and image smoothing methods based on deep learning. For the image smoothing method based on local weighting, the smoothing result of each pixel output is the weighted average of the color values of all pixels within a window covering the pixel. For the image smoothing method based on global optimization, a globally optimized objective function is established, including two terms: one is to try to maintain the color of the pixel in the original image (referred to as the retention term), and the other is to try to reduce the color value gradient between adjacent pixels (referred to as the penalty term); the color value of the pixel is iteratively changed to optimize the objective function, so that the image noise is smoothed out. For the image smoothing method based on deep learning, deep learning methods are used to smooth the image noise by simulating traditional image smoothing methods.

[0003] Currently, great progress has been made in image smoothing processing, but there are still great deficiencies in high-quality noise elimination and maintaining the image structure. Especially for the processing of maintaining weak structures (with small relevant color gradients) and eliminating strong noise (with large relevant color gradients), it is still a very difficult problem. The relevant discussions are as follows:

[0004] The image smoothing method based on local weighting depends on the selection of the size, shape, and position of the window. An overly large window facilitates smoothing calculations but is not conducive to maintaining the structure because the pixels located in the structure will have their smoothed color gradients reduced due to too many surrounding pixels participating in their smoothing calculations, thus hindering the maintenance of the structure. In particular, weak structures are easily smoothed out; conversely, an overly small window is not conducive to eliminating the noise color because too few pixels participate in its smoothing process, making its noise characteristics still more retained and not easily eliminated. Similar difficult problems also exist for the selection of the shape and position of the window.

[0005] The global optimization image smoothing method can generally maintain the structure of the main content of the image well. Recently, some global optimization methods based on frequency domain calculation can also achieve a very high calculation speed. However, the penalty term in its optimization function takes the color gradient of the pixel as the independent variable, hoping to reduce the color gradient between pixels to achieve the purpose of smoothing out the noise. Here, the gradient is gradually reduced through iterative processing. However, the number of iterative processing is difficult to control. A small number of iterations is not conducive to the elimination of strong noise, because its related color gradient will still be large; on the contrary, a large number of iterations will cause weak structures to be smoothed out, because its color gradient will quickly decrease and become very small.

[0006] The image smoothing method based on deep learning is essentially dependent on the local weighted image smoothing method or the global optimization method, so it is difficult to overcome their difficulties. At the same time, the deep learning method is too dependent on training data, and it is difficult to obtain complete and high-quality training data. Therefore, the image smoothing method based on deep learning is also difficult to ensure high-quality image smoothing processing, and it is also difficult to eliminate strong noise and maintain weak structures.

[0007] In short, how to maintain weak structures while eliminating strong noise during image smoothing is an urgent problem to be solved. Summary of the invention

[0008] In order to solve the above technical problems, the present invention provides an intentionally controllable high-speed image smoothing method and system.

[0009] The technical solution of the present invention is: an intentionally controllable high-speed image smoothing method, comprising:

[0010] Step S1: According to the user's wishes, the pixels in the image are divided into pixels to be smoothed q using a measurement method of the image content or a manually specified method. s and the pixel q to be retained r ;

[0011] Step S2: Calculate the weights of the pixels in the image according to the user's wishes, that is, assign a very small weight ω(q s ); and for the pixels to be retained, a very high weight ω(q r );

[0012] Step S3: constructing an objective function of an improved global optimization image smoothing method based on the weights, which is used to perform image smoothing calculations to obtain a smoothed image that meets the user's intentions.

[0013] Compared with the prior art, the present invention has the following advantages:

[0014] The present invention discloses a high-speed image smoothing method with controllable intention, which uses an image content measurement method or other specified methods to divide image pixels into image pixels to be smoothed and image pixels to be retained; thus, according to the user's wishes, the image pixels can be conveniently weighted for smoothing intentionality and the penalty function input value can be modified. In this way, based on the changed penalty function, in a unified global optimization process, the content to be smoothed can be smoothed according to intention and the content to be retained can be retained, such as increasing the smoothing strength of strong noise (accelerating filtering) while reducing the smoothing strength of weak structure (weakening filtering). Since the present invention can strengthen the smoothing processing of the content to be smoothed by weight, image smoothing can be completed by fewer iterations, such as filtering out strong noise in the image while maintaining weak structure in the image. Compared with the existing image smoothing method, the present invention can more efficiently filter out strong noise with high gradient and effectively maintain weak structure with low gradient, thereby solving the problem that the existing method is difficult to filter out strong noise and maintain weak structure in a small number of iterative calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of an intentionally controllable high-speed image smoothing method in an embodiment of the present invention;

[0016] Figure 2 A schematic diagram showing the comparison of smoothing results between the method of the present invention and the existing method;

[0017] Figure 3 A schematic diagram showing the comparison of smoothing results between the method of the present invention and the existing method;

[0018] Figure 4 The structure block diagram of an intentionally controllable high-speed image smoothing system in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention provides an intentionally controllable high-speed image smoothing method, which can effectively filter out strong noise with high gradient in the image while effectively maintaining weak structure with low gradient.

[0020] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0021] Embodiment 1

[0022] like Figure 1 As shown, an embodiment of the present invention provides an intentionally controllable high-speed image smoothing method, comprising the following steps:

[0023] Step S1: According to the user's wishes, the pixels in the image are divided into pixels to be smoothed q using a measurement method of the image content or a manually specified method. s and the pixel q to be retained r;

[0024] Step S2: Calculate the weights of the pixels in the image according to the user's intention, that is, for the pixels to be smoothed, assign a very small weight ω(q s ) close to 0.0; and for the pixels to be retained, assign a very high weight ω(q r ).

[0025] Step S3: Construct an objective function for an improved global optimization image smoothing method based on the weights for performing image smoothing calculations to obtain a smoothed image that meets the user's intention.

[0026] In one embodiment, the above step S1: According to the user's intention, use a method for measuring the image content or a manually specified method to distinguish the pixels in the image into pixels to be smoothed q s and pixels to be retained q r , specifically including:

[0027] In the embodiments of the present invention, the prior art: interval gradient (IG) is used to distinguish the image content to be smoothed and the image content to be retained in the image, or the user can select other methods for distinction. When the user uses the interval gradient method, the scale parameter δ s of the noise to be smoothed can be simply set to estimate the probability γ(q s ) of the pixel q s to be smoothed or the probability γ(q r ) of the pixel q r to be retained for subsequent calculation of the gradient weights of the pixels.

[0028] In one embodiment, the above step S2: Calculate the weights of the pixels in the image according to the user's intention, that is, for the pixels to be smoothed, assign a very small weight ω(q s ) close to 0.0; and for the pixels to be retained, assign a very high weight ω(q r ), specifically including:

[0029] The embodiments of the present invention design a weight calculation formula according to the bilateral texture filtering method (BTF):

[0030]

[0031]

[0032] Among them, ω(q r ) represents the weight of the pixel q r to be retained; ω(q s ) represents the weight of the pixel q s to be smoothed, δ sis the scale parameter set in step S1, which is used to control the sharpness of the weight conversion for the transition from the structure edge to the detail area.

[0033] In one embodiment, the above step S3: constructing an objective function of an improved globally optimized image smoothing method based on weights for performing image smoothing calculation to obtain a smoothed image that meets the user's intention, specifically includes:

[0034] Step S31: constructing an objective function E(u, f) of an improved globally optimized image smoothing method based on weights:

[0035]

[0036]

[0037]

[0038] where f represents the input image, u represents the output smoothed image, s represents the position of the pixel, (u s -f s ) 2 represents the holding term, u s represents the color value of the output image of the pixel located at s, f s represents the color value of the input image of the pixel located at s, ∑ *∈x,y φ p (g *,s ) represents the penalty term, λ represents a parameter for adjusting the effect of the penalty term, p represents the norm power of the penalty function, ∈ represents a small constant, and g represents the input value of the penalty function. represents the gradient value of u along the X / Y axis direction, and ω *,s represents ω(q r ) or ω(q s );

[0039] The present invention improves the existing globally optimized image smoothing method by changing the independent variable of the penalty function therein from the original gradient of the pixel to the gradient strengthened by the weight of the pixel, so that image smoothing can be completed with fewer iterations, such as filtering out strong noise in the image while maintaining weak structures in the image.

[0040] Step S32: performing high-speed image smoothing calculation according to the objective function E(u, f) to obtain a smoothed image that meets the user's intention.

[0041] The embodiment of the present invention adopts the existing global objective function solving technology, that is, the real-time image smoothing algorithm (ILS) based on iterative least squares, to perform high-speed image smoothing calculation, which accelerates the solution of the objective function through frequency domain conversion. Finally, a smoothed image that meets the user's intention is obtained through multiple iterations.

[0042] In order to verify the effectiveness of the method of the present invention, Figure 2 and Figure 3 The image smoothing results of the method of the present invention and the existing methods (BTF, IG, EGF, EAP, ILS, GFES, STDN, Easy2Hard, DeepFSPIS and CSGIS-Net) are compared. It can be seen from these comparison figures that the image smoothing results of the present invention are significantly better than those obtained by the existing image smoothing methods. It is difficult for the existing methods to simultaneously filter out the strong noise of high gradients and maintain the weak structure of low gradients, as shown in the enlarged box at the bottom of the figure, especially where the arrows point.

[0043] Table 1 shows the differences between the method of the present invention and the methods of BTF, IG, EGF, EAP, ILS, GFES, STDN, Easy2Hard, DeepFSPIS and CSGIS-Net in processing Figure 2 and Figure 3 The time cost of smoothing the result is Figure 2 and Figure 3 The pixel resolutions are 615×461 pixels and 919×663 pixels respectively. Obviously, the processing speed of the method of the present invention is significantly better than the existing method, and can also be increased by 0.710 / 0.273=2.6 times compared with the faster deep learning method DeepFSPIS method.

[0044] Table 1 Processing time (seconds) of the present invention compared with the existing method

[0045] Method Figure 2 Figure 3 BTF 10.459 18.046 IG 1.657 3.382 EGF 12.325 14.284 EAP 10.421 23.844 ILS 0.909 0.818 GFES 7.491 14.544 STDN 136.395 299.463 Easy2Hard 1.074 1.595 DeepFSPIS 0.710 0.963 CSGIS-Net 1.084 1.586 The method of the present invention 0.273 0.569

[0046] The present invention discloses a high-speed image smoothing method with controllable intention, which uses an image content measurement method or other specified methods to divide image pixels into image pixels to be smoothed and image pixels to be retained; thus, according to the user's wishes, the image pixels can be conveniently weighted for smoothing intentionality and the penalty function input value can be modified. In this way, based on the changed penalty function, in a unified global optimization process, the content to be smoothed can be smoothed according to intention and the content to be retained can be retained, such as increasing the smoothing strength of strong noise (accelerating filtering) while reducing the smoothing strength of weak structure (weakening filtering). Since the present invention can strengthen the smoothing processing of the content to be smoothed by weight, image smoothing can be completed by fewer iterations, such as filtering out strong noise in the image while maintaining weak structure in the image. Compared with the existing image smoothing method, the present invention can more efficiently filter out strong noise with high gradient and effectively maintain weak structure with low gradient, thereby solving the problem that the existing method is difficult to filter out strong noise and maintain weak structure in a small number of iterative calculations.

[0047] Embodiment 2

[0048] likeFigure 4 As shown in the figure, an embodiment of the present invention provides a high-speed image smoothing system with controllable intention, including the following modules:

[0049] A module 41 for determining image smoothing and retaining pixels, which is used to distinguish pixels in the image into pixels to be smoothed q s and pixels to be retained q r ;

[0050] A module 42 for calculating image pixel weights, which is used to calculate the weights of pixels in the image according to the user's intention, that is, for pixels to be smoothed, a very small weight ω(q s ) close to 0.0 is assigned; while for pixels to be retained, a very high weight ω(q r ) close to 1.0 is assigned;

[0051] An image smoothing module 43, which constructs an objective function of an improved global optimization image smoothing method based on weights and is used to perform image smoothing calculations to obtain a smoothed image that meets the user's intention.

[0052] The above embodiments are provided only for the purpose of describing the present invention, and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims. All equivalent substitutions and modifications made without departing from the spirit and principles of the present invention shall be covered within the scope of the present invention.

Claims

1. A high-speed image smoothing method with controllable intention, characterized in that, Including: Step S1: According to the user's wishes, the pixels in the image are divided into pixels to be smoothed using a measurement method of the image content or a manually specified method. and pixels to be retained ; Step S2: Calculate the weights of the pixels in the image according to the user's intention, that is, assign a very small weight close to 0.0 to the pixel to be smoothed ; while assign a very high weight close to 1.0 to the pixel to be retained ; Step S3: Construct an objective function of an improved globally optimized image smoothing method based on the weights, which is used for image smoothing calculation to obtain a smoothed image that meets the user's intention. Specifically, it includes: Step S31: Construct an objective function of an improved globally optimized image smoothing method based on the weights : Among them, f represents the input image, u represents the output smoothed image, and s represents the position of the pixel. represents the retention term. represents the output image color value of the pixel located at s. represents the input image color value of the pixel located at s. represents the penalty term. represents a parameter for adjusting the effect of the penalty term, p represents the norm power of the penalty function. represents a small constant. represents the input value of the penalty function. ( ) represents the gradient value of u in the X / Y axis direction. represents or ; Step S32: According to the objective function perform high-speed image smoothing calculation to obtain a smoothed image that meets the user's intention.

2. A high-speed image smoothing system with controllable intention, characterized in that, Including the following modules: The module for determining image smoothing and retaining pixels is used to distinguish pixels in the image as pixels to be smoothed according to the user's wishes, using image content measurement methods or manually specified methods and pixels to be retained ; The image pixel weight calculation module is used to calculate the weight of the pixels in the image according to the user's wishes, that is, for the pixels to be smoothed , with a small weight close to 0.0 ; and for the pixels to be retained , with a very high weight close to 1.0 ; An image smoothing module that constructs an objective function of an improved globally optimized image smoothing method based on the weights, which is used for image smoothing calculation to obtain a smoothed image that meets the user's intention. Specifically, it includes: Step S31: Construct an objective function for an improved globally optimized image smoothing method based on the weights : Among them, f represents the input image, u represents the output smoothed image, and s represents the position of the pixel. represents the retention term. represents the output image color value of the pixel located at s. represents the input image color value of the pixel located at s. represents the penalty term. represents a parameter that adjusts the effect of the penalty term, and p represents the norm power of the penalty function. represents a small constant. represents the input value of the penalty function. ( ) represents the gradient value of u in the X / Y axis direction. represents or ; Step S32: According to the objective function perform high-speed image smoothing calculation to obtain a smoothed image that meets the user's intention.

Images