Texture filtering method based on tangential flow field and multi-direction one-dimensional side window frame

Abstract

The invention discloses a texture filtering method based on a tangential flow field and a multi-direction one-dimensional side window frame. According to the method, a tangential flow field and a multi-direction one-dimensional side window frame are constructed, firstly, local structure information of each image point in the tangential direction and the normal direction is obtained by calculating local gradient information and structure tensor of an image, then, one-dimensional side window types in different directions are defined in the tangential direction and the normal direction, and one-dimensional side window filtering is carried out respectively, so that one-dimensional side window filtering is carried out; and the holding capacity of the fine structure is improved. According to the method, experiments are carried out with other methods based on Gaussian filtering, bilateral filtering and BM3D filtering method examples, evaluation is carried out on a BSD500 data set by using PSNR and SSIM evaluation indexes, results show that the method provided by the invention is superior to the existing filtering method, the stability of the structure can be kept while the texture is filtered, and the method is suitable for large-scale popularization and application. And the phenomenon that the edge contour is fuzzy is avoided.

Description

Technical field:

[0001] The invention relates to the technical field of computer vision image processing, and in particular to a texture filtering method for a tangential flow field and a multi-directional one-dimensional side window frame. Background technology:

[0002] In the field of computer vision and computer graphics, texture filtering has a wide range of applications, covering image stylization, image enhancement, image segmentation, and HDR tone mapping. Its core task is to remove redundant information such as noise and texture in the image while ensuring that the core structure of the image is not destroyed. Currently, the mainstream texture filtering algorithms are mainly divided into three categories: local-based texture filtering algorithms, global-based texture filtering algorithms, and deep learning-based texture filtering algorithms.

[0003] The local texture filtering algorithm, also known as the kernel-based texture filtering algorithm, uses the local texture and structural information of the image to achieve the filtering purpose. Common algorithms include Gaussian filtering, mean filtering, median filtering, bilateral filtering, and non-local mean filtering. Although these algorithms can effectively remove noise to a certain extent, it is difficult to maintain the stability of the structure and easily cause a large amount of structural information to be blurred. The global texture filtering algorithm, also known as the optimization-based texture filtering algorithm, smoothes the texture details by optimizing the global objective function. Usually, an objective function containing a data term and a regularization term is defined, in which the data term is used to constrain the similarity between the smoothed image and the input image to retain the main features of the image, and the regularization term is used to smooth the texture and maintain the structure. The texture filtering method based on deep learning mainly uses the powerful feature extraction capability of convolutional neural networks (CNN) to achieve texture filtering and enhancement of images.

[0004] In general, kernel-based texture filtering algorithms use local information of the image for filtering. Compared with global texture filtering algorithms, they perform better in filtering speed and visual perception, but they are not effective when facing complex images. Texture filtering algorithms based on global optimization use gradient information as smoothing parameters and process texture details by optimizing the global objective function. Compared with local texture filtering algorithms, they can effectively alleviate artifacts and edge jaggedness. Texture filtering algorithms based on deep learning can learn complex features of images and are effective in texture smoothing and structure preservation. However, they usually require a large amount of training data and are dependent on the quality of the training data. Summary of the invention:

[0005] In view of the problem that the existing texture filtering method cannot filter out the image texture while maintaining the image structure information, thus making the image blurred, the present invention proposes a texture filtering method based on tangential flow field and multi-directional one-dimensional side window framework, and the implementation steps are as follows:

[0006] Step 1: Convert the input image to RGB2YCbCr color space, extract the Y channel image, and then process the Y channel image.

[0007] Step 2: Create a tangent flow field. First, calculate the local gradient information and structure tensor of the image to obtain the local structure information of each image point in the tangent direction and normal direction.

[0008] Step 3: Based on the bilinear interpolation operation of similarity weight, the image points are sampled in the tangent direction and the normal direction to maximize the similarity between the sampling points and the central image point, making the sampling points and the central image point more similar.

[0009] Step 4: Then define one-dimensional edge window types in different directions in the tangent direction and the normal direction, and perform one-dimensional edge window filtering along the tangent direction and the normal direction.

[0010] Step 5: Finally, the tangential flow field and multi-directional one-dimensional edge window framework proposed in the above steps are applied in the tangent direction and normal direction respectively, and the framework is applied to Gaussian filtering, bilateral filtering and BM3D methods to achieve the purpose of edge-preserving filtering.

[0011] The present invention has the following benefits:

[0012] 1. The edge tangential and normal flow fields are introduced to enhance the continuity and integrity of the structure. The edge fracture phenomenon of the structure is reduced by performing local continuity enhancement filtering along the edge tangential and normal flows.

[0013] 2. A bilinear interpolation strategy based on similarity weight is proposed, that is, similarity weight is introduced on the basis of the original bilinear interpolation strategy, so that the sampling points and the central image points are more similar, thereby enhancing the edge preservation performance of the algorithm.

[0014] 3. A one-dimensional edge window filtering based on multi-direction is proposed. One-dimensional edge window filtering is performed according to the direction of each image point and the neighborhood gradient information. The one-dimensional edge window filtering in the tangent direction can effectively protect the real structure of the image from being destroyed; the one-dimensional edge window filtering in the normal direction can further improve the filtering performance of the entire algorithm.

[0015] 4. A framework based on tangential flow field and multi-directional one-dimensional edge window is proposed. This framework is applied to the classic texture filtering algorithm (Gaussian filtering, bilateral filtering, BM3D, etc.), which can effectively improve the ability to preserve fine structures. Description of the drawings:

[0016] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0017] Figure 2 A schematic diagram of a bilinear interpolation strategy based on similarity weights of the present invention;

[0018] Figure 3 It is a schematic diagram of the original BM3D algorithm flow of the present invention;

[0019] Figure 4 A schematic diagram of the search window and image block of the BM3D algorithm of the present invention;

[0020] Figure 5 Schematic diagram of image block stacking of the BM3D algorithm of the present invention. Specific implementation method:

[0021] The specific implementation steps of the present invention are further described in conjunction with the above-mentioned drawings:

[0022] Figure 1 The overall flow diagram of the present invention is as follows. For the input image, the color space conversion is first performed to convert RGB into the YCbCr color space, and then the Y channel image is extracted; the tangent direction and normal direction fields of the image are calculated, and multi-directional one-dimensional edge window filtering is performed. The specific implementation steps are as follows:

[0023] Step 1: According to the Jacobian matrix Calculate the structure tensor, the calculation formula is

[0024]

[0025] Among them, Φ(p i,j ) represents the image point p i,j The neighborhood of the center; and Represent the first-order partial derivatives calculated in the horizontal and vertical directions respectively;

[0026] Step 2: Since the structure tensor can well describe the structural direction information of the image in the local neighborhood, the relationship between the local structural direction and the eigenvector of the structure tensor matrix can be established to estimate the direction of the local area of ​​the image;

[0027] Step 3: J represents a symmetric and semi-positive definite matrix with two eigenvalues ​​λ 1,p ≥λ 2,p , and the corresponding eigenvectors are denoted as η p and p ξ p represents the tangent direction, η p Indicates the normal direction;

[0028] Step 4: Then, in η p and p Edge window types in different directions are defined to further constrain the continuity and stability of the structure, and one-dimensional edge window filtering is performed separately.

[0029] Figure 2 This is a schematic diagram of the bilinear interpolation strategy based on similarity weights of the present invention. Similarity weights are introduced on the basis of the original bilinear interpolation strategy. Image points are sampled in the tangent direction and the normal direction to maximize the similarity between the sampling points and the center point, so that the sampling points and the center image point are more similar, and the edge preservation performance of the algorithm is enhanced. The calculation formula is:

[0030]

[0031] in, Represents the similarity coefficient.

[0032] Applying the tangential flow field and multi-directional one-dimensional edge window framework to Gaussian filtering, bilateral filtering and BM3D algorithms can improve the edge preservation ability of the original algorithms. The calculation formula is as follows:

[0033] Step 1: Apply the tangential flow field and multi-directional one-dimensional edge window framework to Gaussian filtering. The calculation formulas for the tangential direction and the normal direction are:

[0034]

[0035] Where I is the input image, is the filtering result in the tangent direction, is the filtering result in the normal direction, is the spatial kernel function, σ s represents the spatial smoothing coefficient, T = {T 1 ,T 2 ,…,T l} is the one-dimensional edge window set in the tangent direction, S = {S 1 ,S 2 ,…,S l} is a set of one-dimensional edge windows in the normal direction.

[0036] Step 2: Apply the tangential flow field and multi-directional one-dimensional edge window framework to bilateral filtering. The calculation formulas for the tangential direction and the normal direction are:

[0037]

[0038] in, is the spatial kernel function, σ s represents the spatial smoothing coefficient, is the range kernel function, σ r Represents the range smoothing coefficient.

[0039] Step 3: Apply the tangential flow field and multi-directional one-dimensional side window framework in BM3D. The calculation formulas for the tangential direction and the normal direction are:

[0040]

[0041] in, represents the basic estimation result of the tangent direction, Represents the base estimate of the normal direction.

[0042] Step 1 is to apply the tangential flow field and multi-directional one-dimensional side window framework to Gaussian filtering. The original Gaussian filtering is a linear low-pass filtering algorithm suitable for filtering Gaussian noise. The pixel value of the current pixel is replaced by the weighted average value of the pixel values ​​in the neighborhood of the current pixel. The calculation formula is:

[0043]

[0044] in, σ s represents the spatial smoothing coefficient, S represents the image point p i,j The neighborhood of the center.

[0045] Step 2 is to apply the tangential flow field and multi-directional one-dimensional edge window framework to bilateral filtering. The original bilateral filtering is also a linear low-pass filtering algorithm. Compared with Gaussian filtering, it introduces the range function on the basis of the spatial domain function, which improves the edge preservation ability of the algorithm. The calculation formula is:

[0046]

[0047] in, is the normalization factor, expressed as:

[0048]

[0049] in, is the spatial kernel function, σ s represents the spatial smoothing coefficient, is the range kernel function, σ r represents the range smoothing coefficient, S represents the image point p i,j The neighborhood of the center.

[0050] Figure 3 This is a flow chart of the original BM3D algorithm. Step 3 is to apply the tangential flow field and multi-directional one-dimensional side window framework to BM3D. The original BM3D is a filtering algorithm that combines domain transformation and local thinking, including two stages: basic estimation and final estimation:

[0051] Step 1: In the basic estimation stage, first, for each block in the original image, find other blocks similar to the current block and stack these similar blocks into a 3D image block group;

[0052] Step 2: Then, the domain transform is performed on the 3D image block group, noise is suppressed by hard thresholding the transform coefficients, and the estimates of all grouped blocks are restored by inverse transform and put back to their original positions;

[0053] Step 3: Finally, weighted average is performed on the block estimates of all overlapping areas to obtain the basic estimation result of the image;

[0054] Step 4: In the final estimation stage, first, find other blocks similar to the current block in the original image and the basic estimation results to form two groups of 3D image blocks: one group comes from the original image and the other group comes from the basic estimation results;

[0055] Step 5: Then, the two sets of data are domain transformed, and the energy spectrum of the basic estimation result is used as the true energy spectrum. The Wiener filter is applied to the original image, and the inverse transform is applied to the filtered transform coefficients to restore the estimates of all grouped blocks and return them to their original positions.

[0056] Step 6: Finally, the final estimation result of the real image is obtained by weighted averaging all local estimates.

[0057] Compared with the original BM3D algorithm, the BM3D algorithm based on the tangential flow field and multi-directional one-dimensional side window framework omits the final estimation, that is, the basic estimation is performed in the tangential and normal directions respectively. And the algorithm is converted from two-dimensional space to one-dimensional space in block matching and mixed domain forward and inverse transformation, which greatly reduces the time complexity of the algorithm. Since the tangent direction and the normal direction are the same, the following derivation is only based on the tangent direction as an example:

[0058] Step 1: In the block matching stage, with the image point p as the center, select an image block of size 1×k along the tangent direction as the reference block R and an image block of size 1×|n| as the search window, as follows Figure 4 shown.

[0059] Step 1-1: Search one by one in the search window in units of image blocks to obtain |n|-k+1 image blocks M similar to the reference block R.

[0060] Step 1-2: The similarity measure between two image blocks M and R is obtained by the normalized Euclidean distance, and the calculation formula is:

[0061]

[0062] Among them, X R and X M Represent the grayscale values ​​of the corresponding pixels in image blocks R and M respectively.

[0063] Step 2: In the collaborative filtering stage, the similarity d(R,M) between image blocks M and R is sorted by size, the first N image blocks are selected, and stacked to obtain a two-dimensional image block array G, as shown in Figure 5 shown.

[0064] Step 2-1: After obtaining the stacked two-dimensional image block array, perform a two-dimensional linear transformation on it.

[0065] Step 2-2: First, perform one-dimensional DCT transformation on each image block Q in the two-dimensional image block array.

[0066] Step 2-3: Then, perform one-dimensional DCT transform in the second dimension.

[0067] Step 2-4: Finally, use a hard threshold in the transform domain to shrink the sparse coefficients and filter out the small coefficient components that represent the noise:

[0068]

[0069] Where σ represents the standard deviation of the noise; λ hard Represents the threshold set during the hard thresholding stage.

[0070] Step 3: In the aggregation stage, the two-dimensional image block array after hard threshold processing is restored to the spatial domain by two-dimensional inverse transformation, and each image block is restored to its original position in the search window in the original order. Since there will be overlaps between the image blocks, the image point at the same position may generate multiple estimates, and it is necessary to aggregate the image blocks by weighted averaging to obtain the image after the basic estimate. For a given reference block R, the image block Q participating in the basic estimation in its search window is subjected to two-dimensional inverse transformation, and the result of the pixel estimation at each position can be described by the following formula:

[0071]

[0072] Among them, υ(p) and δ(p) represent the numerator and denominator of the basic estimation result of the pixel p respectively; u hard (p) represents the estimated value of pixel x after two-dimensional inverse transformation; ω hard represents the aggregate weight generated in the hard threshold processing stage, and the expression is:

[0073]

[0074] Where N′hard represents the number of non-zero coefficients of the two-dimensional image block array after hard threshold processing. The aggregation process uses this weight distribution method to suppress the "ringing" effect that occurs after hard threshold filtering in the transform domain. The calculation formula for the basic estimation result based on the tangent direction is:

[0075]

[0076] in,

[0077] To verify the accuracy and robustness of the present invention, the present invention is tested on the public BSD500 data set. Table 1 shows the comparison of the Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) evaluation indicators of the original filtering algorithm, the original edge window filtering algorithm and the edge window filtering algorithm of the present invention by applying the framework of the present invention to Gaussian filtering, bilateral filtering and BM3D algorithms respectively. The framework of the present invention is tested on 3-direction, 4-direction, 5-direction and 6-direction edge windows respectively, and the following results are obtained:

[0078] Table 1 Comparison of PSNR evaluation indicators of the original filtering algorithm, the original edge window filtering algorithm and the edge window filtering algorithm of the present invention

[0079]

[0080] The above bold indicates the optimal value. It can be seen that the edge window filtering algorithm of the present invention achieves larger values ​​in both PSNR and SSIM evaluation indicators. Since the original BM3D algorithm itself is already complex enough, the original edge window framework can no longer adapt to the algorithm, so the indicators based on the original edge window BM3D are not measured.

[0081] The specific implementation methods described above further illustrate the purpose, principle and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A texture filtering method based on tangential flow field and multi-directional one-dimensional edge window framework, characterized in that: include: Step 1: Convert the input image to RGB2YCbCr color space, extract the Y channel image, and then process the Y channel image; Step 2: Create a tangent flow field. First, calculate the local gradient information and structure tensor of the image to obtain the local structure information of each image point in the tangent direction and normal direction. Step 3: Based on the similarity weighted bilinear interpolation operation, the image points are sampled in the tangent direction and the normal direction to maximize the similarity between the sampling points and the central image point, making the sampling points and the central image point more similar; Step 4: Then define one-dimensional edge window types in different directions in the tangent direction and the normal direction, and perform one-dimensional edge window filtering along the tangent direction and the normal direction; Step 5: Finally, the tangential flow field and multi-directional one-dimensional edge window framework proposed in the above steps are applied in the tangent direction and normal direction respectively, and the framework is applied to Gaussian filtering, bilateral filtering and BM3D methods to achieve the purpose of edge-preserving filtering.

2. According to the texture filtering method based on tangential flow field and multi-directional one-dimensional side window framework according to claim 1, it is characterized in that the tangent flow field is created in step 2, and the specific steps are as follows: 2.1 According to the Jacobian matrix Calculate the structure tensor, the calculation formula is Among them, Φ(p i,j ) represents the image point p i,j The neighborhood of the center; and Represent the first-order partial derivatives calculated in the horizontal and vertical directions respectively. 2.2 Since the structure tensor can well describe the structural direction information of the image in the local neighborhood, the relationship between the local structure direction and the eigenvector of the structure tensor matrix can be established to estimate the direction of the local area of ​​the image. 2.3J represents a symmetric and semi-positive definite matrix with two eigenvalues ​​λ 1,p ≥λ 2,p , and the corresponding eigenvectors are denoted as η p and p ξ p represents the tangent direction, η p Indicates the normal direction.

3. The texture filtering method based on tangential flow field and multi-directional one-dimensional edge window framework according to claim 1, characterized in that: In step 3, bilinear interpolation operation of similarity weights is performed in the tangent direction and the normal direction. The implementation steps are as follows: The image points are sampled in the tangent direction and the normal direction to maximize the similarity between the sampling points and the center point, so that the sampling points and the center image point are more similar and the edge preservation performance of the algorithm is enhanced. The calculation formula is: in, Represents the similarity coefficient.

4. The texture filtering method based on tangential flow field and multi-directional one-dimensional edge window framework according to claim 1, characterized in that: In step 4, one-dimensional edge window types in different directions are defined in the tangent direction and the normal direction, and one-dimensional edge window filtering is performed along the tangent direction and the normal direction. The implementation steps are as follows: In the tangent direction p and the normal direction η p Define side window types in different directions to further constrain the continuity and stability of the structure, and then p and the normal direction η p One-dimensional side window filtering is performed on each of the above images.

5. The texture filtering method based on tangential flow field and multi-directional one-dimensional edge window framework according to claim 1, characterized in that: In step 5, the tangential flow field and multi-directional one-dimensional edge window framework proposed in the above steps are applied in the tangential direction and the normal direction respectively, and the framework is applied to Gaussian filtering, bilateral filtering and BM3D methods to achieve the purpose of edge-preserving filtering. The implementation steps are as follows: 5.1 The tangential flow field and multi-directional one-dimensional edge window framework are applied to Gaussian filtering. The calculation formulas for the tangential direction and the normal direction are: Where I is the input image, is the filtering result in the tangent direction, is the filtering result in the normal direction, is the spatial kernel function, σ s represents the spatial smoothing coefficient, T={T1,T2,…,T l } is a set of one-dimensional edge windows in the tangent direction, S = {S1, S2, …, S l } is a set of one-dimensional edge windows in the normal direction. 5.2 The tangential flow field and multi-directional one-dimensional side window framework are applied to bilateral filtering. The calculation formulas for the tangential direction and the normal direction are: in, is the spatial kernel function, σ s represents the spatial smoothing coefficient, is the range kernel function, σ r Represents the range smoothing coefficient. 5.3 The tangential flow field and multi-directional one-dimensional side window framework are applied in BM3D. The calculation formulas for the tangential direction and the normal direction are: in, represents the basic estimation result of the tangent direction, Represents the base estimate of the normal direction. Compared with the original BM3D algorithm, the BM3D algorithm based on tangential flow field and multi-directional one-dimensional side window framework omits the final estimation, and converts from two-dimensional space to one-dimensional space in block matching and mixed domain forward and inverse transform, which greatly reduces the time complexity of the algorithm.

Images