A method for image sharpening by suppressing black and white edges

Abstract

The present invention discloses an image sharpening method for suppressing black and white edges, and belongs to the field of image processing. The method comprises: dividing the image to be sharpened into a plurality of macroblocks of the same size, calculating and obtaining the edge information of the current macroblock, and then screening out the target edge macroblock according to the edge information. Each pixel in the target edge macroblock is traversed, the main pixel line and the reference pixel line are determined within the traversal window range, and the pixel value of the current pixel after sharpening is obtained according to the main pixel line and the reference pixel line. The method provided by the present invention takes the main pixel line as the center in the target edge macroblock, blurs the current pixel in combination with the reference pixel line, and then performs sharpening calculation on the current pixel. The method not only effectively avoids the influence of small noise on sharpening, but also suppresses the phenomenon of black and white edges appearing at the boundary between light and dark in the traditional method.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image sharpening method for suppressing black and white edges. Background Art

[0002] In the process of image shooting and processing, such as smoothing and denoising, the boundaries and contours of the image often become blurred. In order to reduce the impact of such adverse effects, it is necessary to use image sharpening technology to make the edges of the image clear. Therefore, image sharpening is an important part of the image processing process and an effective method to enhance image quality. It needs to be studied and improved continuously.

[0003] Currently, a good and commonly used sharpening method is the USM (unsharpen mask) method, which first blurs the original image, then subtracts the blurred image from the original image to obtain the edge residual, multiplies the residual by a coefficient (the coefficient is the sharpening strength), and then adds it to the original image to obtain the sharpening result. In particular, the general method of blurring the original image is to use the average value of all pixels in the window centered on the current pixel as the pixel value of the current pixel after blurring. As a result, the USM method is easily disturbed by small noise, and it is easy to overshoot and undershoot at the boundary between light and dark. This is manifested in the image as "white edges" and "black edges" at the edges of black and white, resulting in a significant decrease in the visual quality of the output image. Summary of the invention

[0004] Since the existing methods have the above problems, the embodiments of the present invention propose an image sharpening method for suppressing black and white edges. Specifically, the embodiments of the present invention provide the following technical methods:

[0005] Step 101: divide the image to be sharpened into a plurality of macroblocks of the same size, and calculate and obtain edge information of the current macroblock.

[0006] The edge information includes the maximum eigenvalue, the minimum eigenvalue, and the direction of the eigenvector.

[0007] Optionally, the calculating to obtain edge information of the current macroblock includes:

[0008] Get the gradient map I of the pixel points in the current macroblock x , I y :

[0009]

[0010] .

[0011] Then calculate the covariance matrix for each pixel:

[0012]

[0013] in, ; m means the size of the target edge macroblock is m×m; boxblur is the box filter function.

[0014] Compute the eigenvalues ​​and eigenvectors of a matrix, where the eigenvalues ​​satisfy the equation:

[0015]

[0016] but, ,

[0017] Among them, λ 0 is the maximum eigenvalue, λ 1 is the minimum eigenvalue.

[0018] Optionally, the direction of the feature vector is represented by a rotation angle θ. The method for calculating the rotation angle θ is as follows:

[0019]

[0020]

[0021] in, ; m means the size of the target edge macroblock is m×m; boxblur is the box filter function.

[0022] Step 102: Mark target edge macroblocks and non-target edge macroblocks according to edge information of the current macroblock.

[0023] Optionally, marking the target edge macroblock and the non-target edge macroblock according to the edge information of the current macroblock includes:

[0024] It is determined whether a preset first determination condition is met according to the edge information of the current macroblock. If so, the current macroblock is marked as a target edge macroblock; if not, the current macroblock is marked as a non-target edge macroblock.

[0025] The preset first judgment condition is that the minimum eigenvalue of the current macroblock is less than T1 and the maximum eigenvalue is greater than T2, T 2 >T 1 .

[0026] Step 103, traverse each pixel point in the target edge macroblock and execute steps 104-105 for sharpening, and sharpen the pixel points in the non-target edge macroblock using the existing conventional sharpening technology.

[0027] The existing conventional sharpening techniques include but are not limited to: USM method, Sobel operator, and Laplace operator.

[0028] Step 104, determining the main pixel line where the current pixel point is located within the sharpening window range, and determining the reference pixel line of the main pixel line.

[0029] Optionally, determining the main pixel line where the current pixel point is located within the sharpening window range includes:

[0030] According to the current pixel point and the direction θ of the feature vector obtained in step 101, a main pixel line centered on the current pixel point is determined within the sharpening window.

[0031] Optionally, determining the reference pixel line of the main pixel line includes:

[0032] T adjacent pre-selected reference pixel lines are respectively selected on both sides of the main pixel line. The pre-selected reference pixel lines are all parallel to the main pixel line, and the pre-selected reference lines are all within the sharpening window.

[0033] The average pixel value of each pixel line is calculated for the preselected reference pixel line and the main pixel line.

[0034] The average pixel value of the preselected reference pixel line is subtracted from the average pixel value of the main pixel line to obtain an average pixel value difference.

[0035] According to the average pixel value difference, it is determined whether a preset second determination condition is met. If so, the preselected reference pixel line corresponding to the average pixel value difference is used as the reference pixel line; otherwise, it is not used as the reference pixel line.

[0036] The preset second determination condition is that the average pixel value difference is less than the JND curve threshold.

[0037] Step 105, obtaining the sharpened pixel value of the current pixel point according to the main pixel line and the reference pixel line.

[0038] Optionally, obtaining a sharpened pixel value of a current pixel point according to the main pixel line and the reference pixel line includes:

[0039] The average pixel value of the reference pixel line is averaged with the average pixel value of the main pixel line to obtain the average pixel value of the pixel point.

[0040] According to the pixel value of the current pixel point and the average pixel value of the pixel points, a difference between the current pixel point and the average pixel value of the pixel points is calculated.

[0041] The difference between the current pixel point and the average pixel value of the pixel points is multiplied by a sharpening coefficient and superimposed on the pixel value of the current pixel point to obtain the sharpened pixel value of the current pixel point.

[0042] It can be seen from the above technical solution that the present invention uses the rotation angle to represent the direction of the feature vector. In the calculation process, the traditional more complex trigonometric function algorithm is not used, but the principle of fractional optimal consistency approximation is used to establish the calculation method of the rotation angle, which greatly reduces the calculation complexity and improves the calculation efficiency. In addition, the present invention takes the main pixel line as the center in the target edge macroblock and blurs the current pixel in combination with the reference pixel line, which not only effectively avoids the influence of small noise on sharpening, but also avoids the phenomenon of black and white edges appearing at the boundary between light and dark in the traditional method. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 The present invention is a flowchart of an image sharpening method for suppressing black and white edges.

[0045] Figure 2 It is a schematic diagram to determine the main pixel line according to the current pixel point and the direction of the feature vector.

[0046] Figure 3 It is a schematic diagram of the main pixel line and the pre-selected reference pixel line.

[0047] Figure 4 It is a schematic diagram of the JND threshold curve. DETAILED DESCRIPTION

[0048] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0049] Figure 1 A schematic diagram of a process for image sharpening method for suppressing black and white edges is shown, such as Figure 1 As shown, an image sharpening method for suppressing black and white edges provided by an embodiment of the present invention specifically includes the following contents:

[0050] Step 201: divide the image to be sharpened into a plurality of macroblocks of the same size, and calculate and obtain edge information of the current macroblock.

[0051] In this step, the image to be sharpened can be a picture or a video frame in a video. The size of the divided macroblocks is set according to actual conditions.

[0052] The edge information includes the maximum eigenvalue, the minimum eigenvalue, and the direction of the eigenvector.

[0053] Optionally, the calculating to obtain edge information of the current macroblock includes:

[0054] Get the gradient map I of the pixel points in the current macroblock x , I y :

[0055]

[0056] .

[0057] Then calculate the covariance matrix for each pixel:

[0058]

[0059] in, ; m means the size of the target edge macroblock is m×m; boxblur is the box filter function.

[0060] Compute the eigenvalues ​​and eigenvectors of a matrix, where the eigenvalues ​​satisfy the equation:

[0061]

[0062] but, ,

[0063] Among them, λ 0 is the maximum eigenvalue, λ 1 is the minimum eigenvalue.

[0064] In this embodiment, the direction of the eigenvector is represented by the rotation angle θ. Generally, complex trigonometric functions are used to calculate the rotation angle. This embodiment simplifies the calculation of the rotation angle and uses the principle of fractional best consistency approximation to establish a calculation method for the rotation angle, which effectively reduces the calculation complexity and improves the calculation efficiency. The method for calculating the rotation angle θ is as follows:

[0065]

[0066]

[0067] in, ; m means the size of the target edge macroblock is m×m; boxblur is the box filter function.

[0068] Step 202: Mark target edge macroblocks and non-target edge macroblocks according to edge information of the current macroblock.

[0069] Optionally, marking the target edge macroblock and the non-target edge macroblock according to the edge information of the current macroblock includes:

[0070] It is determined whether a preset first determination condition is met according to the edge information of the current macroblock. If so, the current macroblock is marked as a target edge macroblock; if not, the current macroblock is marked as a non-target edge macroblock.

[0071] The preset first determination condition is that the minimum eigenvalue of the current macroblock is less than 1 and the maximum eigenvalue is greater than 6.

[0072] If the maximum eigenvalue and the minimum eigenvalue are both less than 1, it means that the edge in the current macroblock is a weak edge; if the maximum eigenvalue and the minimum eigenvalue are both greater than 6, it means that the direction of the edge in the current macroblock is chaotic and the human eye is not very sensitive to such edges; if the maximum eigenvalue is greater than 6 and the minimum eigenvalue is less than 1, it means that the direction of the edge in the current macroblock is single and it is a strong edge. Compared with weak edges and edges with chaotic directions, the human eye is more sensitive to strong edges in a single direction, and strong edges are more likely to produce black and white edges after sharpening.

[0073] It should be noted that the minimum eigenvalue threshold 1 and the maximum eigenvalue threshold 6 described in this embodiment are both empirical values ​​and have no limiting effect. In practical applications, the minimum eigenvalue threshold and the maximum eigenvalue threshold can be set according to actual conditions.

[0074] Step 203, traverse each pixel point in the target edge macroblock and execute steps 204 and 205 for sharpening, and the pixels in the non-target edge macroblock are sharpened using the existing conventional sharpening technology.

[0075] The existing conventional sharpening techniques include but are not limited to: USM method, Sobel operator, and Laplace operator.

[0076] Step 204, determining the main pixel line where the current pixel point is located within the sharpening window range, and determining the reference pixel line of the main pixel line.

[0077] Optionally, determining the main pixel line where the current pixel point is located within the sharpening window range includes:

[0078] According to the current pixel point and the direction θ of the feature vector obtained in step 201, a main pixel line centered on the current pixel point is determined within the sharpening window, such as Figure 2 As shown in (a) and (b).

[0079] The sharpening window is a window of size n×n centered at the current pixel point, and the size of the sharpening window does not exceed the size of the target edge macroblock.

[0080] It should also be noted that Figure 2 The direction of the main pixel line shown in is only for illustration and has no limiting effect. In practical applications, the main pixel line needs to be determined according to the direction of the actual eigenvector.

[0081] Optionally, determining the reference pixel line of the main pixel line includes:

[0082] like Figure 3 As shown, two adjacent pre-selected reference pixel lines are respectively taken on both sides of the main pixel line. The pre-selected reference pixel lines are all parallel to the main pixel line, and the four pre-selected reference lines are all within the sharpening window.

[0083] It should be noted that in this embodiment, the number of pre-selected reference pixel lines is only an empirical value and has no limiting effect. In actual application, it needs to be set according to the size of the sharpening window and actual needs.

[0084] The average pixel value of each pixel line is calculated for the preselected reference pixel line and the main pixel line.

[0085] In this embodiment, the average value of the pixel line can be calculated by using a Gaussian weighting method or a simple arithmetic mean. For example, the sharpening window size is 5×5, and the average pixel value of the main pixel line is calculated by using a Gaussian weighting method:

[0086] .

[0087] The average pixel value of the preselected reference pixel line is subtracted from the average pixel value of the main pixel line to obtain an average pixel value difference.

[0088] According to the average pixel value difference, it is determined whether a preset second determination condition is met. If so, the preselected reference pixel line corresponding to the average pixel value difference is used as the reference pixel line; otherwise, it is not used as the reference pixel line.

[0089] The preset second determination condition is that the average pixel value difference is less than the JND curve threshold.

[0090] The information points that human vision can perceive have different thresholds under different brightness backgrounds. The JND curve reflects the JND thresholds corresponding to different brightness backgrounds, such as Figure 4 By selecting different judgment thresholds under different brightness backgrounds instead of setting the threshold to a constant, the pre-selected reference pixel lines that the human eye is not sensitive to the pixel difference between the main pixel line and the reference pixel line can be more accurately screened as the reference pixel line.

[0091] Step 205, obtaining the sharpened pixel value of the current pixel point according to the main pixel line and the reference pixel line.

[0092] Optionally, obtaining a sharpened pixel value of a current pixel point according to the main pixel line and the reference pixel line includes:

[0093] The average pixel value of the reference pixel line is averaged with the average pixel value of the main pixel line to obtain the average pixel value of the pixel point.

[0094] According to the pixel value of the current pixel point and the average pixel value of the pixel points, a difference between the current pixel point and the average pixel value of the pixel points is calculated.

[0095] The difference between the current pixel point and the average pixel value of the pixel points is multiplied by a sharpening coefficient and superimposed on the pixel value of the current pixel point to obtain the sharpened pixel value of the current pixel point.

[0096] In the present invention, relational terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image sharpening method for suppressing black and white edges, characterized in that: include: Step 1, dividing the image to be sharpened into multiple macroblocks of the same size, and calculating and obtaining the edge information of the current macroblock; Step 2, marking the target edge macroblock and the non-target edge macroblock according to the edge information of the current macroblock; Step 3, traverse each pixel in the target edge macroblock and perform steps 4 to 5 for sharpening, and the pixels in the non-target edge macroblock are sharpened using the existing conventional sharpening technology; Step 4, determining the main pixel line where the current pixel point is located within the sharpening window range, and determining the reference pixel line of the main pixel line; The direction of the reference pixel line is consistent with the direction of the main pixel line; the sharpening window is a window of size n×n centered at the current pixel point, and the size of the sharpening window does not exceed the size of the target edge macroblock; The determining of the main pixel line where the current pixel point is located within the sharpening window range includes: According to the direction of the feature vector of the current pixel point and the current target edge macroblock, a main pixel line centered on the current pixel point and having a direction consistent with the direction of the feature vector is determined within the sharpening window range; The direction of the eigenvector is calculated by the covariance matrix of the current pixel; Step 5: Obtain the sharpened pixel value of the current pixel point according to the main pixel line and the reference pixel line.

2. The image sharpening method for suppressing black and white edges according to claim 1, characterized in that: The edge information includes the maximum eigenvalue, the minimum eigenvalue, and the direction of the eigenvector; The maximum eigenvalue, the minimum eigenvalue, and the direction of the eigenvector are calculated by the covariance matrix of the current pixel.

3. The image sharpening method for suppressing black and white edges according to claim 1, characterized in that: The step of marking the target edge macroblock and the non-target edge macroblock according to the edge information of the current macroblock includes: Determine whether a preset first determination condition is met according to the edge information of the current macroblock, and if so, mark the current macroblock as a target edge macroblock; if not, mark the current macroblock as a non-target edge macroblock; The preset first determination condition is that the minimum eigenvalue of the current macroblock is less than a threshold value T1 and the maximum eigenvalue is greater than a threshold value T2, T2>T1; The maximum eigenvalue and the minimum eigenvalue are calculated by the covariance matrix of the current pixel.

4. The image sharpening method for suppressing black and white edges according to claim 1, characterized in that: The existing conventional sharpening techniques include: USM method, Sobel operator, and Laplace operator.

5. The image sharpening method for suppressing black and white edges according to claim 1 or 2, characterized in that: The direction of the eigenvector is represented by a rotation angle, and the rotation angle algorithm for calculating the eigenvector is: ， ， in, ; m indicates that the size of the target edge macroblock is m×m; boxblur is the box filter function; I x and I y is the gradient map of the image to be sharpened.

6. The image sharpening method for suppressing black and white edges according to claim 1, characterized in that: The determining of the reference pixel line of the main pixel line comprises: Taking t adjacent pre-selected reference pixel lines on both sides of the main pixel line respectively; The pre-selected reference pixel lines are all parallel to the main pixel line, and the t pre-selected reference lines are all within the sharpening window; Calculating the average pixel value of each pixel line for the preselected reference pixel line and the main pixel line respectively; Differences are calculated between the average pixel value of the preselected reference pixel line and the average pixel value of the main pixel line to obtain average pixel value differences; According to the average pixel value difference, determining whether a preset second determination condition is satisfied, if satisfied, the preselected reference pixel line corresponding to the average pixel value difference is used as the reference pixel line; otherwise, it is not used as the reference pixel line; The preset second determination condition is that the average pixel value difference is less than the JND curve threshold.

7. The image sharpening method for suppressing black and white edges according to claim 1, characterized in that: The step of obtaining a sharpened pixel value of a current pixel point according to the main pixel line and the reference pixel line includes: Averaging the average pixel value of the reference pixel line and the average pixel value of the main pixel line to obtain the average pixel value of the pixel point; Calculate the difference between the current pixel value and the average pixel value of the pixel points according to the pixel value of the current pixel point and the average pixel value of the pixel points; The difference between the current pixel point and the average pixel value of the pixel points is multiplied by a sharpening coefficient and superimposed on the pixel value of the current pixel point to obtain the sharpened pixel value of the current pixel point.

Images