Image sharpening method and device

Abstract

The invention provides an image sharpening method and device. The method comprises the following steps: acquiring an image in a YUV format; calculating local feature information of a Y component of the image in each window; extracting a detail image layer and an edge image layer of a Y component of the image in each window by using filters of different frequency bands; adjusting the detail image layer and the edge image layer by using the local information and the motion information of the image; adding the detail edge layer and the Y component of the image; performing overshoot suppression processing on the sharpened Y component; performing same-proportion correction on the UV component of the image according to the ratio of the Y component after overshoot suppression to the Y component of the image; and outputting image data composed of the Y component after overshoot suppression and the corrected UV component. By fully detecting and counting the local feature information of the Y component of the image and by means of the externally input motion information of the image, the edge contour area can be sharp and smooth, and the detail texture area can be fine and natural.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to an image sharpening method and device. Background Art

[0002] Limited by the resolution of the shooting device or the noise reduction link in image processing, images often suffer from insufficient sharpness and visual blurriness. Given that the human visual system prefers images with high sharpness, image sharpening has gradually become an important link in the image processing process, and image sharpening can significantly improve the visual quality of images.

[0003] In the prior art, high-frequency filters such as Sobel filters and Laplace filters are usually used to extract high-frequency information, or the high-frequency information is obtained by subtracting the low-pass filtering result of the original image from the original image; then the high-frequency information and the original image are superimposed to obtain the final image. Since the high-frequency information contains both edge contours and detailed texture information as well as noise, undifferentiated sharpening will reduce the signal-to-noise ratio while increasing the sharpness. In addition, too strong sharpening intensity will cause artifacts such as black and white edges, reducing the authenticity of the image. Too weak sharpening intensity will result in insufficient detail restoration and limited improvement in image quality. Summary of the Invention

[0004] The present invention provides an image sharpening method and device to solve the technical problem that the existing image sharpening methods will reduce the signal-to-noise ratio and the authenticity of the image.

[0005] To solve the above technical problem, the present invention provides an image sharpening method, including the following steps:

[0006] S1. Obtain an image in YUV format;

[0007] S2. Divide the image into multiple windows, and calculate the local feature information of the Y component of the image within each window;

[0008] S3. Use filters of different frequency bands to extract the detail layer and the edge layer of the Y component of the image within each window;

[0009] S4. Adjust the detail layer and the edge layer by using the local information and the motion information of the image to obtain an adjusted detail-edge layer;

[0010] S5. Add the detail-edge layer and the Y component of the image to obtain a sharpened Y component;

[0011] S6. Perform overshoot suppression processing on the sharpened Y component to obtain an overshoot-suppressed Y component;

[0012] S7. Correct the UV components of the image proportionally according to the ratio between the overshoot-suppressed Y component and the Y component of the image, to obtain the corrected UV components;

[0013] S8. Output the image data composed of the overshoot-suppressed Y component and the corrected UV components.

[0014] Preferably, calculating the local feature information of the Y component of the image in each window in step S2 includes the following steps: respectively calculating the local maximum value, local minimum value, local average value, local standard deviation, local frequency value, and local direction consistency of the Y component of the image in each window.

[0015] Preferably, step S3 includes the following steps: respectively process the Y component of the image using a high-frequency filter, a medium-frequency filter, and a low-frequency band-pass filter to obtain a high-frequency detail layer, a medium-frequency detail layer, a low-frequency detail layer, a high-frequency edge layer, a medium-frequency edge layer, and a low-frequency edge layer.

[0016] Preferably, step S4 includes the following steps: perform a first frequency synthesis on the high-frequency detail layer, the medium-frequency detail layer, the low-frequency detail layer, the high-frequency edge layer, the medium-frequency edge layer, and the low-frequency edge layer in a weighted average manner to obtain a highest-frequency static detail layer, a lowest-frequency static detail layer, a highest-frequency motion detail layer, a lowest-frequency motion detail layer, a highest-frequency static edge layer, a lowest-frequency static edge layer, a highest-frequency motion edge layer, and a lowest-frequency motion edge layer;

[0017] Use the local frequency value to perform a second frequency synthesis on the highest-frequency static detail layer, the lowest-frequency static detail layer, the highest-frequency motion detail layer, the lowest-frequency motion detail layer, the highest-frequency static edge layer, the lowest-frequency static edge layer, the highest-frequency motion edge layer, and the lowest-frequency motion edge layer to obtain a static detail layer, a motion detail layer, a static edge layer, and a dynamic edge layer;

[0018] Use the motion information of the image to perform motion synthesis on the static detail layer, the motion detail layer, the static edge layer, and the dynamic edge layer to obtain a motion-synthesized detail layer and a motion-synthesized edge layer;

[0019] Use the local standard deviation to adjust the sharpening intensity of the motion-synthesized detail layer to obtain an enhanced detail layer;

[0020] Remap the enhanced detail layer to obtain a remapped detail layer;

[0021] Adjust the sharpening intensity of the edge layer after motion synthesis by using the local standard deviation to obtain an enhanced edge layer;

[0022] Remap the enhanced edge layer to obtain a remapped edge layer;

[0023] Perform direction synthesis on the remapped detail layer and the remapped edge layer by using the local direction consistency information to obtain an adjusted detail-edge layer.

[0024] Preferably, step S5 includes the following steps: Add the detail-edge layer and the Y component of the image according to the following formula,

[0025] Ysharp = Yin + AC

[0026] where Ysharp is the sharpened Y component, Yin is the original Y component of the image, and AC is the adjusted detail-edge layer.

[0027] Preferably, step S6 includes the following steps: Perform overshoot suppression processing on the sharpened Y component according to the following formula,

[0028]

[0029] where Ysharpsc is the Y component after overshoot suppression, Ymin is the local minimum of the original Y component, Ymax is the local maximum of the original Y component, and SCGain is the total suppression gain.

[0030] Preferably, step S7 includes the following steps: Perform proportional correction on the UV components of the image according to the following formula,

[0031]

[0032] where UCorrect is the corrected U component value and VCorrect is the corrected V component value.

[0033] The present invention also provides an image sharpening device, including the following units:

[0034] An image acquisition unit for acquiring an image in YUV format;

[0035] A local feature information calculation unit for dividing the image into multiple windows and calculating the local feature information of the Y component of the image in each window;

[0036] A detail-edge extraction unit for extracting the detail layer and the edge layer of the Y component of the image in each window by using filters of different frequency bands;

[0037] A detail edge adjustment unit for adjusting the detail layer and the edge layer by using the local information and the motion information of the image to obtain an adjusted detail edge layer;

[0038] An overlay unit for adding the detail edge layer and the Y component of the image to obtain a sharpened Y component;

[0039] An overshoot suppression unit for performing overshoot suppression processing on the sharpened Y component to obtain an overshoot-suppressed Y component;

[0040] A color correction unit for proportionally correcting the UV components of the image according to the ratio between the overshoot-suppressed Y component and the Y component of the image to obtain corrected UV components;

[0041] An output unit for outputting image data composed of the overshoot-suppressed Y component and the corrected UV components.

[0042] Preferably, the local feature information calculation unit is used to perform the following steps: calculating the local maximum value, local minimum value, local average value, local standard deviation, local frequency value, and local direction consistency of the Y component of the image in each window respectively.

[0043] Preferably, the detail edge extraction unit is used to perform the following steps: processing the Y component of the image by using a high-frequency filter, a medium-frequency filter, and a low-frequency band-pass filter respectively to obtain a high-frequency detail layer, a medium-frequency detail layer, a low-frequency detail layer, a high-frequency edge layer, a medium-frequency edge layer, and a low-frequency edge layer.

[0044] An image sharpening method and device provided by the present invention can apply a reasonable and appropriate enhancement style to different image contents within a local range by fully detecting and statistically analyzing the local feature information of the Y component of the image and by means of the motion information of an externally input image, thereby making the enhancement strengths of the moving and static regions show differences, the edge contour regions show sharpness and smoothness, and the detail texture regions show fineness and naturalness. Through overshoot suppression processing and correction of the UV components, defects and flaws such as black and white edges, reduction of signal-to-noise ratio, noise amplification in flat regions, and saturation loss, which are easily introduced by traditional methods, can be effectively avoided. Description of the Drawings

[0045] Figure 1 is a flowchart of an image sharpening method provided by an embodiment of the present invention.

[0046] Figure 2 is a structural schematic diagram of an image sharpening device provided by an embodiment of the present invention.

[0047] Figure 3It is a schematic structural diagram of a local feature information calculation unit provided by an embodiment of the present invention.

[0048] Figure 4 It is a schematic diagram of four directions defined within a window provided by an embodiment of the present invention.

[0049] Figure 5 It is a schematic structural diagram of a detail edge extraction unit provided by an embodiment of the present invention.

[0050] Figure 6 It is a schematic structural diagram of a detail edge adjustment unit provided by an embodiment of the present invention.

[0051] The reference numerals are as follows:

[0052] Image acquisition unit - 101, local feature information calculation unit - 102, detail edge extraction unit - 103, detail edge adjustment unit - 104, superposition unit - 105, overshoot suppression unit - 106, color correction unit - 107, output unit - 108;

[0053] First information calculator - 102 - 1, second information calculator - 102 - 2, third information calculator - 102 - 3, fourth information calculator - 102 - 4, fifth information calculator - 102 - 5, sixth information calculator - 102 - 6;

[0054] Detail first filter - 103 - 1, detail second filter - 103 - 2, detail third filter - 103 - 3, edge first filter bank - 103 - 4, edge second filter bank - 103 - 5, edge third filter bank - 103 - 6;

[0055] Detail kernelizer - 104 - 1, detail frequency synthesizer - 104 - 2, detail motion synthesizer - 104 - 3, detail regulator - 104 - 4, edge kernelizer - 104 - 5, edge frequency synthesizer - 104 - 6, edge motion synthesizer - 104 - 7, edge regulator - 104 - 8, direction synthesizer - 104 - 9. Specific embodiments

[0056] To make the objectives, advantages and features of the present invention clearer, a method and device for image sharpening proposed by the present invention will be further described in detail below with reference to the accompanying drawings. It should be noted that the accompanying drawings are all in a very simplified form and use non - precise scales, only for conveniently and clearly assisting in explaining the objectives of the embodiments of the present invention.

[0057] In the description of the present invention, the qualifier terms "first", "second", etc. are added for convenience of description and reference, and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with the qualifier terms "first", "second", etc. may explicitly or implicitly include one or more of such features.

[0058] As Figure 1 shown, this embodiment provides an image sharpening method, including the following steps:

[0059] S1. Obtain an image in YUV format; where Y represents luminance (brightness), and U and V represent chrominance. If it is an image in other formats, the image in other formats can be converted into YUV format. According to different configurations, the bit width of the image data can be different, and it is usually set to 10 bit.

[0060] S2. Divide the image into multiple windows, and calculate the local feature information of the Y component of the image within each window.

[0061] In order to adaptively adjust the sharpening according to the image content, it is necessary to detect and statistically analyze the local information of the Y component of the image. Specifically, the image can be first divided into multiple windows of w*w, and the shape of the window can be a square or a rectangle, etc. The following takes a window of 5 rows * 5 columns as an example for illustration; then calculate the local maximum value, local minimum value, local average value, local standard deviation, local frequency value, and local direction consistency of the Y component (original Y component) of the image within each window, as Figure 3 shown, these 6 parameters can be calculated by the first information calculator 102-1, the second information calculator 102-2, the third information calculator 102-3, the fourth information calculator 102-4, the fifth information calculator 102-5, and the sixth information calculator 102-6 respectively.

[0062] The calculation formula of the local maximum value Ymax(x,y) of the Y component is as follows:

[0063] Ymax(x,y) = max(Yin(x+i,y+j)), i,j∈[-2,2] (1)

[0064] Where, (x,y) represents the coordinate position of the currently processed pixel, max() represents taking the maximum value, (x+i,y+j) represents the coordinate positions of each pixel within the current window, and Yin() represents the Y component of the image.

[0065] The calculation formula of the local minimum value of the Y component is as follows:

[0066] Ymin(x,y) = min(Yin(x + i,y + j)), where i,j ∈ [-2,2] (2)

[0067] Among them, min() represents taking the minimum value.

[0068] The calculation formula for the local average value of the Y component is as follows:

[0069]

[0070] The calculation formula for the local standard deviation of the Y component is as follows:

[0071]

[0072] Among them, abs represents taking the absolute value.

[0073] The purpose of calculating the local frequency value of the Y component is to statistically analyze the frequency information of the change of each pixel in each direction, aiming to strengthen the high-frequency component in the high-frequency information area and strengthen the low-frequency component in the low-frequency information area.

[0074] First, calculate the gradient information GradH(x,y), GradV(x,y), GradZ(x,y), GradN(x,y) of four directions point by point. The signs of the four directions are as Figure 4 shown Figure 4 In the figure, the pixel point in the upper left corner is used as the coordinate origin, the horizontal rightward direction is the positive direction of the x-axis, and the vertical downward direction is the positive direction of the y-axis. When performing gradient operations, if some pixel points are outside the image, these pixel points can be supplemented by methods such as replication padding or mirror padding. Replication padding can copy the parameters of the current pixel point to the pixel points that need to be supplemented; mirror padding can use a certain horizontal or vertical pixel point as the axis of symmetry, and mirror the parameters of the existing pixel points to fill the pixel points that need to be supplemented.

[0075] Horizontal direction gradient: GradH(x,y) = Yin(x,y) - Yin(x - 1,y) (5)

[0076] Vertical direction gradient: GradV(x,y) = Yin(x,y) - Yin(x,y - 1) (6)

[0077] 45° direction gradient: GradZ(x,y) = Yin(x,y) - Yin(x + 1,y - 1) (7)

[0078] 135° direction gradient: GradN(x,y) = Yin(x,y) - Yin(x - 1,y - 1) (8)

[0079] Then, count the number of positive and negative jumps of each pixel point in the above four directions respectively, and the calculation process is as shown in formulas (9), (10), (11), and (12):

[0080] Number of jumps in the horizontal direction:

[0081] Number of jumps in the vertical direction:

[0082] Number of jumps in the 45° direction:

[0083] Number of jumps in the 135° direction:

[0084] Among them, xor() represents the exclusive OR operation, and sign() represents the operation of taking the positive or negative sign.

[0085] Finally, the calculation formula for the local frequency value FreqInfo of the Y component is as follows:

[0086] FreqInfo = c1 * (JumpH(x, y) + JumpV(x, y) + JumpZ(x, y) + JumpN(x, y)) (13)

[0087] Among them, c1 is usually set to 20.

[0088] The purpose of calculating the local direction consistency of the Y component is to count the directional information, including the first direction and the second direction of each pixel point, the primary and secondary ratios, and the local direction consistency. The aim is to strengthen the directional information in the edge contour area and the non-directional information in the texture detail area.

[0089] First, perform a convolution operation using the local average value Yavg of the Y component and the Sobel operators in four directions, and the calculation process is as shown in formula (14):

[0090]

[0091] Among them, represents the convolution operation, and the SobelX filter kernel can be set as follows:

[0092]

[0093]

[0094] RespondX is the response result after convolution, abs() represents taking the absolute value, SobelX represents SobelH, SobelV, SobelZ or SobelN, and SobelH, SobelV, SobelZ, SobelN represent the Sobel filter kernel in the horizontal direction, the Sobel filter kernel in the vertical direction, the Sobel filter kernel in the 45° direction, and the Sobel filter kernel in the 135° direction, respectively.

[0095] Then, traverse RespondX, and take the directions corresponding to the maximum value and the second maximum value as the first direction Dir1 and the second direction Dir2 respectively. The four directions {H, V, Z, N} correspond to the indices {1, 2, 3, 4} in sequence. When the maximum response of RespondX is less than the threshold DirTh, which can be set to 40, it proves to be a flat area, and the direction index is recorded as 0.

[0096] Next, perform mode filtering on each pixel in the first direction Dir1, and the calculation process is as shown in formula (15):

[0097] Dir1Filt(x,y) = Vote(Ω(Dir1(x,y))) (15)

[0098] Among them, Dir1Filt(x,y) represents the result after mode filtering of the pixel points in the first direction Dir1; Vote() represents the mode filtering operation; Ω() represents a 3-element neighborhood that is normal to the first direction Dir1(x,y) of the currently processed pixel (x,y). For example, if Dir1(x,y) = 1, Ω() is the set of (x,y), (x,y - 1), (x,y + 1). Specifically, if Dir1(x,y) = 0, Ω() is the set of (x,y), (x,y - 1), (x - 1,y - 1), (x - 1,y), (x - 1,y + 1).

[0099] Next, perform the primary-secondary ratio Ratio calculation, and the calculation process is as shown in formula (16):

[0100] Ratio = Respond1 / (Respond1 + Respond2) (16)

[0101] Among them, Respond1 is the response in the first direction, and Respond2 is the response in the second direction. If Dir1Filt changes compared to Dir1, the primary-secondary ratio Ratio is defaulted to 1.

[0102] Finally, perform the local direction consistency calculation. Statistically analyze the number of pixel points in each direction within a 5*5 window to obtain a histogram containing 4 intervals (bins), and then take the approximate standard deviation of the histogram and normalize it. The calculation process is as shown in formula (17):

[0103]

[0104] Among them, DirConf represents the local direction consistency. The larger it is, the more it indicates that the local neighborhood is a significant edge with a dominant direction. The smaller it is, the more it indicates that the area is undirected texture details; bin(i) represents the number of pixel points in each direction; the calculation process of the average direction value Avgbin is as shown in formula (18):

[0105]

[0106] S3. Use filters of different frequency bands to extract the detail layer and the edge layer of the Y component of the image within each window.

[0107] The purpose of this step is to extract various basic layers of detail edges using filters of different frequency bands and different styles for subsequent steps.

[0108] As Figure 5 shown, this step can configure a detail first filter 103-1, a detail second filter 103-2, a detail third filter 103-3, an edge first filter bank 103-4, an edge second filter bank 103-5, and an edge third filter bank 103-6.

[0109] The detail first filter 103-1, the detail second filter 103-2, and the detail third filter 103-3 are used to extract three detail layers, namely, a high-frequency detail layer (UDH), a medium-frequency detail layer (UDM), and a low-frequency detail layer (UDL).

[0110] Perform a convolution operation on the Y component Yin of the image and the band-pass filter operators of three frequency bands. The calculation process is as shown in formulas (19), (20), and (21):

[0111]

[0112] Among them, UDHFilter represents a high-frequency band-pass filter, UDMFilter represents a medium-frequency band-pass filter, UDLFilter represents a high-frequency band-pass filter, and the filter size is 5*5.

[0113] The edge first, second, and third filter banks are used to extract three edge layers, namely, a high-frequency edge layer (DH), a medium-frequency edge layer (DM), and a low-frequency edge layer (DL).

[0114] The first edge filter bank 103-4 includes a high-frequency edge horizontal filter DHHFilter, a high-frequency edge vertical filter DHVFilter, a high-frequency edge 45° direction filter DHZFilter, and a high-frequency edge 135° direction filter DHNFilter. The second edge filter bank 103-5 includes a medium-frequency edge horizontal filter DMHFilter, a medium-frequency edge vertical filter DMVFilter, a medium-frequency edge 45° direction filter DMZFilter, and a medium-frequency edge 135° direction filter DMNFilter. The third edge filter bank 103-6 includes a low-frequency edge horizontal filter DLHFilter, a low-frequency edge vertical filter DLVFilter, a low-frequency edge 45° direction filter DLZFilter, and a low-frequency edge 135° direction filter DLNFilter. The sizes of these 12 filters are all 5*5. The frequency domain of the first edge filter bank 103-4 is high-pass in the main direction and low-pass in the normal direction of the main direction. The second and third edge filter banks are designed similarly.

[0115] Use the primary and secondary filters corresponding to the primary and secondary directions Dir1Filt(x,y) and Dir2(x,y) in the first, second, and third edge filter banks to perform convolution operations on pixel points. The calculation process is as shown in formulas (22) and (23):

[0116]

[0117] where X is DH, DM, DL; represents the convolution operation; master represents the primary direction corresponding to Dir1Filt(x,y); second represents the secondary direction corresponding to Dir2(x,y); masterXFilter is the filter corresponding to Dir1Filt, and secondXFilter is the filter corresponding to Dir2.

[0118] Finally, the primary and secondary direction filtering results are weighted and summed according to the weight, that is, the primary and secondary ratio Ratio, to obtain the high-frequency edge layer DH, the medium-frequency edge layer DM, and the low-frequency edge layer DL. The calculation process is as shown in formulas (24), (25), and (26):

[0119] DH = DHmaster * Ratio + DHsecond * (1 - Ratio) (24)

[0120] DM = DLmaster * Ratio + DMsecond * (1 - Ratio) (25)

[0121] DL = DLmaster * Ratio + DLm * Ratio(1 - Ratio) (26)

[0122] S4. Adjust the detail layer and the edge layer by using the local information and the motion information of the image to obtain an adjusted detail-edge layer.

[0123] As Figure 6 shown, this step can configure nine sub-modules, namely, a detail nucleator 104-1, a detail frequency synthesizer 104-2, a detail motion synthesizer 104-3, a detail adjuster 104-4, an edge nucleator 104-5, an edge frequency synthesizer 104-6, an edge motion synthesizer 104-7, an edge adjuster 104-8, and a direction synthesizer 104-9. The detail layer and the edge layer are adjusted and fused by these nine sub-modules.

[0124] The detail nucleator 104-1 and the edge nucleator 104-5 are used to adjust the parts in the original detail layer and the original edge layer whose absolute values are lower than the nucleation threshold (CoreTh) to 0. The purpose of nucleation is to suppress the noise in the flat areas. The nucleation thresholds in the layers of different frequency bands can be set differently. For example, the nucleation thresholds for high frequency, medium frequency, and low frequency are 5, 8, and 10 respectively. After such processing, the subsequent image sharpening will not be interfered by the noise in the flat areas of the image, and the flat areas of the sharpened image will remain smooth and natural without jagged edges and obvious noise effects, improving the visual quality of the image.

[0125] The detail frequency synthesizer 104-2 and the edge frequency synthesizer 104-6 are used to synthesize layers of different frequencies. The upper and lower limits of the frequencies adopted for four types of image contents (static details, static edges, motion details, and dynamic edges) can be set according to user preferences. For example, the upper and lower limits of the frequencies for static details are 230-255, the upper and lower limits of the frequencies for static edges are 200-230, the upper and lower limits of the frequencies for motion details are 150-175, and the upper and lower limits of the frequencies for dynamic edges are 120-150. The synthesized layers are the highest-frequency static detail layer (UDstlhigh), the lowest-frequency static detail layer (UDstllow), the highest-frequency motion detail layer (UDmothigh), the lowest-frequency motion detail layer (UDmotlow), the highest-frequency static edge layer (Dstlhigh), the lowest-frequency static edge layer (Dstllow), the highest-frequency motion edge layer (Dmothigh), and the lowest-frequency motion edge layer (Dmotlow). The differential setting of the upper and lower limits of the frequencies can bring the effect of refined style adjustment. The first frequency synthesis adopts weighted average, and the synthesis materials come from the high-frequency detail layer (UDH), the medium-frequency detail layer (UDM), the low-frequency detail layer (UDL), the high-frequency edge layer (DH), the medium-frequency edge layer (DM), and the low-frequency edge layer (DL).

[0126] Then, the local frequency value FreqInfo is mapped through the first mapping LUT1 (LUT1 is a monotonically increasing non-linear mapping) to obtain the frequency weight (FreqWgt), and then the second frequency synthesis is performed. The process of the second frequency synthesis is shown in formulas (27), (28), (29), and (30):

[0127] UDstl = (UDstlhigh * Freqwgt + UDstllow * (255 - Freqwgt)) / 255 (27)

[0128] UDmot = (UDmothigh * Freqwgt + UDmotlow * (255 - Freqwgt)) / 255(28)

[0129] Dstl = (Dstlhigh * Freqwgt + Dstllow * (255 - Feqwgt)) / 255 (29)

[0130] Dmot = (Dmothigh * Fregwgt + Dmotlow * (255 - Freqwgt)) / 255 (30)

[0131] Among them, UDmot is the motion detail layer, UDstl is the static detail layer, Dmot is the dynamic edge layer, and Dstl is the static edge layer.

[0132] The detail motion synthesizer 104-3 and the edge motion synthesizer 104-7 are used for fusing the static and dynamic layers. The motion detection module can determine the motion information (MotInfo) of the image based on the method of pixel difference or the method of motion vector, and then transmit the motion information of the image to the image sharpening system; the image sharpening system maps the motion information of the image through the second mapping LUT2 (LUT2 is a monotonically increasing non-linear mapping) to obtain the motion weight (MotWgt), and then performs the fusion of the static and dynamic layers. The motion synthesis process is shown in formulas (31) and (32):

[0133] UD = (UDmot * MotWgt + UDstl * (255 - MotWgt)) / 255 (31)

[0134] D = (Dmot * MotWgt + Dstl * (255 - MotWgt)) / 255 (32)

[0135] Among them, UD is the detail layer after motion synthesis, and D is the edge layer after motion synthesis.

[0136] Detail adjuster 104-4 is used to precisely enhance different extracted details. First, the local standard deviation (Ystd) is used to control the detail sharpening intensity, and the process is shown in Equation (33):

[0137] UDpost = UDGainbystd * UD / 63 (33)

[0138] where UDGainbystd = LUT3(Ystd) (34)

[0139] UDpost represents the enhanced detail layer, and UDGainbystd represents the standard deviation weight obtained by mapping the local standard deviation (Ystd) through the third mapping LUT3. The third mapping LUT3 is a mapping that first increases and then levels off. Through the third mapping LUT3, the local contrast in the middle and low sections can be empirically amplified to restore more weak details.

[0140] Then, a remapping for distinguishing positive and negative details is performed, and the process is shown in Equation (35):

[0141]

[0142] The fourth mapping LUT4P / LUT4N is designed to empirically amplify the intensity in the middle and low sections to restore more weak details. The fourth mapping can be configured differently, but it must maintain the property of being monotonically increasing, with the slope increasing from 1 and then returning to 1.

[0143] Edge adjuster 104-8 is used to precisely enhance different extracted edges. First, the local standard deviation (Ystd) is used to control the edge sharpening intensity, and the process is shown in Equation (36):

[0144] Dpost = DGainbystd * D / 63 (36)

[0145] where DGainbystd = LUT5(Ystd) (37)

[0146] Dpost is the enhanced edge layer, and DGainbystd represents the standard deviation weight obtained by mapping the local standard deviation (Ystd) through the fifth mapping LUT5. The fifth mapping LUT5 is a mapping that first increases and then levels off. Through the fifth mapping LUT5, the local contrast in the middle and low sections can be empirically amplified to restore more weak edges.

[0147] Then, a remapping for distinguishing positive and negative edges is performed, and the process is shown in Equation (38):

[0148]

[0149] The sixth mapping LUT6P / LUT6N empirically amplifies the intensity in the middle and low segments during design to restore more weak edges. The sixth mapping can be configured differently, but it must maintain a monotonically increasing non-linear property.

[0150] The direction synthesizer 104-9 is used to synthesize the remapped detail layer and edge layer. According to the local direction consistency information (DirConf), the directional weight (DirWgt) is obtained through the seventh mapping LUT7, and then weighted fusion is performed. The direction synthesis process is shown in formula (39):

[0151] AC = (DpostRemap * Dirwgt + UDpsotRemap * (255 - Dirwgt)) / 255 (39)

[0152] Where AC is the adjusted detail edge layer.

[0153] S5. Add the detail edge layer and the Y component of the image to obtain the sharpened Y component.

[0154] Superimpose the adjusted detail edge layer and the Y component of the image, that is, the original Y component, to obtain the sharpened Y component Ysharp, as shown in formula (40):

[0155] Ysharp = Yin + AC (40)

[0156] S6. Perform overshoot suppression processing on the sharpened Y component to obtain the overshoot-suppressed Y component.

[0157] The purpose of this step is to reduce the strong black and white edge effect introduced by oversharpening. Compare the sharpened Y component Ysharp with the local maximum Ymax and local minimum Ymin, and reduce the part exceeding the extreme values instead of using hard clipping.

[0158] First, calculate the suppression gain (SCGain). The suppression gain is controlled by user presets, brightness, and standard deviation.

[0159] The luminance suppression gain (LumaSCGain) is separately controlled according to the black and white edges. The calculation process refers to formula (41):

[0160]

[0161] Where glbscposstr represents the user-preset white edge suppression gain, and glbscnegstr represents the user-preset black edge suppression gain. The eighth positive mapping LUT8P can be designed as a monotonically decreasing broken line, and the eighth negative mapping LUT8N can be designed as a monotonically increasing broken line.

[0162] The standard deviation suppression gain (StdSCGain) only takes effect in the overexposed area. The calculation process refers to formula (42):

[0163]

[0164] Among them, the ninth mapping LUT9 is a non-linear curve that first increases and then levels off, and c2 = 960.

[0165] The calculation process of the total suppression gain (SCGain) is as shown in formula (43):

[0166] SCGain = LumaSCGain * StdSCGain / norm (43)

[0167] Among them, norm = 255 * 255.

[0168] Then overshoot suppression is performed, and the process is as shown in formula (44):

[0169]

[0170] Among them, Ysgarpsc represents the Y component after overshoot suppression.

[0171] S7. According to the ratio between the Y component after overshoot suppression and the Y component of the image, the UV components of the image are corrected proportionally to obtain the corrected UV components.

[0172] The UV components of the image should be adjusted following the sharpened Y component to ensure that the saturation remains consistent before and after sharpening. The specific calculation is as shown in formulas (45) and (46):

[0173]

[0174]

[0175] Among them, Yin, Uin, and Vin are the original data corresponding to the YUV three channels of the image, UCorrect is the value of the corrected U component, and VCorrect is the value of the corrected V component.

[0176] S8. Output the image data composed of the Y component after overshoot suppression and the corrected UV components.

[0177] The finally output image data in YUV format is as shown in formula (47):

[0178]

[0179] An image sharpening method provided by this embodiment can apply a reasonable and appropriate enhancement style to different image contents within a local range by fully detecting and statistically analyzing the local feature information of the Y component of the image and by means of the motion information of the externally input image, so that the motion and static regions exhibit different enhancement intensities, the edge contour regions are sharp and smooth, and the detailed texture regions are delicate and natural. Through overshoot suppression processing and correction of the UV components, defects and flaws such as black and white edges, reduced signal-to-noise ratio, amplified noise in flat regions, and saturation loss, which are easily introduced by traditional methods, can be effectively avoided.

[0180] As Figure 2 shown, based on the same technical concept as the above-mentioned image sharpening method, this embodiment provides an image sharpening device, including the following units:

[0181] An image acquisition unit 101, configured to acquire an image in YUV format;

[0182] A local feature information calculation unit 102, configured to divide the image into multiple windows and calculate the local feature information of the Y component of the image within each window;

[0183] A detail edge extraction unit 103, configured to extract the detail layer and the edge layer of the Y component of the image within each window using filters of different frequency bands;

[0184] A detail edge adjustment unit 104, configured to adjust the detail layer and the edge layer using the local information and the motion information of the image to obtain an adjusted detail edge layer;

[0185] An overlay unit 105, configured to add the detail edge layer and the Y component of the image to obtain a sharpened Y component;

[0186] An overshoot suppression unit 106, configured to perform overshoot suppression processing on the sharpened Y component to obtain an overshoot-suppressed Y component;

[0187] A color correction unit 107, configured to perform proportional correction on the UV components of the image according to the ratio between the overshoot-suppressed Y component and the Y component of the image to obtain corrected UV components;

[0188] An output unit 108, configured to output image data composed of the overshoot-suppressed Y component and the corrected UV components.

[0189] An image sharpening device provided in this embodiment can, by fully detecting and statistically analyzing the local feature information of the Y component of an image and by means of the motion information of an externally input image, apply a reasonable and appropriate enhancement style to different image contents within a local range, thereby making the enhancement strengths of the moving and stationary regions show differences, making the edge contour regions appear sharp and smooth, and making the detail texture regions appear delicate and natural. Through overshoot suppression processing and correction of the UV components, defects and flaws such as black and white edges, reduced signal-to-noise ratio, noise amplification in flat regions, and saturation loss, which are easily introduced by traditional methods, can be effectively avoided.

[0190] Preferably, the local feature information calculation unit 102 is configured to perform the following steps: calculate respectively the local maximum value, local minimum value, local average value, local standard deviation, local frequency value, and local direction consistency of the Y component of the image within each window. Using these local feature information, a reasonable and appropriate enhancement style can be applied to different image contents.

[0191] Preferably, the detail edge extraction unit 103 is configured to perform the following steps: process the Y component of the image respectively using a high-frequency filter, a medium-frequency filter, and a low-frequency band-pass filter to obtain a high-frequency detail layer, a medium-frequency detail layer, a low-frequency detail layer, a high-frequency edge layer, a medium-frequency edge layer, and a low-frequency edge layer. Using filters of different frequency bands can extract various basic layers of the detail edges to be further processed by the detail edge adjustment unit 104.

[0192] In summary, an image sharpening method and device provided by the present invention can, by fully detecting and statistically analyzing the local feature information of the Y component of an image and by means of the motion information of an externally input image, apply a reasonable and appropriate enhancement style to different image contents within a local range, thereby making the enhancement strengths of the moving and stationary regions show differences, making the edge contour regions appear sharp and smooth, and making the detail texture regions appear delicate and natural. Through overshoot suppression processing and correction of the UV components, defects and flaws such as black and white edges, reduced signal-to-noise ratio, noise amplification in flat regions, and saturation loss, which are easily introduced by traditional methods, can be effectively avoided.

[0193] The above description is only a description of the preferred embodiments of the present invention and does not limit the scope of the present invention in any way. Any changes and modifications made by those of ordinary skill in the art based on the above disclosure shall fall within the protection scope of the present invention.

Claims

1. An image sharpening method, characterized in that, It includes the following steps: S1. Obtain an image in YUV format; S2. Divide the image into multiple windows, and calculate the local feature information of the Y component of the image within each window; S3. Use filters of different frequency bands to extract the detail layer and the edge layer of the Y component of the image within each window; S4. Adjust the detail layer and the edge layer by using the local information and the motion information of the image to obtain an adjusted detail edge layer; S5. Add the detail edge layer and the Y component of the image to obtain a sharpened Y component; S6. Perform overshoot suppression processing on the sharpened Y component to obtain an overshoot-suppressed Y component; S7. According to the ratio between the overshoot-suppressed Y component and the Y component of the image, perform proportional correction on the UV components of the image to obtain corrected UV components; S8. Output the image data composed of the overshoot-suppressed Y component and the corrected UV components.

2. The image sharpening method according to claim 1, wherein The calculation of the local feature information of the Y component of the image within each window in step S2 includes the following steps: respectively calculate the local maximum value, local minimum value, local average value, local standard deviation, local frequency value, and local direction consistency of the Y component of the image within each window.

3. The image sharpening method according to claim 2, characterized in that, Step S3 includes the following steps: respectively use a high-frequency filter, a medium-frequency filter, and a low-frequency band-pass filter to process the Y component of the image to obtain a high-frequency detail layer, a medium-frequency detail layer, a low-frequency detail layer, a high-frequency edge layer, a medium-frequency edge layer, and a low-frequency edge layer.

4. The image sharpening method according to claim 3, wherein Step S4 includes the following steps: Perform first-frequency synthesis on the high-frequency detail layer, the medium-frequency detail layer, the low-frequency detail layer, the high-frequency edge layer, the medium-frequency edge layer, and the low-frequency edge layer in a weighted average manner to obtain a highest-frequency static detail layer, a lowest-frequency static detail layer, a highest-frequency motion detail layer, a lowest-frequency motion detail layer, a highest-frequency static edge layer, a lowest-frequency static edge layer, a highest-frequency motion edge layer, and a lowest-frequency motion edge layer; Perform second-frequency synthesis on the highest-frequency static detail layer, the lowest-frequency static detail layer, the highest-frequency motion detail layer, the lowest-frequency motion detail layer, the highest-frequency static edge layer, the lowest-frequency static edge layer, the highest-frequency motion edge layer, and the lowest-frequency motion edge layer by using the local frequency value to obtain a static detail layer, a motion detail layer, a static edge layer, and a dynamic edge layer; Perform motion synthesis on the static detail layer, the motion detail layer, the static edge layer, and the dynamic edge layer by using the motion information of the image to obtain a motion-synthesized detail layer and a motion-synthesized edge layer; Adjust the sharpness intensity of the motion-synthesized detail layer by using the local standard deviation to obtain an enhanced detail layer; Perform remapping on the enhanced detail layer to obtain a remapped detail layer; Adjust the sharpness intensity of the motion-synthesized edge layer by using the local standard deviation to obtain an enhanced edge layer; Remap the enhanced edge layer to obtain a remapped edge layer; Use the local direction consistency information to perform direction synthesis on the remapped detail layer and the remapped edge layer to obtain an adjusted detail edge layer.

5. The image sharpening method according to claim 1, wherein Step S5 includes the following steps: Add the detail edge layer and the Y component of the image according to the following formula, Ysharp = Yin + AC where Ysharp is the sharpened Y component, Yin is the original Y component of the image, and AC is the adjusted detail edge layer.

6. The image sharpening method according to claim 5, wherein, Step S6 includes the following steps: Perform overshoot suppression processing on the sharpened Y component according to the following formula, where Ysharpsc is the Y component after overshoot suppression, Ymin is the local minimum value of the original Y component, Ymax is the local maximum value of the original Y component, and SCGain is the total suppression gain.

7. The image sharpening method according to claim 6, wherein Step S7 includes the following steps: Perform proportional correction on the UV components of the image according to the following formula, where UCorrect is the corrected U component value and VCorrect is the corrected V component value.

8. An image sharpening device, characterized in that Includes the following units: An image acquisition unit for acquiring an image in YUV format; A local feature information calculation unit for dividing the image into multiple windows and calculating the local feature information of the Y component of the image within each window; A detail edge extraction unit for using filters of different frequency bands to extract the detail layer and the edge layer of the Y component of the image within each window; A detail edge adjustment unit for adjusting the detail layer and the edge layer using the local information and the motion information of the image to obtain an adjusted detail edge layer; An overlay unit for adding the detail edge layer and the Y component of the image to obtain a sharpened Y component; An overshoot suppression unit for performing overshoot suppression processing on the sharpened Y component to obtain a Y component after overshoot suppression; A color correction unit for performing proportional correction on the UV components of the image according to the ratio between the Y component after overshoot suppression and the Y component of the image to obtain corrected UV components; An output unit for outputting image data composed of the Y component after overshoot suppression and the corrected UV components.

9. The image sharpening device according to claim 8, wherein The local feature information calculation unit is used to perform the following steps: Calculate the local maximum value, local minimum value, local average value, local standard deviation, local frequency value, and local direction consistency of the Y component of the image within each window respectively.

10. An image sharpening device according to claim 9, wherein, The detail edge extraction unit is used to perform the following steps: Process the Y component of the image using a high-frequency filter, a medium-frequency filter, and a low-frequency band-pass filter respectively to obtain a high-frequency detail layer, a medium-frequency detail layer, a low-frequency detail layer, a high-frequency edge layer, a medium-frequency edge layer, and a low-frequency edge layer.

Images