Image processing method and image processing apparatus

Abstract

An image processing method and an image processing apparatus are provided. The image processing method includes performing a first weighted average calculation on luminance data of a plurality of pixels in a first mask to obtain a background average value with respect to a target pixel, the target pixel being a center pixel of the first mask; performing a second weighted average calculation on luminance data of a plurality of pixels in a second mask to obtain a foreground average value with respect to the target pixel, the target pixel also being a center pixel of the second mask; generating a sharpness level based on a gain value and a difference between the luminance data of the target pixel and the foreground average value with respect to the target pixel; and adjusting the luminance data of the target pixel based on the sharpness level.

Description

Technical Field

[0001] This invention relates to an image processing technique, and more specifically, to an image processing method and apparatus for making lines in an image sharper. Background Technology

[0002] With technological advancements, displays can typically render images at a higher resolution than the original image source. Therefore, it's necessary to convert low-resolution images to high-resolution ones. During this conversion, lines in the image become thicker and blurrier as the resolution increases. Image sharpening filters are commonly used to prevent this blurring caused by resolution adjustments.

[0003] However, image sharpening filters may only make lines in an image sharper, but may not make lines thinner or more delicate. Summary of the Invention

[0004] The present invention provides an image processing method and an image processing apparatus that make lines on an image sharper and clearer, while also allowing the line width to be adjusted to make the lines thinner or more delicate.

[0005] According to an embodiment of the present invention, an image processing method includes the following steps: performing a first weighted average calculation on the brightness data of a plurality of pixels in a first mask to obtain a background average value for a target pixel, wherein the target pixel is the center pixel of the first mask; performing a second weighted average calculation on the brightness data of a plurality of pixels in a second mask to obtain a foreground average value for the target pixel, wherein the target pixel is also the center pixel of the second mask; generating a sharpness level based on a gain value and the difference between the brightness data of the target pixel and the foreground average value for the target pixel; and adjusting the brightness data of the target pixel according to the sharpness level.

[0006] According to an embodiment of the present invention, an image processing apparatus includes an image sharpening filter. The image sharpening filter includes a foreground averaging filter, a first calculation circuit, and a brightness adjustment circuit. The foreground averaging filter is configured to generate a foreground average value with respect to a target pixel. The first calculation circuit is coupled to the foreground averaging filter and configured to generate a sharpness level based on a gain value and the difference between the brightness data of the target pixel and the foreground average value with respect to the target pixel. The brightness adjustment circuit is configured to adjust the brightness data of the target pixel according to the sharpness level. The foreground averaging filter includes a background extractor and a second calculation circuit. The background extractor is configured to perform a first weighted average calculation on the brightness data of a plurality of pixels in a first mask to obtain a background average value with respect to the target pixel, the target pixel being the center pixel of the first mask. The second calculation circuit is configured to perform a second weighted average calculation on the brightness data of a plurality of pixels in a second mask based on the background average value with respect to the target pixel to obtain a foreground average value with respect to the target pixel, the target pixel also being the center pixel of the second mask.

[0007] Based on the above, this disclosure uses a first mask (e.g., a background mask) to perform a first weighted average calculation to obtain the background average value of the target pixel, and then uses a second mask (e.g., a foreground mask) and the foreground average value to perform a second weighted average calculation to obtain the foreground average value of the target pixel. Next, this disclosure generates a sharpness level based on the gain value and the difference between the brightness data of the target pixel and the foreground average value of the target pixel, so as to adjust the brightness data of the target pixel using the sharpness level. Therefore, this disclosure makes lines on an image sharper and clearer through the sharpness level, and also allows the line width to be adjusted by adjusting the gain value of the sharpness level, making the lines thinner or more refined. Attached Figure Description

[0008] Figure 1 This is a circuit block diagram of an image processing apparatus according to an embodiment of the present invention.

[0009] Figure 2 This is a schematic flowchart of an image processing method according to an embodiment of the present invention.

[0010] Figure 3 This is according to an embodiment of the present invention. Figure 2 A schematic diagram of the target pixel, the first mask, and the second mask in the middle step.

[0011] Figure 4 This is according to an embodiment of the present invention. Figure 2 A schematic diagram of the brightness data of the target pixel in the intermediate step, with the gain value set to "1".

[0012] Figure 5 This is according to an embodiment of the present invention. Figure 4 A schematic diagram of the lines on the image.

[0013] Figure 6 This is according to an embodiment of the present invention. Figure 2 A schematic diagram of the brightness data of the target pixel in steps S210 to S240, with the gain value set to "3". Detailed Implementation

[0014] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component reference numerals are used in the drawings and description to denote the same or similar parts.

[0015] The term "coupled (or connected)" as used throughout this application (including the claims) can refer to any direct or indirect connection. For example, if it is described that a first device is coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices or certain connection methods. The terms "first," "second," etc., used throughout this application (including the claims) are used only to name components or distinguish different embodiments or scopes, and are not intended to limit the upper or lower limit of the number of components, nor to limit the order of components. Furthermore, whenever possible, elements / components / steps using the same reference numerals in the drawings and embodiments denote the same or similar parts. In different embodiments, the same reference numerals or the same terms may be used to refer to related descriptions of elements / components / steps.

[0016] Figure 1 This is a circuit block diagram of an image processing apparatus 100 according to an embodiment of the present invention. The image processing apparatus 100 is implemented on a display device, graphics processing chip, display processing chip, or related electronic device. The image processing apparatus 100 can be implemented as various logic blocks, modules, and circuits in one or more controllers, microcontrollers, microprocessors, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), central processing units (CPUs), and / or other processing units. The image processing apparatus 100 includes an image sharpening filter 102. The image sharpening filter 102 generates adjusted brightness data LY' of a target pixel.

[0017] The image sharpening filter 102 includes a foreground averaging filter 110, a first calculation circuit 120, and a brightness adjustment circuit 128, all implemented as digital logic circuits. The foreground averaging filter 110 generates a foreground average value FY with respect to the target pixel Y. The first calculation circuit 120 is coupled to the foreground averaging filter 110.

[0018] In this embodiment, the first calculation circuit 120 includes a first inverter 122, a first adder 124, and a first multiplier 126. The brightness adjustment circuit 128 may be an adder, for example. The input node of the first inverter 122 is coupled to the output node of the foreground averaging filter 110. One input node of the first adder 124 is coupled to the output node of the first inverter 122. The other input node of the first adder 124 is used to receive the brightness data LY of the target pixel Y. One input node of the first multiplier 126 is coupled to the output node of the first adder 124. The other input node of the first multiplier 126 is used to receive the gain value GAIN. One input node of the brightness adjustment circuit 128 is coupled to the output node of the first multiplier 126. The other input node of the brightness adjustment circuit 128 is used to receive the brightness data LY of the target pixel Y.

[0019] The foreground averaging filter 110 includes a background extractor 130 and a second calculation circuit 140. The background extractor 130 performs a first weighted average calculation on multiple brightness data of pixels in the first mask to obtain a background average value ABK. In this embodiment, the second calculation circuit 140 includes a second inverter 142, a second adder 144, an absolute value calculator 146, and a second multiplier 148. An input node of the second inverter 142 is coupled to an output node of the background extractor 130. One input node of the second adder 144 is coupled to an output node of the second inverter 142. Another input node of the second adder 144 is used to receive brightness data LY of the target pixel Y. An input node of the absolute value calculator 146 is coupled to an output node of the second adder 144. One input node of the second multiplier 148 is coupled to an output node of the absolute value calculator 146. Another input node of the second multiplier 148 is used to receive brightness data LY of the target pixel Y.

[0020] Figure 2 This is a schematic flowchart of an image processing method according to an embodiment of the present invention. Figure 2 Image processing methods are provided by Figure 1 The image processing device is implemented. Figure 3 According to an embodiment of the present invention Figure 2 A schematic diagram of the target pixel Y, the first mask BMK, and the second mask FMK in steps S210 and S220. Figure 3 In the diagram, the target pixel Y has its own coordinates, represented as (x, y).

[0021] Please refer to Figures 1 to 3 In step S210, Figure 1 Background extractor 130 pairs Figure 3Multiple brightness data of pixels in the first mask BMK are subjected to a first weighted average calculation to obtain a background average value ABK for a target pixel Y, which is the center pixel of the first mask BMK. For example, in this embodiment, the size of the first mask BMK is 35 pixels in length and 35 pixels in width (e.g., a mask with a size of 35x35 pixels).

[0022] The image processing apparatus using this embodiment can adjust the implementation of the first weighted average calculation as needed. In this embodiment, regarding the brightness data LY of the target pixel Y in the image, a first mask BMK of size 35x35 pixels is applied to the image centered on the target pixel Y(x, y). The first mask BMK is presented as a region (x-17:x+17, y-17:y+17). The first weighted average calculation is performed by calculating a weighted average of multiple brightness data of pixels in the region (x-17:x+17, y-17:y+17) using multiple first weights for each pixel in the region (x-17:x+17, y-17:y+17). In one embodiment, the first weights are assigned based on a Gaussian distribution centered on the target pixel Y(x, y). Pixels closer to the center of the first mask BMK (e.g., the target pixel Y(x, y)) have larger weights, while pixels farther from the center of the first mask BMK (e.g., the target pixel Y(x, y)) have smaller weights. The result of the first weighted average calculation is defined as the background average ABK(x, y). Similarly, the background average ABK(x+1, y) is the result of the first weighted average calculation of the target pixel Y(x+1, y) in the first mask BMK that is presented as the region (x+1-17:x+1+17, y-17:y+17).

[0023] To illustrate an exemplary block-based image, such as an image of size 3x3 pixels (given pixel coordinates E), let the image's brightness data LYA be represented in matrix form as Equation (1), let the first mask BMK be a mask of size 3x3 pixels, and let the first weight W1A (in matrix form) for each pixel be calculated by Equation (2).

[0024] …(1)

[0025] …(2)

[0026] The background average value ABK(1,1) of the target pixel Y(1,1) (e.g., the result of the first weighted average calculation of the target pixel Y(1,1)) is calculated by equation (3):

[0027] …(3)

[0028] In step S220, the second calculation circuit 140 of the foreground averaging filter 110 calculates the background average value ABK for the target pixel Y based on the background average value ABK. Figure 3 In the second mask FMK, the brightness data of multiple pixels are calculated using a second weighted average to obtain the foreground average value FY for the target pixel Y, which is also... Figure 3 The center pixel of the second mask FMK. For example, in an embodiment, the size of the second mask FMK is 11 pixels in length and 11 pixels in width (e.g., a mask with a size of 11x11 pixels).

[0029] In this embodiment, regarding the brightness data LY of the target pixel Y in the image, a second mask FMK of size n x n pixels is applied to the region centered on the target pixel Y(x, y). The following example is described based on a second mask FMK of size 3x3 pixels corresponding to the region (0:2, 0:2). In the second weighted average calculation, the second weight W2 for the target pixel Y(x, y) is a weight matrix and is calculated by equation (4):

[0030] …(4)

[0031] Here, [LY(x-1:x+1, y-1:y+1) - ABK(x,y)] represents the absolute difference ABSD(x-1:x+1, y-1:y+1), which is the result of subtracting the background average ABK(x-1:x+1, y-1:y+1) from the brightness data LY(x-1:x+1, y-1:y+1) of the target pixel Y. Specifically, the second inverter 142 inverts the background average ABK(x, y) into a negative background average -ABK(x, y), the second adder 144 adds the brightness data LY of the target pixel Y and the negative background average -ABK(x, y) as the result of the second adder 144, and the absolute value calculator 146 converts the result of the second adder 144 into its absolute value.

[0032] In the second mask FMK, pixels with a large difference from the background average ABK(x, y) (e.g., pixels or lines in the foreground) are given larger weights. The result of the second weighted average calculation for the target pixel Y(x, y) is defined as the foreground average FY(x, y).

[0033] The second multiplier 148 calculates the foreground average value FY(x, y) of the target pixel Y(x, y) based on the second weight W2(x, y) of the target pixel Y(x, y) and the brightness data LY of the target pixel Y(x, y). The second weight W2(x, y) is calculated based on the absolute value of the result of the second adder 144. In detail, the foreground average value FY(x, y) of the target pixel Y(x, y) is calculated by equation (5):

[0034] …(5)

[0035] Based on the example image of size 3x3 pixels presented in Equation (1), let the brightness data LYA of the center pixel (i.e., the target pixel) of the 3x3 pixel image be presented in matrix form in Equation (1) as described above, and let the background average value ABK(1, 1) of the target pixel Y(1, 1) be 4. Therefore, the absolute difference ABSD(0:2, 0:2) between the brightness data LY(0:2, 0:2) of the region centered on the target pixel Y(1, 1) and the background average value ABK(1, 1) is calculated by Equation (6):

[0036] …(6)

[0037] The second weight W2(1, 1) for the target pixel Y(1, 1) is calculated by equation (7) based on equation (4):

[0038] …(7)

[0039] For example, the foreground average value FY(1, 1) for the target pixel Y(1, 1) is calculated by equation (8) based on equation (5).

[0040] …(8)

[0041] In step S230, the first calculation circuit 120 generates a sharpness level based on the gain value GAIN and the difference between the brightness data LY of the target pixel Y and the foreground average value FY of the target pixel Y.

[0042] For example, let the brightness data LY of the target pixel Y be 5, let the foreground average value FY of the target pixel Y be 5.95 as presented in equation (8), and let the gain value GAIN be 1. Therefore, the sharpness level SHPLV is calculated as equation (9):

[0043] …(9)

[0044] In detail, the first inverter 122 inverts the foreground average value FY of the target pixel Y into the negative foreground average value -FY, the first adder 124 adds the brightness data LY of the target pixel Y to the negative foreground average value -FY as the result of the first adder 124, and the first multiplier 126 multiplies the result of the first adder 124 by the gain value GAIN to produce the sharpness level SHPLV.

[0045] In step S240, the brightness adjustment circuit 128 of the first calculation circuit 120 adjusts the brightness data LY of the target pixel Y according to the sharpness level SHPLV. The brightness adjustment circuit 128 adds the brightness data LY of the target pixel Y to the sharpness level SHPLV to generate the adjusted brightness data LY' of the target pixel. For example, if the brightness data LY of the target pixel Y is 5 and the sharpness level SHPLV is -0.95, then the adjusted brightness data LY' of the target pixel Y is the result of adding the sharpness level SHPLV to the brightness data LY of the target pixel Y, which is 4.05 (equal to 5 + (-0.95)).

[0046] Figure 4 According to one embodiment of the present invention, when the gain value is set to 1, Figure 2 A schematic diagram of the target pixel brightness data in steps S210 to S240. Figure 5 According to an embodiment of the present invention, Figure 4 A schematic diagram of the lines on the image. Figure 4 The X-axis represents the horizontal or vertical axis of the image in the embodiment, for example... Figure 5 The horizontal axis of the image, while Figure 4 The Y-axis display embodiment shows the brightness data of each pixel along one axis. Figure 5 Image 500 and line L0 in the image display the original image and its lines without processing by image sharpening filter 102. Figure 5 Image 510 and line L1 in the image mainly show Figure 5 Part (A) and Figure 4 The width WD1 of line L1 shown in section (A). Figure 5 The image 511 and line L1' in the image mainly display the image processed by the image processing device 100. Figure 5 Part (B) and Figure 4 The width WD2 of line L1' shown in section (B) is also shown. In this embodiment, for clarity of illustration, Figure 5 Lines L0, L1, and L1' are depicted as dark lines, and the areas in images 500, 510, and 520 other than lines L0, L1, and L1' are white. However, Figure 4 It is based on Figure 5 The lines L0, L1, and L1' are drawn; in fact, these lines are brighter relative to the background of the image.

[0047] Reference Figure 4 and Figure 5 ,exist Figure 4 In the upper section (A), line 410 displays the brightness data LY for each pixel in the image, and line 420 displays the foreground average value FY for each pixel in step S210. The difference between the foreground average value FY and the brightness data LY is shown, for example, by arrow 425 for target pixel Y1 and arrow 427 for another target pixel Y2. Figure 4 In the lower section (B), line 430 displays the adjusted brightness data LY' for each pixel along an axis (such as the horizontal axis) in the image when the gain value GAIN is 1. For example, the sharpness level SHPLV of pixel Y1 is shown with arrow 425', while the sharpness level SHPLV of pixel Y2 is shown with arrow 427'. Therefore, the width WD2 of line L1' is narrower than the width WD1 of lines L0 and L1.

[0048] Adjusting the gain value (GAIN) allows control over the width of the sharpening lines based on the target pixel's Y region. A higher gain value (GAIN) results in narrower sharpening lines. For example, Figure 6 According to one embodiment of the present invention, when the gain value is set to 3, Figure 2 A schematic diagram of the target pixel luminance data for steps S210 to S240. Line 630 shows the adjusted luminance data LY' of each pixel along one axis of the image when the applied gain value GAIN is 3. For example, in step S230, the luminance data LY of the target pixel Y is set to 5, the foreground average value FY of the target pixel Y is set to 5.95 calculated by equation (8), and the gain value GAIN is set to 3. Therefore, the calculation of the sharpness level SHPLV is presented as equation (10):

[0049] …(10)

[0050] In step S240 and Figure 6 In the image, the adjusted brightness data LY' of the target pixel Y is 2.15 (equal to 5-2.85), therefore... Figure 6 The width WD2' shown in the image is greater than... Figure 5 The width WD2 in part (B) is narrower.

[0051] Based on the above, this disclosure performs a first weighted average calculation using a first mask (e.g., a background mask) to obtain a background average value for the target pixel, and then performs a second weighted average calculation using a second mask (e.g., a foreground mask) and the foreground average value to obtain a foreground average value for the target pixel. Next, this disclosure generates a sharpness level based on the gain value and the difference between the brightness data of the target pixel and the foreground average value of the target pixel, so as to adjust the brightness data of the target pixel using the sharpness level. Therefore, this disclosure makes lines on an image sharper and clearer through the sharpness level, and also allows the line width to be adjusted by adjusting the gain value of the sharpness level, making the lines thinner or more refined.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that, include: A first weighted average calculation is performed on the brightness data of multiple pixels in the first mask to obtain the background average value for the target pixel, which is the center pixel of the first mask; A second weighted average calculation is performed on the brightness data of multiple pixels in the second mask based on the background average value of the target pixel, which is also the center pixel of the second mask; A sharpness level is generated based on the difference between the gain value and the brightness data of the target pixel and the foreground average value for the target pixel; as well as Adjust the brightness data of the target pixel according to the sharpness level.

2. The image processing method according to claim 1, characterized in that, The steps for performing the first weighted average calculation include: The background extractor applies a first weighted average filter to the brightness data of multiple pixels in the first mask.

3. The image processing method according to claim 1, characterized in that, The steps for performing the second weighted average calculation include: A second weighted averaging filter is applied to the brightness data of the plurality of pixels in the second mask, wherein the second weighted averaging filter is formed by the absolute difference between each brightness data of the plurality of pixels in the second mask and the average value of the background.

4. The image processing method according to claim 1, characterized in that, The step of generating the sharpness level based on the gain value and the difference between the brightness data of the target pixel and the foreground average value of the target pixel includes: The difference between the luminance data of the target pixel and the foreground average value for the target pixel is calculated as the original sharpness level; and The original sharpness level is multiplied by the gain value to produce the sharpness level.

5. The image processing method according to claim 1, characterized in that, The steps of adjusting the brightness data of the target pixel according to the sharpness level include: The brightness data of the target pixel is multiplied by the sharpness level to generate the adjusted brightness data of the target pixel.

6. An image processing apparatus, characterized in that, include: Image sharpening filter, The image sharpening filter mentioned above includes: A foreground averaging filter, configured to produce a foreground average value for the target pixel; A first computing circuit, coupled to the foreground averaging filter and configured to generate a sharpness level based on a gain value and the difference between the brightness data of the target pixel and the foreground average value with respect to the target pixel; and A brightness adjustment circuit is configured to adjust the brightness data of the target pixel according to the sharpness level. The foreground averaging filter includes a background extractor and a second calculation circuit. The background extractor is configured to perform a first weighted average calculation on multiple brightness data of multiple pixels in the first mask to obtain a background average value with respect to the target pixel, which is the center pixel of the first mask. The second calculation circuit is configured to perform a second weighted average calculation on multiple brightness data of multiple pixels in the second mask based on the background average value with respect to the target pixel, so as to obtain a foreground average value with respect to the target pixel, which is also the center pixel of the second mask.

7. The image processing apparatus according to claim 6, characterized in that, The background extractor applies a first weighted average filter to the brightness data of the plurality of pixels in the first mask.

8. The image processing apparatus according to claim 6, characterized in that, The second calculation circuit applies a second weighted averaging filter to the brightness data of the plurality of pixels in the second mask, wherein the second weighted averaging filter is formed by the absolute difference between each of the brightness data of the plurality of pixels in the second mask and the average value of the background.

9. The image processing apparatus according to claim 6, characterized in that, The first computing circuit calculates the difference between the brightness data of the target pixel and the foreground average value of the target pixel as the original sharpness level, and multiplies the original sharpness level by the gain value to generate the sharpness level.

10. The image processing apparatus according to claim 6, characterized in that, The brightness adjustment circuit adjusts the brightness data of the target pixel by multiplying the brightness data of the target pixel by the sharpness level, so as to generate the adjusted brightness data of the target pixel.

11. The image processing apparatus according to claim 6, characterized in that, The first computing circuit includes: A first inverter is coupled to the foreground averaging filter, wherein the first inverter inverts the foreground average value on the target pixel to a negative foreground average value; A first adder, coupled to the output of the first inverter, wherein the first adder adds the brightness data of the target pixel and the average value of the negative foreground as the result of the first adder; and A first multiplier, coupled to the first adder and receiving the gain value, wherein the first multiplier multiplies the result of the first adder by the gain value to produce the sharpness level.

12. The image processing apparatus according to claim 6, characterized in that, The second computing circuit includes: A second inverter is coupled to the background extractor, wherein the second inverter inverts the background average value to a negative background average value; A second adder is coupled to the second inverter, wherein the second adder adds the brightness data of the target pixel and the average value of the negative background as the result of the second adder; An absolute value calculator, coupled to the output of the second adder, wherein the absolute value calculator converts the result of the second adder into an absolute value; and A second multiplier, coupled to the output of the absolute value calculator, wherein the second multiplier calculates the foreground average value on the target pixel based on the second weight of the target pixel and the brightness data of the target pixel, and the second weight of the target pixel is the absolute value based on the result of the second adder.

Images