Image processing method and apparatus

Abstract

The application discloses an image processing method and device. In an edge image of an original image, an original image edge pixel position coordinate of a first pixel block occupied by an image edge is determined; in an enlarged image of the edge image, a new image edge pixel position coordinate of a second pixel block occupied by an enlarged image edge corresponding to the first pixel block is calculated; a new edge slope of the enlarged image edge in the enlarged image is calculated; a gray value of the second pixel block corresponding to the new image edge pixel position coordinate is calculated according to the new edge slope and the new image edge pixel position coordinate; and a sub-pixel rendering is performed on surrounding pixel blocks along the enlarged image edge. The technical scheme of the application can be used in image enlargement and character display enlargement processes, and can effectively reduce the sawtooth problem of a result image and smooth the image region edge.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method and apparatus. Background Technology

[0002] Nowadays, with the rapid development of display devices, in order to meet the requirements of various high-resolution display devices such as 1080P, 4K or 8K, many flat panel display devices include an image magnification process, which magnifies non-standard low-resolution images and displays them on high-resolution devices.

[0003] As display resolutions continue to increase, the requirements for image display quality become more stringent. Therefore, the image magnification algorithm used by the display screen is a key factor determining display quality. Magnified images often exhibit increased jagged edges, blurred image edges, and blurred text. Directly using these images will severely affect the display effect of the display device.

[0004] To enlarge a smaller image, we can use the color values ​​of the original NxM neighboring pixels to find the positions of new pixels according to the magnification factor, and then estimate and fill in the pixels between them—this is pixel interpolation. There are many different algorithms for interpolation methods used for unknown pixels, the most common being nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. Nearest neighbor interpolation is the simplest and fastest, but the enlarged image may exhibit noticeable distortions such as pixelation and jagged edges. Bilinear and bicubic interpolation increase the number of pixels calculated, significantly improving the visual effect, but they can cause some blurring of image edges and text.

[0005] Image upscaling algorithms have a long history and come in many forms. A common approach is to take 4 or 9 pixels and recalculate new interpolation points based on the correlation with surrounding pixels. To achieve better image upscaling quality, many new algorithms have been proposed. These new algorithms typically improve the realism of pixel interpolation by expanding the range of reference pixels and using more complex interpolation calculations. However, they all suffer from the inherent defects of blurred edges or jagged edges found in traditional image upscaling algorithms. Whether it's a linear weighted averaging algorithm or a multi-step, convolutional iterative algorithm, these approaches only address the image itself and still cannot solve the aforementioned inherent defects.

[0006] In addition, subpixel anti-aliasing rendering technology has been used in products such as character display, like Microsoft's ClearType, but it has not been used for edge smoothing of magnified images. Summary of the Invention

[0007] One of the objectives of this invention is to overcome the shortcomings of the prior art and to provide an image processing method and apparatus to address the problems of blurred edges and jagged edges in images magnified by image magnification algorithms in the prior art.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0009] In a first aspect, the present invention provides an image processing method, the image processing method comprising:

[0010] In the edge image of the original image, determine the original image edge pixel position coordinates of the first pixel block occupied by the image edge;

[0011] In the magnified image of the edge image, calculate the new image edge pixel position coordinates of the second pixel block occupied by the magnified image edge corresponding to the first pixel block;

[0012] Calculate the new edge slope of the magnified image's edges;

[0013] Based on the new edge slope and the new image edge pixel position coordinates, calculate the gray value of the second pixel block corresponding to the new image edge pixel position coordinates;

[0014] Subpixel rendering is performed on the surrounding pixel blocks along the edge of the magnified image.

[0015] In a preferred embodiment of this application, the process of obtaining the edge image is as follows:

[0016] Set the image magnification factor;

[0017] Based on the image magnification ratio, the original image is divided into multiple pixel macroblocks of the same size;

[0018] Based on the image magnification ratio, edge detection is performed on the multiple pixel macroblocks to obtain the edge image of the original image.

[0019] In a preferred embodiment of this application, the process for determining the original image edge pixel position coordinates of the first pixel block is as follows:

[0020] The RGB threshold of the image edges is preset;

[0021] Scan each pixel macroblock, and determine the position where the edge operator calculation result of the pixel macroblock is greater than the RGB threshold as the first pixel block occupied by the image edge;

[0022] Calculate the coordinates of the edge pixels in the original image corresponding to the first pixel block.

[0023] In a preferred embodiment of this application, the coordinates of the edge pixels of the new image corresponding to the second pixel block are the coordinates obtained by taking the product of the coordinates of the edge pixels of the original image corresponding to the first pixel block and the image magnification factor of the magnified image, and rounding it up.

[0024] In a preferred embodiment of this application, the slope of the new edge is the quotient of the sum of the y-values ​​and the sum of the x-values ​​of all new edge pixel position coordinates in the magnified image.

[0025] In a preferred embodiment of this application, the grayscale value of the second pixel block is calculated in the following manner:

[0026] When the coordinates of the second pixel block are less than the coordinates of the edge pixel position of the new image (Xs-1, Ys-1), equal to the coordinates of the edge pixel position of the new image (Xs, Ys), or greater than the coordinates of the edge pixel position of the new image (Xs+1, Ys+1), the gray value of the second pixel block is calculated using the pixel proximity method.

[0027] When the coordinates of the second pixel block are less than the coordinates (Xs, Ys) of the edge pixel position of the new image,

[0028] If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) of the new image edge is calculated using the x-distance weighted mixing method.

[0029] If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs, Ys-1) of the new image edge is calculated using the y-distance weighted mixing method.

[0030] If the slope of the new edge is 1 or is a straight line in the x or y direction, the pixel proximity method is used to calculate the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) and (Xs, Ys-1) of the new image edge;

[0031] When the coordinates of the second pixel block are greater than the coordinates (Xs, Ys) of the edge pixel position of the new image,

[0032] If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs+1, Ys) of the new image edge is calculated using the x-distance weighted mixing method.

[0033] If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs, Ys+1) of the new image edge is calculated using the y-distance weighted mixing method.

[0034] If the new edge slope = 1, use a first-order or second-order sub-pixel rendering anti-aliasing method to calculate the grayscale value of the second pixel block corresponding to the edge pixel position coordinates (Xs-1, Ys) and (Xs, Ys-1) of the new image.

[0035] In a preferred embodiment of this application, the process of obtaining the magnified image is as follows:

[0036] Set the image magnification factor;

[0037] Based on the image magnification factor, the edge image is magnified along the image scanning direction to obtain a magnified image.

[0038] In a second aspect, the present invention provides an image processing device, the image processing device comprising a first coordinate calculation module, a second coordinate calculation module, an edge operator and an edge slope calculation module, a grayscale value calculation module, and a sub-pixel rendering module;

[0039] The first coordinate calculation module is used to determine the original image edge pixel position coordinates of the first pixel block occupied by the image edge in the edge image of the original image;

[0040] The second coordinate calculation module is used to calculate the new image edge pixel position coordinates of the second pixel block occupied by the magnified image edge corresponding to the first pixel block in the magnified image of the edge image;

[0041] The edge operator and edge slope calculation module are used to calculate the new edge slope of the magnified image edge;

[0042] The grayscale value calculation module is used to calculate the grayscale value of the second pixel block corresponding to the edge pixel position coordinates of the new image based on the new edge slope and the new image edge pixel position coordinates;

[0043] The subpixel rendering module is used to perform subpixel rendering on the surrounding pixel blocks along the edge of the magnified image.

[0044] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the image processing method as described in the first aspect.

[0045] Fourthly, the present invention provides a computer program product comprising a computer program that, when run on a computer, causes the computer to perform the image processing method as described in the first aspect.

[0046] The image processing method and apparatus disclosed in this invention can be used in the process of image magnification and character display magnification, and can effectively reduce the jagged edges of the resulting image and smooth the edges of the image area. Attached Figure Description

[0047] The present invention is described with reference to the following figures:

[0048] Figure 1 This is a flowchart of the image processing method in Embodiment 1 of the present invention;

[0049] Figure 2 This is a display effect diagram of the image edges after the magnified image has been processed using bilinear interpolation algorithm or bicubic interpolation algorithm in the existing technology;

[0050] Figure 3 This is a display diagram showing the image processing effect of the image processing method of Embodiment 1 of the present invention on the magnified image;

[0051] Figure 4 This is a schematic diagram of the original image to be processed in the image processing method of Embodiment 1 of the present invention;

[0052] Figure 5 This is a schematic diagram of the edge image in the image processing method of Embodiment 1 of the present invention;

[0053] Figure 6 This is a schematic diagram of pixel macroblocks of the original image in the image processing method of Embodiment 1 of the present invention;

[0054] Figure 7 This is a schematic diagram of magnifying pixel macroblocks in the image processing method of Embodiment 1 of the present invention;

[0055] Figure 8 This is a schematic diagram of image processing using ClearType in existing technology;

[0056] Figure 9 This is a schematic diagram of sub-pixel rendering of a pixel macroblock in the image processing method of Embodiment 1 of the present invention;

[0057] Figure 10 This is an actual display effect diagram of the first implementation of the image processing method of Embodiment 1 of the present invention, which performs image edge sub-pixel rendering on the magnified image;

[0058] Figure 11 This is an actual display effect diagram of the second implementation of the image processing method of Embodiment 1 of the present invention, which performs image edge sub-pixel rendering on the magnified image;

[0059] Figure 12 This is an actual display effect diagram of the third implementation of the image processing method of Embodiment 1 of the present invention, which performs image edge sub-pixel rendering on the magnified image;

[0060] Figure 13 This is an actual display effect diagram of the fourth implementation of the image processing method of Embodiment 1 of the present invention, which performs image edge sub-pixel rendering on the magnified image;

[0061] Figure 14 This is a schematic diagram of the image processing device in Embodiment 2 of the present invention.

[0062] Explanation of reference numerals in the attached figures:

[0063] 10 - First coordinate calculation module; 20 - Second coordinate calculation module; 30 - Edge operator and edge slope calculation module; 40 - Gray value calculation module; 50 - Sub-pixel rendering module. Detailed Implementation

[0064] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0065] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0066] Example 1

[0067] Embodiment 1 of the present invention discloses an image processing method that breaks away from the inherent image processing methods of traditional image magnification algorithms. It takes the perspective of flat panel display devices, namely the characteristics or shortcomings of display devices where single-pixel display cannot be infinitely small or infinitely segmented.

[0068] See Figure 1 The image processing method of this embodiment 1 includes:

[0069] S101: In the edge image of the original image, determine the original image edge pixel position coordinates of the first pixel block occupied by the image edge;

[0070] S102: In the magnified image of the edge image, calculate the new image edge pixel position coordinates of the second pixel block occupied by the magnified image edge corresponding to the first pixel block;

[0071] S103: Calculate the slope of the new edge of the magnified image.

[0072] S104: Calculate the gray value of the second pixel block corresponding to the new edge pixel position coordinates based on the new edge slope and the new image edge pixel position coordinates;

[0073] S105: Perform subpixel rendering on the surrounding pixel blocks along the edge of the magnified image.

[0074] Specifically, the image processing method in Embodiment 1 aims to overcome the edge blurring in enlarged images caused by traditional algorithms (even though the edges in the original image are sharp). It abandons the traditional fixed isotropic calculation expression for all calculation regions (4 or 9 points) of the entire image, which inevitably produces new gray levels not present in the original image. These new gray levels are distributed on both sides of the edges where light and dark meet in the enlarged image, resulting in certain image edge blurring and text blurring.

[0075] The image processing method in Embodiment 1, based on the characteristics of visual perception, stipulates that any curve or straight line with a width of 1 pixel will still have a width of 1 pixel after magnification. The determination rule is that any curve or straight line with a width of 1 pixel in the original image that remains 1 pixel wide after magnification will have its enclosed shape magnified. Conversely, any curve or straight line in the original image with a width greater than 1 pixel will have its magnification multiplied by the original line width, and its enclosed shape will also be magnified. This method compromises on jagged edges and edge blurring. While considering signal spectrum fidelity, it also takes into account human subjective perception and preferences; for example, image edges should be sharp, while eliminating the jaggedness of long beveled edges. This maintains the original image edge width, preventing it from widening due to scaling up, and preserving the original image's original appearance. After the edge image of the original image is magnified into a magnified image, the first pixel block occupied by the image edge will be magnified into a second pixel block in the magnified image. As described above, both the first and second pixel blocks are 1 pixel wide. This ensures that the image information expressed by the pixel blocks in the magnified image will not be distorted or deformed compared to the pixel blocks in the original image, thus eliminating jagged edges and blurred edges. The image edge in the original image becomes the magnified image edge in the magnified image after magnification. The magnified image edge has a new edge slope. By combining the new edge slope with the new edge pixel position coordinates of the second pixel block in the magnified image, the grayscale value of the first pixel block closest to the second pixel block can be calculated.

[0076] The image processing method in Embodiment 1, based on image edge detection and edge coordinate expansion and the sub-pixel structure of flat panel display devices, adjusts the R / G / B sub-pixel weight ratio of the pixels on both sides of the pixel edge at the magnified image edge, visually achieving the purpose of reducing jaggedness and smoothing edges. Figure 2 and Figure 3 As shown, compared to Figure 2 The existing methods for displaying image edges after processing magnified images using bilinear or bicubic interpolation algorithms are as follows. Figure 3The image processing method adopted in Embodiment 1 further reduces jagged edges when processing the magnified image edges through sub-pixel anti-aliasing rendering, while maintaining the sharpness of the magnified character or image edges, making the magnified image edges smoother. It effectively balances and solves the problems of image edge blurring and jagged edges. When applied to integrated circuits, it provides good image magnification display effect and is easy to implement at low cost. Under the premise of slightly increasing the amount of computation, it obtains relatively clear magnified image quality, especially with a significant improvement effect on the display of details and characters in the magnified image.

[0077] In the image processing method of this embodiment 1, the process of obtaining the edge image is as follows: first, set the image magnification factor; then, according to the image magnification factor, divide the original image into multiple pixel macroblocks of the same size; finally, according to the image magnification factor, perform edge detection on the multiple pixel macroblocks to obtain the edge image of the original image. Figure 4 and Figure 5 As shown, Figure 4 It is one of the pixel macroblocks in the original image without edge processing. Figure 5 It is the edge image corresponding to the pixel macroblock obtained after edge processing. Figure 4 The rice grains shown in the image are... Figure 5 The image displayed shows the edge contours of rice grains. For image magnification less than 2x, the original image can be divided using a (2*2)x(2*2) method for macroblock edge detection; for image magnification less than 3x but greater than 2x, the original image can be divided using a (2*3)x(2*3) method for macroblock edge detection; and for image magnification greater than 3x, the original image can be divided using a (2*4)x(2*4) method for macroblock edge detection. After detecting the image edges of the original image, the magnified image edges are reconstructed using an expansion method based on the edge trends of the original image edges.

[0078] In the image processing method of this embodiment 1, the process of obtaining the magnified image is as follows: first, the image magnification factor is set; then, according to the image magnification factor, the edge image is magnified along the image scanning direction to obtain the magnified image. During the magnification of the original image, the input of the original image and the output of the magnified image are kept synchronized. First, N rows of pixel regions are read into the row buffer according to the scanning direction of the original image; then, the input video pixel values ​​are read from the original image according to the raster scanning order. These data will be stored in the on-chip row buffer, where each operates as a FIFO (First Input First Output). When detecting the edges of the original image, the edge detection algorithm used is, for example, the Sobel operator, the Kirsch operator, or the Laplacian operator.

[0079] In the image processing method of this embodiment 1, the process of determining the original image edge pixel position coordinates of the first pixel block is as follows: First, a pre-set RGB threshold for the image edge is established; then, each pixel macroblock is scanned, and the position where the edge operator calculation result of the pixel macroblock is greater than the RGB threshold is determined as the first pixel block occupied by the image edge; then, the original image edge pixel position coordinates corresponding to the first pixel block are calculated. Taking a 3x3 pixel macroblock as an example, such as... Figure 6 As shown, the image edges occupy three first pixel blocks in the edge image, and their original image edge pixel position coordinates are (x0, y0), (x1, y1), and (x2, y2), respectively.

[0080] In the image processing method of this embodiment 1, the coordinates of the edge pixels of the new image corresponding to the second pixel block are the product of the coordinates of the edge pixels of the original image corresponding to the first pixel block and the image magnification factor of the magnified image, rounded up. See also Figure 7 , Figure 7 The left side shows a Cartesian coordinate system established with the magnified image, using the top left corner (0,0) as the image reference point and the bottom right corner as the direction of image magnification, i.e., the image scanning direction. Figure 7 The right side shows a macroblock of a zoomed-in image, with the edges of the zoomed-in image occupying four second-pixel blocks. Figure 6 The coordinates (Xo, Yo) of the edge pixel position of the first pixel block in the original image. Figure 7 The coordinates (Xs, Ys) of the new image edge pixels of the second pixel block can be calculated based on the image magnification factor, using the first pixel block as a reference. That is, the new image edge pixel coordinates (Xs, Ys) = [rounded up](original image edge pixel coordinates (Xo, Yo) * image magnification factor). For example, with an image magnification factor of 4 / 3 = 1.33 times... Figure 6 The first pixel block with edge pixel coordinates (x0, y0) in the original image, after being magnified, is... Figure 7 The corresponding second pixel block in the new image still has the edge pixel coordinates (X0, Y0); the first pixel block with the edge pixel coordinates (x1, y1) in the original image, after being enlarged, appears... Figure 7 The corresponding second pixel block in the new image should have edge pixel coordinates that are (X2, Y2) obtained by rounding up from (X1.33, Y1.33); the first pixel block with edge pixel coordinates of (x2, y2) in the original image, after being enlarged, will have... Figure 7 The coordinates of the new image edge pixel position of the corresponding second pixel block should be (X3, Y3) obtained by rounding up (X2.66, Y2.66). Here, the y-distance weight is equal to the fractional part of Yo, and the x-distance weight is equal to the fractional part of Xo. It can be observed that... Figure 7The second pixel block corresponding to the edge pixel position coordinates (X1, Y1) in the new image is in Figure 6 Since there is no corresponding first pixel block in the image, pixel interpolation is required to calculate its grayscale value. The image processing method in this embodiment 1 calculates the grayscale value of each second pixel block in the pixel macroblock under various conditions by calculating the new edge slope of the edge trend formed by each second pixel block in the magnified image.

[0081] In the image processing method of this embodiment 1, the slope of the new edge is the quotient of the sum of the y-coordinates and the sum of the x-coordinates of all new edge pixel positions in the magnified image. The formula for calculating the slope of the new edge is: Slope of the new edge = sum(y-coordinate value of the magnified image edge) / sum(x-coordinate value of the magnified image edge). "Y-coordinate value of the magnified image edge" refers to the sum of the y-coordinates of the new edge pixel positions in each second pixel block occupied by the magnified image edge in the magnified image, and "x-coordinate value of the magnified image edge" refers to the sum of the x-coordinates of the new edge pixel positions in each second pixel block occupied by the magnified image edge in the magnified image. The quotient of the two is the slope of the new edge. If all the x-coordinate values ​​or all the y-coordinate values ​​of the magnified image edge are the same, then the line indicator in the x or y direction is 1.

[0082] In the image processing method of this embodiment 1, the grayscale value of the second pixel block is calculated in the following way:

[0083] 1. When the coordinates of the second pixel block are less than (Xs-1, Ys-1), equal to (Xs, Ys), or greater than (Xs+1, Ys+1) the coordinates of the edge pixel of the new image, the gray value of the second pixel block is calculated using the pixel proximity method.

[0084] 2. When the coordinates of the second pixel block are less than the coordinates (Xs, Ys) of the edge pixel position in the new image,

[0085] If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) of the new image edge is calculated using the x-distance weighted mixing method.

[0086] If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinate (Xs, Ys-1) of the new image edge is calculated using the y-distance weighted mixing method.

[0087] If the slope of the new edge is 1 or is a straight line in the x or y direction, use the pixel proximity method to calculate the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) of the new image edge and (Xs, Ys-1);

[0088] 3. When the coordinates of the second pixel block are greater than the coordinates (Xs, Ys) of the edge pixel position in the new image,

[0089] If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs+1, Ys) of the new image edge is calculated using the x-distance weighted mixing method.

[0090] If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinate (Xs, Ys+1) of the new image edge is calculated using the y-distance weighted mixing method.

[0091] If the slope of the new edge is 1, use first-order or second-order subpixel rendering anti-aliasing to calculate the grayscale value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) and (Xs, Ys-1) of the new image edge.

[0092] Using the above method, for the second pixel block in the magnified image that does not correspond to the first pixel block, different pixel interpolation algorithms are used according to the different pixel position coordinates of the new image edge of the second pixel block, such as the nearest neighbor repeating method or bilinear interpolation method, to fill the pixel position coordinates of the new image edge of the magnified image and calculate the gray value of the remaining second pixel block in the magnified image.

[0093] In the image processing method of this embodiment 1, a sub-pixel anti-aliasing rendering method applied to character display, such as Microsoft's ClearType, is used to achieve a smoothing effect on the edges of the magnified image. For Figure 5 For edge images, if a pixel region containing image edges exists within a single pixel macroblock, then the third pixel block occupied by these pixel regions needs to be filled. Based on the image position coordinates of these third pixel blocks in the magnified image and the distance between the new image edge pixel position coordinates of the second pixel block, the grayscale weight ratios of the R / G / B sub-pixels of the third pixel blocks located on both sides of the image edge are adjusted. The adjustment methods include strengthening or weakening. The adjustment rules for image edge sub-pixel rendering (SPR) are as follows:

[0094] 1. If the image position coordinates of the third pixel block are greater than the new image edge pixel position coordinates (Xs-1, Ys-1) of the second pixel block, but less than the new image edge pixel position coordinates (Xs-0, Ys-0),

[0095] If the slope of the new edge is less than 1, then the pixel position coordinates (Xs-1, Ys) of the new image edge are subjected to sprAA (anti-aliasing) of 2 sub-pixels using the x-distance weighted mixing method.

[0096] If the slope of the new edge is 1 (45° / 135°), then the pixel position coordinates (Xs-1, Ys) of the new image edge will be sprAA for one sub-pixel.

[0097] If the slope of the new edge is 1 (45° / 135°), then the pixel position coordinates (Xs, Ys-1) of the new image edge will be sprAA for one sub-pixel.

[0098] If the slope of the new edge is greater than 1, then the pixel position coordinates (Xs, Ys-1) of the new image edge are sprAA performed on two sub-pixels using the y-distance weighted mixing method.

[0099] If the third pixel block is located on a straight line in the x or y direction, the edge pixel position coordinates (Xs, Ys-1) and (Xs-1, Ys) of the new image will not be spr.

[0100] 2. If the image position coordinates of the third pixel block are equal to the new image edge pixel position coordinates (Xs, Ys) of the second pixel block, then no spr is performed;

[0101] 3. If the image position coordinates of the third pixel block are greater than the position coordinates (Xs+1, Ys+1) of the new image edge pixel of the second pixel block, then SPR is not performed.

[0102] 4. If the image position coordinates of the third pixel block are greater than the new image edge pixel position coordinates (Xs+0, Ys+0) of the second pixel block, but less than the position (Xs+1, Ys+1),

[0103] If the slope of the new edge is less than 1, then the pixel position coordinates (Xs+1, Ys) of the new image edge are sprAA performed on two sub-pixels using the x-distance weighted mixing method.

[0104] If the slope of the new edge is 1 (45° / 135°), then the pixel position coordinates (Xs+1,Ys) of the new image edge will be sprAA for one sub-pixel.

[0105] If the slope of the new edge is 1 (45° / 135°), then the pixel position coordinates (Xs, Ys+1) of the new image edge will be sprAA for one sub-pixel.

[0106] If the slope of the new edge is greater than 1, then the pixel position coordinates (Xs, Ys+1) of the new image edge are sprAA performed on two sub-pixels using the y-distance weighted mixing method.

[0107] If the third pixel block is located on a straight line in the x or y direction, the edge pixel position coordinates (Xs, Ys-1) and (Xs-1, Ys) of the new image will not be spr.

[0108] The image processing method in Embodiment 1 eliminates points absent from the edge trend of the original image and replaces them with sub-pixel rendering of the image edge, maintaining the original edge direction or differential slope, and adding sub-pixel grayscale variations to the pixel positions surrounding the magnified image edge. (See [link to previous section]). Figures 9 to 13 When the PPI (pixels per inch) becomes too high, the human eye cannot distinguish subpixels; instead, it perceives the combined brightness of several neighboring subpixels. For example, with Microsoft's ClearType, only after the subpixels are widened to several pixels (unlike the method used in Example 1, which involves enlarging the image before rendering subpixels) can the human eye perceive reddish and bluish edges. Figure 8 As shown, Figure 8 Line 1 is a ClearType line, line 2 is a regular anti-aliased line; lines 3 and 4 are magnified four times, respectively, of lines 1 and 2; line 5 is a magnified view of line 1 actually displayed on an LCD screen. ClearType fully utilizes the color bar arrangement characteristics of LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode) displays a more perfect diagonal line. Figure 9 Taking a 2x2 pixel macroblock as an example, the gray area in the diagram represents the brightness of the pixel at the image edge after mixing three R / G / B sub-pixels. The R, G, and B colored bars represent the R / G / B sub-pixels within a single pixel. Adding one sub-pixel to the edge creates one spr AA (subpixel rendering anti-aliasing); adding two sub-pixels creates two spr AA. If spr needs adjustment, the grayscale values ​​of the pixel blocks adjacent to the image edge are analyzed, and the R / G / B sub-pixel grayscale adjustment values ​​are calculated. In the schematic diagram of image edge subpixel rendering spr, the grayscale changes of the colored subpixels represent the processed subpixel grayscale. The colored subpixel grayscale is slightly darker or brighter than the uncolored subpixel grayscale. For sloping image edges, the direction or differential slope of the image edge in the original image is maintained, and subpixel grayscale changes are added to the positions of the pixel blocks surrounding the image edge to reduce the edge jaggedness effect, thus completing the edge smoothing process. During processing, avoid increasing the brightness of points R or G along vertical lines, as this can easily lead to a reddish or greenish tinge at the image edges. For diagonal lines, the staggered arrangement of highlighted subpixels will create an average brightness effect across adjacent rows, reducing the reddish or greenish tinge at the edges; therefore, edge slope detection is necessary. Figure 10 The image processing method of Embodiment 1 is used to process... Figure 7 The image shown is an actual display effect of rendering the image edge subpixels using one spr AA. Figure 11 The image processing method of Embodiment 1 is used to process... Figure 7 The image shown is an actual display effect of rendering the image edge sub-pixels using two spr AA techniques on the magnified image. Figure 12 The image processing method of Embodiment 1 is used to process... Figure 7 The image shown is an actual display effect of rendering the image edge sub-pixels using one sprAA on the magnified image. Figure 12 The grayscale of a colored subpixel is darker than that of an uncolored subpixel, or the grayscale value of a subpixel in the original pixel is changed. Figure 13 The image processing method of Embodiment 1 is used to process... Figure 7 The image shown is a magnified image rendered using alternating 1 and 2 sprAA techniques to render the image edge subpixels. Compared to... Figure 7 Enlarged image, Figures 10 to 13 The magnified image appears smoother to the human eye.

[0109] Example 2

[0110] Embodiment 2 of the present invention discloses an image processing device that breaks away from the inherent image processing methods of traditional image magnification algorithms. It takes the perspective of flat panel display devices, namely the characteristics or shortcomings of display devices where single-pixel display cannot be infinitely small or infinitely segmented.

[0111] See Figure 14 The image processing device of this embodiment 2 includes a first coordinate calculation module 10, a second coordinate calculation module 20, an edge operator and edge slope calculation module 30, a grayscale value calculation module 40, and a sub-pixel rendering module 50. The first coordinate calculation module 10 is used to determine the original image edge pixel position coordinates of the first pixel block occupied by the image edge in the edge image of the original image. The second coordinate calculation module 20 is used to calculate the new image edge pixel position coordinates of the second pixel block occupied by the magnified image edge corresponding to the first pixel block in the magnified image. The edge operator and edge slope calculation module 30 is used to calculate the new edge slope of the magnified image edge in the magnified image. The grayscale value calculation module 40 is used to calculate the grayscale value of the second pixel block corresponding to the new image edge pixel position coordinates based on the new edge slope and the new image edge pixel position coordinates. The sub-pixel rendering module 50 is used to perform sub-pixel rendering on the surrounding pixel blocks along the edge of the magnified image.

[0112] Specifically, the image processing device in Embodiment 2 aims to overcome the edge blurring in enlarged images caused by traditional algorithms (even though the edges in the original image are sharp). It abandons the traditional fixed isotropic calculation expression for all calculation regions (4 or 9 points) of the entire image, which inevitably produces new gray levels not present in the original image. These new gray levels are distributed on both sides of the edges where light and dark meet in the enlarged image, resulting in certain image edge blurring and text blurring.

[0113] The image processing device in Embodiment 2, based on the characteristics of visual perception, stipulates that any curve or straight line with a width of 1 pixel will still have a width of 1 pixel after magnification. The determination rule is that any curve or straight line with a width of 1 pixel in the original image that remains 1 pixel wide after magnification will have its enclosed shape magnified. Conversely, any curve or straight line in the original image with a width greater than 1 pixel will have its magnification multiplied by the original line width, and its enclosed shape will also be magnified. This method compromises on jagged edges and edge blurring. While considering signal spectrum fidelity, it also takes into account human subjective perception and preferences; for example, image edges should be sharp, while eliminating the jaggedness of long beveled edges. This maintains the original image edge width, preventing it from widening due to scaling up, and preserving the original image's original appearance. After the edge image of the original image is magnified into a magnified image, the first pixel block occupied by the image edge will be magnified into a second pixel block in the magnified image. As described above, both the first and second pixel blocks are 1 pixel wide. This ensures that the image information expressed by the pixel blocks in the magnified image will not be distorted or deformed compared to the pixel blocks in the original image, thus eliminating jagged edges and blurred edges. The image edge in the original image becomes the magnified image edge in the magnified image after magnification. The magnified image edge has a new edge slope. By combining the new edge slope with the new edge pixel position coordinates of the second pixel block in the magnified image, the grayscale value of the first pixel block closest to the second pixel block can be calculated.

[0114] The image processing device in Embodiment 2 is based on image edge detection and edge coordinate expansion, and the sub-pixel structure of a flat panel display device. At the edge of the magnified image, the R / G / B sub-pixel weight ratio of the pixels on both sides of the pixel edge is adjusted to visually reduce jagged edges and achieve edge smoothing. Through sub-pixel anti-aliasing rendering, jagged edges are further reduced when processing the magnified image edge, while maintaining sharp edges of magnified characters or images, resulting in smoother edges. This effectively balances and solves the problems of image edge blurring and jagged edges. When applied to integrated circuits, it provides good image magnification and display effects and is easy to implement at low cost. With a slight increase in computational load, it achieves relatively clear magnified image quality, especially showing a significant improvement in the display of details and characters in the magnified image.

[0115] Example 3

[0116] Embodiment 3 of the present invention discloses a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the image processing method disclosed in Embodiment 1.

[0117] Example 4

[0118] Embodiment 4 of the present invention discloses a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the image processing method disclosed in Embodiment 1.

[0119] The image processing method and apparatus disclosed in this invention can be used in the process of image magnification and character display magnification, and can effectively reduce the jagged edges of the resulting image and smooth the edges of the image area.

[0120] It should be understood that the above description of specific embodiments of the present invention is only for illustrating the technical approach and features of the present invention, and is intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. However, the present invention is not limited to the specific embodiments described above. All changes or modifications made within the scope of the claims of the present invention should be covered within the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, The image processing method includes: In the edge image of the original image, determine the original image edge pixel position coordinates of the first pixel block occupied by the image edge; In the magnified image of the edge image, calculate the new image edge pixel position coordinates of the second pixel block occupied by the magnified image edge corresponding to the first pixel block; Calculate the new edge slope of the magnified image's edges; Based on the new edge slope and the pixel position coordinates of the new image edge, the grayscale value of the second pixel block corresponding to the pixel position coordinates of the new image edge is calculated; the grayscale value of the second pixel block is calculated in the following way: When the coordinates of the second pixel block are less than (Xs-1, Ys-1), equal to (Xs, Ys), or greater than (Xs+1, Ys+1), the grayscale value of the second pixel block is calculated using the pixel proximity method. When the coordinates of the second pixel block are less than the coordinates (Xs, Ys) of the edge pixel position of the new image, If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) of the new image edge is calculated using the x-distance weighted mixing method. If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs, Ys-1) of the new image edge is calculated using the y-distance weighted mixing method. If the slope of the new edge is 1 or is a straight line in the x or y direction, the pixel proximity method is used to calculate the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) of the new image edge and (Xs, Ys-1). When the coordinates of the second pixel block are greater than the coordinates (Xs, Ys) of the edge pixel position of the new image, If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs+1, Ys) of the new image edge is calculated using the x-distance weighted mixing method. If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs, Ys+1) of the new image edge is calculated using the y-distance weighted mixing method. If the new edge slope = 1, use the first-order or second-order sub-pixel rendering anti-aliasing method to calculate the gray value of the second pixel block corresponding to the edge pixel position coordinates (Xs-1, Ys) and (Xs, Ys-1) of the new image. Subpixel rendering is performed on the surrounding pixel blocks along the edge of the magnified image.

2. The image processing method according to claim 1, characterized in that, The process of obtaining the edge image is as follows: Set the image magnification factor; Based on the image magnification ratio, the original image is divided into multiple pixel macroblocks of the same size; Based on the image magnification ratio, edge detection is performed on the multiple pixel macroblocks to obtain the edge image of the original image.

3. The image processing method according to claim 1, characterized in that, The process of determining the position coordinates of the edge pixels in the original image of the first pixel block is as follows: The RGB threshold of the image edges is preset; Scan each pixel macroblock, and determine the position where the edge operator calculation result of the pixel macroblock is greater than the RGB threshold as the first pixel block occupied by the image edge; Calculate the coordinates of the edge pixels in the original image corresponding to the first pixel block.

4. The image processing method according to claim 1, characterized in that, The coordinates of the edge pixels of the new image corresponding to the second pixel block are the product of the coordinates of the edge pixels of the original image corresponding to the first pixel block and the magnification factor of the magnified image, and then rounded up.

5. The image processing method according to claim 1, characterized in that, The slope of the new edge is the quotient of the sum of the y-values ​​and the sum of the x-values ​​of all the new edge pixel position coordinates in the magnified image.

6. The image processing method according to claim 5, characterized in that, The process of obtaining the magnified image is as follows: Set the image magnification factor; Based on the image magnification factor, the edge image is magnified along the image scanning direction to obtain a magnified image.

7. An image processing apparatus, characterized in that, The image processing device includes a first coordinate calculation module, a second coordinate calculation module, an edge operator and edge slope calculation module, a grayscale value calculation module, and a sub-pixel rendering module. The first coordinate calculation module is used to determine the original image edge pixel position coordinates of the first pixel block occupied by the image edge in the edge image of the original image; The second coordinate calculation module is used to calculate the new image edge pixel position coordinates of the second pixel block occupied by the magnified image edge corresponding to the first pixel block in the magnified image of the edge image; The edge operator and edge slope calculation module are used to calculate the new edge slope of the magnified image edge; The grayscale value calculation module is used to calculate the grayscale value of the second pixel block corresponding to the edge pixel position coordinates of the new image based on the new edge slope and the new image edge pixel position coordinates; the grayscale value of the second pixel block is calculated in the following way: When the coordinates of the second pixel block are less than (Xs-1, Ys-1), equal to (Xs, Ys), or greater than (Xs+1, Ys+1), the grayscale value of the second pixel block is calculated using the pixel proximity method. When the coordinates of the second pixel block are less than the coordinates (Xs, Ys) of the edge pixel position of the new image, If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) of the new image edge is calculated using the x-distance weighted mixing method. If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs, Ys-1) of the new image edge is calculated using the y-distance weighted mixing method. If the slope of the new edge is 1 or is a straight line in the x or y direction, the pixel proximity method is used to calculate the gray value of the second pixel block corresponding to the pixel position coordinates (Xs-1, Ys) of the new image edge and (Xs, Ys-1). When the coordinates of the second pixel block are greater than the coordinates (Xs, Ys) of the edge pixel position of the new image, If the slope of the new edge is less than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs+1, Ys) of the new image edge is calculated using the x-distance weighted mixing method. If the slope of the new edge is greater than 1, the gray value of the second pixel block corresponding to the pixel position coordinates (Xs, Ys+1) of the new image edge is calculated using the y-distance weighted mixing method. If the new edge slope = 1, use the first-order or second-order sub-pixel rendering anti-aliasing method to calculate the gray value of the second pixel block corresponding to the edge pixel position coordinates (Xs-1, Ys) and (Xs, Ys-1) of the new image. The subpixel rendering module is used to perform subpixel rendering on the surrounding pixel blocks along the edge of the magnified image.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the image processing method as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when run on a computer, causes the computer to perform the image processing method as described in any one of claims 1 to 6.

Images