Image anti-aliasing method, device, apparatus, and computer program

Abstract

This application discloses an image anti-aliasing method, apparatus, device, medium, and program product, which belong to the image rendering field. The method includes the steps of: obtaining a first edge pixel of an image under HDR space (310); generating a first scaling factor according to a local brightness environment where the first edge pixel is located (320); generating a target color ratio corresponding to the first edge pixel according to brightness values ​​of a plurality of pixels in an image block where the first edge pixel is located (330); scaling the target color ratio according to the first scaling factor to obtain a first color ratio (340); mixing with the first color ratio according to a color value of the first edge pixel and a color value of a target pixel to obtain a first color value (350); and updating the first color value as the color value of the first edge pixel (360). The above solution resolves the conflict between the sharpness of image details and the anti-aliasing effect of highlight areas under the detail anti-aliasing branch.

Description

[Technical Field]

[0001] FIELD Embodiments of the present application relate to the field of image rendering, and more particularly to image anti-aliasing methods, apparatus, devices, media, and program products.

[0002] This application claims priority to a Chinese patent application filed on August 30, 2022, bearing application number 202211045046.6 and entitled "Image anti-aliasing method, device, equipment, medium, and program product," the entire contents of which are incorporated herein by reference. [Background technology]

[0003] Aliasing refers to the jagged appearance of drawn geometric lines due to a lack of pixel accuracy in the drawing pass of an image. The technical means of eliminating aliasing is called anti-aliasing (also known as anti-aliasing). In the screen rendering process, one frame of the screen can be drawn through multiple drawing passes, and each drawing pass is one complete drawing pass for one frame of the screen.

[0004] In the related art, an integrated rendering path FXAA (Fast Approximate Antialiasing) solution is provided, which completes tone mapping and the FXAA algorithm within one rendering path, where tone mapping refers to tone mapping from HDR (High-Dynamic Range) space to LDR (Low-Dynamic Range) space, and in the integrated rendering path FXAA solution, the FXAA algorithm directly operates on edges in HDR space.

[0005] The integrated drawing path FXAA solution includes a main anti-aliasing branch and a detail anti-aliasing branch, where the main anti-aliasing branch is used to macroscopically remove aliasing of major edge lines, and the detail anti-aliasing branch is used to perform independent anti-aliasing processing on each edge pixel to compensate for edges missed by the main anti-aliasing branch. Due to the specific algorithm of the main anti-aliasing branch, the main anti-aliasing branch cannot remove aliasing in highlight areas in HDR space, and aliasing in highlight areas in HDR space must be removed by relying on the detail anti-aliasing branch.

[0006] However, the detail anti-aliasing branch is based on a subpixel anti-aliasing quality parameter (SubPixQuality). When the subpixel anti-aliasing quality parameter is set to 0, no anti-aliasing is performed on the edges of image details, resulting in sharp image details but relatively obvious aliasing in highlight areas. When the subpixel anti-aliasing quality parameter is set to 1, the anti-aliasing effect on the edges of image details is maximized, resulting in optimal anti-aliasing effect on the edges of highlight areas but unclear image details. Therefore, in the integrated rendering path FXAA solution, the value of the subpixel anti-aliasing quality parameter is constantly adjusted to balance the sharpness of image details and the anti-aliasing effect in highlight areas. However, constantly adjusting the value of the subpixel anti-aliasing quality parameter cannot effectively achieve both the sharpness of image details and the anti-aliasing effect in highlight areas. Summary of the Invention [Means for solving the problem]

[0007] This application provides an image anti-aliasing method, device, equipment, medium, and program product, which resolves the contradiction between the sharpness of image details and the anti-aliasing effect of highlight areas under the detail anti-aliasing branch. The above technical solutions are as follows:

[0008] According to one aspect of the present application, there is provided a method for anti-aliasing an image, the method comprising: obtaining a first edge pixel of the image in HDR space; generating a first scaling factor based on the local brightness environment in which the first edge pixel is located; generating a color ratio to be processed corresponding to the first edge pixel based on brightness values ​​of a plurality of pixels in the image block in which the first edge pixel is located; scaling the color proportions to be processed based on a first scaling factor to obtain first color proportions; A step of blending a first color value based on a color value of a first edge pixel and a color value of a target pixel together with a first color ratio, wherein the target pixel is a pixel among a plurality of surrounding pixels of the first edge pixel that has the largest color value difference from the first edge pixel; updating the first color value as the color value of the first edge pixel.

[0009] According to another aspect of the present application, there is provided an apparatus for anti-aliasing an image, the apparatus comprising: an acquisition module used for acquiring a first edge pixel of the image in HDR space; a first generation module adapted to generate a first scaling factor based on a local brightness environment in which the first edge pixel is located; a second generation module used to generate a target color ratio corresponding to the first edge pixel according to brightness values ​​of a plurality of pixels in the image block where the first edge pixel is located; a scaling module used for scaling the color proportion of the processing target according to a first scaling factor to obtain a first color proportion; a blending module for blending a color value of a first edge pixel and a color value of a target pixel together with a first color ratio to obtain a first color value, wherein the target pixel is a pixel among a plurality of surrounding pixels of the first edge pixel that has the largest color value difference from the first edge pixel; an updating module used for updating the first color value as the color value of the first edge pixel.

[0010] According to one aspect of the present application, there is provided a computer device including a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to realize the above-mentioned image anti-aliasing method.

[0011] According to another aspect of the present application, there is provided a computer-readable storage medium having a computer program stored therein, the computer program being loaded and executed by a processor to implement the above-described image anti-aliasing method.

[0012] According to another aspect of the present application, there is provided a computer program product, the computer program product including computer instructions stored in a computer-readable storage medium, wherein a processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computing device to perform the image anti-aliasing method provided in the aspect. [Effects of the Invention]

[0013] The beneficial effects of the technical solutions provided by the embodiments of the present application include at least the following:

[0014] By generating a scaling factor for each edge pixel for an image in HDR space and generating an individual customized scaling factor for each edge pixel, and each edge pixel performs color mixing based on its own scaling factor, the contradiction between the sharpness of image details and the anti-aliasing effect in highlight areas caused by the use of only one sub-pixel anti-aliasing quality parameter for all frame images in the detail anti-aliasing branch is effectively resolved.

[0015] According to the anti-aliasing method of the present application, the detail anti-aliasing branch under the FXAA solution of the integrated drawing path not only has a relatively good anti-aliasing effect on the edges of the highlight areas, but also ensures the sharpness of the image details. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a schematic diagram of the anti-aliasing effect of the FXAA solution under the integrated drawing path in the related art; [Figure 2] 1 shows a schematic diagram of an image anti-aliasing framework provided by one exemplary embodiment. [Figure 3] 1 shows a flowchart of a method for anti-aliasing an image provided by one exemplary embodiment. [Figure 4] 1 shows a schematic diagram of an image block provided by one exemplary embodiment; [Figure 5] 1 illustrates a schematic diagram of a scaling factor generation process provided by one exemplary embodiment. [Figure 6] 1 shows a schematic diagram of intensity weights of image blocks provided by one exemplary embodiment; [Figure 7] 1 shows a schematic diagram of a mapping relationship between brightness difference and color ratio to be processed provided by one exemplary embodiment; [Figure 8] 1 illustrates a schematic diagram of an image anti-aliasing framework provided by another exemplary embodiment. [Figure 9]1 illustrates a schematic diagram of an image anti-aliasing framework provided by another exemplary embodiment. [Figure 10] 1 shows a schematic diagram of the depth filtering and illumination compensation process provided by one exemplary embodiment. [Figure 11] 1 illustrates a schematic diagram of an image anti-aliasing framework provided by another exemplary embodiment. [Figure 12] 1 shows a schematic comparison of the FXAA solution of the integrated drawing path provided by one example embodiment and the anti-aliasing effect of the present application. [Figure 13] 1 shows a schematic comparison diagram of the FXAA solution of sub-drawing passes provided by one exemplary embodiment and the anti-aliasing effect of the present application. [Figure 14] 1 shows a structural block diagram of an image anti-aliasing device provided by one exemplary embodiment; [Figure 15] 1 illustrates a structural block diagram of a computer device provided by one exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] First, a brief introduction will be given to the nouns involved in the embodiments of the present application.

[0018] High Dynamic Range (HDR): HDR is a rendering technology that is closer to realistic lighting. Compared to low dynamic range (LDR), the maximum pixel brightness value in HDR can exceed 1. HDR technology is used to simulate and calculate realistic lighting intensity, while the maximum pixel brightness value in LDR is 1. Images in LDR space are applied to screen rendering. Related technologies use tone mapping to map HDR colors to LDR and finally render them on the screen.

[0019] Post-processing stage: A drawing stage that occurs after scene object drawing and character drawing in game rendering, where adjustments can be made to the final rendered screen, such as tone adjustment and anti-aliasing processing. In related technologies, tone mapping is a post-processing stage.

[0020] Rendering pass: In the rendering process, a frame of the screen can be rendered using multiple rendering passes, and each pass can complete a complete rendering pass, including a series of rendering processes such as vertex rendering and fragment rendering. In related technology, tone mapping is a complete rendering pass.

[0021] Fast Approximate Anti-Aliasing (FXAA): A type of anti-aliasing solution based on post-processing proposed by NVIDIA. FXAA performs anti-aliasing processing on the currently rendered screen using a single post-processing drawing pass. In the post-processing drawing pass, the shader receives the input subpixel anti-aliasing quality parameter (SubPixQuality) and performs detailed anti-aliasing processing based on the subpixel anti-aliasing quality parameter.

[0022] When the input subpixel anti-aliasing quality parameter value is 0, screen details (relatively detailed edges on the screen, such as tree leaves and thin textures on wall surfaces) are not anti-aliased and the screen details are described as sharp, and when the input subpixel anti-aliasing quality parameter value is 1, the anti-aliasing effect of screen details reaches its maximum and the screen details are described as unclear. The subpixel anti-aliasing quality parameter is effective on screen details only in the detail anti-aliasing branch, and does not macroscopically affect the anti-aliasing effect on major lines (such as the outline edges of relatively large objects in a scene).

[0023] In the FXAA solution proposed by NVIDIA, pixel colors in HDR space must first be tone-mapped to pixel colors in LDR space in one rendering pass, and then FXAA anti-aliasing processing must be performed in another rendering pass, i.e., FXAA anti-aliasing is performed using sub-rendering passes. The sub-rendering pass solution consumes bandwidth corresponding to two rendering passes, which, when used in a mobile terminal, results in poor performance due to the limited processor bandwidth of the mobile terminal.

[0024] In order to solve the problem of excessive performance consumption caused by the sub-drawing path solution in the related art, a unified drawing path solution is further proposed, which mainly includes the following two types:

[0025] Local tone mapping solution: Because the FXAA algorithm can achieve a relatively good anti-aliasing effect in LDR space, this solution tone maps the current pixel in HDR space to LDR space within one drawing pass, and then performs FXAA calculation on the current calculation pixel. At this time, the calculation result is the same as the anti-aliasing effect of the solution of the sub-drawing pass.

[0026] However, tone mapping a current pixel to an LDR space requires looking up and obtaining the color value of the current pixel in HDR space (the HDR value to be mapped) and the color values ​​of multiple pixels including the current pixel in HDR space (the multiple HDR values ​​required for mapping) from an LUT (Lookup Table). Therefore, the HDR value of the current pixel needs to be sampled twice from the LUT, and the bandwidth consumed by this solution is still too high for a mobile terminal.

[0027] Parameter adjustment solution: This solution directly applies the FXAA algorithm to images in HDR space. Under this solution, when the subpixel anti-aliasing quality parameter value is 0, no anti-aliasing processing is performed on the image details, making the image appear sharp overall, but the anti-aliasing effect in the highlights is very poor. When the subpixel anti-aliasing quality parameter value is 1, the strongest anti-aliasing processing is performed on the image details, making the image appear blurred overall, but the blurring effect in the highlights results in a relatively good anti-aliasing effect.

[0028] 1 Schematically, parts (a) and (b) of Fig. 1 respectively show the anti-aliasing effect of an image highlight when the subpixel anti-aliasing quality parameter takes the values ​​of 1 and 0. As can be seen, the anti-aliasing effect of the edge line of the highlight when the subpixel anti-aliasing quality parameter is equal to 1 is better than the anti-aliasing effect of the edge line of the highlight when the subpixel anti-aliasing quality parameter is equal to 0. Parts (c) and (d) of Fig. 1 show the representation of an image detail when the subpixel anti-aliasing quality parameter takes the values ​​of 1 and 0. As can be seen, the anti-aliased image when the subpixel anti-aliasing quality parameter is equal to 1 is less clear than the anti-aliased image when the subpixel anti-aliasing quality parameter is equal to 0.

[0029] Therefore, when the FXAA algorithm is directly used in the HDR space, there is a contradiction between image sharpness and the anti-aliasing effect in the highlight area. A sub-pixel anti-aliasing quality parameter that is too small will cause the anti-aliasing effect in the highlight area to deteriorate, while a sub-pixel anti-aliasing quality parameter that is too large will cause the details on the screen to become unclear.

[0030] As described above, both the sub-rendering path solution and the integrated-rendering path solution have their own shortcomings. Based on this, the present application provides an image anti-aliasing method in HDR space.

[0031] FIG. 2 shows a schematic diagram of an image anti-aliasing framework provided by one exemplary embodiment of the present application, where the image anti-aliasing framework 200 includes an edge pixel detection stage 201, a scaling factor generation stage 202, a detail anti-aliasing stage 203, and a blended color generation stage 204.

[0032] The edge pixel detection step 201 is used to detect and obtain all edge pixels in an image in HDR space, and all edge pixels obtained in the edge pixel detection step 201 are input to the detail anti-aliasing step 203.

[0033] The scaling factor generation step 202 is used to generate a scaling factor corresponding to each pixel for all edge pixels of the image in HDR space, and the scaling factor is used to dynamically adjust the color proportion of each edge pixel.

[0034] The detail anti-aliasing step 203 is used to generate color proportions for all edge pixels. For each edge pixel, the detail anti-aliasing step 203 generates a target color proportion corresponding to the edge pixel, and adjusts the target color proportion of the edge pixel according to the scaling factor corresponding to the edge pixel to obtain the adjusted color proportion corresponding to the edge pixel.

[0035] The mixed color generation step 204 is used to color-mix all edge pixels of the image to obtain the anti-aliasing effect of the image. For each edge pixel, a target pixel corresponding to the edge pixel is determined to be used for color mixing, and the final color of the edge pixel is obtained by mixing the color of the edge pixel and the color of the target pixel together with the determined color ratio. After color mixing is performed on all edge pixels of the image, an anti-aliased image can be obtained.

[0036] In one embodiment, the image anti-aliasing framework 200 runs on a terminal. Optionally, the image anti-aliasing framework 200 is executed by a shader on the terminal. Optionally, the image anti-aliasing framework 200 is used to render image frames of an application program, and the application program runs on an operating system platform (Android or IOS). Optionally, the terminal includes at least one of a smartphone, a tablet PC, a wearable device, a smart TV, an in-car terminal, an e-book reader, an MP3 player, an MP4 player, a laptop portable computer, and a desktop computer. In the following embodiment, the terminal includes a smartphone.

[0037] FIG. 3 shows a flowchart of an image anti-aliasing method provided by one exemplary embodiment of the present application, which is illustrated by taking the method as being executed by a terminal, and includes:

[0038] Step 310: Obtain the first edge pixel of the image in HDR space.

[0039] The first edge pixel refers to a pixel in the image block in which the pixel difference reaches a threshold. Optionally, the first edge pixel refers to a pixel in the image block in which the pixel difference reaches a threshold. Optionally, the first edge pixel refers to a pixel in the image block in which the pixel difference reaches a threshold.

[0040] The image block in which the first edge pixel is located refers to a square pixel array obtained by dividing the image block with the first edge pixel at the center. Also refer to Figure 4, which shows the image block in which the first edge pixel is located, where the pixel indicated by M is the first edge pixel, N, S, W, and E indicate pixels above, below, left, and right of the first edge pixel, and NW, SW, NE, and SE indicate pixels above and to the left, below and to the right of the first edge pixel.

[0041] The first edge pixel is any one of all edge pixels of the image in the HDR space.

[0042] Step 310 is performed by the edge pixel detection stage 201 in the anti-aliasing framework 200 shown in FIG.

[0043] In one embodiment, before acquiring a first edge pixel of an image in HDR space, it is necessary to further detect all edge pixels of the image. The step of detecting all edge pixels of the image includes, after acquiring the image in HDR space, acquiring, for a jth pixel of the image, a pixel with the largest brightness value and a pixel with the smallest brightness value among a plurality of pixels in an image block in which the jth pixel is located, where j is a positive integer; determining that the jth pixel is an edge pixel when a brightness value difference between the largest and smallest brightness values ​​among a plurality of pixels in the image block in which the jth pixel is located reaches a jth threshold; and determining and acquiring a plurality of edge pixels of the image, where the jth pixel is any one pixel of the image. Optionally, the jth threshold is a predetermined brightness value difference threshold, and optionally, the same brightness value difference threshold is used for edge pixel detection for the same image.

[0044] Referring also to Figure 4, Figure 4 shows one image block in an image, where the pixel indicated by M is the jth pixel, N, S, W, and E indicate pixels above, below, to the left, and to the right of the jth pixel, and NW, SW, NE, and SE indicate pixels above and to the left, below and to the right of the jth pixel.

[0045] Selectably, the pixel with the maximum brightness value and the pixel with the minimum brightness value are determined from all pixels in the image block shown in Figure 4, selectably, the pixel with the maximum brightness value and the pixel with the minimum brightness value are determined from multiple pixels indicated by N, S, W, and E, and selectably, the pixel with the maximum brightness value and the pixel with the minimum brightness value are determined from multiple pixels indicated by NW, SW, NE, and SE.

[0046] Step 320: Generate a first scaling factor based on the local brightness environment in which the first edge pixel resides.

[0047] Step 320 is performed by the scaling factor generation stage 202 in the anti-aliasing framework 200 shown in FIG.

[0048] The first scaling factor refers to a scaling factor used to adjust the color proportion of the target pixel corresponding to the first edge pixel, and the color proportion is used to mix the color of the first edge pixel with the color of the target pixel.

[0049] The local brightness environment refers to the brightness situation within the image block in which the first edge pixel is located. Optionally, the local brightness environment is obtained by measuring the maximum and minimum brightness values ​​within the image block in which the first edge pixel is located. Optionally, the local brightness environment refers to the brightness situation of a plurality of pixels within the image block in which the first edge pixel is located. Optionally, the local brightness environment is obtained by measuring the maximum and minimum brightness values ​​within the image block in which the first edge pixel is located.

[0050] Referring also to Figure 4, Figure 4 can also be understood as an image block in which the first edge pixel is located, where M is the first edge pixel, N, S, W, and E indicate pixels above, below, left, and right of the first edge pixel, and NW, SW, NE, and SE indicate pixels above, below, above, right, and below the first edge pixel.

[0051] Selectively, the local brightness environment in which the first edge pixel is located refers to the brightness conditions of all pixels in the image block shown in Figure 4, selectively, the local brightness environment in which the first edge pixel is located refers to the brightness conditions of the four pixels indicated by N, S, W, and E, and selectively, the local brightness environment in which the first edge pixel is located refers to the brightness conditions of the four pixels indicated by NW, SW, NE, and SE.

[0052] Step 330: Generate a color proportion to be processed corresponding to the first edge pixel based on the brightness values ​​of a plurality of pixels in the image block where the first edge pixel is located.

[0053] Step 330 is performed by the detail anti-aliasing stage 203 in the anti-aliasing framework 200 shown in FIG.

[0054] Optionally, the image block in which the first edge pixel is located is a square pixel array, as shown in Figure 4. Based on the brightness values ​​of all pixels in the image block, a color proportion to be processed corresponding to the first edge pixel is generated.

[0055] Step 340: Scale the color proportion to be processed based on a first scaling factor to obtain a first color proportion.

[0056] Step 340 is performed by the detail anti-aliasing stage 203 in the anti-aliasing framework 200 shown in FIG.

[0057] Optionally, a first color proportion corresponding to the first edge pixel is obtained based on the first scaling factor obtained in step 320 and the color proportion to be processed obtained in step 330. Optionally, the first color proportion can be obtained by multiplying the first scaling factor by the color proportion to be processed.

[0058] Schematically, the first color ratio is calculated and obtained by the following formula:

[0059] AA sub =AA sub0 *f Here, AA sub is the first color fraction obtained after scaling, and AA sub0 is the color fraction to be processed, and f is the first scaling factor.

[0060] Step 350: According to the color value of the first edge pixel and the color value of the target pixel, together with the first color proportion, mix to obtain a first color value.

[0061] Step 350 is performed by the mixed color generation stage 204 in the anti-aliasing framework 200 shown in FIG.

[0062] Here, the target pixel is a pixel among a plurality of surrounding pixels of the first edge pixel that has the largest difference in color value from the first edge pixel. As also shown in FIG. 4 , the first edge pixel is a pixel indicated by M, and the target pixel is a pixel among a plurality of surrounding pixels indicated by N, S, W, and E that has the largest difference in color value from the pixel indicated by M, and the target pixel is a pixel among a plurality of surrounding pixels indicated by NW, SW, NE, and SE that has the largest difference in color value from the pixel indicated by M, and the target pixel is a pixel among a plurality of surrounding pixels indicated by N, S, W, E, NW, SW, NE, and SE that has the largest difference in color value from the pixel indicated by M.

[0063] Optionally, the target pixel is determined and acquired based on the edge direction of the first edge pixel. Optionally, after detecting and acquiring all edge pixels of the image, the edge direction corresponding to each edge pixel is calculated and stored in a lookup table. In one embodiment, the edge direction is vertical or horizontal, and the target pixel is a pixel that straddles the edge, is located vertically to the edge direction, and is closest to the first edge pixel. In one embodiment, if the brightness difference between the pixels above and below the edge pixel is greater than the brightness difference between the pixels to the left and right of the edge pixel, the edge direction of the edge pixel is determined to be horizontal. If the brightness difference between the pixels to the left and right of the edge pixel is greater than the brightness difference between the pixels above and below the edge pixel, the edge direction of the edge pixel is determined to be vertical. In one embodiment, if the edge direction of the first edge pixel is horizontal and the color of the first edge pixel is closer to the color of the pixel above it than the color of the pixel below it, the pixel below the first edge pixel is determined to be the target pixel, which is called edge straddling. In one embodiment, if the edge direction of the first edge pixel is vertical and the color of the first edge pixel is closer to the color of the right pixel than the color of the left pixel, the pixel to the left of the first edge pixel is determined as the target pixel, which is called edge straddling. For example, if the edge direction of the first edge pixel is left to right and the vertical edge direction is bottom to top, the target pixel is the first pixel above the first edge pixel. For example, if the edge direction of the first edge pixel is top to bottom and the vertical edge direction is left to right, the target pixel is the first pixel to the right of the first edge pixel.

[0064] For the target pixel determined and obtained based on the edge direction, the color value of the target pixel is the color value of the first edge pixel that is the most different from the color value of the first edge pixel among the plurality of surrounding pixels of the first edge pixel.

[0065] Based on the color value of the first edge pixel and the color value of the target pixel, and in conjunction with the first color proportion, a first color value can be blended to obtain the first color value.

[0066] Step 360: Update the first color value as the color value of the first edge pixel.

[0067] The first color value obtained by blending in step 350 is updated as the color value of the first edge pixel. Based on this, if a similar operation is performed on all edge pixels of the image in HDR space, anti-aliasing processing for the image in HDR space can be realized.

[0068] It should be noted that between the above step 320 and step 330, step 320 may be executed first, or step 330 may be executed first, or step 320 may be executed second, or step 320 and step 330 may be executed simultaneously, and this is not a limitation of the present application.

[0069] As described above, a scaling factor is generated for each edge pixel of an image in HDR space, and an individual customized scaling factor is generated for each edge pixel. Each edge pixel performs color mixing based on its own scaling factor, thereby effectively resolving the contradiction between the sharpness of image details and the anti-aliasing effect in highlight areas that occurs when all frame images in the detail anti-aliasing branch use only one sub-pixel anti-aliasing quality parameter.

[0070] According to the anti-aliasing method of the present application, the detail anti-aliasing branch under the FXAA solution of the integrated drawing path not only has a relatively good anti-aliasing effect on the edges of the highlight areas, but also ensures the sharpness of the image details.

[0071] Next, the relevant contents of the steps in the embodiment of the method shown in FIG. 3 above will be introduced in detail.

[0072] Based on the alternative embodiment shown in FIG. 3, in step 320, "generate a first scaling factor based on the local brightness environment in which the first edge pixel is located" may be replaced with "generate a first scaling factor based on the brightness value of the first surrounding pixel and the brightness value of the second surrounding pixel."

[0073] Here, the first surrounding pixel is the pixel with the greatest brightness value among the multiple pixels in the image block in which the first edge pixel is located, and the second surrounding pixel is the pixel with the smallest brightness value among the multiple pixels in the image block in which the first edge pixel is located.

[0074] As also shown in FIG. 4, M in FIG. 4 indicates a first edge pixel, and optionally the first and second surrounding pixels are determined and obtained from all pixels in the image block shown in FIG. 4, and optionally the first and second surrounding pixels are determined and obtained from a plurality of surrounding pixels indicated by N, S, W, and E shown in FIG. 4, and optionally the first and second surrounding pixels are determined and obtained from a plurality of surrounding pixels indicated by NW, SW, NE, and SE shown in FIG. 4.

[0075] In one embodiment, the brightness value of the first surrounding pixel and the brightness value of the second surrounding pixel are input to a scaling function to obtain a first scaling factor, where the scaling function satisfies the following: the larger the brightness value of the first surrounding pixel, the larger the value of the first scaling factor; the smaller the brightness value of the second surrounding pixel, the smaller the value of the first scaling factor; and when the brightness value of the second surrounding pixel is zero, the value of the first scaling factor is equal to zero.

[0076] The scaling function is derived as follows.

[0077] Since the purpose of the scaling factor is to adjust the color proportions of the edge pixels, the scaling function needs to be able to sense the brightness environment the edge pixels are in, so that edge pixels with different brightness values ​​get different scaling factors.

[0078] First, the general formula of the scaling function is set as follows:

[0079] F=(a*Luma min +b) x *(c*Luma max +d) y +e where F is the scaling factor obtained by the scaling function, and Luma min , and Luma max are the pixel with the smallest brightness value and the pixel with the largest brightness value among the multiple pixels in the image block in which the edge pixel is located, respectively, and a, c, x, y, b, d, and e are undetermined terms in the general formula.

[0080] The scaling function then needs to satisfy two conditions:

[0081] 1. It has a relatively good anti-aliasing effect in the highlight area of ​​the image, namely, Luma max and F show a positive correlation, and Luma max The larger is, the larger the value of F will be.

[0082] 2. The sharpness of the low-brightness edges of the image is protected, i.e., Luma min and F show a positive correlation, and Luma min The smaller is, the smaller the value of F is, and Luma min As approaches 0, F also approaches 0.

[0083] In order to satisfy the above condition 2, e and b in the general formula are discarded. Then, in order to reduce the time consumed in calculating the shader, x and y in the general formula are simplified to 1, and the general formula is simplified as follows:

[0084] F=(a*Luma min )*(c*Luma max +d)

[0085] In order to make the calculation as simple as possible and at the same time ensure good effect, we further simplify the general formula of the function to obtain the final scaling function as follows:

[0086] F=Luma min *(Luma max +1)

[0087] As can be seen, the scaling function only performs one multiplication and one addition, consuming only one MAD calculation instruction in the calculation process of the shader, and the MAD calculation instruction does not consume too much calculation resources.

[0088] After the scaling function has been constructed, the process of calculating and obtaining the first scaling factor for the first edge pixel of the image in HDR space is as follows: multiply the first brightness value by the second brightness value to obtain the first sub-scaling factor, use the second brightness value as the second sub-scaling factor, add the first sub-scaling factor and the second sub-scaling factor, and then perform normalization to obtain the first scaling factor.

[0089] Here, the first brightness value is the brightness value of the first surrounding pixel, and the second brightness value is the brightness value of the second surrounding pixel.

[0090] As shown in Fig. 5, part (a) of Fig. 5 shows the original image in HDR space, part (b) of Fig. 5 shows the scaling factors corresponding to each of all pixels of the image in HDR space, and the brightness values ​​of the pixels in part (b) of Fig. 5 are the numerical values ​​of the scaling factors. As can be seen, the numerical values ​​of the scaling factors calculated in the highlight areas of the original image are relatively large, and the numerical values ​​of the scaling factors calculated in the areas of the original image with relatively low brightness are relatively small.

[0091] As described above, the above method provides specific content of the scaling function, and to ensure that the detail anti-aliasing branch can be used for anti-aliasing the edges of the highlight area, the larger the maximum brightness value in the environment where the first edge pixel of the highlight area is located, the larger the first scaling factor corresponding to the first edge pixel should be. To protect the sharpness of low-brightness edge pixels, the smaller the minimum brightness value in the environment where the first edge pixel is located, the smaller the first scaling factor should be. And when the minimum brightness value in the environment where the first edge pixel is located is close to 0, the first scaling factor should also be close to 0.

[0092] Based on an optional embodiment shown in FIG. 3 , step 330 may be replaced by obtaining a plurality of brightness weights of a plurality of pixels in the image block in which the first edge pixel is located, generating a first brightness difference based on the brightness and brightness weights of each pixel in the image block, and mapping the first brightness difference to obtain the color proportion to be processed.

[0093] Here, the pixels in the image block and the intensity weights have a one-to-one correspondence, and the first intensity difference is used to represent the local intensity environment in which the first edge pixel is located.

[0094] Referring also to Figure 4, based on the brightness value of each pixel in the image block shown in Figure 4 and the brightness weight corresponding to each pixel, a first brightness difference of the first edge pixel is generated, and the first brightness difference is used to represent the difference between the brightness of the first edge pixel and the overall brightness of the image block.

[0095] In one embodiment, the image block is a square pixel array centered on the first edge pixel, and the intensity weights of the pixels in the image block satisfy the following conditions: The pixels in the square pixel array that have the same distance to the first edge pixel have the same corresponding brightness weights, The closer a pixel is to the first edge pixel in the square pixel array, the smaller the corresponding brightness weight. The brightness weights corresponding to the pixels other than the first edge pixel in the square pixel array are positive numbers, and the brightness weights corresponding to the first edge pixel are negative numbers; The sum of the brightness weights of all pixels in a square pixel array is zero.

[0096] 4 and 6, Fig. 6 shows the brightness weight corresponding to each pixel in the square pixel array of Fig. 4. After obtaining the brightness weight of each pixel in the square pixel array, a first brightness difference is generated based on the brightness and brightness weight of each pixel in the image block.

[0097] In one embodiment, the first brightness difference is obtained by multiplying the brightness of each pixel in the image block by its respective brightness weight, adding the products of each pixel in the image block, and then dividing by the absolute value of the brightness weight corresponding to the first edge pixel. Referring to Figures 4 and 6, the calculation formula for the first brightness difference corresponding to the first edge pixel (represented by M) is as follows:

[0098] F=(NW*1+N*2+NE*1+E*2+SE*1+S*2+SW*1+W*2-M*12) / 12

[0099] After calculating and obtaining the first brightness difference, the first brightness difference is further normalized and mapped to obtain the target color proportion, the value range of the target color proportion is set to [0, 1], and the target color proportion and the first brightness difference exhibit a positive correlation. As shown in FIG. 7, the horizontal axis of FIG. 7 represents the first brightness difference, and the vertical axis of FIG. 7 represents the target color proportion.

[0100] As described above, the above method provides a method for generating a target color ratio, which is further used in conjunction with a scaling factor to generate a first color ratio by configuring the detail anti-aliasing branch of the present invention.

[0101] Based on the optional embodiment shown in FIG. 3 , step 350 may be replaced by multiplying the second color proportion by the color value of the first edge pixel to obtain a first color component, where the sum of the second color proportion and the first color proportion is 1; multiplying the first color proportion by the color value of the target pixel to obtain a second color component; and adding the first color component and the second color component to obtain a first color value.

[0102] Schematically, if the value of the first color proportion is p, the value of the second color proportion is 1-p, the color value of the first edge pixel is x, and the color value of the target pixel is x', the first color value can be expressed by the following formula:

[0103] F=(1-p)*x+p*x' F represents the first color value obtained by calculation.

[0104] As can be seen, the above method is a specific color blending algorithm for each edge pixel provided by the detail anti-aliasing branch, thereby realizing the anti-aliasing processing of edge pixels under the detail anti-aliasing branch.

[0105] Based on the image anti-aliasing framework shown in Fig. 2, Fig. 8 shows a schematic diagram of another image anti-aliasing framework provided by the present application, which mainly adds a main anti-aliasing branch 804 compared with Fig. 2. The image anti-aliasing framework 800 includes an edge pixel detection stage 801, a scaling factor generation stage 802, a detail anti-aliasing branch 803, a main anti-aliasing branch 804, a color proportion determination stage 805, and a mixed color generation stage 806.

[0106] The edge pixel detection step 801 is used to detect and obtain all edge pixels in the image in HDR space. Optionally, the edge pixel detection step 801 is further used to generate at least one linear edge based on all edge pixels, where the linear edge is formed by a plurality of adjacent edge pixels connected in series. All edge pixels obtained in the edge pixel detection step 801 are input to a detail anti-aliasing branch 803, and the at least one linear edge obtained in the edge pixel detection step 801 is input to a main anti-aliasing branch 804.

[0107] The scaling factor generation step 802 is used to generate a scaling factor corresponding to each pixel for all edge pixels of the image in HDR space, and the scaling factor is used to dynamically adjust the color proportion of each edge pixel.

[0108] The detail anti-aliasing branch 803 is used to generate color proportions for all edge pixels. For each edge pixel, the detail anti-aliasing branch 803 generates a target color proportion corresponding to the edge pixel, and performs an adjustment on the target color proportion of the edge pixel according to the scaling factor corresponding to the edge pixel to obtain the color proportion corresponding to the edge pixel.

[0109] The main anti-aliasing branch 804 is used to generate color percentages for edge pixels on a linear edge. For an edge pixel on a linear edge, the main anti-aliasing branch 804 generates a color percentage corresponding to the edge pixel based on the edge pixel's position on the linear edge.

[0110] The color ratio determination step 805 is used to determine the final color ratio of all edge pixels. For each edge pixel, the maximum value of the color ratio of the edge pixel generated by the detail anti-aliasing branch 803 and the color ratio of the edge pixel generated by the main anti-aliasing branch 804 is determined, and the maximum value is determined as the final color ratio.

[0111] The mixed color generation step 806 is used to color-mix all edge pixels of the image to obtain the anti-aliasing effect of the image. For each edge pixel, a target pixel corresponding to the edge pixel is determined to be used for color mixing, and the final color of the edge pixel is obtained by mixing the color of the edge pixel and the color of the target pixel together with the determined color ratio. After color mixing is performed on all edge pixels of the image, an anti-aliased image can be obtained.

[0112] Based on the anti-aliasing framework 800 shown in Fig. 8, in the alternative embodiment shown in Fig. 3, the image anti-aliasing method further includes the following steps S1 and S2, and step 350 may be replaced by step S3. Step S1 is realized by the main anti-aliasing branch 804 shown in Fig. 8, step S2 is realized by the color proportion determination step 805, and step S3 is realized by the mixed color generation step 806.

[0113] S1: Under the circumstance where a first edge pixel is used to form a linear edge, obtain a third color proportion corresponding to the first edge pixel according to the position of the first edge pixel on the linear edge; Here, a linear edge includes a plurality of adjacent edge pixels.

[0114] In one embodiment, the linear edge is obtained by repeatedly expanding to adjacent edge pixels. After determining an edge pixel, it is determined whether the adjacent pixel of the edge pixel is an edge pixel, and if it is an edge pixel, it connects the adjacent pixel in series, and then expands to the adjacent pixel again. The repeated expansion is stopped until the expanded adjacent pixel does not belong to an edge pixel or the iteration number reaches a threshold, and finally a linear edge is obtained.

[0115] In one embodiment, a third color proportion corresponding to the first edge pixel is obtained based on the position of the first edge pixel on the linear edge. Schematically, the linear edge includes 10 pixels, and the color of the five pixels on the left is close to the color of the pixel below the linear edge, and the color of the five pixels on the right is close to the color of the pixel above the linear edge. Starting from the leftmost pixel, the five pixels from left to right on the linear edge gradually shift from the color of the pixel below, and the five pixels are color-mixed with the pixel above the linear edge, so that the color proportions corresponding to the five pixels gradually increase. For example, the color proportions are 0.1, 0.2, 0.3, 0.4, and 0.5, in order.

[0116] Starting from the rightmost pixel, the five pixels from the right to the left of the linear edge gradually deviate from the color of the upper pixel, and the five pixels are color-mixed with the lower pixel of the linear edge, and the color proportions corresponding to the five pixels gradually increase, for example, the color proportions are 0.1, 0.2, 0.3, 0.4, and 0.5 in order.

[0117] S2: Determine the maximum value of the first color ratio and the third color ratio as the maximum color ratio corresponding to the first edge pixel.

[0118] S3: Based on the color value of the first edge pixel and the color value of the target pixel, combine with the maximum color proportion to obtain a first color value by blending.

[0119] The fourth color proportion is multiplied by the color value of the first edge pixel to obtain a third color component, the sum of the fourth color proportion and the maximum color proportion is 1, the maximum color proportion is multiplied by the color value of the target pixel to obtain a fourth color component, and the third color component and the fourth color component are added to obtain a first color value.

[0120] Schematically, if the value of the maximum color proportion is p, the value of the fourth color proportion is 1-p, the color value of the first edge pixel is x, and the color value of the target pixel is x', the first color value can be expressed by the following formula:

[0121] F=(1-p)*x+p*x' F represents the first color value obtained by calculation.

[0122] As described above, the above steps S1, S2, and S3 are actually related to the main anti-aliasing branch 804 shown in Fig. 8. The main anti-aliasing branch 804 is a branch that performs anti-aliasing on linear edges, and in fact, the main anti-aliasing branch cannot be applied to anti-aliasing in highlight regions of an image in HDR space, because linear mixing is adopted for color mixing, but converting HDR space to LDR space is a nonlinear conversion.

[0123] For example, suppose there is a horizontal linear edge, and the brightness of the pixel above the linear edge is 1 and the brightness of the pixel below the linear edge is 0.8, and the brightness of the linear edge is linearly distributed from left to right as 1 to 0.8.

[0124] Assume that in HDR space, the brightness of the upper pixel of a linear edge in a highlight area is 10, which is much greater than 1, and the brightness of the lower pixel of the linear edge is 0.8. After calculation using the FXAA algorithm, the brightness of the linear edge is linearly distributed from left to right from 10 to 0.8, and the brightness of most edge pixels on this linear edge is greater than 1. In this case, when this linear edge is converted from HDR space to LDR space using tone mapping, the color values ​​of most edge pixels are mapped to values ​​close to 1. Tone mapping is nonlinear with respect to color mapping, and in this case, the linear edge in LDR space after tone mapping actually simply has a shift in the aliasing position, rather than the aliasing disappearing.

[0125] Based on the image anti-aliasing framework shown in FIG. 2, FIG. 9 shows a schematic diagram of an image anti-aliasing framework 900 provided by one exemplary embodiment of the present application. Compared with FIG. 2, FIG. 9 mainly adds a depth filtering and brightness compensation stage 905, and the edge pixel detection stage 901, the scaling factor generation stage 902, the detail anti-aliasing branch 903, and the mixed color generation stage 904 in the image anti-aliasing framework 900 are similar to the corresponding contents shown in FIG. 2.

[0126] The depth filtering and brightness compensation step 905 is used to perform depth filtering on all pixels of the image in HDR space to obtain areas with obvious changes in depth information through filtering. However, since the depth filtering only considers depth information, it is easy to filter highlight edges, so brightness compensation needs to be further performed to compensate for the filtered highlight edges.

[0127] The depth filtering and brightness compensation step 905 obtains an edge pixel cluster of the image, and if the current pixel does not belong to the edge pixel cluster, the scaling factor value of the current pixel is set to 0. If the current pixel belongs to the edge pixel cluster, the previously calculated scaling factor is subsequently input to the detail anti-aliasing branch 903.

[0128] Based on the image anti-aliasing framework 900 shown in FIG. 9, in the alternative embodiment shown in FIG. 3, after step 320, the following steps 321 and 322 are further included.

[0129] Step 321: Perform depth filtering and brightness compensation operations on the image to obtain edge pixel clusters.

[0130] The depth filtering operation refers to an operation of filtering out areas in an image where depth changes are not obvious and leaving areas in the image where depth changes are obvious. At this time, the remaining areas with obvious depth changes may be called depth edges (edges determined based on depth changes). When the image undergoes the depth filtering operation, a first edge pixel cluster is obtained, and the depth value difference between the pixels in the first edge pixel cluster and the surrounding pixels reaches a threshold.

[0131] The depth filtering operation is performed based on the original image, and the first edge pixel cluster is obtained through the depth filtering operation. The depth filtering operation filters pixels whose depth value difference with surrounding pixels does not reach a threshold. Therefore, the first edge pixel cluster refers to a cluster composed of edge pixels determined from the original image based on depth information.

[0132] Referring also to Figure 10, part (a) of Figure 10 is the original image, and part (b) of Figure 10 is the image obtained after performing the depth filtering operation, from which it can be seen that the depth filtering operation not only filters out the wall and ground textures, but also filters out the luminous rectangles (transparent objects do not leave any depth information).

[0133] The brightness compensation operation refers to the operation of identifying pixels other than the first edge pixel cluster in an image as edge pixels under a certain brightness environment that satisfies certain conditions, and the edge pixels may be called brightness edges. After the depth filtering operation, the image is subjected to brightness compensation, and the edge pixels obtained by brightness compensation are added to the second edge pixel cluster.

[0134] It is necessary to explain that the brightness compensation operation is performed based on the pixels filtered by the depth filtering operation, and selects pixels from the pixels filtered by the depth operation whose brightness difference with the image block where the pixels are located meets a threshold, and adds the selected pixels to a second edge pixel cluster. Therefore, the second edge pixel cluster refers to a cluster composed of edge pixels determined from the pixels filtered by the depth filtering operation based on brightness information.

[0135] Schematically, for an i-th pixel in an image other than a first edge pixel cluster, a pixel with the highest brightness value and a pixel with the lowest brightness value among the pixels in an image block in which the i-th pixel is located are determined, and when a brightness value difference between the pixel with the highest brightness value and the pixel with the lowest brightness value among the pixels in the image block in which the i-th pixel is located reaches an i-th threshold, the i-th pixel is added to a second edge pixel cluster, where the i-th pixel is any pixel in the image other than the first edge pixel cluster, and the first edge pixel cluster and the second edge pixel cluster jointly constitute an edge pixel cluster, the i-th threshold is a predetermined threshold, where i is a positive integer. Optionally, the i-th threshold is a predetermined brightness value difference threshold, and optionally, the same brightness value difference threshold is used for determining whether all pixels in the image other than the first edge pixel cluster belong to the second edge pixel cluster.

[0136] Referring also to Figure 4, the pixel indicated by M in Figure 4 can be understood to be the i-th pixel, and selectably determines the pixel with the largest brightness value and the pixel with the smallest brightness value from among all the pixels in the image block shown in Figure 4, selectably determines the pixel with the largest brightness value and the pixel with the smallest brightness value from among the multiple pixels indicated by N, S, W, and E, and selectably determines the pixel with the largest brightness value and the pixel with the smallest brightness value from among the multiple pixels indicated by NW, SW, NE, and SE.

[0137] 10, part (c) of FIG. 10 shows an image obtained after the depth filtering operation and further performing a brightness compensation operation, which can be seen to recompensate the filtered luminous rectangle. After compensation, the non-zero brightness pixels shown in part (c) of FIG. 10 constitute an edge pixel cluster.

[0138] Step 322: Determine that the first edge pixel is one pixel of the edge pixel cluster.

[0139] The edge pixel cluster is a cluster composed of edge pixels determined from an image together with depth information and brightness information. Specifically, first, depth edges are determined from the original image by a depth filtering operation based on the depth information, and then brightness edges are determined from the filtered pixels by the depth filtering operation based on the brightness information, and all the depth edges and brightness edges jointly constitute an edge pixel cluster.

[0140] For example, if an original image contains 50 pixels, 30 depth edges are determined based on the depth information of the 50 pixels, and then 5 brightness edges are determined from 20 pixels other than the 30 depth edges, in this case, 30 depth edges and 5 brightness edges are obtained in total, and the 35 edges constitute an edge pixel cluster.

[0141] The edge pixel cluster is used to set the scaling factors corresponding to pixels in the image other than the edge pixel cluster to 0. After determining that the first edge pixel belongs to the edge pixel cluster obtained in step 321, the generated first scaling factor is subsequently used to scale the color proportions to be processed.

[0142] As described above, an edge pixel cluster is obtained through the depth filtering and brightness compensation operations, and the edge pixel cluster includes the edge pixels obtained through the depth filtering and brightness compensation. After determining that the first edge pixel falls within the edge pixel cluster, the calculated and obtained first scaling factor is used. If the first edge pixel does not fall within the edge pixel cluster, the scaling factor of the first edge pixel is set to 0, and the subsequent steps are continued.

[0143] In practice, steps 321 and 322 filter relatively low-intensity pixels among the multiple edge pixels obtained in the initial edge detection stage, and perform anti-aliasing on these relatively low-intensity pixels to avoid unsharpness of these relatively low-intensity pixels. That is, the depth filtering and brightness compensation aim to leave true highlight regions, filter unnecessary low-intensity regions, and use a relatively large scaling factor only for highlight regions, while setting the scaling factor value used for other regions to 0. The depth filtering and brightness compensation do not affect the edge detection stage, that is, the depth filtering and brightness compensation do not affect the main anti-aliasing branch.

[0144] FIG. 11 shows an anti-aliasing framework 1100 provided by one exemplary embodiment of the present application, where the image anti-aliasing framework 1100 can be obtained by combining the above-mentioned anti-aliasing framework 200, anti-aliasing framework 800, and anti-aliasing framework 900, where the edge pixel detection stage 1101, scaling factor generation stage 1102, detail anti-aliasing branch 1103, main anti-aliasing branch 1104, color proportion determination stage 1105, blended color generation stage 1106, and depth filtering and brightness compensation stage 1107 in the anti-aliasing framework 1100 have all been introduced in detail and will not be described in detail again here.

[0145] Figure 12 shows a comparison of anti-aliasing effects. Part (a) of Figure 12 is a schematic diagram of the anti-aliasing effect of an integrated rendering path FXAA solution in the related art. Part (a) of Figure 12 shows aliasing in a rectangular frame 1201, and the value of the subpixel anti-aliasing quality parameter (SubPixQuality) of the FXAA solution used in part (a) of Figure 12 is equal to 1. As can be seen, in part (a) of Figure 12, the image is sharp overall, and as can be seen from the local enlarged view on the right, aliasing in the highlight area of ​​the image is relatively obvious.

[0146] Part (b) of Figure 12 shows a schematic diagram of the effect of anti-aliasing performed by the anti-aliasing framework 1100 shown in Figure 11, and part (b) of Figure 12 shows aliasing in a rectangular box 1202. As can be seen, in part (b) of Figure 12, the overall sharpness of the screen is consistent with part (a) of Figure 12, and as can be seen from the local enlargement on the right, there is no aliasing in the highlight area of ​​the image, the anti-aliasing effect is better, and the optimization effect is obvious.

[0147] FIG. 13 shows another anti-aliasing effect comparison diagram. Part (a) of FIG. 13 shows an original image used to compare the anti-aliasing effect. Part (b) of FIG. 13 shows the anti-aliasing effect of the FXAA solution under the sub-rendering pass when the sub-pixel anti-aliasing quality parameter is equal to 0. Part (c) of FIG. 13 shows the anti-aliasing effect of the FXAA solution under the sub-rendering pass when the sub-pixel anti-aliasing quality parameter is equal to 1. Part (d) of FIG. 13 shows the anti-aliasing effect of the anti-aliasing method provided in the present application.

[0148] As can be seen from this, for the highlighted portion shown in the rectangular frame area, the anti-aliasing effect of the present application on the highlighted portion is the same as when the subpixel anti-aliasing quality parameter is 1, and is better than the effect when the subpixel anti-aliasing quality parameter is 0. As can be seen from this, for the screen details shown in the elliptical area, the sharpness of the screen details of the present application is the same as when the subpixel anti-aliasing quality parameter is 0, and is sharper than when the subpixel anti-aliasing quality parameter is 1.

[0149] Through testing, it has been found that the anti-aliasing solution provided by this application has excellent anti-aliasing effect and does not consume too much performance.

[0150] Tests were conducted on a mobile device commonly found on the market, and when the FXAA solution of the integrated drawing path in the related technology performed tone mapping and anti-aliasing, the drawing path of the mobile device took a total of 2.86 ms, the read data bandwidth was 1.759 MB, and the write data bandwidth was 68 KB.

[0151] Tests were performed on the same mobile device, and when the anti-aliasing solution of the present application performed tone mapping and anti-aliasing, the mobile device's drawing pass took a total of 2.85 ms, with a read data bandwidth of 1.765 MB and a write data bandwidth of 69 KB.

[0152] As can be seen, the anti-aliasing solution provided by the present application is almost identical to the FXAA solution of the integrated drawing path in the related art in terms of various consumption indexes, but the anti-aliasing effect is obviously improved.

[0153] FIG. 14 is a structural block diagram of an image anti-aliasing device provided by one exemplary embodiment of the present application, the device including: an acquisition module 1401, a generation module 1402, and a processing module 1403; The obtaining module 1401 is used for obtaining a first edge pixel of an image under HDR space; The generating module 1402 is used to generate a first scaling factor based on a local brightness environment in which the first edge pixel is located; The generating module 1402 is used to generate a target color ratio corresponding to the first edge pixel according to brightness values ​​of a plurality of pixels in the image block where the first edge pixel is located; The processing module 1403 is used for scaling the target color ratio according to a first scaling factor to obtain a first color ratio; The processing module 1403 is used to obtain a first color value by blending the color value of the first edge pixel and the color value of the target pixel together with a first color proportion, where the target pixel is a pixel among a plurality of surrounding pixels of the first edge pixel that has the largest color value difference with the first edge pixel; The processing module 1403 is used to update the first color value as the color value of the first edge pixel.

[0154] In one alternative embodiment, the generation module 1402 is further used to generate a first scaling factor based on the brightness value of a first surrounding pixel and the brightness value of a second surrounding pixel, where the first surrounding pixel is the pixel with the largest brightness value among the multiple pixels of the image block in which the first edge pixel is located, and the second surrounding pixel is the pixel with the smallest brightness value among the multiple pixels of the image block in which the first edge pixel is located.

[0155] In one alternative embodiment, the generation module 1402 is further used to input the brightness value of the first surrounding pixel and the brightness value of the second surrounding pixel into a scaling function to obtain a first scaling factor, where the scaling function satisfies the following: the larger the brightness value of the first surrounding pixel, the larger the value of the first scaling factor; the smaller the brightness value of the second surrounding pixel, the smaller the value of the first scaling factor; and when the brightness value of the second surrounding pixel is zero, the value of the first scaling factor is equal to zero.

[0156] In one alternative embodiment, the brightness value of the first surrounding pixel is a first brightness value, and the brightness value of the second surrounding pixel is a second brightness value. The generating module 1402 is further configured to multiply the first brightness value and the second brightness value to obtain a first sub-scaling factor, use the second brightness value as the second sub-scaling factor, and add the first sub-scaling factor and the second sub-scaling factor, and then perform a normalization process to obtain the first scaling factor.

[0157] In one alternative embodiment, the generation module 1402 further obtains a plurality of brightness weights of a plurality of pixels in the image block in which the first edge pixel is located, where the plurality of pixels in the image block and the plurality of brightness weights correspond one-to-one; generates a first brightness difference based on the brightness and brightness weight of each pixel in the image block, where the first brightness difference is used to represent the local brightness environment in which the first edge pixel is located; and maps the first brightness difference to obtain a color proportion to be processed.

[0158] In one alternative embodiment, the image block is a square pixel array centered on the first edge pixel, and the intensity weights of the pixels in the image block satisfy the following condition: The pixels in the square pixel array that have the same distance to the first edge pixel have the same corresponding brightness weights; The closer a pixel is to the first edge pixel in the square pixel array, the smaller the corresponding brightness weight is. The brightness weights corresponding to the pixels other than the first edge pixel in the square pixel array are positive numbers, and the brightness weights corresponding to the first edge pixel are negative numbers; The sum of the intensity weights of all pixels in a square pixel array is zero.

[0159] In one alternative embodiment, the generation module 1402 is further used for calculating the product of the brightness of each pixel in the image block and its respective brightness weight, and adding the products of each pixel in the image block and then dividing by the absolute value of the brightness weight corresponding to the first edge pixel to obtain a first brightness difference.

[0160] In one alternative embodiment, the generation module 1402 is further used for normalizing the first lightness difference; and mapping the normalized first lightness difference to obtain a target color proportion, such that the target color proportion belongs to a closed interval from zero to one, and the target color proportion and the first lightness difference have a positive correlation.

[0161] In one alternative embodiment, the processing module 1403 is further used for multiplying the second color proportion by the color value of the first edge pixel to obtain a first color component, wherein the sum of the second color proportion and the first color proportion is 1; multiplying the first color proportion by the color value of the target pixel to obtain a second color component; and adding the first color component and the second color component to obtain a first color value.

[0162] In one alternative embodiment, the generating module 1402 is further configured to, when the first edge pixel is used to form a linear edge, obtain a third color proportion corresponding to the first edge pixel based on the position of the first edge pixel on the linear edge, and determine the maximum value among the first color proportion and the third color proportion as the maximum color proportion corresponding to the first edge pixel. The processing module 1403 is further configured to obtain a first color value by blending the maximum color proportion based on the color value of the first edge pixel and the color value of the target pixel, where the linear edge includes multiple adjacent edge pixels.

[0163] In one alternative embodiment, the acquisition module 1401 is further used for acquiring an image under HDR space; for a j-th pixel of the image, acquiring a pixel with the largest brightness value and a pixel with the smallest brightness value among multiple pixels of an image block in which the j-th pixel is located; determining that the j-th pixel is an edge pixel under a circumstance where a brightness value difference between the pixel with the largest brightness value and the pixel with the smallest brightness value among multiple pixels of the image block in which the j-th pixel is located reaches a j-th threshold; and determining and acquiring multiple edge pixels of the image, wherein the j-th pixel is any one pixel of the image.

[0164] In one alternative embodiment, the processing module 1403 is further used to perform a depth filtering operation and a brightness compensation operation on the image to obtain an edge pixel cluster, and to determine that the first edge pixel is one pixel of the edge pixel cluster, where the edge pixel cluster is used to set scaling factors corresponding to pixels in the image other than the edge pixel cluster to zero.

[0165] In one alternative embodiment, the processing module 1403 is further used to: perform a depth filtering operation on the image to obtain a first edge pixel cluster, where a depth value difference between a pixel in the first edge pixel cluster and surrounding pixels reaches a threshold; for an i-th pixel in the image other than the first edge pixel cluster, determine a pixel with the largest brightness value and a pixel with the smallest brightness value among multiple pixels in the image block in which the i-th pixel is located; and add the i-th pixel to a second edge pixel cluster under the condition that a brightness value difference between the pixel with the largest brightness value and the pixel with the smallest brightness value among multiple pixels in the image block in which the i-th pixel is located reaches an i-th threshold, where the i-th pixel is any pixel in the image other than the first edge pixel cluster, and the first edge pixel cluster and the second edge pixel cluster jointly constitute an edge pixel cluster.

[0166] In one alternative embodiment, the apparatus further includes a lookup module 1404. The lookup module 1404 is further used for: looking up and obtaining an edge direction corresponding to the first edge pixel, where the edge direction is a vertical direction or a horizontal direction; and determining a pixel that straddles the edge, is located in a vertical direction of the edge direction, and has a minimum distance from the first edge pixel as a target pixel.

[0167] As described above, a scaling factor is generated for each edge pixel of an image in HDR space, and an individual customized scaling factor is generated for each edge pixel. Each edge pixel performs color mixing based on its own scaling factor, thereby effectively resolving the contradiction between the sharpness of image details and the anti-aliasing effect in highlight areas that occurs when all frame images in the detail anti-aliasing branch use only one sub-pixel anti-aliasing quality parameter.

[0168] According to the anti-aliasing device of the present application, the detail anti-aliasing branch under the FXAA solution of the integrated drawing path not only has a relatively good anti-aliasing effect on the edges of the highlight area, but also ensures the sharpness of the image details.

[0169] 15 shows a structural block diagram of a computer device 1500 provided by one exemplary embodiment of the present application. The computer device 1500 may be a portable mobile terminal, such as a smartphone, a tablet PC, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop, or a desktop PC. The computer device 1500 may also be referred to by other names, such as a user device, a portable terminal, a laptop terminal, and a desktop terminal.

[0170] Typically, the computing device 1500 includes a processor 1501 and a memory 1502 .

[0171] The processor 1501 may include one or more processing cores, such as a 4-core processor or an 8-core processor. The processor 1501 may be implemented using at least one hardware format selected from the group consisting of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1501 may include a main processor and a coprocessor. The main processor, also referred to as a CPU (Central Processing Unit), is used to process data in a wake state. The coprocessor is a low-power processor used to process data in a standby state. In some embodiments, the processor 1501 may be integrated with a GPU (Graphics Processing Unit), which is used to render and draw content to be displayed on a screen. In some embodiments, the processor 1501 may further include an AI (Artificial Intelligence) processor, which is used to process computational operations related to machine learning.

[0172] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may further include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1502 is used to store at least one instruction that is executed by the processor 1501 to implement the image anti-aliasing method provided by the method embodiments herein.

[0173] In some embodiments, computing device 1500 optionally further includes a peripheral device port 1503 and at least one peripheral device. As will be appreciated by those skilled in the art, the structure shown in Figure 15 does not constitute a limitation on computing device 1500, which may include more or fewer components than those shown, may combine certain components, or may employ a different component arrangement.

[0174] The present application further provides a computer-readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, code set, or instruction set being loaded and executed by a processor to implement the image anti-aliasing method provided by the embodiment of the method.

[0175] The present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions, thereby causing the computer device to perform the image anti-aliasing method provided by the above method embodiment.

Claims

1. A method for image anti-aliasing executed by a computer device, the method for image anti-aliasing comprising: Obtaining a first edge pixel of the image under a high dynamic range (HDR) space; generating a first scaling factor based on the local brightness environment in which the first edge pixel resides; generating a color ratio to be processed corresponding to the first edge pixel based on brightness values ​​of a plurality of pixels in an image block in which the first edge pixel is located; scaling the target color proportion based on the first scaling factor to obtain a first color proportion; A step of obtaining a first color value by mixing the color value of the first edge pixel and the color value of a target pixel together with the first color ratio, wherein the target pixel is a pixel among a plurality of surrounding pixels of the first edge pixel that has the largest color value difference from the first edge pixel; updating the first color value as the color value of the first edge pixel.

2. generating a first scaling factor based on a local brightness environment in which the first edge pixel resides; 2. The method for anti-aliasing an image of claim 1, further comprising: generating the first scaling factor based on a brightness value of a first surrounding pixel and a brightness value of a second surrounding pixel, wherein the first surrounding pixel is the pixel with the largest brightness value among a plurality of pixels of the image block in which the first edge pixel is located, and the second surrounding pixel is the pixel with the smallest brightness value among a plurality of pixels of the image block in which the first edge pixel is located.

3. generating a first scaling factor based on the brightness values ​​of the first surrounding pixels and the brightness values ​​of the second surrounding pixels; inputting the brightness values ​​of the first surrounding pixels and the brightness values ​​of the second surrounding pixels into a scaling function to obtain the first scaling factor; 3. The image anti-aliasing method of claim 2, wherein the scaling function satisfies the following: the larger the brightness value of the first surrounding pixel, the larger the value of the first scaling factor; the smaller the brightness value of the second surrounding pixel, the smaller the value of the first scaling factor; and when the brightness value of the second surrounding pixel is zero, the value of the first scaling factor is equal to zero.

4. the brightness value of the first surrounding pixel is a first brightness value, the brightness value of the second surrounding pixel is a second brightness value; The step of inputting the brightness values ​​of the first surrounding pixels and the brightness values ​​of the second surrounding pixels into a scaling function to obtain the first scaling factor comprises: multiplying the first brightness value and the second brightness value to obtain a first sub-scaling factor; a step of setting the second brightness value as a second sub-scaling factor; 4. The image anti-aliasing method according to claim 3, further comprising the step of: adding the first sub-scaling factor and the second sub-scaling factor together, and then performing a normalization process to obtain the first scaling factor.

5. The step of generating a color ratio to be processed corresponding to the first edge pixel based on brightness values ​​of a plurality of pixels in an image block in which the first edge pixel is located includes: obtaining a plurality of brightness weights of a plurality of pixels in an image block in which the first edge pixel is located, wherein the plurality of pixels in the image block and the plurality of brightness weights correspond to each other one-to-one; generating a first brightness difference based on brightness and brightness weights of individual pixels in the image block, the first brightness difference being used to represent a local brightness environment in which the first edge pixel is located; and mapping the first brightness difference to obtain the target color percentage.

6. The image block is a square pixel array centered on the first edge pixel, and a plurality of brightness weights of a plurality of pixels in the image block satisfy the following condition: Pixels in the square pixel array that have the same distance to the first edge pixel have the same corresponding brightness weights; a pixel in the square pixel array that is closer to the first edge pixel has a smaller corresponding brightness weight; a brightness weight corresponding to a pixel other than the first edge pixel in the square pixel array is a positive number, and a brightness weight corresponding to the first edge pixel is a negative number; 6. The method of claim 5, wherein the sum of the intensity weights of all pixels in the square pixel array is zero.

7. generating a first brightness difference based on brightness and brightness weights of individual pixels in the image block; calculating the product of the brightness of each pixel in the image block and a respective brightness weight; and adding the products of the individual pixels in the image block and then dividing by the absolute value of the intensity weight corresponding to the first edge pixel to obtain the first intensity difference.

8. The image anti-aliasing method includes: The method further includes a step of normalizing the first brightness difference; The step of mapping the first lightness difference to obtain the color ratio of the processing target includes:

8. The image anti-aliasing method of claim 7, further comprising: mapping the first brightness difference after normalization to obtain the target color proportion, such that the target color proportion belongs to a closed interval from zero to one, and the target color proportion and the first brightness difference have a positive correlation.

9. The step of obtaining a first color value by blending the color values ​​of the first edge pixels and the color values ​​of the target pixels together with the first color proportion includes: multiplying a second color proportion by the color value of the first edge pixel to obtain a first color component, wherein the sum of the second color proportion and the first color proportion is 1; multiplying the first color proportion by the color value of the target pixel to obtain a second color component; and adding the first color component and the second color component to obtain the first color value.

10. The image anti-aliasing method includes: Under a circumstance where the first edge pixel is used to form a linear edge, obtaining a third color proportion corresponding to the first edge pixel based on a position of the first edge pixel on the linear edge; determining a maximum value of the first color ratio and the third color ratio as a maximum color ratio corresponding to the first edge pixel; The step of obtaining a first color value by blending the color values ​​of the first edge pixels and the color values ​​of the target pixels together with the first color proportion includes: According to the color value of the first edge pixel and the color value of the target pixel, the first color value is obtained by blending together with the maximum color percentage; The method of claim 1 , wherein the linear edge comprises a plurality of adjacent edge pixels.

11. The image anti-aliasing method includes: acquiring an image in the high dynamic range (HDR) space; For a j-th pixel of the image, obtaining a pixel with the maximum brightness value and a pixel with the minimum brightness value from among a plurality of pixels of an image block in which the j-th pixel is located, where j is a positive integer; determining that the jth pixel is an edge pixel when a difference in brightness between a pixel with the maximum brightness value and a pixel with the minimum brightness value among a plurality of pixels in the image block in which the jth pixel is located reaches a jth threshold value; determining and obtaining a plurality of edge pixels of the image; The method for anti-aliasing an image according to claim 1 , wherein the jth pixel is any one pixel of the image.

12. After the step of generating a first scaling factor based on the local brightness environment in which the first edge pixel is located, performing a depth filtering operation and a brightness compensation operation on the image to obtain edge pixel clusters; determining that the first edge pixel is a pixel of the edge pixel cluster; 2. The method of claim 1, wherein the edge pixel clusters are used to set scaling factors corresponding to pixels in the image other than the edge pixel clusters to zero.

13. performing a depth filtering operation and a brightness compensation operation on the image to obtain edge pixel clusters; performing a depth filtering operation on the image to obtain a first edge pixel cluster, wherein a depth value difference between pixels in the first edge pixel cluster and surrounding pixels reaches a threshold; for an i-th pixel outside the first edge pixel cluster in the image, determining a pixel with a maximum brightness value and a pixel with a minimum brightness value among a plurality of pixels in an image block in which the i-th pixel is located, where i is a positive integer; adding the i-th pixel to a second edge pixel cluster when a difference in brightness between a pixel with the maximum brightness value and a pixel with the minimum brightness value among a plurality of pixels in an image block in which the i-th pixel is located reaches an i-th threshold; 13. The method for anti-aliasing an image according to claim 12, wherein the i-th pixel is any one pixel in the image other than the first edge pixel cluster, and the first edge pixel cluster and the second edge pixel cluster jointly constitute the edge pixel cluster.

14. Before the step of blending the color values ​​of the first edge pixels and the color values ​​of the target pixels together with the first color proportion to obtain a first color value, looking up and obtaining an edge direction corresponding to the first edge pixel, the edge direction being a vertical direction or a horizontal direction; The image anti-aliasing method according to claim 1 , further comprising the step of determining, as the target pixel, a pixel that straddles the edge, is located in a direction perpendicular to the edge direction, and has a minimum distance from the first edge pixel.

15. 15. An apparatus for anti-aliasing an image, configured to perform the method for anti-aliasing an image according to any one of claims 1 to 14.

16. A computer device including a processor and a memory, wherein a computer program is stored in the memory, and wherein the computer program is loaded and executed by the processor to implement the image anti-aliasing method according to any one of claims 1 to 14.

17. A computer program that, when loaded and executed by a processor, implements the method for anti-aliasing an image according to any one of claims 1 to 14.

Images