Image Processing Method, Apparatus, Server, and Storage Medium

Abstract

The present application provides an image processing method, apparatus, server and storage medium. First, the initial image to be processed is enlarged to obtain an enlarged image, and the gradient value and gradient direction corresponding to the enlarged image are calculated; further, the enlarged image is subjected to segmented anti-aliasing filtering according to the gradient direction to obtain an initial anti-aliasing filtered image; and the enlarged image is sharpened to obtain an initial sharpened image; finally, the initial sharpened image and the initial anti-aliasing filtered image are fused to obtain a target image after initial image processing. By performing different processes on the enlarged image and then fusing multiple images after different processes, the initial image undergoes a variety of different processes, effectively improving the image clarity while also reducing the sawtooth artifact phenomenon in the image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and specifically to an image processing method, device, server and storage medium. Background Art

[0002] Traditional super-resolution reconstruction algorithms include interpolation-based super-resolution reconstruction, degradation model-based super-resolution reconstruction, and learning-based super-resolution reconstruction. Traditional super-resolution algorithms mainly rely on basic digital image processing technology for reconstruction. Super-resolution algorithms based on deep learning use the SRCNN model to enlarge the low-resolution image by interpolation and then restore it through the model.

[0003] Interpolation-based methods treat each pixel on the image as a point on the image plane, and use known pixel information to fit unknown pixel information on the plane, usually using a predefined transformation function or interpolation kernel. Common interpolation-based methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation.

[0004] The interpolation-based method is simple to calculate and easy to understand, but it also has some obvious defects. First, it assumes that the change of pixel grayscale value is a continuous and smooth process, but in fact this assumption is not completely true. Second, in the reconstruction process, the super-resolution image is calculated only based on a pre-defined conversion function without considering the image degradation model, which often causes the restored image to appear blurry and jagged. Summary of the invention

[0005] The present application aims to provide an image processing method, device, server and storage medium, aiming to solve the problem that image processing in the prior art still produces aliasing and artifacts.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0007] Enlarging the initial image to be processed to obtain an enlarged image, and calculating the gradient value and gradient direction corresponding to the enlarged image;

[0008] According to the gradient direction, performing segmented anti-aliasing filtering on the enlarged image to obtain an initial anti-aliasing filtered image;

[0009] sharpening the enlarged image to obtain an initial sharpened image;

[0010] The initial sharpened image is fused with the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image.

[0011] In a possible embodiment, the method of magnifying the initial image to be processed to obtain a magnified image, and calculating the gradient value and gradient direction corresponding to the magnified image includes:

[0012] Magnify the initial image using bicubic linear interpolation to obtain the magnified image;

[0013] Calculate the gradient value of the magnified image using a preset gradient operator, and obtain the gradient direction of the magnified image.

[0014] In a possible embodiment, the method of calculating the gradient value of the magnified image using a preset gradient operator and obtaining the gradient direction of the magnified image includes:

[0015] Calculate the horizontal gradient value of the magnified image using a preset first gradient operator;

[0016] Calculate the vertical gradient value of the magnified image using a preset second gradient operator;

[0017] Calculate the gradient value of the magnified image and the gradient direction of the magnified image based on the horizontal gradient value and the vertical gradient value.

[0018] In a possible embodiment, the method of performing segmented anti-aliasing filtering on the magnified image according to the gradient direction to obtain an initial anti-aliasing filtered image includes:

[0019] Obtain a filtering operator corresponding to the gradient direction, and perform anti-aliasing filtering on the magnified image according to the filtering operator to obtain the initial anti-aliasing filtered image;

[0020] Among them, when the gradient direction is within different angular ranges, the filtering operators are different, and there are multiple filtering operators.

[0021] In a possible embodiment, the method of sharpening the magnified image to obtain an initial sharpened image includes:

[0022] Sharpen the magnified image using a preset sharpening operator to obtain the initial sharpened image.

[0023] In a possible embodiment, the method of fusing the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image includes:

[0024] Determine a first weight corresponding to the initial anti-aliasing filtered image and a second weight corresponding to the initial sharpened image;

[0025] Obtain a target anti-aliasing filtered image according to the first weight and the initial anti-aliasing filtered image;

[0026] Obtain a target sharpened image based on the second weight and the initial sharpened image;

[0027] Fuse the target anti-aliasing filtered image and the target sharpened image to obtain the target image.

[0028] In a possible embodiment, there are multiple gradient values;

[0029] The determining the first weight corresponding to the initial anti-aliasing filtered image and the second weight corresponding to the initial sharpened image includes:

[0030] Determine the maximum gradient value among multiple gradient values, so as to determine a first gradient value threshold and a second gradient value threshold according to the maximum gradient value;

[0031] Obtain the corresponding relationship between the preset gradient value and the weight;

[0032] Determine the first weight and the second weight according to the first gradient threshold, the second gradient threshold and the preset corresponding relationship between the gradient value and the weight.

[0033] In a second aspect, an embodiment of the present application provides an image processing apparatus, and the apparatus includes:

[0034] An amplification module, configured to amplify an initial image to be processed to obtain an amplified image, and calculate the gradient value and the gradient direction corresponding to the amplified image;

[0035] A filtering module, configured to perform segmented anti-aliasing filtering on the amplified image according to the gradient direction to obtain an initial anti-aliasing filtered image;

[0036] A sharpening module, configured to sharpen the amplified image to obtain an initial sharpened image;

[0037] A fusion module, configured to fuse the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image.

[0038] In a third aspect, an embodiment of the present application provides a server, and the server includes:

[0039] One or more processors;

[0040] A memory; and

[0041] One or more applications, where the one or more applications are stored in the memory and are configured to be executed by the processor to implement the image processing method as described in any one of the above.

[0042] Fourthly, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program is loaded by a processor to execute the steps in the image processing method described in any one of the above.

[0043] An embodiment of the present application provides an image processing method, device, server and storage medium. First, an initial image to be processed is enlarged to obtain an enlarged image, and the gradient value and gradient direction corresponding to the enlarged image are calculated; further, the enlarged image is subjected to segmented anti-aliasing filtering according to the gradient direction to obtain an initial anti-aliasing filtered image; and the enlarged image is sharpened to obtain an initial sharpened image; finally, the initial sharpened image and the initial anti-aliasing filtered image are fused to obtain a target image after initial image processing. By performing different processes on the enlarged image and then fusing multiple images after different processes, the initial image undergoes a variety of different processes, effectively improving the image clarity while also reducing the sawtooth artifact phenomenon in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0045] Figure 1 It is a schematic diagram of the scenario of the image processing system provided by the embodiment of the present application;

[0046] Figure 2 It is a schematic flowchart of an embodiment of the image processing device provided by the embodiment of the present application;

[0047] Figure 3 It is a schematic flowchart of an embodiment of calculating the gradient value and gradient direction of an image provided by the embodiment of the present application;

[0048] Figure 4 It is a schematic diagram of the gradient direction provided by the embodiment of the present application;

[0049] Figure 5 It is a schematic flowchart of an embodiment of image fusion provided by the embodiment of the present application;

[0050] Figure 6 It is a schematic diagram of an embodiment of the image processing device provided by the embodiment of the present application;

[0051] Figure 7 It shows a schematic diagram of the structure of the server involved in the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the scope of protection of the present application.

[0053] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present application. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, "a plurality" means two or more, unless otherwise specifically defined.

[0054] In the present application, the term "exemplary" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "exemplary" in the present application is not necessarily to be construed as more preferred or more advantageous than other embodiments. In order for any person skilled in the art to implement and use the present application, the following description is given. In the following description, details are set forth for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid unnecessary details from obscuring the description of the present application. Therefore, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the present application.

[0055] It should be noted that since the method of the embodiments of the present application is executed in an electronic device, the processing objects of each electronic device exist in the form of data or information. For example, time, which is actually time information. It can be understood that in subsequent embodiments, if dimensions, quantities, positions, etc. are mentioned, they are all corresponding data existences for the electronic device to process, and specific details are not elaborated here.

[0056] The embodiments of the present application provide an image processing method, apparatus, server, and storage medium, which will be described in detail below respectively.

[0057] Please refer to Figure 1 , Figure 1 which is a schematic diagram of the scenario of the image processing system provided by the embodiment of the present application. The image processing system may include an electronic device 100, and an image processing device is integrated in the electronic device 100, such as Figure 1 the electronic device in

[0058] In the embodiment of the present application, the electronic device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the electronic device 100 described in the embodiment of the present application includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. Among them, the cloud server is composed of a large number of computers or network servers based on cloud computing (Cloud Computing).

[0059] Those skilled in the art can understand that Figure 1 the application environment shown in Figure 1 is only one application scenario of the solution of the present application, and does not constitute a limitation on the application scenario of the solution of the present application. Other application environments may also include more or fewer electronic devices than those shown in Figure 1 For example, only 1 electronic device is shown in

[0060] And as shown in Figure 1 the image processing system may further include a storage module 200 for storing data.

[0061] It should be noted that Figure 1 the schematic diagram of the scenario of the image processing system shown is only an example. The image processing system and scenario described in the embodiment of the present application are for more clearly explaining the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. Those skilled in the art know that with the evolution of the image processing system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.

[0062] First, an image processing method is provided in the embodiment of the present application. The execution subject of the image processing method is an image processing device, and the image processing device is applied to an electronic device. The image processing method includes:

[0063] Enlarge the initial image to be processed to obtain an enlarged image, and calculate the gradient value and gradient direction corresponding to the enlarged image; perform segmented anti-aliasing filtering on the enlarged image according to the gradient direction to obtain an initial anti-aliasing filtered image; sharpen the enlarged image to obtain an initial sharpened image; fuse the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image.

[0064] As Figure 2 shown, it is a schematic flowchart of an embodiment of an image processing method provided by an embodiment of the present application, which may include:

[0065] 21. Enlarge the initial image to be processed to obtain an enlarged image, and calculate the gradient value and gradient direction corresponding to the enlarged image.

[0066] In the embodiment of the present application, before processing the initial image, it is necessary to perform a certain scaling on the initial image to improve the resolution and clarity of the image, facilitating subsequent processing.

[0067] In some embodiments, the initial image can be enlarged using an interpolation algorithm, which is a basic and important algorithm in image scaling. In image scaling, the pixel coordinates of the output image may correspond to positions between several pixels on the input image. At this time, it is necessary to calculate the gray value of the output point through gray interpolation processing. The quality of the interpolation algorithm also directly affects the degree of image distortion; the commonly used interpolation algorithms are the following three: nearest neighbor interpolation algorithm, bilinear interpolation algorithm, and bicubic linear interpolation algorithm.

[0068] In the embodiment of the present application, the initial image can be enlarged using the bicubic linear interpolation algorithm to obtain an enlarged image; then, the gradient value of the enlarged image after enlargement is calculated using a preset gradient operator, and the gradient direction of the enlarged image is obtained.

[0069] Among them, the reason for enlarging the initial image using the bicubic linear interpolation algorithm is that bicubic linear interpolation uses the gray values of 16 points around the point to be sampled for cubic interpolation, taking into account not only the gray influence of the directly adjacent points of the sampling point but also the influence of the gray value change rate between adjacent points. The enlarged image obtained using bicubic linear interpolation has higher accuracy.

[0070] Among them, the specific process of enlarging the initial image using the bicubic linear interpolation algorithm can refer to the prior art and will not be limited here.

[0071] 22. Perform segmented anti-aliasing filtering on the enlarged image according to the gradient direction to obtain an initial anti-aliasing filtered image.

[0072] 23. Sharpen the enlarged image to obtain an initial sharpened image.

[0073] 24. Fuse the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image.

[0074] For the embodiments of the present application, after magnifying the initial image to obtain a magnified image, it is necessary to process the magnified image from different angles to obtain images processed by different means, and then fuse the processed images to avoid jagged edges and artifacts in the images obtained by a single processing method.

[0075] Specifically, the magnified image can be first subjected to segmented anti-aliasing filtering according to the gradient direction of the image to obtain an anti-aliasing filtered image.

[0076] In the above embodiments, the magnified image is mainly subjected to anti-aliasing filtering. It is also necessary to sharpen the magnified image to obtain a sharpened image; then fuse the sharpened image and multiple gradient regions. In this way, the processed target image obtained not only undergoes sharpening but also filtering, effectively improving the accuracy of the target image.

[0077] It should be noted that in the embodiments of the present application, the image is not sharpened after anti-aliasing filtering; nor is the image subjected to anti-aliasing filtering after sharpening. Instead, the magnified image is respectively subjected to segmented anti-aliasing filtering and sharpening, and then the image obtained by segmented anti-aliasing filtering and the image obtained by sharpening are fused to finally obtain a processed image. In this way, the magnified image is only processed once, and the obtained image has better accuracy and better processing effect.

[0078] The image processing method provided by the embodiments of the present application first magnifies the initial image to be processed to obtain a magnified image, and calculates the gradient value and gradient direction corresponding to the magnified image; further performs segmented anti-aliasing filtering on the magnified image according to the gradient direction to obtain an initial anti-aliasing filtered image; and sharpens the magnified image to obtain an initial sharpened image; finally, fuses the initial sharpened image and the initial anti-aliasing filtered image to obtain a target image after processing the initial image. By performing different processes on the magnified image and then fusing multiple images processed differently, the initial image undergoes multiple different processes, effectively reducing the jagged artifact phenomenon in the image while improving the image clarity.

[0079] In the embodiments of the present application, the magnitude of the gradient value of the image represents the speed of change of the image gray value. Since an image includes multiple pixels, there are differences between the gray values corresponding to any two adjacent pixels, which results in the fact that each pixel in the image actually corresponds to a gradient value. Therefore, calculating the gradient value and gradient direction of the image in the embodiments of the present application is actually calculating the gradient value and gradient direction corresponding to each pixel in the image.

[0080] Moreover, since the gradient is a vector, in the embodiments of the present application, the gradient value of the image is actually the magnitude corresponding to the gradient, and the gradient direction is the direction corresponding to the gradient. That is, the gradient value and the gradient direction are two specific parameters in the gradient.

[0081] As Figure 3 shown, it is a schematic flowchart of an embodiment for calculating the gradient value and gradient direction of an image provided by the embodiments of the present application, which may include:

[0082] 31. Calculate the horizontal gradient value of the enlarged image using a preset first gradient operator.

[0083] 32. Calculate the vertical gradient value of the enlarged image using a preset second gradient operator.

[0084] 33. Calculate the gradient value of the enlarged image and the gradient direction of the enlarged image according to the horizontal gradient value and the vertical gradient value.

[0085] In the embodiments of the present application, to calculate the gradient value and gradient direction of the enlarged image, it is first necessary to sample in the enlarged image to obtain any sampling point. The calculation of the gradient value or gradient direction of the image described in the embodiments of the present application is actually the calculation of the gradient value and gradient direction of this sampling point.

[0086] The embodiments of the present application provide different gradient operators to calculate the horizontal gradient value and vertical gradient value corresponding to the sampling point respectively. Specifically, the Sobel operator can be used to calculate the horizontal gradient value and vertical gradient value of the sampling point. Among them, the first gradient operator can be:

[0087]

[0088] And the calculation of the horizontal gradient value of the sampling point using the first gradient operator can be:

[0089] G x = S bx * I bc

[0090] And G x is the horizontal gradient value corresponding to the sampling point, S bx is the first gradient operator, and I bc is the grayscale value (or pixel value) corresponding to the sampling point.

[0091] And the second gradient operator can be:

[0092]

[0093] Similarly, the calculation of the vertical gradient value of the sampling point using the second gradient operator can be:

[0094] G y = Sby *I bc

[0095] And G y is the vertical gradient value corresponding to the sampling point, and S by is the second gradient operator, and I bc is the grayscale value (or pixel value) corresponding to the sampling point.

[0096] In the above embodiments, the horizontal gradient value and the vertical gradient value of the pixel are determined. In fact, the gradient value corresponding to the pixel needs to be obtained by using the horizontal gradient value and the vertical gradient value. Specifically, the gradient value corresponding to the pixel can be:

[0097]

[0098] That is, in the embodiments of the present application, it is necessary to first determine the horizontal gradient and the vertical gradient corresponding to the pixel, and then calculate the gradient value corresponding to the pixel according to the horizontal gradient and the vertical gradient.

[0099] After calculating the horizontal gradient value and the vertical gradient value corresponding to the sampling point, the gradient direction corresponding to the sampling point can be further calculated by using the horizontal gradient value and the vertical gradient value. Specifically, the gradient direction corresponding to the sampling point can be:

[0100]

[0101] That is, in the embodiments of the present application, the ratio value of the horizontal gradient value and the vertical gradient value can be solved by using the arctangent function, and the obtained result is the angle size corresponding to the gradient, and then the gradient direction can be determined. In the embodiments of the present application, the value range of the gradient direction θ is (-π, π). As Figure 4 shown, it is a schematic diagram of the gradient direction provided by the embodiments of the present application.

[0102] In the actual image processing process, the grayscale value (or pixel value) corresponding to each pixel in the image is relatively easy to determine. Therefore, the horizontal gradient value and the vertical gradient value corresponding to the pixel can be determined by using a preset gradient operator. After determining the horizontal gradient value and the vertical gradient value, the gradient value and the gradient direction corresponding to the pixel are determined.

[0103] After determining the gradient direction corresponding to the sampling point, it is also necessary to perform segmented anti-aliasing filtering on the enlarged image according to the gradient direction. Since the gradient direction is the direction in which the grayscale value rises or falls fastest, performing segmented anti-aliasing filtering on the enlarged image based on the gradient direction can effectively reduce or remove the jaggedness in the image.

[0104] In some embodiments, a filtering operator corresponding to the gradient direction is also obtained to perform anti-aliasing filtering on the enlarged image according to the filtering operator. Only when the gradient direction is within different angular ranges, the filtering operators are different, so there are multiple filtering operators.

[0105] Specifically, the formula for performing segmented anti-aliasing filtering on the enlarged image can be:

[0106] I AAFn =AAF n *I bc

[0107] Wherein, I AAFn is the gray scale value corresponding to the sampled point after filtering, and AAF n is the filtering operator provided by the embodiments of the present application. Generally speaking, there are multiple filtering operators. And I bc is the original gray scale value corresponding to the sampled point.

[0108] In the embodiments of the present application, the range of the gradient direction is between (-π, π), and the absolute value of the gradient direction needs to be taken during calculation. At this time, the value range of θ = abs(θ) becomes (0, π).

[0109] In a specific embodiment, four different filtering operators, namely AAF1, AAF2, AAF3, and AAF4, can be selected according to the gradient direction θ within the range of (0, π). And the four different gradient operators are respectively:

[0110]

[0111]

[0112] At this time, the formula for segmented anti-aliasing filtering can be:

[0113]

[0114] Taking the gradient direction θ = abs(θ) corresponding to the sampled point as π / 4 as an example, substituting it into the above formula for segmented anti-aliasing filtering, t1 can be obtained as 1, and then I AAF =(1 - 1)*I AAF1 + 1*I AAF2 =I AAF2 . And according to I AAFn =AAF n *I bc the size of I AAF2 can also be determined, so as to determine the size of the gray scale value after segmented anti-aliasing filtering corresponding to the sampled point when θ = π / 4.

[0115] The above embodiments mainly perform anti-aliasing filtering on the enlarged image, aiming to remove sawtooth shadows and the like in the enlarged image after enlargement. In the embodiments of the present application, the image also needs to be sharpened to obtain a sharpened image, and the sharpened image is fused with the anti-aliased filtered image after anti-aliasing filtering.

[0116] In some embodiments, sharpening the enlarged image to obtain an initial sharpened image may include: sharpening the enlarged image using a preset sharpening operator to obtain an initial sharpened image.

[0117] Specifically, the enlarged image can be sharpened using a preset Laplacian operator to obtain a sharpened image. The sharpening formula can be: I lp = I bc + G lp *(S lp * I bc )

[0118] where G lp is the sharpening coefficient, and the sharpening coefficient G lp is a value that can be adjusted. The sharpening coefficient G lp can be adjusted according to the actual sharpening situation to obtain images with different sharpening degrees. In the embodiments of the present application, the sharpening coefficient G lp can be adjusted within the range of (1, 4). In a specific embodiment, the sharpening coefficient G lp can be 2, and S lp is the preset Laplacian operator. When the value of the Laplacian operator is determined, S lp * I bc is also determined. In the embodiments of the present application, the sharpening degree of the image is mainly adjusted by adjusting the sharpening coefficient G lp ; therefore, the sharpening coefficient G lp in the embodiments of the present application is a value that can be adjusted.

[0119] In the above embodiments, I bc is also the grayscale value corresponding to the sampling point. In the embodiments of the present application, the grayscale value corresponding to the sampling point is first processed using a filtering operator and then multiplied by the adjustable sharpening coefficient G lp .

[0120] In a specific embodiment, the Laplacian operator S lp can be:

[0121]

[0122] It should be noted that the specific values of the first gradient operator, the second gradient operator, and the Laplace operator provided in the embodiments of the present application are the values conventionally used. In other embodiments, the first gradient operator, the second gradient operator, and the Laplace operator may also take other values, which are not limited herein.

[0123] In the above embodiments, the anti-aliasing filtered image is obtained by performing anti-aliasing filtering on the magnified magnified image respectively, and the sharpened image is obtained by sharpening the magnified image; the embodiments of the present application also need to fuse the anti-aliasing filtered image and the sharpened image, so that the obtained image is both anti-aliasing filtered and sharpened.

[0124] As Figure 5 shown, it is a schematic flowchart of an embodiment of image fusion provided by the embodiments of the present application, which may include:

[0125] 51. Determine the first weight corresponding to the initial anti-aliasing filtered image and the second weight corresponding to the initial sharpened image.

[0126] Specifically, in the embodiments of the present application, when fusing the image obtained by anti-aliasing filtering and the image obtained by sharpening, for different pixels, the weights corresponding to the initial anti-aliasing filtered image and the initial sharpened image are different.

[0127] In some embodiments, determining the first weight corresponding to the initial anti-aliasing filtered image and the second weight corresponding to the initial sharpened image may include: determining the maximum gradient value among multiple gradient values, so as to determine the first gradient value threshold and the second gradient value threshold according to the maximum gradient value; obtaining the corresponding relationship between the preset gradient value and the weight; and determining the first weight and the second weight according to the first gradient threshold, the second gradient threshold, and the preset corresponding relationship between the gradient value and the weight.

[0128] Specifically, since the gradient value and the gradient direction corresponding to each pixel in the magnified image are determined in the present application, there are actually multiple gradient values; and there is a maximum gradient value with the largest numerical value among the multiple different gradient values.

[0129] The present application also provides a preset corresponding relationship between the gradient value and the weight; in this preset corresponding relationship between the gradient value and the weight, when the gradient value corresponding to the sampling point is within different angular ranges, the first weight and the second weight corresponding to the sampling point are also different; therefore, the initial anti-aliasing filtered image and the initial sharpened image for fusion are also different.

[0130] Meanwhile, the embodiments of the present application also determine the first gradient threshold and the second gradient threshold according to the maximum gradient value, and the specific weight values in the aforementioned preset corresponding relationship between the gradient value and the weight can be obtained by further calculating the first gradient threshold and the second gradient threshold.

[0131] In a specific embodiment, the maximum gradient value can be: max(G). The first gradient threshold and the second gradient threshold obtained based on the maximum gradient value can be:

[0132] E icor = a * max(G);

[0133] E ith = b * max(G);

[0134] where E icor is the first gradient threshold, and E ith is the second gradient threshold; the values of a and b can be set arbitrarily; generally, the value of a is less than b. In a specific embodiment, a can be 0.2 and b can be 0.4.

[0135] And based on the first gradient threshold, the second gradient threshold, and the preset correspondence between the gradient value and the weight, determining the first weight and the second weight can be:

[0136]

[0137] E in the above formula i is the weight parameter; and the first weight can be 1 - E i , and the second weight is E i .

[0138] 52. Obtain the target anti-aliasing filtered image according to the first weight and the initial anti-aliasing filtered image.

[0139] At this time, obtaining the target anti-aliasing filtered image according to the first weight and the initial anti-aliasing filtered image can be: (1 - E i ) * I AAF .

[0140] 53. Obtain the target sharpened image according to the second weight and the initial sharpened image.

[0141] And obtaining the target sharpened image according to the second weight and the initial sharpened image can be: E i * I lp .

[0142] 54. Fuse the target anti-aliasing filtered image and the target sharpened image to obtain the target image.

[0143] At this time, fusing the target anti-aliasing filtered image and the target sharpened image can be:

[0144] I fusion = (1 - E i ) * I AAF + E i * I lp

[0145] The above formula is the image fusion formula provided by the embodiments of the present application. Among them, I fusion is the pixel gray scale value corresponding to the fused sampling point, (1 - E i ) * I AAF is the target anti-aliasing filtered image, and E i * I lp is the target sharpened image. The weight parameter E i needs to be determined according to the maximum gradient value in the enlarged image.

[0146] The embodiments of the present application also provide an image processing apparatus, as Figure 6 shown, which is a schematic diagram of an embodiment of the image processing apparatus provided by the embodiments of the present application, and may include:

[0147] An enlargement module 601, configured to enlarge an initial image to be processed to obtain an enlarged image, and calculate the gradient value and gradient direction corresponding to the enlarged image;

[0148] A filtering module 602, configured to perform segmented anti-aliasing filtering on the enlarged image according to the gradient direction to obtain an initial anti-aliasing filtered image;

[0149] A sharpening module 603, configured to sharpen the enlarged image to obtain an initial sharpened image;

[0150] A fusion module 604, configured to fuse the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image.

[0151] For the image processing apparatus provided by the embodiments of the present application, first, the initial image to be processed is enlarged to obtain an enlarged image, and the gradient value and gradient direction corresponding to the enlarged image are calculated; further, segmented anti-aliasing filtering is performed on the enlarged image according to the gradient direction to obtain an initial anti-aliasing filtered image; and the enlarged image is sharpened to obtain an initial sharpened image; finally, the initial sharpened image and the initial anti-aliasing filtered image are fused to obtain a target image after processing the initial image. By performing different processes on the enlarged image and then fusing multiple images after different processes, the initial image undergoes a variety of different processes, effectively improving the image clarity while also reducing the sawtooth artifact phenomenon in the image.

[0152] In some embodiments, the enlargement module 601 may specifically be configured to: enlarge the initial image by using a preset bicubic linear interpolation method to obtain an enlarged image; calculate the gradient value of the enlarged image by using a preset gradient operator, and obtain the gradient direction of the enlarged image.

[0153] In some embodiments, the amplification module 601 may specifically be configured to: calculate the horizontal gradient value of the amplified image by using a preset first gradient operator; calculate the vertical gradient value of the amplified image by using a preset second gradient operator; calculate the gradient value of the amplified image and the gradient direction of the amplified image based on the horizontal gradient value and the vertical gradient value.

[0154] In some embodiments, the filtering module 602 may specifically be configured to: obtain a filtering operator corresponding to the gradient direction, and perform anti-aliasing filtering on the amplified image according to the filtering operator to obtain an initial anti-aliasing filtered image;

[0155] Among them, when the gradient direction is within different angular ranges, the filtering operators are different, and there are multiple filtering operators.

[0156] In some embodiments, the sharpening module 603 may specifically be configured to: sharpen the amplified image by using a preset sharpening operator to obtain an initial sharpened image.

[0157] In some embodiments, the fusion module 604 may specifically be configured to: determine a first weight corresponding to the initial anti-aliasing filtered image and a second weight corresponding to the initial sharpened image; obtain a target anti-aliasing filtered image according to the first weight and the initial anti-aliasing filtered image; obtain a target sharpened image according to the second weight and the initial sharpened image; fuse the target anti-aliasing filtered image and the target sharpened image to obtain a target image.

[0158] In some embodiments, there are multiple gradient values; the fusion module 604 may specifically be configured to: determine the maximum gradient value among the multiple gradient values, and determine a first gradient value threshold and a second gradient value threshold according to the maximum gradient value; obtain the corresponding relationship between the preset gradient value and the weight; determine the first weight and the second weight according to the first gradient threshold, the second gradient threshold, and the corresponding relationship between the preset gradient value and the weight.

[0159] An embodiment of the present application further provides an electronic device, which integrates any one of the image processing apparatuses provided in the embodiments of the present application. As Figure 7 shown, it shows a schematic structural diagram of a server involved in the embodiments of the present application. Specifically:

[0160] The electronic device may include a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, an input unit 704, and other components. Those skilled in the art can understand that the structural diagram of the electronic device shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Among them:

[0161] The processor 701 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 702, and by invoking the data stored in the memory 702, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 701 either.

[0162] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.); the data storage area can store data created according to the use of the electronic device. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0163] The electronic device further includes a power supply 703 for supplying power to each component. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 703 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0164] The electronic device may further include an input unit 704, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0165] Although not shown, the electronic device may further include a display unit, etc., which will not be elaborated here. Specifically, in this embodiment, the processor 701 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application programs stored in the memory 702 to realize various functions as follows:

[0166] Enlarge the initial image to be processed to obtain an enlarged image, and calculate the gradient value and gradient direction corresponding to the enlarged image; according to the gradient direction, perform segmented anti-aliasing filtering on the enlarged image to obtain an initial anti-aliasing filtered image; sharpen the enlarged image to obtain an initial sharpened image; fuse the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image.

[0167] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions or by controlling relevant hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0168] For this reason, an embodiment of the present application provides a computer-readable storage medium, which may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), a magnetic disk or an optical disc, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the image processing methods provided by the embodiments of the present application. For example, when the computer program is loaded by a processor, the following steps may be executed:

[0169] Enlarge the initial image to be processed to obtain an enlarged image, and calculate the gradient value and gradient direction corresponding to the enlarged image; according to the gradient direction, perform segmented anti-aliasing filtering on the enlarged image to obtain an initial anti-aliasing filtered image; sharpen the enlarged image to obtain an initial sharpened image; fuse the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image.

[0170] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the detailed descriptions of other embodiments above, and details will not be repeated here.

[0171] In specific implementation, the above-mentioned various units or structures can be implemented as independent entities, or can be combined arbitrarily to be implemented as the same or several entities. The specific implementation of the above-mentioned various units or structures can refer to the method embodiments above, and details will not be repeated here.

[0172] The specific implementation of the above operations can refer to the previous embodiments, and details will not be repeated here.

[0173] The above has introduced in detail an image processing method, apparatus, server, and storage medium provided by the embodiments of the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An image processing method, characterized in that, The method includes: Enlarging an initial image to be processed to obtain an enlarged image, and calculating a gradient value and a gradient direction corresponding to each pixel in the enlarged image; Performing segmented anti-aliasing filtering on the enlarged image according to the gradient direction to obtain an initial anti-aliasing filtered image; Sharpening the enlarged image to obtain an initial sharpened image; Fusing the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image; The fusing the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image includes: Determining a first weight corresponding to the initial anti-aliasing filtered image and a second weight corresponding to the initial sharpened image; Obtaining a target anti-aliasing filtered image according to the first weight and the initial anti-aliasing filtered image; Obtaining a target sharpened image according to the second weight and the initial sharpened image; Fusing the target anti-aliasing filtered image and the target sharpened image to obtain the target image; There are multiple gradient values; the determining a first weight corresponding to the initial anti-aliasing filtered image and a second weight corresponding to the initial sharpened image includes: Determining a maximum gradient value among the multiple gradient values, and determining a first gradient value threshold and a second gradient value threshold according to the maximum gradient value; Obtaining a corresponding relationship between a preset gradient value and a weight; Determining the first weight and the second weight according to the first gradient value threshold, the second gradient value threshold, and the preset corresponding relationship between the gradient value and the weight; Wherein, the calculation methods of the first weight and the second weight are: 1- E i is the first weight, E i is the second weight, E icor is the first gradient value threshold, E ith is the second gradient value threshold, G is the gradient value corresponding to the pixel.

2. The image processing method according to claim 1, wherein The enlarging an initial image to be processed to obtain an enlarged image, and calculating a gradient value and a gradient direction corresponding to the enlarged image includes: Enlarging the initial image by using bicubic linear interpolation to obtain the enlarged image; Calculating the gradient value of the enlarged image by using a preset gradient operator, and obtaining the gradient direction of the enlarged image.

3. The image processing method according to claim 2, wherein The calculating the gradient value of the enlarged image by using a preset gradient operator, and obtaining the gradient direction of the enlarged image includes: Calculating the horizontal gradient value of the enlarged image by using a preset first gradient operator; Calculating the vertical gradient value of the enlarged image by using a preset second gradient operator; Calculating the gradient value of the enlarged image and the gradient direction of the enlarged image according to the horizontal gradient value and the vertical gradient value.

4. The image processing method according to claim 1, wherein The performing segmented anti-aliasing filtering on the enlarged image according to the gradient direction to obtain an initial anti-aliasing filtered image includes: Obtaining a filtering operator corresponding to the gradient direction, and performing anti-aliasing filtering on the enlarged image according to the filtering operator to obtain the initial anti-aliasing filtered image; Wherein, when the gradient direction is within different angular ranges, the filtering operators are different, and there are multiple filtering operators.

5. The image processing method according to claim 1, wherein The sharpening the enlarged image to obtain an initial sharpened image includes: Sharpening the enlarged image by using a preset sharpening operator to obtain the initial sharpened image.

6. An image processing apparatus, characterized in that, The device includes: An amplification module, configured to amplify an initial image to be processed to obtain an amplified image, and calculate a gradient value and a gradient direction corresponding to each pixel in the amplified image; A filtering module, configured to perform segmented anti-aliasing filtering on the amplified image according to the gradient direction to obtain an initial anti-aliasing filtered image; A sharpening module, configured to sharpen the amplified image to obtain an initial sharpened image; A fusion module, configured to fuse the initial sharpened image and the initial anti-aliasing filtered image to obtain a processed target image corresponding to the initial image; The fusion module is further configured to determine a first weight corresponding to the initial anti-aliasing filtered image and a second weight corresponding to the initial sharpened image; obtain a target anti-aliasing filtered image according to the first weight and the initial anti-aliasing filtered image; obtain a target sharpened image according to the second weight and the initial sharpened image; and fuse the target anti-aliasing filtered image and the target sharpened image to obtain the target image; There are multiple gradient values; the fusion module is further configured to determine a maximum gradient value among the multiple gradient values, so as to determine a first gradient value threshold and a second gradient value threshold according to the maximum gradient value; obtain a preset corresponding relationship between the gradient value and the weight; and determine the first weight and the second weight according to the first gradient value threshold, the second gradient value threshold, and the preset corresponding relationship between the gradient value and the weight; Wherein, the calculation methods of the first weight and the second weight are: 1- E i is the first weight, E i is the second weight, E icor is the first gradient value threshold, E ith is the second gradient value threshold, and G is the gradient value corresponding to the pixel.

7. A server, characterized in that, The server includes: One or more processors; A memory; and One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to implement the image processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is loaded by the processor to execute the steps in the image processing method according to any one of claims 1 to 5.

Images