Image processing method and device, electronic equipment and storage medium

Abstract

The application discloses an image processing method and device, electronic equipment and storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first image frame; determining a spatial domain kernel and a pixel domain kernel of the first image frame according to pixel points in the first image frame; determining at least one sawtooth point in the first image frame which needs to be subjected to smoothing processing according to a gray value of a pixel point in the spatial domain kernel and a gray value of a pixel point in the pixel domain kernel; determining a target sampling step according to a difference value between a pixel value of the sawtooth point and a pixel value of a neighborhood pixel point; and performing smoothing processing on the sawtooth point based on the target sampling step and coordinate information of the sawtooth point, to obtain a sawtooth point subjected to smoothing processing, wherein the neighborhood pixel point is a pixel point in a neighboring preset range of the sawtooth point.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to an image processing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the rapid development of electronic equipment technology, when displaying an image using an electronic equipment, the user's requirement for the quality of the displayed image is also increasingly high. However, there are many jaggies in the image. How to process these jaggies to improve the quality of the image is a problem to be solved at present. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide an image processing method and device, electronic equipment and a storage medium. The jaggies in the image are processed to improve the quality of the image.

[0004] In a first aspect, the embodiments of the present application provide an image processing method, which comprises:

[0005] obtaining a first image frame;

[0006] determining a spatial domain kernel and a pixel domain kernel of the first image frame according to the pixel points in the first image frame;

[0007] determining at least one jaggie point in the first image frame that needs to be processed by smoothing according to the gray values of the pixel points in the spatial domain kernel and the gray values of the pixel points in the pixel domain kernel;

[0008] determining a target sampling step length according to the difference between the pixel value of the jaggie point and the pixel value of the neighborhood pixel point;

[0009] processing the jaggie point by smoothing based on the target sampling step length and the coordinate information of the jaggie point to obtain a jaggie point processed by smoothing, wherein the neighborhood pixel point is a pixel point within a predetermined range adjacent to the jaggie point.

[0010] In a second aspect, the embodiments of the present application provide an image processing device, which comprises:

[0011] an obtaining module configured to obtain a first image frame;

[0012] a determining module configured to determine a spatial domain kernel and a pixel domain kernel of the first image frame according to the pixel points in the first image frame;

[0013] The determining module is further configured to determine at least one jaggie point in the first image frame that needs to be processed by smoothing according to the gray values of the pixel points in the spatial domain kernel and the gray values of the pixel points in the pixel domain kernel.

[0014] The determination module is further configured to determine a target sampling step length according to a difference between a pixel value of the jaggy point and a pixel value of a neighboring pixel point;

[0015] The determination module is further configured to perform smoothing processing on the jaggy point based on the target sampling step length and coordinate information of the jaggy point to obtain a jaggy point after the smoothing processing, wherein the neighboring pixel point is a pixel point within a preset range adjacent to the jaggy point.

[0016] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores programs or instructions executable on the processor. When the programs or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.

[0017] In a fourth aspect, a readable storage medium is provided, which stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the method according to the first aspect are implemented.

[0018] In a fifth aspect, a chip is provided, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run programs or instructions to implement the method according to the first aspect.

[0019] In a sixth aspect, a computer program product is provided, which is stored in a storage medium. The program product is executed by at least one processor to implement the method according to the first aspect.

[0020] In the embodiments of the present application, at least one jaggy point in the first image frame that needs to be smoothed is determined according to the gray value of the pixel point in the spatial domain kernel and the gray value of the pixel point in the pixel domain kernel of the first image frame. Then, a target sampling step length is determined according to the difference between the pixel value of the jaggy point and the pixel value of the neighboring pixel point. The jaggy point is smoothed based on the target sampling step length and the coordinate information of the jaggy point to obtain a jaggy point after the smoothing processing. Since each jaggy point has a different target sampling step length when being smoothed, the jaggy point is smoothed according to the different target sampling step length, instead of being smoothed according to the same sampling step length. The accuracy of the smoothing of the jaggy points is improved, and the quality of the first image frame is improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a flowchart of an image processing method provided by some embodiments of the present application;

[0022] Figure 2 is a schematic diagram of a jaggy point provided by some embodiments of the present application;

[0023] Figure 3 is a schematic diagram of a sliding window provided by some embodiments of the present application;

[0024] Figure 4 is a schematic diagram of a first pixel point and a preset range adjacent to the first pixel point provided by some embodiments of the present application;

[0025] Figure 5 is a schematic diagram of a first image frame after jaggy point smoothing provided by some embodiments of the present application;

[0026] Figure 6 is a flowchart of an image processing method provided by some embodiments of the present application;

[0027] Figure 7 is a structural schematic diagram of an image processing apparatus shown by some embodiments of the present application;

[0028] Figure 8 is a structural schematic diagram of an electronic device shown by some embodiments of the present application;

[0029] Figure 9 is a hardware structural schematic diagram of an electronic device shown by some embodiments of the present application DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0031] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", and the like are generally of a kind and are not limited in number, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0032] Before introducing the technical solutions of the embodiments of the present application, the background art of the embodiments of the present application is first introduced:

[0033] When displaying images using electronic devices, there will be many jagged edges in the image. Usually, all jagged points are smoothed to the same degree. This not only causes blurring in areas with small jagged edges and insufficient smoothing in areas with large jagged edges, but also wastes performance.

[0034] To address the aforementioned issues, embodiments of this application provide an image processing method, apparatus, electronic device, and storage medium. The method determines at least one jagged point in the first image frame that requires smoothing based on the grayscale values ​​of pixels in the spatial domain kernel and the pixel domain kernel. Then, it determines a target sampling step size based on the difference between the pixel value of the jagged point and the pixel values ​​of its neighboring pixels. Finally, it smooths the jagged point based on the target sampling step size and the coordinate information of the jagged point, obtaining a smoothed jagged point. Because each jagged point has a different target sampling step size, and the smoothing is performed according to these different target sampling step sizes, rather than applying the same sampling step size to all jagged points, the accuracy of the smoothing process for each jagged point is improved, thus enhancing the image quality of the first image frame.

[0035] The technical solutions of this application embodiment can be applied to scenarios where images have jagged edges and the jagged edges need to be processed, such as the game screen processing scenario of mobile games.

[0036] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios. The method of this application embodiment can be applied to electronic devices, which may be, but are not limited to, personal computers (PCs), smartphones, tablet computers, or personal digital assistants (PDAs).

[0037] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this application, as shown below. Figure 1 As shown, the image processing method provided in this application embodiment may include steps 110-150.

[0038] Step 110: Obtain the first image frame.

[0039] The first image frame can be an image frame to be processed, and this image frame may include at least one jagged point. For example, the first image frame may be a game image with jagged points in a certain frame of a mobile game.

[0040] In some embodiments of this application, the first image frame may be captured by the user through an image acquisition device such as a camera, or it may be obtained directly from the local database of the electronic device. No limitation is made in the embodiments of this application.

[0041] Step 120: Determine the spatial domain kernel and pixel domain kernel of the first image frame based on the pixels in the first image frame.

[0042] In some embodiments of this application, the first image frame can be transformed into spatial domain and pixel domain based on the pixels in the first image frame to obtain the spatial domain kernel and pixel domain kernel of the first image frame respectively.

[0043] It should be noted that, based on the pixels in an image, the image is transformed into both the spatial domain and the pixel domain to obtain the spatial domain kernel and the pixel domain kernel of the image, respectively. The specific transformation of the spatial domain and the pixel domain is existing technology and will not be described in detail in the embodiments of this application.

[0044] Step 130: Based on the gray values ​​of the pixels in the spatial domain kernel and the gray values ​​of the pixels in the pixel domain kernel, determine at least one jagged point in the first image frame that needs to be smoothed.

[0045] In some embodiments of this application, at least one jagged point in the first image frame that needs to be smoothed can be determined based on the gray values ​​of all pixels in the spatial domain kernel and the gray values ​​of all pixels in the pixel domain kernel.

[0046] In one example, reference Figure 2 , Figure 2 If the pixels in region 21 are blurred, then the pixels in region 21 are the pixels that need to be smoothed, i.e., jagged pixels.

[0047] In some embodiments of this application, in order to accurately determine at least one jagged point in the first image frame that needs to be smoothed, step 120 may specifically include:

[0048] Based on the pixels within a preset range adjacent to the first pixel in the first image frame, determine the spatial domain kernel and pixel domain kernel of the preset range;

[0049] Correspondingly, step 130 may specifically include:

[0050] Get the maximum and minimum gray values ​​among all the gray values ​​of all pixels in the spatial domain kernel within the preset range, and get the maximum and minimum gray values ​​among all the gray values ​​of all pixels in the pixel domain kernel within the preset range.

[0051] The larger of the maximum gray value in the spatial kernel and the maximum gray value in the pixel domain kernel is determined as the first gray value;

[0052] The smaller of the minimum gray value in the spatial kernel and the minimum gray value in the pixel domain kernel is determined as the second gray value;

[0053] If the difference between the first gray value and the second gray value meets the preset conditions, the jagged points in the first image frame that need to be smoothed are determined based on the first pixel.

[0054] The first pixel can be any pixel in the first image frame, or more specifically, a pixel in the first image frame that is observable to the user and may be jagged.

[0055] The adjacent preset range of the first pixel can be a certain range adjacent to the first pixel. This preset range can be set by the user according to their needs, and is not limited in this embodiment.

[0056] In some embodiments of this application, a sliding window can be constructed within a preset range adjacent to the first pixel. The sliding window is generally centered on the first pixel, and its size is generally less than or equal to the preset range adjacent to the first pixel. Thus, when determining the jagged points, the calculation can be performed only within the sliding window, without needing to determine them over the entire first image frame, which simplifies the calculation and improves the efficiency of jagged point determination.

[0057] In one example, reference Figure 3 , Figure 3 Pixel 31 is the first pixel. Within a preset range 32 of pixel 31, there can be a sliding window 33, and the center point of the sliding window 33 is pixel 34.

[0058] The first gray value can be the larger of the maximum gray value in the spatial kernel and the maximum gray value in the pixel domain kernel.

[0059] The second gray value can be the smaller of the minimum gray value in the spatial kernel and the minimum gray value in the pixel domain kernel.

[0060] The preset conditions can be pre-set conditions that the first grayscale value and the second grayscale value must meet when determining the sawtooth point.

[0061] In some embodiments of this application, the pixel value difference between a jagged pixel and its surrounding pixels is generally large. Therefore, based on the pixel value difference, the region with the large pixel value difference can be selected as the region where jagged pixels may occur. A pixel is selected in this region as the first pixel. Then, based on the pixels within a preset range adjacent to the first pixel, a spatial kernel and a pixel kernel within the preset range are determined. The larger of the maximum gray value in the spatial kernel and the maximum gray value in the pixel kernel is determined as the first gray value. The smaller of the minimum gray value in the spatial kernel and the minimum gray value in the pixel kernel is determined as the second gray value. When the difference between the first gray value and the second gray value satisfies a preset condition, the jagged pixels in the first image frame that need to be smoothed can be determined based on the first pixel.

[0062] In the embodiments of this application, by first determining the spatial domain kernel and pixel domain kernel within a preset range adjacent to the first pixel in the first image frame, the region where jagged points may occur is identified and processed in advance. Only the pixels within this region need to be checked for jagged points, eliminating the need to process all pixels in the entire first image frame, thus saving computational power. After determining the spatial domain kernel and pixel domain kernel within the preset range, the larger of the maximum gray value in the spatial kernel and the maximum gray value in the pixel domain kernel can be determined as the first gray value, and the smaller of the minimum gray value in the spatial kernel and the minimum gray value in the pixel domain kernel can be determined as the second gray value. If the difference between the first gray value and the second gray value satisfies a preset condition, the jagged points in the first image frame that need to be smoothed can be accurately determined based on the first pixel.

[0063] In some embodiments of this application, in order to accurately determine jagged points, the step of determining jagged points in the first image frame that need to be smoothed based on the first pixel, after determining that the difference between the first gray value and the second gray value meets a preset condition, may specifically include:

[0064] Obtain the preset brightness coefficient and the preset grayscale threshold of the first image frame;

[0065] The first value is obtained based on the preset brightness coefficient and the first gray value;

[0066] Extract the maximum value between the first value and the preset grayscale threshold, and use it as the second value;

[0067] If the difference between the first gray value and the second gray value is less than the second value, the first pixel is determined to be a jagged point in the first image frame that needs to be smoothed.

[0068] The preset brightness coefficient can be a pre-set maximum brightness coefficient that can clearly show the jagged points of the first image frame. This preset brightness coefficient can be determined based on the maximum brightness coefficient of the first image frame. Specifically, it can be smaller than the maximum brightness coefficient of the first image frame, just enough to ensure that the jagged points of the first image frame can be clearly seen.

[0069] The preset grayscale threshold can be a pre-set minimum grayscale value that allows the jagged points of the first image frame to be clearly seen. This preset grayscale threshold is determined based on the minimum grayscale value of the first image frame.

[0070] In some embodiments of this application, if an image frame is too bright, it will appear very white, making the jagged edges invisible. Therefore, when setting the preset brightness coefficient, it should be set smaller than the maximum brightness coefficient of the first image frame to ensure that the jagged edges of the first image frame are visible. Similarly, if the grayscale value of an image frame is too small, such as 0, it will be completely black, making the jagged edges of the first image frame invisible. Therefore, when setting the preset grayscale threshold, it should be set larger than the minimum grayscale value of the first image frame to ensure that the jagged edges of the first image frame are visible.

[0071] The first value can be determined based on a preset brightness coefficient and a first grayscale value, specifically, it can be the product of the preset brightness coefficient and the first grayscale value.

[0072] The second value can be the maximum value between the first value and the preset grayscale threshold.

[0073] In some embodiments of this application, when determining jagged points, it is first necessary to ensure the brightness and grayscale value of the first image frame are adequate to ensure that the jagged points in the first image frame are clearly visible. Based on this, a first value can be obtained according to a preset brightness coefficient and a first grayscale value. Then, the maximum value between the first value and a preset grayscale threshold is extracted as a second value. If the difference between the first grayscale value and the second grayscale value is less than the second value, the first pixel can be determined as a jagged point in the first image frame that requires smoothing.

[0074] In the embodiments of this application, a preset brightness coefficient and a preset grayscale threshold of the first image frame are obtained. Then, based on the preset brightness coefficient and the first grayscale value, a first value is obtained. The maximum value between the first value and the preset grayscale threshold is extracted as a second value. If the difference between the first grayscale value and the second grayscale value is less than the second value, the first pixel is determined to be a jagged point in the first image frame that needs to be smoothed. Thus, when determining the jagged point, not only the first grayscale value and the second grayscale value are considered, but also the brightness of the first image frame is considered to determine that the jagged point can be clearly seen in the first image frame. In this way, the jagged point can be accurately determined.

[0075] Step 140: Determine the target step size based on the difference between the pixel value of the jagged point and the pixel value of the neighboring pixels.

[0076] Among them, neighboring pixels are pixels within a preset range adjacent to the jagged points.

[0077] The target sampling step size can be defined as the number of sampling points to be sampled when smoothing jagged points. This target sampling step size includes a preset number of sampling points, which can be a pre-defined number of sampling points corresponding to the target sampling step size. Different target sampling step sizes can have different numbers of sampling points.

[0078] In some embodiments of this application, the target sampling step size for smoothing the jagged point can be determined based on the difference between the pixel value of the jagged point and the pixel values ​​of its neighboring pixels. Specifically, if the difference between the pixel value of the jagged point and the pixel values ​​of its neighboring pixels is large, more neighboring pixels should be selected to smooth the jagged point, i.e., the sampling step size should be reduced to increase the sampling density. If the difference between the pixel value of the jagged point and the pixel values ​​of its neighboring pixels is small, fewer neighboring pixels should be selected to smooth the jagged point, i.e., the sampling step size should be increased to reduce the sampling density.

[0079] In some embodiments of this application, the preset quantity may include a first preset quantity and a second preset quantity. Step 140 may specifically include:

[0080] If the difference between the pixel value of the jagged point and the pixel value of the neighboring pixel is greater than a preset difference threshold, the target sampling step size is determined as the first sampling step size, wherein the first sampling step size includes a first preset number of sampling points.

[0081] If the difference between the pixel value of the jagged point and the pixel value of the neighboring pixel is less than or equal to a preset difference threshold, the target sampling step size is determined as the second sampling step size, wherein the second sampling step size includes a second preset number of sampling points, and the second preset number is less than the first preset number.

[0082] The first sampling step size can be the sampling step size for smoothing the jagged points when the difference between the pixel value of the jagged point and the pixel value of the neighboring pixels is greater than a preset difference threshold.

[0083] The second sampling step size can be determined when the difference between the pixel value of the jagged point and the pixel value of the neighboring pixel is less than or equal to a preset difference threshold, and the sampling step size for smoothing the jagged point is determined.

[0084] The preset difference threshold can be a pre-set threshold for the difference between the pixel value of the jagged point and the pixel value of the neighboring pixels.

[0085] In some embodiments of this application, when the difference between the pixel value of the jagged point and the pixel value of its neighboring pixels is greater than a preset difference threshold, the sampling step size for smoothing the jagged point is determined as a first sampling step size, which includes a first preset number of sampling points. When the difference between the pixel value of the jagged point and the pixel value of its neighboring pixels is less than or equal to the preset difference threshold, the sampling step size for smoothing the jagged point is determined as a second sampling step size, which includes a second preset number of sampling points. The second preset number is less than the first preset number. That is, if the difference between the pixel value of the jagged point and the pixel value of its neighboring pixels is large, more neighboring pixels are selected to smooth the jagged point, i.e., the sampling step size is reduced to increase the sampling density. If the difference between the pixel value of the jagged point and the pixel value of its neighboring pixels is small, fewer neighboring pixels are selected to smooth the jagged point, i.e., the sampling step size is increased to reduce the sampling density.

[0086] In the embodiments of this application, based on the relationship between the difference between the pixel value of the jagged point and the pixel value of the neighboring pixel and a preset difference threshold, different target sampling step sizes are selected for the sampling points to smooth the jagged points. In this way, different degrees of smoothing processing are performed on each jagged point, which improves the accuracy of smoothing the jagged points of the first image frame and thus improves the quality of the first image frame.

[0087] Step 150: Smooth the jagged points based on the target sampling step size and the coordinate information of the jagged points to obtain smoothed jagged points. In some embodiments of this application, the jagged points can be smoothed according to the target sampling step size and the coordinate information of the jagged points to obtain smoothed jagged points. Specifically, in order to accurately improve the image quality of the first image frame, step 150 may specifically include:

[0088] Based on the coordinate information of the sawtooth points and the coordinate information of a preset number of sampling points, calculate the offset between each sampling point and the sawtooth points, as well as the weight corresponding to each sampling point;

[0089] Based on the offset and weight, the jagged points are smoothed to obtain the target grayscale value of the jagged points.

[0090] The target grayscale value can be the grayscale value of the jagged points after smoothing the jagged points according to the offset and weight.

[0091] The offset can be the deviation of each sampling point from the sawtooth point, that is, the difference between each sampling point and the sawtooth point.

[0092] The weight can be the proportion of each sampling point used to smooth the sawtooth points when using each sampling point to smooth the sawtooth points.

[0093] In some embodiments of this application, the jagged points can be smoothed according to the following formula (1) based on the offset and weight to obtain the target gray value of the jagged points:

[0094] It should be noted that formula (1) is explained using a preset number of 4 as an example, that is, for a certain sawtooth point, the number of sampling points in the target sampling step size is 4.

[0095]

[0096] Where p(i,j) is the sawtooth point, pixel p(i,j) The target grayscale value of the jagged points is given by weight[0], weight[1], weight[2] and weight[3], which are the weights of the four sampling points, respectively. P i+offset[0],j+offset[0] P i+offset[1],j+offset[1] P i+offset[2],j+offset[2] and P i+offset[3],j+offset[3] These are the offsets of the four sampling points and the sawtooth points, respectively.

[0097] In the embodiments of this application, the offset between each sampling point and the sawtooth point is calculated based on the coordinate information of the sawtooth point and the coordinate information of a preset number of sampling points, as well as the weight corresponding to each sampling point. Based on the offset and weight, the sawtooth point is smoothed to obtain the target gray value of the sawtooth point. Thus, for each sawtooth point, the sawtooth point is smoothed based on the coordinate information of the sawtooth point and the target sampling step size corresponding to the sawtooth point. That is, different degrees of smoothing are performed on different sawtooth points, which improves the accuracy of smoothing the sawtooth points of the first image frame, thereby improving the quality of the first image frame.

[0098] In some embodiments of this application, in order to further improve the image quality of the first image frame, before calculating the offset between each sampling point and the jagged point, and the weight corresponding to each sampling point, based on the coordinate information of the jagged points and the coordinate information of a preset number of sampling points, the method described above may further include:

[0099] The correlation between the sawtooth points and the preset number of sampling points is obtained by fitting the sawtooth points and the preset number of sampling points.

[0100] The step of calculating the offset between each sampling point and the sawtooth point, as well as the weight corresponding to each sampling point, based on the coordinate information of the sawtooth point and the coordinate information of a preset number of sampling points, may specifically include:

[0101] Based on the coordinate information of the sawtooth points, the coordinate information of a preset number of sampling points, and their correlation, the offset between each sampling point and the sawtooth points, as well as the weight corresponding to each sampling point, are calculated.

[0102] In some embodiments of this application, before determining the offset between each sampling point and the sawtooth point, and the weight corresponding to each sampling point, it is necessary to first obtain the correlation between each sampling point and the sawtooth point. Only then can the offset between each sampling point and the sawtooth point, and the weight corresponding to each sampling point, be obtained based on the correlation. Specifically, each sampling point can be fitted to the sawtooth point, that is, each sampling point can be connected to the sawtooth point with a smooth curve, thus obtaining the correlation between the sawtooth point and a preset number of sampling points.

[0103] In some embodiments of this application, each sampling point can be fitted with the sawtooth point using the following formula (2) to obtain the correlation between the sawtooth point and a preset number of sampling points:

[0104] p n =p0*(-0.5*t) 3 +t 2 -0.5*t)+p1*(1.5*t 3 -2.5*t 2 +1.0)+

[0105] p2*(-1.5*t 3 +2.0*t 2 +0.5*t)+p3*(0.5*t 3 -0.5*t 2 (2)

[0106] In the above formula (2), t∈[0,1] is the hyperparameter of formula (2), which is used to control the number of line segments. The other values ​​in formula (2) are related to the type of fitting curve used.

[0107] The t mentioned above, used to control the number of line segments, can be understood as follows: there are line segments connecting each sawtooth point to each sampling point, and the number of line segments is equal to the number of sampling points.

[0108] It should be noted that, taking the preset quantity of 4 as an example, the sampling points in the above formula (2) are p0, p1, p2 and p3 respectively. n These are sawtooth points.

[0109] In some embodiments of this application, after obtaining the correlation between the sawtooth point and a preset number of sampling points, the offset between each sampling point and the sawtooth point, as well as the weight corresponding to each sampling point, can be calculated based on the coordinate information of the sawtooth point, the coordinate information of the preset number of sampling points, and the correlation.

[0110] In the embodiments of this application, by fitting the sawtooth points and a preset number of sampling points, the correlation between the sawtooth points and the preset number of sampling points is obtained. In this way, based on the coordinate information of the sawtooth points, the coordinate information of the preset number of sampling points, and the correlation, the offset between each sampling point and the sawtooth points, as well as the weight corresponding to each sampling point, can be accurately obtained.

[0111] In some embodiments of this application, to accurately obtain the offset between each sampling point and the sawtooth point, and the weight corresponding to each sampling point, the step of calculating the offset between each sampling point and the sawtooth point, and the weight corresponding to each sampling point, based on the coordinate information of the sawtooth point, the coordinate information of a preset number of sampling points, and their correlation relationships, may specifically include:

[0112] The coordinate information of the sawtooth points and the coordinate information of a preset number of sampling points are both normalized to obtain the normalized coordinate information;

[0113] Based on the normalized coordinate information and correlation, the offset between each sampling point and the sawtooth point is obtained;

[0114] The weight corresponding to each sampling point is determined based on the offset between each sampling point and the sawtooth point.

[0115] In some embodiments of this application, the coordinate information of the sawtooth point and the coordinate information of a preset number of sampling points can be normalized to obtain normalized coordinate information. For example, taking the sawtooth point as an example, the sawtooth point is normalized to obtain the normalized coordinate information as: f = vec2(i,j,1.0-(i,j)). Then, the coordinate information of the normalized sawtooth point and the coordinate information of the preset number of sampling points are substituted into formula (2) to obtain the offset between each sampling point and the sawtooth point. Then, the weight of each sampling point for smoothing the sawtooth point can be determined based on the offset.

[0116] In one example, reference Figure 4 , Figure 4 This is a schematic diagram of a sliding window corresponding to a certain sawtooth point. In this sliding window, sawtooth point 41 is located at the center of the sliding window. Taking a preset number of 4 as an example, the preset number of sampling points is... Figure 4Given sampling points 42, 43, 44, and 45, and the sawtooth point 41, and formula (1), we can obtain the offsets of sampling points 42, 43, 44, and 45 from the sawtooth point 41, as well as the weights of sampling points 42, 43, 44, and 45.

[0117] It should be noted that, Figure 4 Only four sampling points related to the smoothing of jagged points were selected in this embodiment. It is understood that in the actual processing, it is not limited to four points. The specific number can be selected according to the user's needs and is not limited in this embodiment.

[0118] In the embodiments of this application, the coordinate information of the sawtooth points and the coordinate information of a preset number of sampling points are normalized to obtain normalized coordinate information. Then, based on the normalized coordinate information and the correlation, the offset between each sampling point and the sawtooth points can be accurately obtained. Furthermore, based on the offset between each sampling point and the sawtooth points, the weight corresponding to each sampling point is accurately determined, thereby accurately improving the image quality of the first image frame.

[0119] In some embodiments of this application, when the first image frame is a mobile game, i.e., an image frame in a mobile game, mobile games are often limited in terms of image quality processing due to the limited computing resources and graphics processing capabilities of mobile devices.

[0120] To solve the above problem, given that there are N sawtooth points, steps 140-150 may specifically include:

[0121] The following operations are performed on N sawtooth points in parallel using N processing threads:

[0122] The target sampling step size is determined based on the difference between the pixel value of the sawtooth point and the pixel value of the neighboring pixels. The sawtooth point is then smoothed based on the target sampling step size and the coordinate information of the sawtooth point to obtain the smoothed sawtooth point.

[0123] There are N processing threads that correspond one-to-one with N sawtooth points, where N is a positive integer greater than 1.

[0124] In some embodiments of this application, when there are multiple aliasing points, the Compute Shader program in the graphics processing unit (GPU) of the electronic device can create multiple processing threads, each thread corresponding to one aliasing point, and the above-mentioned smoothing process can be performed on the N aliasing points in parallel based on N processing threads.

[0125] In the embodiments of this application, when there are multiple sawtooth points, the smoothing process of the N sawtooth points is performed in parallel by N processing threads. In this way, each traversal operation is a separate calculation process. By utilizing the high concurrency of the GPU, the smoothing process of all sawtooth points can be completed at once, thereby improving the efficiency of sawtooth point smoothing.

[0126] In some embodiments of this application, after all jagged points have been processed, the processing result can be returned to the central processing unit of the electronic device, and then the processed first image frame can be displayed on the display of the electronic device.

[0127] In one example, continue to refer to Figure 2 ,right Figure 2 After processing the jagged points in region 21, the following result is obtained: Figure 5 The results shown are from Figure 5 It can be seen that, Figure 2 The jagged points in region 21 are smoothed out.

[0128] In some embodiments of this application, to better understand the technical solutions of the embodiments of this application, another possible implementation of the image processing method is also provided, such as... Figure 6 As shown, the image processing method may include steps 610-650.

[0129] Step 610: Obtain the first image frame.

[0130] Step 610 is the same as step 110 in the above embodiment, and will not be described again here.

[0131] Step 620: Determine whether the image quality of the first image frame needs to be improved. If yes, proceed to step 630; otherwise, proceed to step 650.

[0132] In step 620, it is determined whether the image quality of the first image frame needs to be improved, that is, whether the first image frame contains jagged edges. If jagged edges are present, the first image frame needs to be improved; if there are no jagged edges, no improvement is required. The determination of jagged edges is the same as that in the above embodiments, and will not be repeated here.

[0133] Step 630: Based on the difference between the pixel value of the jagged point and the pixel value of the neighboring pixels, the jagged point is smoothed using an adaptive sampling step size anti-aliasing method to obtain the smoothed jagged point.

[0134] The anti-aliasing method with adaptive sampling step size in step 630 is the same as that in the above embodiment, which determines the target sampling step size based on the difference between the pixel value of each jagged point and the pixel value of the neighboring pixels, and smooths the jagged points based on the target sampling step size and the coordinate information of the jagged points. That is, step 630 is consistent with the implementation process of steps 140-150 in the above embodiment, and will not be described again here.

[0135] Step 640: Mobile GPU acceleration.

[0136] Step 640 is the process of using N processing threads to process N sawtooth points in parallel in the above embodiment, which will not be described in detail here.

[0137] Step 650: Output the first image frame after smoothing.

[0138] In step 650, after smoothing the jagged points in the first image frame, the smoothed first image frame can be output and displayed.

[0139] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0140] Figure 7 This is a schematic diagram of the structure of an image processing apparatus according to an exemplary embodiment.

[0141] like Figure 7 As shown, the image processing apparatus 700 may include:

[0142] Acquisition module 710 is used to acquire the first image frame;

[0143] The determining module 720 is used to determine the spatial domain kernel and pixel domain kernel of the first image frame based on the pixels in the first image frame;

[0144] The determining module 720 is further configured to determine at least one jagged point in the first image frame that needs to be smoothed based on the gray values ​​of the pixels in the spatial domain kernel and the gray values ​​of the pixels in the pixel domain kernel.

[0145] The determining module 720 is further configured to determine the target sampling step size based on the difference between the pixel value of the sawtooth point and the pixel value of the neighboring pixel point;

[0146] The determining module 720 is further configured to perform smoothing processing on the sawtooth points based on the target sampling step size and the coordinate information of the sawtooth points to obtain smoothed sawtooth points, wherein the neighboring pixels are the pixels in the pixel neighborhood of the sawtooth points.

[0147] In the embodiments of this application, at least one jagged point in the first image frame that needs to be smoothed is determined based on the gray values ​​of the pixels in the spatial domain kernel and the pixel domain kernel of the first image frame. Then, the target sampling step size is determined based on the difference between the pixel value of the jagged point and the pixel value of the neighboring pixels. The jagged point is then smoothed based on the target sampling step size and the coordinate information of the jagged point to obtain the smoothed jagged point. Since each jagged point has a different target sampling step size when smoothing, and the jagged point is smoothed according to the different target sampling step size, instead of smoothing all jagged points uniformly according to the same sampling step size, the accuracy of smoothing each jagged point is improved, and the image quality of the first image frame is improved.

[0148] In some embodiments of this application, the determining module 720 is specifically used to: determine the spatial domain kernel and pixel domain kernel of the preset range based on the pixels adjacent to the first pixel in the first image frame, wherein the first pixel is any pixel in the first image frame;

[0149] The acquisition module 710 is further configured to acquire the maximum and minimum gray values ​​among the gray values ​​of all pixels in the spatial domain kernel of the preset range, and the maximum and minimum gray values ​​among the gray values ​​of all pixels in the pixel domain kernel of the preset range.

[0150] The determining module 720 is specifically used to determine the larger of the maximum gray value in the spatial kernel and the maximum gray value in the pixel domain kernel as a first gray value; to determine the smaller of the minimum gray value in the spatial kernel and the minimum gray value in the pixel domain kernel as a second gray value; and, if the difference between the first gray value and the second gray value satisfies a preset condition, to determine the jagged points in the first image frame that need to be smoothed based on the first pixel.

[0151] In some embodiments of this application, the acquisition module 710 is further configured to acquire a preset brightness coefficient of the first image frame and a preset grayscale threshold of the first image frame, wherein the preset brightness coefficient is determined based on the maximum brightness coefficient of the first image frame, and the preset grayscale threshold is determined based on the minimum grayscale value of the first image frame.

[0152] The determining module 720 is specifically used to obtain a first value based on the preset brightness coefficient and the first gray value; extract the maximum value between the first value and the preset gray value threshold as a second value; and determine the first pixel as a jagged point in the first image frame that needs to be smoothed when the difference between the first gray value and the second gray value is less than the second value.

[0153] In some embodiments of this application, the target sampling step size includes a preset number of sampling points, and the determining module 720 is specifically used for:

[0154] Based on the coordinate information of the sawtooth point and the coordinate information of the preset number of sampling points, the offset between each sampling point and the sawtooth point, as well as the weight corresponding to each sampling point, are calculated; based on the offset and the weight, the sawtooth point is smoothed to obtain the target gray value of the sawtooth point.

[0155] In some embodiments of this application, the preset quantity includes a first preset quantity and a second preset quantity, and the determining module 720 is specifically used for:

[0156] If the difference between the pixel value of the sawtooth point and the pixel value of the neighboring pixel point is greater than a preset difference threshold, the target sampling step size is determined to be the first sampling step size, wherein the first sampling step size includes a first preset number of sampling points.

[0157] If the difference between the pixel value of the sawtooth point and the pixel value of the neighboring pixel is less than or equal to the preset difference threshold, the target sampling step size is determined to be the second sampling step size, wherein the second sampling step size includes a second preset number of sampling points, and the second preset number is less than the first preset number.

[0158] In some embodiments of this application, the apparatus described above may further include:

[0159] The fitting module is used to fit the sawtooth points and the preset number of sampling points to obtain the correlation between the sawtooth points and the preset number of sampling points;

[0160] The module 720 is specifically used for:

[0161] Based on the coordinate information of the sawtooth point, the coordinate information of the preset number of sampling points, and the correlation relationship, the offset between each sampling point and the sawtooth point, as well as the weight corresponding to each sampling point, are calculated respectively.

[0162] In some embodiments of this application, the determining module 720 is specifically used for:

[0163] The coordinate information of the sawtooth points and the coordinate information of the preset number of sampling points are both normalized to obtain normalized coordinate information;

[0164] Based on the normalized coordinate information and the correlation, the offset between each sampling point and the sawtooth point is obtained;

[0165] The weight corresponding to each sampling point is determined based on the offset between each sampling point and the sawtooth point.

[0166] In some embodiments of this application, when the number of sawtooth points is determined to be N, the determining module 720 is specifically used for:

[0167] Based on the N processing threads performing the following operations in parallel on the N sawtooth points:

[0168] The target sampling step size is determined based on the difference between the pixel value of the sawtooth point and the pixel value of the neighboring pixels. The sawtooth point is then smoothed based on the target sampling step size and the coordinate information of the sawtooth point to obtain the smoothed sawtooth point. The N processing threads correspond one-to-one with the N sawtooth points, and N is a positive integer greater than 1.

[0169] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0170] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0171] The image processing apparatus provided in this application embodiment can achieve... Figure 1 and Figure 6 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0172] Optionally, such as Figure 8As shown, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores a program or instructions that can run on the processor 801. When the program or instructions are executed by the processor 801, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0173] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0174] Figure 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0175] The electronic device 900 includes, but is not limited to, components such as: radio frequency unit 901, network module 902, audio output unit 903, input unit 904, sensor 905, display unit 906, user input unit 907, interface unit 908, memory 909, and processor 910.

[0176] Those skilled in the art will understand that the electronic device 900 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 910 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0177] The radio frequency unit 901 is used to acquire the first image frame;

[0178] The processor 910 is configured to: determine a spatial domain kernel and a pixel domain kernel of the first image frame based on the pixels in the first image frame; determine at least one jagged point in the first image frame that needs to be smoothed based on the gray values ​​of the pixels in the spatial domain kernel and the pixel domain kernel; determine a target sampling step size based on the difference between the pixel value of the jagged point and the pixel value of its neighboring pixels; and perform smoothing processing on the jagged point based on the target sampling step size and the coordinate information of the jagged point to obtain a smoothed jagged point, wherein the neighboring pixels are the pixels within the pixel neighborhood of the jagged point.

[0179] Thus, by determining at least one jagged point in the first image frame that needs smoothing based on the gray values ​​of pixels in the spatial domain kernel and the pixel domain kernel of the first image frame, and then determining the target sampling step size based on the difference between the pixel value of the jagged point and the pixel value of its neighboring pixels, the jagged point is smoothed based on the target sampling step size and the coordinate information of the jagged point, resulting in a smoothed jagged point. Since each jagged point has a different target sampling step size when smoothing, and the jagged point is smoothed according to the different target sampling step size, rather than smoothing all jagged points uniformly with the same sampling step size, the accuracy of smoothing each jagged point is improved, thus improving the image quality of the first image frame.

[0180] Optionally, the processor 910 is further configured to determine the spatial domain kernel and pixel domain kernel of the preset range based on the pixels within the preset range adjacent to the first pixel in the first image frame, wherein the first pixel is any pixel in the first image frame.

[0181] The radio frequency unit 901 is also used to acquire the maximum and minimum gray values ​​among the gray values ​​of all pixels in the spatial domain kernel within the preset range, and the maximum and minimum gray values ​​among the gray values ​​of all pixels in the pixel domain kernel within the preset range.

[0182] The processor 910 is further configured to determine the larger of the maximum gray value in the spatial kernel and the maximum gray value in the pixel domain kernel as a first gray value; determine the smaller of the minimum gray value in the spatial kernel and the minimum gray value in the pixel domain kernel as a second gray value; and, if the difference between the first gray value and the second gray value satisfies a preset condition, determine the jagged points in the first image frame that need to be smoothed based on the first pixel.

[0183] Thus, by first determining the spatial domain kernel and pixel domain kernel within a preset range adjacent to the first pixel in the first image frame, the region where jagged points might appear is identified and processed first. Only the pixels within this region need to be checked for jagged points, eliminating the need to process all pixels in the entire first image frame, saving computational resources. After determining the preset range of spatial and pixel domain kernels, the larger of the maximum grayscale value in the spatial kernel and the maximum grayscale value in the pixel domain kernel is determined as the first grayscale value. The smaller of the minimum grayscale value in the spatial kernel and the minimum grayscale value in the pixel domain kernel is determined as the second grayscale value. If the difference between the first and second grayscale values ​​meets a preset condition, the jagged points in the first image frame that require smoothing can be accurately determined based on the first pixel.

[0184] Optionally, the radio frequency unit 901 is further configured to acquire a preset brightness coefficient of the first image frame and a preset grayscale threshold of the first image frame, wherein the preset brightness coefficient is determined based on the maximum brightness coefficient of the first image frame, and the preset grayscale threshold is determined based on the minimum grayscale value of the first image frame.

[0185] The processor 910 is further configured to obtain a first value based on the preset brightness coefficient and the first gray value; extract the maximum value between the first value and the preset gray value threshold as a second value; and determine the first pixel as a jagged point in the first image frame that needs to be smoothed if the difference between the first gray value and the second gray value is less than the second value.

[0186] Thus, by obtaining the preset brightness coefficient and the preset grayscale threshold of the first image frame, and then obtaining the first value based on the preset brightness coefficient and the first grayscale value, the maximum value between the first value and the preset grayscale threshold is extracted as the second value. If the difference between the first grayscale value and the second grayscale value is less than the second value, the first pixel is determined to be a jagged point in the first image frame that needs to be smoothed. In this way, when determining the jagged point, not only the first grayscale value and the second grayscale value are considered, but also the brightness of the first image frame is considered to determine that the jagged point can be clearly seen in the first image frame. Thus, the jagged point can be accurately determined.

[0187] Optionally, the target sampling step includes a preset number of sampling points; the preset number includes a first preset number and a second preset number. The processor 910 is further configured to determine the target sampling step as a first sampling step when the difference between the pixel value of the sawtooth point and the pixel value of the neighboring pixel is greater than a preset difference threshold, wherein the first sampling step includes a first preset number of sampling points.

[0188] If the difference between the pixel value of the sawtooth point and the pixel value of the neighboring pixel is less than or equal to the preset difference threshold, the target sampling step size is determined to be the second sampling step size, wherein the second sampling step size includes a second preset number of sampling points, and the second preset number is less than the first preset number.

[0189] Thus, based on the relationship between the difference between the pixel value of the jagged point and the pixel value of the neighboring pixels and the preset difference threshold, different target sampling step sizes are selected for the sampling points to smooth the jagged points. In this way, different degrees of smoothing are performed on each jagged point, which improves the accuracy of smoothing the jagged points of the first image frame and thus improves the quality of the first image frame.

[0190] Optionally, the processor 910 is further configured to calculate the offset between each sampling point and the sawtooth point, and the weight corresponding to each sampling point, based on the coordinate information of the sawtooth point and the coordinate information of the preset number of sampling points; and to perform smoothing processing on the sawtooth point based on the offset and the weight to obtain the target gray value of the sawtooth point.

[0191] Thus, based on the coordinate information of the jagged points and the coordinate information of a preset number of sampling points, the offset between each sampling point and the jagged points, as well as the weight corresponding to each sampling point, are calculated. Based on the offset and weight, the jagged points are smoothed to obtain the target grayscale value of the jagged points. In this way, for each jagged point, the jagged point is smoothed according to its coordinate information and the target sampling step size corresponding to the jagged point. That is, different degrees of smoothing are performed for different jagged points, which improves the accuracy of smoothing the jagged points of the first image frame, thereby improving the quality of the first image frame.

[0192] Optionally, the processor 910 is further configured to fit the sawtooth point and the preset number of sampling points to obtain the correlation between the sawtooth point and the preset number of sampling points; and to calculate the offset between each sampling point and the sawtooth point, and the weight corresponding to each sampling point, based on the coordinate information of the sawtooth point, the coordinate information of the preset number of sampling points and the correlation.

[0193] Thus, by fitting the sawtooth points and a preset number of sampling points, the correlation between the sawtooth points and the preset number of sampling points is obtained. Based on the coordinate information of the sawtooth points, the coordinate information of the preset number of sampling points, and the correlation, the offset between each sampling point and the sawtooth points, as well as the weight corresponding to each sampling point, can be accurately obtained.

[0194] Optionally, the processor 910 is further configured to normalize the coordinate information of the sawtooth point and the coordinate information of the preset number of sampling points to obtain normalized coordinate information; obtain the offset between each sampling point and the sawtooth point based on the normalized coordinate information and the correlation relationship; and determine the weight corresponding to each sampling point based on the offset between each sampling point and the sawtooth point.

[0195] In this way, the coordinate information of the jagged points and the coordinate information of the preset number of sampling points are normalized to obtain normalized coordinate information. Then, based on the normalized coordinate information and the correlation, the offset between each sampling point and the jagged points can be accurately obtained. Then, based on the offset between each sampling point and the jagged points, the weight corresponding to each sampling point can be accurately determined, thereby accurately improving the image quality of the first image frame.

[0196] Optionally, if the number of sawtooth points is determined to be N, the processor 910 is further configured to perform the following operations on the N sawtooth points in parallel using N processing threads: determine a target sampling step size based on the difference between the pixel value of the sawtooth point and the pixel value of its neighboring pixels; and perform smoothing processing on the sawtooth points based on the target sampling step size and the coordinate information of the sawtooth points to obtain smoothed sawtooth points; wherein, the N processing threads correspond one-to-one with the N sawtooth points, and N is a positive integer greater than 1.

[0197] Thus, when there are multiple sawtooth points, the process of smoothing N sawtooth points in parallel based on N processing threads is such that each traversal operation is a separate calculation process. By utilizing the high concurrency of the GPU, the smoothing of all sawtooth points can be completed at once, thereby improving the efficiency of sawtooth point smoothing.

[0198] It should be understood that, in this embodiment, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042. The GPU 9041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0199] The memory 909 can be used to store software programs and various data. The memory 909 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 909 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0200] Processor 910 may include one or more processing units; optionally, processor 910 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 910.

[0201] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0202] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0203] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0204] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0205] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0206] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0207] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0208] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a first image frame; determining a spatial domain kernel and a pixel domain kernel of the first image frame according to pixel points in the first image frame; determining at least one jaggy point in the first image frame that needs to be smoothed according to a gray value of a pixel point in the spatial domain kernel and a gray value of a pixel point in the pixel domain kernel; determining a target sampling step according to a difference between a pixel value of the jaggy point and a pixel value of a neighborhood pixel point of the jaggy point; smoothing the jaggy point based on the target sampling step and coordinate information of the jaggy point to obtain a smoothed jaggy point, wherein the neighborhood pixel point is a pixel point within a preset range adjacent to the jaggy point; in a case where it is determined that the jaggy point is N, the determining of the target sampling step according to the difference between the pixel value of the jaggy point and the pixel value of the neighborhood pixel point of the jaggy point, and the smoothing of the jaggy point based on the target sampling step and the coordinate information of the jaggy point to obtain the smoothed jaggy point, comprises: based on N processing threads, performing the following operations on the N jaggy points in parallel: determining a target sampling step according to a difference between a pixel value of the jaggy point and a pixel value of a neighborhood pixel point of the jaggy point, and smoothing the jaggy point based on the target sampling step and coordinate information of the jaggy point to obtain a smoothed jaggy point; wherein the N processing threads correspond one-to-one to the N jaggy points, and N is a positive integer greater than 1.

2. The method of claim 1, wherein, The determining of the spatial domain kernel and the pixel domain kernel of the first image frame according to the pixel points in the first image frame comprises: determining a spatial domain kernel and a pixel domain kernel of the preset range according to pixel points within a preset range adjacent to a first pixel point in the first image frame, wherein the first pixel point is any one pixel point in the first image frame; The determining of the jaggy point in the first image frame that needs to be smoothed according to the gray value of the pixel point in the spatial domain kernel and the gray value of the pixel point in the pixel domain kernel comprises: acquiring a maximum gray value and a minimum gray value among the gray values of all pixel points in the spatial domain kernel of the preset range, and a maximum gray value and a minimum gray value among the gray values of all pixel points in the pixel domain kernel of the preset range; determining a larger one of the maximum gray value in the spatial domain kernel and the maximum gray value in the pixel domain kernel as a first gray value; determining a smaller one of the minimum gray value in the spatial domain kernel and the minimum gray value in the pixel domain kernel as a second gray value; in a case where it is determined that a difference between the first gray value and the second gray value satisfies a preset condition, determining the jaggy point in the first image frame that needs to be smoothed according to the first pixel point.

3. The method of claim 2, wherein, The determining of the jaggy point in the first image frame that needs to be smoothed according to the first pixel point in a case where it is determined that a difference between the first gray value and the second gray value satisfies a preset condition comprises: obtaining a preset brightness coefficient of the first image frame and a preset gray threshold of the first image frame, wherein the preset brightness coefficient is determined based on a maximum brightness coefficient of the first image frame, and the preset gray threshold is determined based on a minimum gray value of the first image frame; obtaining a first value based on the preset brightness coefficient and the first gray value; extracting a maximum value of the first value and the preset gray threshold as a second value; in a case where a difference between the first gray value and the second gray value is less than the second value, determining that the first pixel point is a jaggy point in the first image frame that needs to be smoothed.

4. The method of claim 1, wherein, The target sampling step includes a preset number of sampling points; the preset number includes a first preset number and a second preset number; and the target sampling step is determined according to a difference between a pixel value of the jaggy point and pixel values of neighboring pixel points, including: in a case where the difference between the pixel value of the jaggy point and the pixel values of the neighboring pixel points is greater than a preset difference threshold, determining that the target sampling step is a first sampling step, wherein the first sampling step includes a first preset number of sampling points; in a case where the difference between the pixel value of the jaggy point and the pixel values of the neighboring pixel points is less than or equal to the preset difference threshold, determining that the target sampling step is a second sampling step, wherein the second sampling step includes a second preset number of sampling points, and the second preset number is less than the first preset number.

5. The method of claim 4, wherein, The smoothing processing of the jaggy point based on the target sampling step and the coordinate information of the jaggy point includes: calculating, according to the coordinate information of the jaggy point and the coordinate information of the preset number of sampling points, an offset between each sampling point and the jaggy point, and a weight corresponding to each sampling point; performing smoothing processing on the jaggy point according to the offset and the weight to obtain a target gray value of the jaggy point.

6. The method of claim 5, wherein, Before the calculation of the offset between each sampling point and the jaggy point and the weight corresponding to each sampling point according to the coordinate information of the jaggy point and the coordinate information of the preset number of sampling points, the method further includes: fitting the jaggy point and the preset number of sampling points to obtain a correlation between the jaggy point and the preset number of sampling points; The calculation of the offset between each sampling point and the jaggy point and the weight corresponding to each sampling point according to the coordinate information of the jaggy point and the coordinate information of the preset number of sampling points includes: calculating, according to the coordinate information of the jaggy point, the coordinate information of the preset number of sampling points, and the correlation, the offset between each sampling point and the jaggy point, and the weight corresponding to each sampling point, respectively.

7. The method of claim 6, wherein, The calculation of the offset between each sampling point and the jaggy point and the weight corresponding to each sampling point according to the coordinate information of the jaggy point, the coordinate information of the preset number of sampling points, and the correlation includes: The coordinate information of the sawtooth point and the coordinate information of the preset number of sampling points are both normalized to obtain normalized coordinate information; According to the normalized coordinate information and the association relationship, an offset between each sampling point and the sawtooth point is obtained; According to the offset between each sampling point and the sawtooth point, a weight corresponding to each sampling point is determined.

8. An image processing apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire a first image frame; A determination module is configured to determine a spatial domain kernel and a pixel domain kernel of the first image frame according to a pixel point in the first image frame; The determination module is further configured to determine at least one sawtooth point in the first image frame that needs to be smoothed according to a gray value of the pixel point in the spatial domain kernel and a gray value of the pixel point in the pixel domain kernel; The determination module is further configured to determine a target sampling step according to a difference between a pixel value of the sawtooth point and a pixel value of a neighboring pixel point; The determination module is further configured to smooth the sawtooth point based on the target sampling step and coordinate information of the sawtooth point to obtain a smoothed sawtooth point, wherein the neighboring pixel point is a pixel point within a preset range adjacent to the sawtooth point; In a case where it is determined that the sawtooth point is N, the determination module is specifically configured to: Based on N processing threads, the N sawtooth points are operated in parallel as follows: A target sampling step is determined according to a difference between a pixel value of the sawtooth point and a pixel value of a neighboring pixel point, and the sawtooth point is smoothed based on the target sampling step and coordinate information of the sawtooth point to obtain a smoothed sawtooth point; wherein the N processing threads correspond to the N sawtooth points one by one, and N is a positive integer greater than 1.

9. The apparatus of claim 8, wherein, The determination module is specifically configured to: Determine a spatial domain kernel and a pixel domain kernel of the preset range according to a pixel point within a neighboring preset range of a first pixel point in the first image frame, wherein the first pixel point is any one pixel point in the first image frame; The acquisition module is further configured to acquire a maximum gray value and a minimum gray value in gray values of all pixel points in the spatial domain kernel of the preset range, and a maximum gray value and a minimum gray value in gray values of all pixel points in the pixel domain kernel of the preset range; The determination module is specifically configured to determine a larger one of the maximum gray value in the spatial domain kernel and the maximum gray value in the pixel domain kernel as a first gray value, and determine a smaller one of the minimum gray value in the spatial domain kernel and the minimum gray value in the pixel domain kernel as a second gray value; in a case where it is determined that a difference between the first gray value and the second gray value satisfies a preset condition, determine a sawtooth point in the first image frame that needs to be smoothed according to the first pixel point.

10. The apparatus of claim 9, wherein, The acquisition module is further configured to acquire a preset luminance coefficient of the first image frame and a preset gray threshold of the first image frame, wherein the preset luminance coefficient is determined based on a maximum luminance coefficient of the first image frame, and the preset gray threshold is determined based on a minimum gray value of the first image frame; The determining module is specifically configured to: obtain a first value based on the preset luminance coefficient and the first gray value; extract a maximum value of the first value and the preset gray threshold value as a second value; and determine the first pixel point as a jaggy point in the first image frame that needs to be smoothed in a case where a difference between the first gray value and the second gray value is less than the second value.

11. The apparatus of claim 8, wherein, The target sampling step includes a preset number of sampling points, and the preset number includes a first preset number and a second preset number. The determining module is specifically configured to: In a case where a difference between a pixel value of the jaggy point and a pixel value of a neighborhood pixel point is greater than a preset difference threshold, determine the target sampling step as a first sampling step, wherein the first sampling step includes a first preset number of sampling points. In a case where the difference between the pixel value of the jaggy point and the pixel value of the neighborhood pixel point is less than or equal to the preset difference threshold, determine the target sampling step as a second sampling step, wherein the second sampling step includes a second preset number of sampling points, and the second preset number is less than the first preset number.

12. The apparatus of claim 11, wherein, The determining module is specifically configured to: According to the coordinate information of the jaggy point and the coordinate information of the preset number of sampling points, calculate an offset between each sampling point and the jaggy point, and a weight corresponding to each sampling point; and perform smoothing processing on the jaggy point according to the offset and the weight, to obtain a target gray value of the jaggy point.

13. The apparatus of claim 12, wherein, The device further includes: a fitting module configured to fit the jaggy point and the preset number of sampling points to obtain a correlation between the jaggy point and the preset number of sampling points; The determining module is specifically configured to: According to the coordinate information of the jaggy point, the coordinate information of the preset number of sampling points, and the correlation, calculate an offset between each sampling point and the jaggy point, and a weight corresponding to each sampling point.

14. The apparatus of claim 13, wherein, The determining module is specifically configured to: perform normalization processing on the coordinate information of the jaggy point and the coordinate information of the preset number of sampling points to obtain normalized coordinate information; According to the normalized coordinate information and the correlation, obtain an offset between each sampling point and the jaggy point; According to the offset between each sampling point and the jaggy point, determine a weight corresponding to each sampling point.

15. An electronic device, comprising: A processor and a memory, the memory stores programs or instructions that can be run on the processor, and the programs or instructions are executed by the processor to implement the steps of the image processing method according to any one of claims 1-7.

16. A readable storage medium, characterized by, The readable storage medium stores programs or instructions, and the programs or instructions are executed by the processor to implement the steps of the image processing method according to any one of claims 1-7.

Images