Image processing method, device and electronic equipment

Abstract

The present application discloses an image processing method, device, and electronic device, belonging to the field of image processing technology. The method comprises: acquiring a first image, wherein the first image is an image obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera; determining a first local high-frequency energy of edge pixels in a foreground image region of the first image, and a second local high-frequency energy of pixels adjacent to the edge pixels; determining a first value of an image blur parameter of the first image based on the first local high-frequency energy and the second local high-frequency energy, wherein the image blur parameter is used to indicate an image blur processing intensity; and adjusting the image blur processing intensity of the image processing algorithm when the first value is less than or equal to a first threshold value.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device and electronic equipment. Background Art

[0002] The research and development of image processing algorithms involves an iterative process of continuous optimization and reconstruction. During this process, it's necessary to frequently evaluate images processed by the algorithm before and after iteration to ensure that the image doesn't degrade due to algorithm iteration. For example, the blurring effect of an image processed by an image processing algorithm can be evaluated.

[0003] In related technologies, the evaluation method is usually human eye evaluation, which relies on the experience of the observer. The evaluation results are not objective and have low reliability, which in turn leads to low reliability in the adjustment of the image processing algorithm. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method, device, and electronic device that can solve the problem in the prior art of low reliability of adjustments to image processing algorithms.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Acquire a first image, where the first image is an image obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera;

[0007] determining a first local high-frequency energy of an edge pixel in a foreground image region of the first image, and a second local high-frequency energy of a pixel adjacent to the edge pixel;

[0008] determining a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy, wherein the image blur parameter is used to indicate a blur processing intensity of the image;

[0009] When the first value is less than or equal to a first threshold, the image blurring processing intensity of the image processing algorithm is adjusted.

[0010] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0011] A first acquisition module is configured to acquire a first image, where the first image is obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera;

[0012] a first determining module, configured to determine a first local high-frequency energy of an edge pixel in a foreground image region of the first image, and a second local high-frequency energy of a pixel adjacent to the edge pixel;

[0013] a second determining module, configured to determine a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy, wherein the image blur parameter is used to indicate an image blur processing intensity;

[0014] The first adjustment module is configured to adjust the image blur processing intensity of the image processing algorithm when the first value is less than or equal to a first threshold.

[0015] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0018] In a sixth aspect, an embodiment of the present application provides a program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0019] In an embodiment of the present application, after obtaining a first image processed by an image processing algorithm, the first value of the image blur parameter of the first image can be determined by determining the first local high-frequency energy of the edge pixel points in the foreground image area of ​​the first image, and the second local high-frequency energy of the adjacent pixel points of the edge pixel points. The image blur parameter is used to indicate the image blur processing intensity. When the first value is less than or equal to the first threshold, that is, when the blurring effect of the image processing algorithm is poor, the image blur processing intensity of the image processing algorithm can be adjusted to improve the blurring effect of the image processing algorithm. Compared with human eye evaluation, by determining the local high-frequency energy of the edge pixel points and their adjacent pixel points in the foreground image area of ​​the image processed by the image processing algorithm to evaluate the image blur processing intensity of the image processing algorithm, the evaluation reliability of the image processed by the image processing algorithm can be improved, and the adjustment reliability of the image processing algorithm can be thereby improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is one of the flow charts of the image processing method provided in the embodiment of the present application;

[0021] Figure 2 is one of the schematic diagrams of the first image provided in the embodiment of the present application;

[0022] Figure 3 This is the second schematic diagram of the first image provided in the embodiment of the present application;

[0023] Figure 4 This is the second flowchart of the image processing method provided in the embodiment of the present application;

[0024] Figure 5 is a structural diagram of an image processing device provided in an embodiment of the present application;

[0025] Figure 6 This is one of the structural diagrams of the electronic device provided in the embodiment of the present application;

[0026] Figure 7 This is the second structural diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0028] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0029] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0030] Figure 1This is one of the flow charts of the image processing method provided in the embodiment of the present application. The image processing method can be applied to electronic devices. The electronic device can capture images through a camera and process the images through image processing algorithms, such as blurring, color adjustment, etc. Figure 1 As shown, the image processing method may include:

[0031] Step 101: Acquire a first image, where the first image is an image obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera.

[0032] The second image can be an image captured by the electronic device's camera, or an image captured by the camera of another electronic device. In other words, the second image can be an image captured by the electronic device's own camera, an image captured by the camera of another electronic device, or an image received by the camera of another electronic device. Furthermore, the camera can be a front-facing camera; or the camera can be a rear-facing camera.

[0033] After acquiring the second image, the electronic device may use its own image processing algorithm to process the second image to obtain the first image.

[0034] Step 102: determining a first local high-frequency energy of an edge pixel in a foreground image region of the first image and a second local high-frequency energy of a pixel adjacent to the edge pixel.

[0035] In an embodiment of the present application, the blurring effect of the image processing algorithm can be evaluated by determining the local high-frequency energy of edge pixels and their adjacent pixels in the foreground image area of ​​the image processed by the image processing algorithm.

[0036] It is understandable that the number of edge pixels included in the foreground image area in the first image is greater than 1. In a specific implementation, the blurring effect of the image processing algorithm can be evaluated by the local high-frequency energy of at least two edge pixels and their adjacent pixels in the foreground image area in the first image.

[0037] For different edge pixel points, the method of determining their adjacent pixel points can be the same. The adjacent pixel points of an edge pixel point may include: the pixel points closest to the top, bottom, left and / or right of the edge pixel point. Further, the top may include at least one of the following: the top left, directly above and the top right; the bottom may include at least one of the following: the bottom left, directly below and the bottom right, and the left and right are similar. As an example, assuming that the size of a pixel point is 1×1, and the coordinates of the edge pixel point are (x, y), its adjacent pixel points may include adjacent pixel points in the X-axis and Y-axis directions, such as: a pixel point with coordinates (x-1, y), a pixel point with coordinates (x+1, y), a pixel point with coordinates (x, y-1), and a pixel point with coordinates (x, y+1).

[0038] In step 102, the electronic device may first determine at least two edge pixels and their adjacent pixels in the foreground image area of ​​the first image. Afterwards, for each determined pixel, its local high-frequency energy may be determined. The embodiment of the present application does not limit the method for determining the local high-frequency energy of the pixel, and any method that can be used to determine the local high-frequency energy of the pixel may fall within the protection scope of the embodiment of the present application. For ease of distinction, in the embodiment of the present application, the local high-frequency energy of the edge pixel in the foreground image area is referred to as the first local high-frequency energy, and the local high-frequency energy of the adjacent pixel of the edge pixel is referred to as the second local high-frequency energy, but it is worth noting that the determination methods of the first local high-frequency energy and the second local high-frequency energy may be the same.

[0039] Step 103: Determine a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy, where the image blur parameter is used to indicate an image blur processing intensity.

[0040] In step 103, for each edge pixel, whether the transition between the edge pixel and its adjacent pixels is natural can be determined based on the local high-frequency energy of the edge pixel and the local high-frequency energy of its adjacent pixels.

[0041] In the embodiment of the present application, if an edge pixel point is too natural with respect to its adjacent pixel points, the edge pixel point may be referred to as a virtual natural point or a first pixel point. If an edge pixel point is too natural with respect to its adjacent pixel points, the edge pixel point may be referred to as a virtual unnatural point.

[0042] In this manner, the number of unnaturally blurred points in the first image can be determined based on the first local high-frequency energy and the second local high-frequency energy. Subsequently, a first value of an image blur parameter for the first image can be determined based on the number of unnaturally blurred points in the first image. The first value of the image blur parameter for the first image can be negatively or positively correlated with the number of unnaturally blurred points in the first image. Specifically, the greater the number of unnaturally blurred points in the first image, the smaller the first value of the image blur parameter for the first image, and vice versa.

[0043] The first value of the image blur parameter of the first image is used to characterize the image blur processing intensity of the first image. Since the first image is processed using an image processing algorithm, the image blur processing intensity of the image processing algorithm can be evaluated by the first value of the image blur parameter of the first image.

[0044] The larger the first value of the image blur parameter of the first image, the higher the image blur processing intensity of the first image, and the higher the image blur processing intensity of the image processing algorithm. Conversely, the smaller the first value of the image blur parameter of the first image, the lower the image blur processing intensity of the first image, and the lower the image blur processing intensity of the image processing algorithm. The relationship between the image blur processing intensity and the image blur effect can be positively correlated, that is, the higher the image blur processing intensity, the better the image blur effect, and vice versa.

[0045] Therefore, after obtaining the value of the image blur parameter of the first image, i.e., the first value, in step 103, the first value can be compared with a first threshold. The first threshold is the minimum value at which the image blur effect of the image processing algorithm meets the requirements, and can be set based on the actual requirements for the image blur effect, and is not limited in this embodiment of the present application.

[0046] If the first value is less than or equal to the first threshold, the image blurring effect of the image processing algorithm is poor, and step 104 may be performed. If the first value is greater than the first threshold, the image blurring effect of the image processing algorithm is good, and the image blurring intensity of the image processing algorithm may not be adjusted.

[0047] Step 104 : When the first value is less than or equal to a first threshold, adjust the image blur processing intensity of the image processing algorithm.

[0048] If the first value of the image blur parameter of the first image is less than or equal to the first threshold, it indicates that the blurring effect of the image processing algorithm is poor and the naturalness of the blurring needs to be improved. The image processing algorithm can be optimized and the image processing intensity of the image processing algorithm can be adjusted to improve the blurring effect of the image processing algorithm. It can be understood that the blurring effect of the image obtained by processing with the adjusted image processing algorithm is due to the blurring effect of the image obtained by processing with the image processing algorithm before the adjustment.

[0049] The image processing method of the embodiment of the present application, after obtaining a first image processed by an image processing algorithm, can determine a first value of an image blur parameter of the first image by determining a first local high-frequency energy of edge pixels in a foreground image region of the first image, and a second local high-frequency energy of adjacent pixels of the edge pixels, wherein the image blur parameter is used to indicate the intensity of image blur processing. When the first value is less than or equal to a first threshold value, that is, when the blurring effect of the image processing algorithm is poor, the intensity of image blur processing of the image processing algorithm can be adjusted to improve the blurring effect of the image processing algorithm. In this way, compared to manual evaluation, by determining the local high-frequency energy of edge pixels and their adjacent pixels in the foreground image region of the image processed by the image processing algorithm to evaluate the intensity of image blur processing of the image processing algorithm, the reliability of the evaluation of the image processed by the image processing algorithm can be improved, thereby improving the reliability of the adjustment of the image processing algorithm.

[0050] In some embodiments, determining first local high-frequency energy of edge pixels in a foreground image region of the first image and second local high-frequency energy of pixels adjacent to the edge pixels may include:

[0051] For each of the at least two pixels, determining a local area corresponding to the pixel, where the local area includes the pixel and at least one adjacent pixel of the pixel;

[0052] Determine a first local grayscale value corresponding to each pixel point according to the grayscale value of each pixel point in the local area;

[0053] Performing Gaussian filtering on the first local grayscale value to obtain a second local grayscale value corresponding to the pixel point;

[0054] determining a local high-frequency energy of the pixel point according to the first local grayscale value and the second local grayscale value;

[0055] In which, the at least two pixel points include an edge pixel point in the foreground image area of ​​the first image, and an adjacent pixel point of the edge pixel point; when the pixel point is the edge pixel point, the local high-frequency energy of the pixel point is the first local high-frequency energy; when the pixel point is the adjacent pixel point of the edge pixel point, the local high-frequency energy of the pixel point is the second local high-frequency energy.

[0056] In a specific implementation, the local area of ​​a pixel can be used as the center point. In an optional implementation, the center point of the local area of ​​a pixel can be the pixel, and the size can be predetermined. As an example, assuming that the local area of ​​each pixel includes 3×3 pixels, the local area of ​​the pixel includes itself, as well as the first pixel to its upper left, directly above, upper right, directly left, directly right, lower left, directly below, and lower left.

[0057] Afterwards, the grayscale value of each pixel in the local area of ​​the pixel may be used to determine a first local grayscale value corresponding to the pixel.

[0058] In some optional implementations, the first local grayscale value s(x, y) of the pixel point (x, y) may be determined by formula (1):

[0059]

[0060] Where U(x, y) is the local region of pixel (x, y); j is any pixel in the local region of pixel (x, y); and f(j) is the grayscale value of pixel j. The grayscale value of pixel j can be the value of any color channel of pixel j or the average of all color channels of pixel j.

[0061] In some other optional implementations, the average of the grayscale values ​​of each pixel in a local area of ​​the pixel may be determined as the first local grayscale value corresponding to the pixel.

[0062] Afterwards, the first local grayscale value may be Gaussian filtered to obtain the second local grayscale value corresponding to the pixel point. In a specific implementation, the first local grayscale value may be input into a Gaussian filter, and the first local grayscale value may be Gaussian filtered using the standard deviation σ of the Gaussian filter to obtain the second local grayscale value.

[0063] Afterwards, the local high-frequency energy of the pixel point may be determined according to the first local grayscale value and the second local grayscale value.

[0064] In some optional implementations, the local high-frequency energy h(x, y) of the pixel point (x, y) can be determined by formula (2):

[0065] h(x,y)=(s(x,y)-s σ (x,y)) 2 (2)

[0066] Among them, s σ (x, y) is the second local grayscale value of the pixel (x, y).

[0067] In some other optional implementations, the difference between the first local grayscale value and the second local grayscale value may be used to directly determine the local high-frequency energy of the pixel.

[0068] Through the above method, the local high-frequency energy of the pixel point is obtained through the local grayscale value of the local area of ​​the pixel point and the local grayscale value after Gaussian filtering. In this way, the local high-frequency energy of the pixel point can be obtained by extracting the high-frequency information of the local area of ​​the pixel point, thereby improving the reliability of determining the local high-frequency energy of the pixel point.

[0069] In some embodiments, determining a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy includes:

[0070] For each edge pixel in the foreground image area, if a first local high-frequency energy of the edge pixel is greater than an average of second local high-frequency energies of adjacent pixels of the edge pixel, determining the edge pixel as a first pixel;

[0071] A first value of an image blur parameter of the first image is determined according to the number of the first pixel points in the at least two edge pixel points.

[0072] In this embodiment, whether the edge pixel is the first pixel is determined by comparing the local high-frequency energy of the edge pixel and the average of the local high-frequency energies of adjacent pixels.

[0073] In some optional implementations, h(x, y) may be compared with 1 / 2(h(x-1, y)+h(x+1, y)) to determine whether the pixel point (x, y) is the first pixel point. In other optional implementations, h(x, y) may be compared with 1 / 2(h(x, y-1), h(x, y+1)) to determine whether the pixel point (x, y) is the first pixel point.

[0074] When the local high-frequency energy of an edge pixel is greater than the average of the local high-frequency energies of its adjacent pixels, it indicates that the energy gradient of the edge pixel is large, and the transition between the edge pixel and the background is unnatural, so the edge pixel can be determined as an unnaturally blurred point.

[0075] When the local high-frequency energy of an edge pixel is less than or equal to the average of the local high-frequency energies of its adjacent pixels, it indicates that the energy gradient of the edge pixel is small, and the transition between the edge pixel and the background is natural, so the edge pixel can be determined as the first pixel.

[0076] Afterwards, the number of first pixels in the first image may be counted and compared with a first preset value to determine a first value of the image blur parameter of the first image.

[0077] In some optional implementations, the first preset value may be pre-set. In other optional implementations, the first preset value may be based on the number N of edge pixels of the foreground image area of ​​the selected first image. he OK, N he is an integer greater than 1, such as the first preset value can be 1 / 4 (N he ).

[0078] The number of first pixel points N hn When the value is greater than the first preset value, the first value of the image blur parameter of the first image may be determined as unqualified, that is, less than or equal to the first threshold value.

[0079] In N hn When the value is less than or equal to the first preset value, the first value of the image blur parameter of the first image may be determined to be qualified, that is, greater than the first threshold value.

[0080] In the above manner, the number of blurred unnatural points in the first image is determined by comparing the local high-frequency energy of each edge pixel point and the average of the local high-frequency energy of its adjacent pixel points. Then, based on the comparison result of the number of blurred unnatural points in the first image and the first threshold, the first value of the image blur parameter of the first image is determined. In this way, the first value of the image blur parameter of the first image can truly reflect the blur effect of the first image, thereby improving the evaluation reliability of the blur effect of the first image and thereby improving the adjustment reliability of the image processing algorithm.

[0081] In the embodiments of the present application, it is considered that image quality is affected not only by the naturalness of image blur but also by other factors such as chromatic aberration, facial shadows, composition tilt, imaging distance, and image edge offset. Therefore, the embodiments of the present application can also adjust the image processing algorithm or guide image capture by evaluating other image factors.

[0082] In some embodiments, after acquiring the first image, the method further includes:

[0083] determining mean values ​​of color gamut parameters of a first region in the first image, where the first region is a region including the first element;

[0084] determining a color difference value of the first area according to an average value of each color gamut parameter of the first area and a reference value of each color gamut parameter of the first area;

[0085] determining a second value of an image beautification parameter of the first image according to the color difference value, wherein the image beautification parameter is used to indicate an image beautification processing intensity;

[0086] When the second value is less than or equal to a second threshold, the image beautification processing intensity of the image processing algorithm is adjusted.

[0087] In this embodiment, the image beautification processing strength of the image processing algorithm may be evaluated by calculating the value of the image beautification parameter of the image processed by the image processing algorithm.

[0088] The first area may be an area including a first element. The first element may be an element for which the user has high requirements for natural color, such as a human face, green plants, etc., and may be preset.

[0089] The color gamut parameters of the first region may be obtained by averaging the values ​​of the color gamut parameters of the pixels in the first region.

[0090] The reference values ​​of the color gamut parameters of the first region can be pre-set and can serve as reference colors for determining whether the colors of the first region are natural, and can also be called memory colors. Therefore, the average value of the color gamut parameters of the first region can be compared with the reference values ​​of the color gamut parameters of the first region to obtain a color difference value for the first region.

[0091] The average value of each color gamut parameter of the first area and the reference value of each color gamut parameter of the first area belong to the same color gamut, such as the RGB color gamut, the Lab color gamut, etc.

[0092] Take the Lab color gamut as an example for explanation. In this example, the values ​​of the various color gamut parameters of the pixel points in the first area can be averaged first to obtain the mean values ​​C(r, g, b) of the various color gamut parameters of the RGB color gamut, where C(r) represents the color mean of the red channel of the first area, C(g) represents the color mean of the color channel of the first area, and C(b) represents the color mean of the blue channel of the first area. Afterwards, C(r, g, b) of the RGB color gamut is converted to C(L, a, b) of the Lab color gamut, where C(L) represents the illuminance mean of the first area, C(a) represents the color mean of the range from red to green in the first area, and C(b) represents the color mean of the range from blue to yellow in the first area. The various color gamut parameters of the first area are denoted as V(L, a, b). The color difference value D of the first area can be calculated by formula (3):

[0093]

[0094] The second value of the image beautification parameter of the first image is used to characterize the image beautification processing intensity of the first image. Since the first image is processed using an image processing algorithm, the image beautification processing intensity of the image processing algorithm can be evaluated using the second value of the image beautification parameter of the first image.

[0095] The larger the second value of the image beautification parameter for the first image, the higher the image beautification intensity of the first image, and the higher the image beautification intensity of the image processing algorithm. Conversely, the lower the second value of the image beautification parameter for the first image, the lower the image beautification intensity of the first image, and the lower the image beautification intensity of the image processing algorithm. The image beautification intensity can be positively correlated with the image toning effect, that is, the higher the image beautification intensity, the better the image effect, and vice versa.

[0096] The second value of the image beautification parameter of the first image is negatively correlated with the color difference value of the first region in the first image, that is, the larger the color difference value of the first region in the first image, the smaller the second value of the image beautification parameter of the first image, and vice versa.

[0097] After obtaining the color difference value of the first area, the color difference value can be compared with a second preset value, wherein the second preset value is a preset maximum value for representing the undistorted color of the area, which can be set according to needs and is not limited in this embodiment of the application.

[0098] If the color difference value is greater than the second preset value, it indicates that the color of the first region is distorted, and the second value of the image beautification parameter of the first image can be determined to be unqualified, that is, less than or equal to the second threshold. The second value is the minimum value that indicates that the image color adjustment effect of the image processing algorithm meets the requirements. It can be set according to the actual requirements for the color adjustment effect and is not limited in this embodiment of the application.

[0099] When the color difference value is less than or equal to the second threshold, it indicates that the color of the first region is not distorted, and the second value of the image beautification parameter of the first image can be determined as qualified, that is, greater than the second threshold.

[0100] If the second value of the image beautification parameter of the first image is less than or equal to the second threshold, it indicates that the color adjustment effect of the image processing algorithm is poor and needs to be improved. The image beautification processing intensity of the image processing algorithm can be optimized to improve the color adjustment effect of the image processing algorithm. It can be understood that the color adjustment effect of the image obtained by using the adjusted image processing algorithm is due to the color adjustment effect of the image obtained by using the image processing algorithm before the adjustment.

[0101] Through the above method, the color adjustment effect of the image processing algorithm can be evaluated by evaluating the beautification intensity of the first region in the image processed by the image processing algorithm. When the image beautification parameter value of the image is low, the image processing algorithm can be adjusted to optimize the color adjustment effect of the image processing algorithm. This can improve the reliability of the image processing algorithm evaluation and, in turn, the reliability of the image processing algorithm adjustment.

[0102] In some embodiments, after acquiring the first image, the method further includes:

[0103] determining a grayscale mean of a second region in the first image, where the second region is a region including a second element;

[0104] For each pixel point in the second area, if the grayscale value of the pixel point is less than the grayscale mean value, determine the pixel point as a second pixel point;

[0105] determining a third value of an image shadow parameter of the first image according to the number of the second pixels in the second area, where the image shadow parameter is used to indicate an image shadow processing intensity;

[0106] When the third value is less than or equal to a third threshold, performing a first operation, where the first operation includes at least one of the following:

[0107] Outputting first prompt information, where the first prompt information is used to prompt the user to adjust the shooting position;

[0108] Adjust the image shadow processing strength of the image processing algorithm.

[0109] In this embodiment, the second image may be an image captured by a camera of the electronic device itself. Furthermore, the electronic device may be in a shooting state.

[0110] The second area may be an area including a second element. The second element may be an element for which the user has high requirements on the naturalness of shadow, such as a face, and may be preset.

[0111] The shadow in the second area may be caused by improper processing of the image processing algorithm or by an inappropriate shooting position of the user. Therefore, the electronic device can perform at least one of the following by calculating the value of the image shadow parameter of the second area of ​​the image processed by the image processing algorithm: evaluate the image shadow processing intensity of the image processing algorithm, and decide whether to guide the user to adjust the shooting position to avoid affecting the image quality due to excessive shadow in the second area.

[0112] In a specific implementation, the grayscale mean value of the second region in the first image may be determined first. The grayscale mean value of the second region may be obtained by averaging the grayscale values ​​of each pixel in the second region.

[0113] Afterwards, the grayscale values ​​of each pixel in the second area can be compared with the grayscale mean I m The pixel whose gray value is less than the gray mean is determined as the second pixel, which can also be called the shadow point. m , less than I m The pixel point is determined as the shadow point.

[0114] The value of the image shadow parameter of the first image is used to represent the intensity of image shadow processing for the first image. Since the first image is processed using an image processing algorithm, the value of the image shadow parameter of the first image can be used to evaluate the intensity of image shadow processing by the image processing algorithm. Furthermore, the value of the image shadow parameter of the first image can be used to determine whether to guide the user to adjust the shooting position. A higher value of the image shadow parameter of the first image indicates a higher intensity of image shadow processing for the first image, and thus an image shadow processing intensity of the image processing algorithm is higher. Conversely, a lower value of the image shadow parameter of the first image indicates a lower intensity of image shadow processing for the first image, and thus an image shadow processing intensity of the image processing algorithm is higher. The intensity of image shadow processing can be positively correlated with the image shadow effect; a higher intensity of image shadow processing indicates a better image shadow effect, and vice versa.

[0115] The value of the image shadow parameter of the first image is negatively correlated with the number of shadow points in the second region of the first image. That is, the more shadow points there are in the second region of the first image, the lower the second value of the image beautification parameter of the first image, and vice versa. In a specific implementation, the number of shadow points in the second region can be counted and compared with a third preset value to determine the value of the image shadow parameter of the first image.

[0116] The third preset value is a preset maximum value within the allowable range for characterizing the shadow of the second area, which can be set according to needs and is not limited in the embodiment of the present application. In an optional implementation, the third preset value can be preset. In other optional implementations, the third preset value can be based on the number N of pixels in the second area. h Determine, for example, the third preset value may be 1 / 3 (N h ).

[0117] If the number of shadow points is greater than the third preset threshold, it indicates that the shadow area of ​​the second region is too large, and the value of the image shadow parameter of the first image can be determined to be unqualified, that is, less than or equal to the third threshold. The third threshold can be set according to the actual requirements for the shadow effect and is not limited in this embodiment of the application.

[0118] When the number of shadow points is less than or equal to the third threshold, it indicates that the shadow area of ​​the second region is within the allowable range, and the value of the image shadow parameter of the first image can be determined to be qualified, that is, greater than the third threshold.

[0119] If the value of the image shadow parameter of the first image is less than or equal to the third threshold, it indicates that the shadow effect of the image processing algorithm is poor and needs to be improved. The image processing algorithm can be optimized to improve the shadow effect of the image processing algorithm. Alternatively, it indicates that the user's shooting position is inappropriate and needs to be adjusted. A first prompt message can be output to prompt the user to adjust the shooting position to avoid affecting the image quality due to the excessive shadow area of ​​the second area.

[0120] Through the above method, the shadow effect of the image processing algorithm can be evaluated by evaluating the value of the shadow processing parameter in the image processed by the image processing algorithm, and / or determining whether to guide the user to adjust the shooting position. When the value of the image shadow parameter is low, the image processing algorithm can be adjusted to perform at least one of the following: optimizing the shadow effect of the image processing algorithm, thereby improving the quality of the image processed by the image processing algorithm; or outputting a prompt message prompting the user to adjust the shooting position. In this way, it is possible to avoid increasing the shadow effect of the image processing algorithm or improve the shooting effect.

[0121] In some embodiments, after acquiring the first image, the method further includes:

[0122] Determining a first key point and a second key point of the face region, where the first key point and the second key point satisfy: when the face is not tilted, an angle between a line connecting the first key point and the second key point and a vertical direction is less than a fourth threshold;

[0123] Determining a target angle between a connecting line of the first key point and the second key point and a vertical direction;

[0124] When the target angle is greater than the fourth threshold, second prompt information is output, where the second prompt information is used to prompt the user to adjust the shooting angle.

[0125] In this embodiment, the second image may be an image captured by a camera of the electronic device itself. Furthermore, the electronic device may be in a shooting state.

[0126] When taking a photo, users may rotate their electronic device to find a good angle. However, if the rotation angle is too large, the image quality may be poor. Therefore, in this embodiment, the tilt of the facial region in the image can be detected to determine whether to prompt the user to adjust the shooting angle to avoid image quality degradation caused by excessive tilt of the facial region.

[0127] In a specific implementation, two key points in the facial region can be pre-selected to determine the tilt of the facial region. It is worth noting that when the face is not tilted, the line connecting these two key points is parallel to the vertical direction. For example, the first key point can be the center of the nose tip, and the second key point can be the center of the two eyes.

[0128] To determine the tilt of the face area in the first image, as Figure 2 As shown, the first key point 21 and the second key point 22 of the face region in the first image can be found first. Then, the angle θ25 between the connecting line 23 of the first key point 21 and the second key point 22 and the vertical direction 24 in the face region in the first image is determined.

[0129] Afterwards, the angle is compared with a fourth threshold value to determine whether to output second prompt information for prompting the user to adjust the shooting angle.

[0130] The fourth threshold is a preset maximum value of the shooting angle within the allowable range for representing the face area, which can be set according to needs and is not limited in this embodiment of the present application.

[0131] When the included angle is greater than the fourth threshold, it indicates that the inclination of the face region is too large, and the second prompt information may be output.

[0132] When the included angle is less than or equal to the fourth threshold, it indicates that the inclination of the face region is within the allowable range, and the second prompt information may not be output.

[0133] Through the above method, the tilt of the facial region in the first image can be determined to determine whether to guide the user to adjust the shooting angle. If the tilt is too large, a prompt message can be output to prompt the user to adjust the shooting angle. This can avoid excessive tilt of the facial region, thereby improving image quality.

[0134] In some embodiments, after acquiring the first image, the method further includes:

[0135] Determining the number of pixels included in the face area in the first image;

[0136] When the number of pixels included in the face area is greater than a fifth threshold, a third prompt message is output, where the third prompt message is used to prompt the user to stay away from the camera.

[0137] In this embodiment, the second image may be an image captured by a camera of the electronic device itself. Furthermore, the electronic device may be in a shooting state.

[0138] If you are too close to the camera when shooting, the scene will be stretched, resulting in an unnatural magnification. To prevent unnatural stretching caused by the user being too close to the camera, in this embodiment, the proportion of the face area in the image can be detected to determine whether to prompt the user to adjust the distance between themselves and the camera, so as to avoid unnatural stretching caused by being too close to the camera and affecting image quality.

[0139] In a specific implementation, the number of pixels included in the face area in the first image can be counted, and then compared with the fifth threshold to determine whether to output the third prompt information for prompting the user to stay away from the camera.

[0140] The fifth threshold is a pre-set minimum value used to indicate that the face accounts for an excessively large proportion of the image. It can be set as needed and is not limited in the embodiments of the present application. In some optional implementations, the fifth threshold can be pre-set. In other optional implementations, the fifth threshold can be determined based on the total number of pixels N included in the first image, for example, the fifth threshold can be (1 / 3)N.

[0141] When the number of pixels included in the face area is greater than the fifth threshold, it means that the face area occupies too large a proportion in the image and there is a high possibility that the image is unnaturally stretched, and the third prompt information can be output.

[0142] When the number of pixels included in the face area is less than or equal to the fifth threshold, it means that the proportion of the face area in the image is within the allowable range, the possibility of the image being unnaturally stretched is low, and the third prompt information may not be output.

[0143] This approach can determine the proportion of the face area in the image and then decide whether to guide the user to adjust their distance from the camera. If the face area accounts for too large a proportion of the image, a prompt message can be output to prompt the user to move away from the camera. This can avoid the risk of unnatural stretching caused by being too close to the camera, thereby improving image quality.

[0144] In some embodiments, after acquiring the first image, the method further includes:

[0145] Determine a target pixel point located in an edge area of ​​the first image among the pixels of the face area of ​​the first image;

[0146] When the number of the target pixel points is greater than the sixth threshold, fourth prompt information is output, where the fourth prompt information is used to prompt the user to adjust the camera position.

[0147] In this embodiment, the second image may be an image captured by a camera of the electronic device itself. Furthermore, the electronic device may be in a shooting state.

[0148] Due to the limitations of convex lens imaging and the size of the camera sensor, the imaged portion at the edge of the sensor may be squeezed, affecting the visual quality of the image. In this embodiment, the facial imaging position can be detected to determine whether to prompt the user to adjust the camera position to avoid squeezing the face due to being too close to the edge and affecting image quality.

[0149] In specific implementation, the edge area of ​​the first image can be delineated first, taking a 1:1 imaging ratio as an example, such as Figure 3 As shown, the first image is divided into 7×7 uniform blocks, a total of 49 blocks, of which 24 edge blocks are marked as edge areas. For 16:9 or 4:3 imaging ratios, the image is divided into 8×6 blocks, a total of 48 blocks, and the 24 edge blocks are also marked as edge areas.

[0150] Afterwards, the number of target pixel points in the face area of ​​the first image located in the edge area of ​​the first image can be determined and compared with the sixth threshold to determine whether to output the fourth prompt information for prompting the user to adjust the camera.

[0151] The sixth threshold is a pre-set minimum value used to characterize the proximity of a face to an edge, which can be set as needed and is not limited in this embodiment of the present application. In some optional implementations, the sixth threshold can be pre-set. In other optional implementations, the sixth threshold can be based on the total number of pixels N included in the face area in the first image. h Determine, for example, the sixth threshold may be (1 / 2)N h .

[0152] When the number of target pixel points is greater than the sixth threshold, it indicates that the face is close to the edge, and a fourth prompt message may be output to prompt the user to adjust the camera position so that the imaging position of the person is away from the edge.

[0153] When the number of target pixels is less than or equal to the sixth threshold, it indicates that the face is not close to the edge, and the fourth prompt information may not be output.

[0154] Through the above method, the proportion of target pixels in the face area located at the edge of the image can be determined to determine whether to guide the user to adjust the camera position. If the proportion is too large, an output can be output to prompt the user to adjust the camera position so that the person's image position is away from the edge, thereby improving image quality.

[0155] It should be noted that the various optional embodiments introduced in the embodiments of the present application can be implemented in combination with each other or separately if they do not conflict with each other, and the embodiments of the present application do not limit this.

[0156] To facilitate understanding of the image processing method provided by the above embodiment, the above image processing method is described below using a specific scenario embodiment.

[0157] In order to provide users with a better selfie experience, this scenario embodiment can evaluate the aesthetic quality of selfie photos from multiple perspectives.

[0158] There are many factors that affect the selfie effect. In this scenario embodiment, the six most important factors are selected for detection, such as Figure 4 As shown, these include chromatic aberration, bokeh naturalness, facial shadows, composition tilt, imaging distance, and image edge shift.

[0159] In this scenario embodiment, the severity of the six influencing factors is first accurately quantified, and then a decision is made based on the quantification results whether feedback is required to the user or the image processing algorithm.

[0160] As for chromatic aberration and blur naturalness, the main reasons come from the algorithm processing. When the impact is high, it should be fed back to the algorithm for parameter adjustment.

[0161] For composition tilt, imaging distance and imaging edge offset, the main factor is the shooting environment or shooting angle selected by the user. If the impact is serious, timely feedback should be provided to the user for adjustment.

[0162] As for the impact of facial shadows, it is due to both user choice and should be within the scope of algorithm processing. At this time, the results should be fed back to both the processing algorithm and the user in real time.

[0163] This scenario embodiment can achieve user selfie aesthetic perception evaluation through memory color offset calculation, blur naturalness detection, face shadow detection, composition tilt detection, imaging distance detection, and portrait edge offset detection. It may include the following:

[0164] 1. Color difference detection.

[0165] With the development of photography technology, users are beginning to pay attention to the authenticity of their photos, rather than just pursuing high saturation images. For example, in the past, people pursued whitening effects on human skin and bright colors on plants. Now, users are more concerned about whether the camera's effects can restore the true colors of nature. To this end, this scenario embodiment will use the memory color system to compare the color of the target area in the selfie photo to measure whether the image processing algorithm has over-beautified the captured image. The comparison method can be divided into the following steps:

[0166] (1) Detect memory color areas: Use the target recognition algorithm to identify the face area and green plant area in the image.

[0167] (2) Calculate the color of the memory color area: calculate the average value of the pixels in the face area and the green plant area respectively, and obtain the color mean C of the corresponding RGB color domain h (r, g, b) and C g (r, g, b).

[0168] Then the C of the RGB color gamut h (r, g, b) and C g (r, g, b) is converted to C in Lab color space h (L, a, b) and C g (L, a, b).

[0169] (3) Memory color area comparison: Calculate C using formula (3) h (L, a, b) and C g (L, a, b) and face memory color V h (L, a, b), green plant memory color V h The deviation of (L, a, b) is obtained as D h With D g .

[0170] (4) Feedback evaluation: If D h With D g If one or two of the values ​​are greater than 10, they are fed back to the image processing algorithm; if they are less than 10, no feedback is given.

[0171] 2. Blurred naturalness detection.

[0172] In order to provide users with better selfie effects, the selfie mode of mobile phone cameras usually uses dual-camera depth map calculation technology to add a blur effect to the background of the character to achieve the purpose of highlighting the main character. However, excessive blurring can easily lead to an unnatural transition between the foreground and background, separating the character from the background, and affecting the visual aesthetic perception. To this end, this scenario embodiment proposes to evaluate the blur effect of the image processing algorithm by calculating the high-frequency energy gradient between the foreground edge and the background of the character. The specific steps are as follows:

[0173] (1) Detect edge points: Use the target recognition algorithm to identify the face area in the selfie photo, list the pixel points on the edge of the face area as candidate points (the high-frequency gradient naturalness is calculated point by point), and count the number of candidate points as N he .

[0174] (2) Calculate high-frequency energy: Calculate the local high-frequency energy of the candidate pixel point (x, y) and the local high-frequency energy of its adjacent pixels in the X and Y axis directions, where the calculation formula for the local high-frequency energy h can be formula (2).

[0175] The local high-frequency energies of the pixel point (x, y) and its adjacent pixels in the X and Y axis directions h(x, y), h(x-1, y), h(x+1, y), h(x, y-1), and h(x, y+1) can be calculated respectively.

[0176] (3) Calculate the energy gradient naturalness: Compare the local high-frequency energy of the pixel (x, y) with that of its neighboring pixels. If h(x, y) > 1 / 2(h(x-1, y) + h(x+1, y)) or h(x, y) > 1 / 2(h(x, y-1), h(x, y+1)), the candidate point is marked as a blurred unnatural point; otherwise, it is marked as a blurred natural point.

[0177] (4) Feedback evaluation: Through (2) and (3), all candidate points can be divided into blurred natural points and blurred unnatural points. The number of blurred unnatural pixels is counted and recorded as N hn , if N hn >1 / 4(N he ), it is considered that the naturalness of the blur in the selfie photo needs to be improved. In this case, the result is fed back to the image processing algorithm for optimization, otherwise no feedback is given.

[0178] 3. Facial shadow detection.

[0179] When taking selfies, users are restricted by the scene, such as when the light source is above and behind the head, and the light is blocked and the facial area cannot be directly illuminated, causing the facial area to be darker than the surrounding environment, resulting in a shadow area, and ultimately poor imaging aesthetics. To this end, this scenario embodiment proposes to use brightness thresholds to distinguish the proportion of facial shadow areas and non-shadow areas, thereby determining whether the shadow area of ​​the photo is too large to affect the aesthetic perception quality. The specific steps are as follows:

[0180] (1) Counting the size of the face area: Use the face recognition algorithm to identify the face area, and calculate the grayscale mean of the area, which is recorded as I m , the total number of pixels in the face area is recorded as N h .

[0181] (2) Statistical shadow area size: grayscale values ​​of pixels in the face area I and I m Pixel-by-pixel comparison, if (1 / 2)I m <I<I m It is marked as the shaded area and is denoted as N s .

[0182] (3) Feedback evaluation: quantification Ns The proportion of N s >(1 / 3)N h , it means that the selfie photo has too heavy shadows, which is caused by improper image algorithm processing and poor light source position of the user's shooting scene. At this time, the evaluation results should be fed back to the user and the image processing algorithm at the same time.

[0183] 4. Composition tilt detection.

[0184] When taking selfies, users often rotate their phones to find a good shooting angle, but if the rotation angle is too large, the imaging visual effect will be poor. In addition, if the rotation exceeds 90 degrees, the image processing algorithm will automatically rotate the cropped photo to adjust it to a centered angle. In order to assist users in selecting a suitable shooting angle, this scenario embodiment proposes the following method to detect the tilt of the shooting composition. The specific steps are as follows:

[0185] (1) Constructing the slope line: Using the face recognition algorithm to detect key points of the face, such as Figure 2 Select Figure 2 The two key points marked in the figure are connected by a straight line as the slope line, and the vertical line as the baseline.

[0186] (2) Calculate the inclination angle: Calculate the angle formed by the two straight lines and record it as θ.

[0187] (3) Feedback evaluation: If θ is greater than 40°, the composition is considered to have a large tilt. The result is fed back to the user and the user is advised to adjust the shooting angle.

[0188] 5. Imaging distance detection.

[0189] Due to the physical limitations of camera sensors and lenses, subjects appearing too close to the camera may be stretched, resulting in an unnatural magnification. To prevent this from happening when users are too close to the camera, this embodiment proposes estimating the imaging distance by calculating the facial area ratio.

[0190] The specific steps are as follows:

[0191] (1) Counting the face area: Use the existing face detection algorithm to identify the face area and count the number of pixels in the area, denoted as N h , and the total number of pixels in the photo is counted as N.

[0192] (2) Feedback evaluation: Comparison N h and N, if N h >(1 / 3)N, it is considered that the imaging distance of the person is too close, and the user should be reminded to move away from the lens.

[0193] 6. Imaging edge offset detection

[0194] Due to the limitations of convex lens imaging and camera sensor size, the image at the edge of the sensor will be squeezed, affecting the visual effect of the image. To help users take more aesthetically pleasing images, this scenario embodiment proposes the following method to detect the imaging position of the person to assist the user in taking pictures.

[0195] The specific steps are as follows:

[0196] (1) Imaging area division: Taking the 1:1 imaging ratio as an example, Figure 3 The imaging picture is divided into uniform 7×7 blocks, a total of 49 blocks, of which 24 edge blocks are marked as edge areas (for 16:9 and 4:3 imaging ratios, it is divided into 8×6 blocks, a total of 48 blocks, and similarly 24 edge blocks are marked as edge areas).

[0197] (2) Face area edge ratio: Use the existing face detection algorithm to identify the face area and count the number of pixels in the area as N h , and count the number of pixels in the edge area of ​​the face area as N e .

[0198] (3) Feedback evaluation: Calculate N e The edge ratio, if N e >(1 / 2)N h , it is considered that the person’s image is too close to the edge, and the user is advised to adjust the camera position.

[0199] This scenario embodiment can perform aesthetic evaluation on user selfies, detect whether there are areas for improvement in six aspects, including color difference, naturalness of blur, facial shadows, and composition tilt, and then promptly feed back the detection results to the algorithm or user for adjustment to obtain better shooting results.

[0200] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, the image processing device provided in the embodiment of the present application is described by taking the image processing device executing the image processing method as an example.

[0201] like Figure 5 As shown, the image processing apparatus 500 may include:

[0202] A first acquisition module 501 is configured to acquire a first image, where the first image is obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera;

[0203] A first determining module 502 is configured to determine a first local high-frequency energy of an edge pixel in a foreground image region of the first image, and a second local high-frequency energy of a pixel adjacent to the edge pixel;

[0204] A second determining module 503 is configured to determine a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy, wherein the image blur parameter is used to indicate an image blur processing intensity;

[0205] The first adjustment module 504 is configured to adjust the image blurring intensity of the image processing algorithm when the first value is less than or equal to a first threshold.

[0206] In some embodiments, the first determining module includes:

[0207] A first determining unit is configured to determine, for each of at least two pixel points, a local area corresponding to the pixel point, where the local area includes the pixel point and at least one adjacent pixel point of the pixel point;

[0208] a second determining unit, configured to determine a first local grayscale value corresponding to each pixel point in the local area according to the grayscale value of each pixel point in the local area;

[0209] a first acquiring unit, configured to perform Gaussian filtering on the first local grayscale value to obtain a second local grayscale value corresponding to the pixel point;

[0210] The third determining unit is used to

[0211] determining a local high-frequency energy of the pixel point according to the first local grayscale value and the second local grayscale value;

[0212] In which, the at least two pixel points include an edge pixel point in the foreground image area of ​​the first image, and an adjacent pixel point of the edge pixel point; when the pixel point is the edge pixel point, the local high-frequency energy of the pixel point is the first local high-frequency energy; when the pixel point is the adjacent pixel point of the edge pixel point, the local high-frequency energy of the pixel point is the second local high-frequency energy.

[0213] In some embodiments, the second determining module includes:

[0214] a fourth determining unit, configured to, for each edge pixel in the foreground image area, determine the edge pixel as a first pixel if a first local high-frequency energy of the edge pixel is greater than an average of second local high-frequency energies of adjacent pixels of the edge pixel;

[0215] A fifth determining unit is configured to determine a first value of an image blur parameter of the first image according to the number of the first pixel points in the at least two edge pixel points.

[0216] In some embodiments, the apparatus further comprises:

[0217] a third determining module, configured to determine a mean value of each color gamut parameter of a first region in the first image, where the first region is a region including the first element;

[0218] determining a color difference value of the first area according to an average value of each color gamut parameter of the first area and a reference value of each color gamut parameter of the first area;

[0219] a fourth determining module, configured to determine a second value of an image beautification parameter of the first image according to the color difference value, wherein the image beautification parameter is used to indicate an image beautification processing intensity;

[0220] The second adjustment module is configured to adjust the image beautification processing intensity of the image processing algorithm when the second value is less than or equal to a second threshold.

[0221] In some embodiments, the apparatus further comprises:

[0222] a fifth determining module, configured to determine a grayscale mean of a second region in the first image, where the second region is a region including a second element;

[0223] a sixth determining module, configured to, for each pixel in the second area, determine the pixel as a second pixel when the grayscale value of the pixel is less than the grayscale mean;

[0224] a seventh determining module, configured to determine a third value of an image shadow parameter of the first image according to the number of the second pixels in the second area, wherein the image shadow parameter is used to indicate an image shadow processing intensity;

[0225] an execution module, configured to, when the third value is less than or equal to a third threshold, perform a first operation, the first operation comprising at least one of the following:

[0226] Outputting first prompt information, where the first prompt information is used to prompt the user to adjust the shooting position;

[0227] Adjust the image shadow processing strength of the image processing algorithm.

[0228] In some embodiments, the apparatus further comprises:

[0229] an eighth determining module, configured to determine a first key point and a second key point of the facial region, wherein the first key point and the second key point satisfy: when the face is not tilted, an angle between a line connecting the first key point and the second key point and a vertical direction is less than a fourth threshold;

[0230] a ninth determining module, configured to determine a target angle between a connecting line of the first key point and the second key point and a vertical direction;

[0231] The first output module is configured to output second prompt information when the target angle is greater than the fourth threshold, wherein the second prompt information is used to prompt the user to adjust the shooting angle.

[0232] In some embodiments, the apparatus further comprises:

[0233] a tenth determining module, configured to determine the number of pixels included in the face area in the first image;

[0234] The second output module is used to output third prompt information when the number of pixels included in the face area is greater than a fifth threshold, and the third prompt information is used to prompt the user to stay away from the camera.

[0235] In some embodiments, the apparatus further comprises:

[0236] an eleventh determining module, configured to determine a target pixel located in an edge area of ​​the first image among the pixels in the face area of ​​the first image;

[0237] The third output module is used to output fourth prompt information when the number of the target pixel points is greater than the sixth threshold, and the fourth prompt information is used to prompt the user to adjust the camera position.

[0238] The image processing device 500 provided in the embodiment of the present application can implement each process implemented in the method embodiment. To avoid repetition, they will not be described here.

[0239] The image processing device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or a device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0240] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0241] Alternatively, as Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602, wherein the memory 602 stores a program or instruction that can be run on the processor 601, and when the program or instruction is executed by the processor 601, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0242] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0243] Figure 7 A hardware structure diagram of an electronic device implementing an embodiment of the present application.

[0244] The electronic device 700 includes but is not limited to components such as a radio frequency unit 701 , a network module 702 , an audio output unit 703 , an input unit 704 , a sensor 705 , a display unit 706 , a user input unit 707 , an interface unit 708 , a memory 709 , and a processor 710 .

[0245] Those skilled in the art will understand that the electronic device 700 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 710 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0246] The processor 710 is configured to:

[0247] Acquire a first image, where the first image is an image obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera;

[0248] determining a first local high-frequency energy of an edge pixel in a foreground image region of the first image, and a second local high-frequency energy of a pixel adjacent to the edge pixel;

[0249] determining a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy, wherein the image blur parameter is used to indicate an image blur processing intensity;

[0250] When the first value is less than or equal to a first threshold, the image blurring processing intensity of the image processing algorithm is adjusted.

[0251] In some embodiments, the processor 710 is configured to:

[0252] For each of the at least two pixels, determining a local area corresponding to the pixel, where the local area includes the pixel and at least one adjacent pixel of the pixel;

[0253] Determining a first local grayscale value corresponding to each pixel point according to the grayscale value of each pixel point in the local area;

[0254] Performing Gaussian filtering on the first local grayscale value to obtain a second local grayscale value corresponding to the pixel point;

[0255] determining a local high-frequency energy of the pixel point according to the first local grayscale value and the second local grayscale value;

[0256] In which, the at least two pixel points include an edge pixel point in the foreground image area of ​​the first image, and an adjacent pixel point of the edge pixel point; when the pixel point is the edge pixel point, the local high-frequency energy of the pixel point is the first local high-frequency energy; when the pixel point is the adjacent pixel point of the edge pixel point, the local high-frequency energy of the pixel point is the second local high-frequency energy.

[0257] In some embodiments, the processor 710 is configured to:

[0258] For each edge pixel in the foreground image area, if a first local high-frequency energy of the edge pixel is greater than an average of second local high-frequency energies of adjacent pixels of the edge pixel, determining the edge pixel as a first pixel;

[0259] A first value of an image blur parameter of the first image is determined according to the number of the first pixel points in the at least two edge pixel points.

[0260] In some embodiments, the processor 710 is configured to:

[0261] determining mean values ​​of color gamut parameters of a first region in the first image, where the first region is a region including the first element;

[0262] determining a color difference value of the first area according to an average value of each color gamut parameter of the first area and a reference value of each color gamut parameter of the first area;

[0263] determining a second value of an image beautification parameter of the first image according to the color difference value, wherein the image beautification parameter is used to indicate an image beautification processing intensity;

[0264] When the second value is less than or equal to a second threshold, the image beautification processing intensity of the image processing algorithm is adjusted.

[0265] In some embodiments, the processor 710 is configured to:

[0266] determining a grayscale mean of a second region in the first image, where the second region is a region including a second element;

[0267] For each pixel point in the second area, if the grayscale value of the pixel point is less than the grayscale mean value, determine the pixel point as a second pixel point;

[0268] determining a third value of an image shadow parameter of the first image according to the number of the second pixels in the second area, where the image shadow parameter is used to indicate an image shadow processing intensity;

[0269] When the third value is less than or equal to a third threshold, performing a first operation, where the first operation includes at least one of the following:

[0270] Outputting first prompt information, where the first prompt information is used to prompt the user to adjust the shooting position;

[0271] Adjust the image shadow processing strength of the image processing algorithm.

[0272] In some embodiments, the processor 710 is configured to:

[0273] Determining a first key point and a second key point of the face region, where the first key point and the second key point satisfy: when the face is not tilted, an angle between a line connecting the first key point and the second key point and a vertical direction is less than a fourth threshold;

[0274] Determining a target angle between a connecting line of the first key point and the second key point and a vertical direction;

[0275] When the target angle is greater than the fourth threshold, second prompt information is output, where the second prompt information is used to prompt the user to adjust the shooting angle.

[0276] In some embodiments, the processor 710 is configured to:

[0277] Determining the number of pixels included in the face area in the first image;

[0278] When the number of pixels included in the face area is greater than a fifth threshold, a third prompt message is output, where the third prompt message is used to prompt the user to stay away from the camera.

[0279] In some embodiments, the processor 710 is configured to:

[0280] Determine a target pixel point located in an edge area of ​​the first image among the pixels of the face area of ​​the first image;

[0281] When the number of the target pixel points is greater than the sixth threshold, fourth prompt information is output, where the fourth prompt information is used to prompt the user to adjust the camera position.

[0282] The electronic device provided in the embodiment of the present application can implement each process implemented in the method embodiment, and to avoid repetition, it will not be described here.

[0283] It should be understood that in an embodiment of the present application, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and at least one of other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0284] The memory 709 can be used to store software programs and various data. The memory 709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 709 may include a volatile memory or a non-volatile memory, or the memory 709 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0285] Processor 710 may include one or more processing units. Optionally, processor 710 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 710.

[0286] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0287] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0288] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0289] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0290] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0291] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0292] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0293] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An image processing method, characterized in that: include: Acquire a first image, where the first image is an image obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera; determining a first local high-frequency energy of an edge pixel in a foreground image region of the first image, and a second local high-frequency energy of a pixel adjacent to the edge pixel; determining a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy, wherein the image blur parameter is used to indicate an image blur processing intensity; When the first value is less than or equal to a first threshold, adjusting the image blur processing intensity of the image processing algorithm; The determining, according to the first local high-frequency energy and the second local high-frequency energy, a first value of an image blur parameter of the first image includes: For each edge pixel in the foreground image area, if a first local high-frequency energy of the edge pixel is greater than an average of second local high-frequency energies of adjacent pixels of the edge pixel, determining the edge pixel as a first pixel; A first value of an image blur parameter of the first image is determined according to the number of the first pixel points in at least two edge pixel points.

2. The method according to claim 1, characterized in that The determining of first local high-frequency energy of edge pixels in a foreground image region of the first image and second local high-frequency energy of pixels adjacent to the edge pixels includes: For each of the at least two pixels, determining a local area corresponding to the pixel, where the local area includes the pixel and at least one adjacent pixel of the pixel; Determining a first local grayscale value corresponding to each pixel point according to the grayscale value of each pixel point in the local area; Performing Gaussian filtering on the first local grayscale value to obtain a second local grayscale value corresponding to the pixel point; determining a local high-frequency energy of the pixel point according to the first local grayscale value and the second local grayscale value; In which, the at least two pixel points include an edge pixel point in the foreground image area of ​​the first image, and an adjacent pixel point of the edge pixel point; when the pixel point is the edge pixel point, the local high-frequency energy of the pixel point is the first local high-frequency energy; when the pixel point is the adjacent pixel point of the edge pixel point, the local high-frequency energy of the pixel point is the second local high-frequency energy.

3. The method according to claim 1, characterized in that After acquiring the first image, the method further includes: determining mean values ​​of color gamut parameters of a first region in the first image, where the first region is a region including the first element; determining a color difference value of the first area according to an average value of each color gamut parameter of the first area and a reference value of each color gamut parameter of the first area; determining a second value of an image beautification parameter of the first image according to the color difference value, wherein the image beautification parameter is used to indicate an image beautification processing intensity; When the second value is less than or equal to a second threshold, the image beautification processing intensity of the image processing algorithm is adjusted.

4. The method according to claim 1, wherein After acquiring the first image, the method further includes: determining a grayscale mean of a second region in the first image, where the second region is a region including a second element; For each pixel point in the second area, if the grayscale value of the pixel point is less than the grayscale mean value, determine the pixel point as a second pixel point; determining a third value of an image shadow parameter of the first image according to the number of the second pixels in the second area, where the image shadow parameter is used to indicate an image shadow processing intensity; When the third value is less than or equal to a third threshold, performing a first operation, where the first operation includes at least one of the following: Outputting first prompt information, where the first prompt information is used to prompt the user to adjust the shooting position; Adjust the image shadow processing strength of the image processing algorithm.

5. The method according to claim 1, wherein The first image includes a face area. After acquiring the first image, the method further includes: Determining a first key point and a second key point of the face region, where the first key point and the second key point satisfy: when the face is not tilted, an angle between a line connecting the first key point and the second key point and a vertical direction is less than a fourth threshold; Determining a target angle between a connecting line of the first key point and the second key point and a vertical direction; When the target angle is greater than the fourth threshold, second prompt information is output, where the second prompt information is used to prompt the user to adjust the shooting angle.

6. The method according to claim 1, characterized in that The first image includes a face area. After acquiring the first image, the method further includes: Determining the number of pixels included in the face area in the first image; When the number of pixels included in the face area is greater than a fifth threshold, a third prompt message is output, where the third prompt message is used to prompt the user to stay away from the camera.

7. The method according to claim 1, characterized in that The first image includes a face area. After acquiring the first image, the method further includes: Determine a target pixel point located in an edge area of ​​the first image among the pixels of the face area of ​​the first image; When the number of the target pixel points is greater than the sixth threshold, fourth prompt information is output, where the fourth prompt information is used to prompt the user to adjust the camera position.

8. An image processing device, characterized in that: include: A first acquisition module is configured to acquire a first image, where the first image is obtained by processing a second image using an image processing algorithm, and the second image is an image captured by a camera; a first determining module, configured to determine a first local high-frequency energy of an edge pixel in a foreground image region of the first image, and a second local high-frequency energy of a pixel adjacent to the edge pixel; a second determining module, configured to determine a first value of an image blur parameter of the first image according to the first local high-frequency energy and the second local high-frequency energy, wherein the image blur parameter is used to indicate an image blur processing intensity; a first adjustment module, configured to adjust the image blur processing intensity of the image processing algorithm when the first value is less than or equal to a first threshold; The second determining module includes: a fourth determining unit, configured to, for each edge pixel in the foreground image area, determine the edge pixel as a first pixel if a first local high-frequency energy of the edge pixel is greater than an average of second local high-frequency energies of adjacent pixels of the edge pixel; A fifth determining unit is configured to determine a first value of an image blur parameter of the first image according to the number of the first pixel points in at least two edge pixel points.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

Images