Face image processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to a facial image processing method, device, electronic device and storage medium. The method comprises: obtaining a facial edge image and a plurality of original facial feature points corresponding to an original facial image, and then determining a plurality of original facial edge points from the facial edge image based on the plurality of original facial feature points, and then obtaining facial adjustment information based on the plurality of original facial edge points and adjustment parameters, and finally adjusting the facial contour of the original facial image based on the facial adjustment information to obtain a target facial image. By determining a plurality of original facial edge points from the facial edge image based on a plurality of original facial features, it is possible to obtain original facial edge points with higher precision, so that the obtained plurality of original facial edge points can accurately describe the facial contour line, and then improve the precision of the obtained facial adjustment information, so as to accurately process the facial image and make the processed facial image more natural.

Description

Technical Field

[0001] The present disclosure relates to image processing technology, and in particular to a face image processing method, device, electronic device and storage medium. Background Art

[0002] At present, after people obtain facial images, they usually beautify the facial images, for example, they perform operations such as thinning the face, retracting the cheekbones and chin on the taken photos.

[0003] In the related art, some original feature points of a face image are detected, and then the position of the original feature points is changed to process the face image to obtain a processed face image. Since the number of original feature points is small and the accuracy of the original feature points is poor, the face contour curve cannot be accurately described, and thus the face image cannot be accurately processed, resulting in the processed face image not being natural enough. Summary of the invention

[0004] The present disclosure provides a facial image processing method, device, electronic device and storage medium to at least solve the problem that the facial image is not natural enough due to the inability to accurately process the facial image in the related art. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a face image processing method is provided, comprising:

[0006] Obtaining a face edge image and a plurality of original face feature points corresponding to the original face image;

[0007] Based on the multiple original face feature points, determining multiple original face edge points from the face edge image;

[0008] Obtaining facial adjustment information according to the plurality of original facial edge points and the adjustment parameter, wherein the adjustment parameter represents an adjustment range of the facial contour;

[0009] According to the face adjustment information, the face contour of the original face image is adjusted to obtain a target face image.

[0010] Optionally, based on the multiple original face feature points, determining multiple original face edge points from the face edge image comprises:

[0011] Connecting a plurality of pairs of adjacent two original face feature points among the plurality of original face feature points to obtain a plurality of connecting lines;

[0012] For each of the multiple connecting lines, obtaining a plurality of pixel points in the face edge image whose distance from the connecting line is within a preset distance, and selecting a plurality of pixel points with the largest edge strength from the plurality of pixel points;

[0013] The acquired plurality of pixel points with the largest edge strength are determined as the original face edge points.

[0014] Optionally, obtaining face adjustment information according to the plurality of original face edge points and the adjustment parameters includes:

[0015] According to the positions of the plurality of original face edge points in the original face image, the plurality of original face edge points are divided into a plurality of groups of original face edge points;

[0016] According to the multiple groups of original face edge points, edge function curves corresponding to the multiple groups of original face edge points are respectively fitted;

[0017] According to the adjustment parameters, the edge function curves corresponding to the plurality of groups of original face edge points are adjusted respectively to obtain the face adjustment information.

[0018] Optionally, according to a plurality of groups of original face edge points, edge function curves corresponding to the plurality of groups of original face edge points are respectively fitted, including:

[0019] For each group of original face edge points, a line connecting two original face edge points at both ends of each group of original face edge points is used as an X-axis, and a perpendicular bisector of a line connecting two original face edge points at both ends of the group of original face edge points is used as a Y-axis, to establish a coordinate system corresponding to the group of original face edge points;

[0020] Determine the coordinate points of each group of original face edge points in the corresponding coordinate system;

[0021] Each group of original face edge points is fitted according to the coordinate points of each group of original face edge points in the corresponding coordinate system to obtain the edge function curve corresponding to each group of original face edge points.

[0022] Optionally, adjusting the original face image according to the face adjustment information to obtain a target face image includes:

[0023] According to the face adjustment information, a plurality of adjusted face edge points corresponding to the plurality of original face edge points are obtained;

[0024] Downsampling the plurality of adjusted face edge points according to a preset step length to obtain a plurality of adjusted sampling feature points;

[0025] Acquire a plurality of original sampling feature points corresponding to the plurality of adjusted sampling feature points from the plurality of original face edge points;

[0026] The original face image is adjusted according to the plurality of adjusted sampling feature points and the plurality of original sampling feature points to obtain a target face image.

[0027] Optionally, adjusting the original face image according to the plurality of adjusted sampling feature points and the plurality of original sampling feature points to obtain a target face image includes:

[0028] Acquire first pixel points on both sides of the plurality of adjusted sampling feature points from the original face image;

[0029] Acquire second pixel points on both sides of the plurality of original sampling feature points from the original face image, wherein the number of the first pixel points is equal to the number of the second pixel points;

[0030] Constructing a first triangulated network according to the plurality of adjusted sampling feature points and the first pixel point;

[0031] Constructing a second triangulated network according to the plurality of original sampling feature points and the second pixel points;

[0032] The original face image is adjusted according to the first triangular network and the second triangular network to obtain a target face image.

[0033] Optionally, adjusting the original face image according to the first triangulated network and the second triangulated network to obtain a target face image includes:

[0034] Establishing an association relationship between a plurality of first triangular sub-meshes included in the first triangular network and a plurality of second triangular sub-meshes included in the second triangular network;

[0035] Acquire image contents within the plurality of second triangular sub-grids on the original face image;

[0036] For the first triangular network area on the original image, according to the association relationship, the image content in each second triangular sub-mesh is filled into the associated first triangular sub-mesh to obtain a target face image.

[0037] According to a second aspect of an embodiment of the present disclosure, there is provided a face image processing device, comprising:

[0038] An acquisition module is configured to acquire a face edge image and a plurality of original face feature points corresponding to the original face image;

[0039] a determination module configured to determine a plurality of original face edge points from the face edge image based on the plurality of original face feature points;

[0040] A fitting module, configured to obtain face adjustment information according to the plurality of original face edge points and adjustment parameters, wherein the adjustment parameters represent an adjustment range of the face contour;

[0041] The adjustment module is configured to adjust the facial contour of the original facial image according to the facial adjustment information to obtain a target facial image.

[0042] Optionally, the determining module includes:

[0043] A first obtaining submodule is configured to connect multiple pairs of adjacent two original face feature points among the multiple original face feature points to obtain multiple connecting lines;

[0044] A selection submodule is configured to obtain, for each of the multiple connecting lines, a plurality of pixel points in the face edge image whose distance from the connecting line is within a preset distance, and select a plurality of pixel points with the largest edge strength from the plurality of pixel points;

[0045] The determination submodule is configured to determine the acquired multiple pixel points with the largest edge strength as the original face edge points.

[0046] Optionally, the fitting module includes:

[0047] a grouping submodule, configured to divide the plurality of original face edge points into a plurality of groups of original face edge points according to positions of the plurality of original face edge points in the original face image;

[0048] A fitting submodule is configured to fit, according to a plurality of groups of original face edge points, edge function curves corresponding to the plurality of groups of original face edge points respectively;

[0049] The second obtaining submodule is configured to adjust the edge function curves corresponding to the plurality of groups of original face edge points respectively according to the adjustment parameters to obtain the face adjustment information.

[0050] Optionally, the fitting submodule includes:

[0051] an establishing subunit configured to establish, for each group of original face edge points, a coordinate system corresponding to the group of original face edge points by taking a line connecting two original face edge points at two ends of each group of original face edge points as an X-axis and a perpendicular bisector of a line connecting two original face edge points at two ends of the group of original face edge points as a Y-axis;

[0052] A determination subunit is configured to determine the coordinate points of each group of original face edge points in the corresponding coordinate system;

[0053] The fitting subunit is configured to fit each group of original face edge points according to the coordinate points of each group of original face edge points in the corresponding coordinate system to obtain the edge function curve corresponding to each group of original face edge points.

[0054] Optionally, the adjustment module includes:

[0055] a third obtaining submodule, configured to obtain, according to the face adjustment information, a plurality of adjusted face edge points corresponding to the plurality of original face edge points;

[0056] A downsampling submodule is configured to downsample the plurality of adjusted face edge points according to a preset step length to obtain a plurality of adjusted sampling feature points;

[0057] An acquisition submodule is configured to acquire a plurality of original sampling feature points corresponding to the plurality of adjusted sampling feature points from the plurality of original face edge points;

[0058] The adjustment submodule is configured to adjust the original face image according to the multiple adjusted sampling feature points and the multiple original sampling feature points to obtain a target face image.

[0059] Optionally, the adjustment submodule includes:

[0060] A first acquisition subunit is configured to acquire first pixel points on both sides of the plurality of adjusted sampling feature points from the original face image;

[0061] A second acquisition subunit is configured to acquire second pixel points on both sides of the plurality of original sampling feature points from the original face image, wherein the number of the first pixel points is equal to the number of the second pixel points;

[0062] A first construction subunit is configured to construct a first triangulated network according to the plurality of adjusted sampling feature points and the first pixel point;

[0063] A second construction subunit is configured to construct a second triangulated network according to the plurality of original sampling feature points and the second pixel points;

[0064] The adjustment subunit is configured to adjust the original face image according to the first triangular network and the second triangular network to obtain a target face image.

[0065] Optionally, according to the adjustment subunit, it includes:

[0066] Establishing a subnode, configured to establish an association relationship between a plurality of first triangular sub-meshes included in the first triangular network and a plurality of second triangular sub-meshes included in the second triangular network;

[0067] An acquisition subnode is configured to acquire image contents within the plurality of second triangular sub-grids on the original face image;

[0068] The filling subnode is configured to fill the image content in each second triangular sub-mesh into the associated first triangular sub-mesh according to the association relationship for the first triangular network area on the original image, so as to obtain a target face image.

[0069] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0070] processor;

[0071] a memory for storing instructions executable by the processor;

[0072] The processor is configured to execute the instructions to implement the facial image processing method as described in the first aspect.

[0073] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the facial image processing method as described in the first aspect.

[0074] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the face image processing method as described in the first aspect is implemented.

[0075] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0076] The present invention obtains a face edge image and a plurality of original face feature points corresponding to an original face image, and then determines a plurality of original face edge points from the face edge image based on the plurality of original face feature points, and then obtains face adjustment information based on the plurality of original face edge points and adjustment parameters, and finally adjusts the face contour of the original face image based on the face adjustment information to obtain a target face image. By determining a plurality of original face edge points from the face edge image based on a plurality of original face features, it is possible to obtain original face edge points with high precision, so that the obtained plurality of original face edge points can accurately describe the face contour line, thereby improving the precision of the obtained face adjustment information, so as to accurately process the face image and make the processed face image more natural.

[0077] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0079] Figure 1 is a flowchart of a method for processing a face image according to an exemplary embodiment;

[0080] Figure 2 is a schematic diagram of original feature points of an original face image according to an exemplary embodiment;

[0081] Figure 3 is a schematic diagram of a face edge image according to an exemplary embodiment;

[0082] Figure 4 is a flow chart of a method for determining face edge points according to an exemplary embodiment;

[0083] Figure 5 is a flowchart of a method for determining face adjustment information according to an exemplary embodiment;

[0084] Figure 6 is a schematic diagram of adjusting sampling feature points of an original face image according to an exemplary embodiment;

[0085] Figure 7 is a block diagram of a face image processing device according to an exemplary embodiment;

[0086] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0087] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0088] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0089] Figure 1 is a flowchart of a method for processing a face image according to an exemplary embodiment. Figure 2 is a schematic diagram of original feature points of an original face image according to an exemplary embodiment. Figure 3is a schematic diagram of a face edge image according to an exemplary embodiment. Figures 1 to 3 As shown, the following steps are included:

[0090] In step S11, a face edge image and a plurality of original face feature points corresponding to the original face image are obtained.

[0091] The original facial image is a facial image that needs to be processed. Specifically, the original facial image may be a photo containing a face taken by a user. Usually, the user will perform beautification processing on the original facial image, specifically, face slimming processing, for example, face slimming, cheekbone and chin slimming operations on the taken photo. In some entertainment functions, the facial image may also be processed to make the face fatter to achieve entertainment effects.

[0092] Specifically, the original face image can be feature extracted to obtain multiple original face feature points. For example, by using a face feature point detection algorithm, the original face image can be feature extracted to obtain multiple original feature points. Specifically, a commonly used face feature point detection algorithm can be used. This is an existing technology and will not be described in detail here. Figure 2 The white dots near the numbers in the figure are the original feature points, and the face is partially blocked in the figure.

[0093] Specifically, the original face image can be filtered to obtain a face edge image. For example, the original face image can be subjected to image edge detection, specifically, the original face image can be filtered using an operator in image filtering to obtain a face edge image, wherein the specific operator in image filtering can be Soble, or an operator such as Prewitt, Roberts, or Canny. Among the multiple pixel points of the obtained face edge image, the edge strength at the edge is higher. The higher the edge strength, the more obvious the difference from the background color, so that the face edge can be more clearly displayed. Among them, Figure 3 The white lines in the figure are edge outlines. The higher the edge intensity, the more obvious the white color is. The face in the figure is partially occluded.

[0094] In step S12, based on the multiple original face feature points, multiple original face edge points are determined from the face edge image.

[0095] In the face edge image, the face edge is represented by multiple pixel points with different edge strengths, but the edges are not obvious in some places, and some other edges such as hair contours will be displayed, resulting in the inability to obtain a more accurate face edge. As for multiple original face feature points, the number of original face feature points is small, and the multiple original face feature points obtained by face feature point detection are not accurate enough. At this time, by combining the original face feature points with the face edge image, more accurate original face edge points can be obtained. Specifically, based on the multiple original face feature points, multiple pixel points with higher edge strength can be obtained from the vicinity of the original face feature points, and the obtained multiple pixel points are determined as original face edge points, so that more accurate original face edge points can be obtained, wherein the number of original face edge points is greater than the number of original face feature points, so that the multiple original face edge points can more accurately describe the face contour curve.

[0096] In step S13, facial adjustment information is obtained according to the plurality of original facial edge points and adjustment parameters, wherein the adjustment parameters represent the adjustment range of the facial contour.

[0097] The multiple original face edge points obtained by the above method are not a complete and continuous face contour, and the integrity is poor. At this time, it is necessary to fit the multiple original face edge points and obtain face adjustment information based on the adjustment parameters. At this time, the obtained face adjustment information is relatively accurate and smooth. Specifically, the face adjustment information can be a function curve used to adjust the face, and the function curve can be a univariate quadratic function curve.

[0098] In step S14, the facial contour of the original facial image is adjusted according to the facial adjustment information to obtain a target facial image.

[0099] After the face adjustment information is obtained, the original face image can be adjusted according to the face adjustment information. Since the obtained face adjustment information is relatively accurate and smooth, the face image can be accurately adjusted to make the adjusted face image more natural.

[0100] By determining multiple original face edge points from a face edge image based on multiple original features, it is possible to obtain original face edge points with higher precision, so that the obtained multiple original face edge points can accurately describe the face contour line, thereby improving the accuracy of the obtained face adjustment information, so as to accurately process the face image and make the processed face image more natural.

[0101] Figure 4 FIG. 1 is a flowchart of a method for determining a face edge point according to an exemplary embodiment. Figure 4As shown, on the basis of the above technical solution, based on multiple original face feature points, the specific method for determining multiple original face edge points from the face edge image can be:

[0102] In step S41, multiple pairs of adjacent original face feature points among the multiple original face feature points are connected to obtain multiple connecting lines.

[0103] After obtaining multiple original facial feature points, since the number of original facial feature points is small, the spacing is large, and the accuracy is poor, multiple more accurate original face edge points can be obtained based on the multiple original facial feature points. Specifically, multiple original face edge points can be obtained between every two adjacent original facial feature points. First, multiple pairs of adjacent original facial feature points in the multiple original facial feature points are connected to obtain multiple continuous connecting lines.

[0104] In step S42, for each connecting line in the multiple connecting lines, multiple pixel points in the face edge image whose distance to the connecting line is within a preset distance are obtained, and multiple pixel points with the largest edge strength are selected from the multiple pixel points.

[0105] For each connecting line, multiple pixel points in the face edge image whose distance from the connecting line is less than a preset distance are obtained. The preset distance can be set according to actual conditions. Specifically, the preset distance can be a distance of 5 pixel points. From the multiple pixel points obtained, multiple pixel points with the largest edge strength are selected.

[0106] In step S43, the acquired plurality of pixel points with the largest edge strength are determined as the original face edge points.

[0107] In a face edge image, the greater the edge strength, the more likely the pixel is to be an edge point, and the original facial feature points are approximate points on the face edge contour, so multiple pixels with the largest edge strength obtained near the line connecting the original facial feature points can be determined as the original face edge points, which can be used to accurately describe the face edge contour.

[0108] Figure 5 is a flowchart of a method for determining face adjustment information according to an exemplary embodiment. Figure 5 As shown, based on the above technical solution, according to a plurality of original face edge points and adjustment parameters, a specific method for obtaining face adjustment information may be:

[0109] In step S51, the plurality of original face edge points are divided into a plurality of groups of original face edge points according to their positions in the original face image.

[0110] According to the positions of the original face edge points in the original face image, the multiple original face edge points can be divided into multiple groups of original face edge points, so that the face adjustment information can be obtained more accurately. For example, the multiple original face edge points can be divided into two groups to obtain left face edge points and right face edge points. In order to obtain more accurate face adjustment information, the left face edge points and the right face edge points can be further divided into multiple groups of original face edge points on the basis of the left face edge points and the right face edge points. For example, the left face edge points and the right face edge points can be further divided into cheekbone edge points, cheek edge points and chin edge points according to the face parts. Of course, the left face edge points and the right face edge points can also be grouped according to any other grouping method.

[0111] In step S52, edge function curves corresponding to the multiple groups of original face edge points are fitted respectively according to the multiple groups of original face edge points.

[0112] For each group of original face edge points in the multiple groups of original face edge points, fitting is performed respectively, and each group of original face edge points is fitted to obtain a corresponding edge function curve.

[0113] Specifically, according to the multiple groups of original face edge points, the edge function curves corresponding to the multiple groups of original face edge points are respectively fitted, including:

[0114] For each group of original face edge points, a line connecting two original face edge points at both ends of each group of original face edge points is used as an X-axis, and a perpendicular bisector of a line connecting two original face edge points at both ends of the group of original face edge points is used as a Y-axis, to establish a coordinate system corresponding to the group of original face edge points;

[0115] Determine the coordinate points of each group of original face edge points in the corresponding coordinate system;

[0116] Each group of original face edge points is fitted according to the coordinate points of each group of original face edge points in the corresponding coordinate system to obtain the edge function curve corresponding to each group of original face edge points.

[0117] First, in order to make the edge function curve obtained by fitting each group of original face edge points more accurate, it is necessary to establish a respective coordinate system for each group of original face edge points. Specifically, for each group of original face edge points, the line connecting the two original face edge points at both ends of each group of original face edge points is used as the X-axis, and the perpendicular bisector of the line connecting the two original face edge points at both ends of the group of original face edge points is used as the Y-axis to establish the coordinate system corresponding to the group of original face edge points.

[0118] Then, the coordinate points of each group of original face edge points in the corresponding coordinate system are determined, specifically, the coordinate points of each original face edge point in each group of original face edge points in the corresponding coordinate system are determined.

[0119] Then, according to the coordinate points of each group of original face edge points in the corresponding coordinate system, each group of original face edge points is fitted to obtain the edge function curve corresponding to each group of original face edge points. Specifically, the edge function curve (i.e., face contour) can be fitted using the least squares algorithm. The fitting function can be a quadratic function. For example, the fitted edge function curve is: y=a*x*x+b*x+c, wherein a, b and c are constants. In the edge function curves obtained by fitting the original face edge points of different groups, there are differences in the values ​​of a, b and c.

[0120] In step S53, the edge function curves corresponding to the plurality of groups of original face edge points are adjusted respectively according to the adjustment parameters to obtain the face adjustment information.

[0121] Among them, the adjustment parameter is used to characterize the degree of adjustment of the facial contour. The larger the adjustment parameter, the greater the adjustment of the facial contour. If the adjustment parameter is positive, it represents a fat face adjustment for the face. If the adjustment parameter is negative, it represents a thin face adjustment for the face. Specifically, the adjustment parameter can be one or more. When the adjustment parameter is one, all edge function curves are adjusted according to the adjustment parameter to obtain multiple adjusted edge function curves. The multiple adjusted edge function curves are combined to form facial adjustment information.

[0122] When there are multiple adjustment parameters, each group of original face edge points may correspond to one adjustment parameter, and then, according to the adjustment parameters corresponding to each group of original face edge points, the edge function curve corresponding to the group of original face edge points may be adjusted to obtain the adjusted edge function curve corresponding to the group of original face edge points. The adjustment parameter may be input by the user or may be a preset default value.

[0123] For example: the fitted edge function curve in the new coordinate is y=a*x*x+b*x+c, then thin cheekbones means the function shrinks inward, which is reflected in the function expression as y=(a*x*x+b*x+c)*lamda, where lamda is the adjustment parameter, which here is the degree of thinness. For example, lamda=0.5 means that the cheekbones are 50% thinner.

[0124] On the basis of the above technical solution, adjusting the original face image according to the face adjustment information to obtain the target face image may specifically include:

[0125] According to the face adjustment information, a plurality of adjusted face edge points corresponding to the plurality of original face edge points are obtained;

[0126] Downsampling the plurality of adjusted face edge points according to a preset step length to obtain a plurality of adjusted sampling feature points;

[0127] Acquire a plurality of original sampling feature points corresponding to the plurality of adjusted sampling feature points from the plurality of original face edge points;

[0128] The original face image is adjusted according to the plurality of adjusted sampling feature points and the plurality of original sampling feature points to obtain a target face image.

[0129] In this embodiment, after obtaining the facial adjustment information, the original facial image can be adjusted according to the facial adjustment information. In order to make the target facial image obtained after adjustment more natural, the image needs to be processed according to the multiple contour points before adjustment and the multiple contour points after adjustment. Specifically, according to the facial adjustment information, multiple adjusted facial edge points (multiple contour points after adjustment) corresponding to the multiple original facial edge points (multiple contour points before adjustment) can be obtained. The specific method is: the horizontal coordinate of each original facial edge point is substituted into the facial adjustment information, specifically, the edge function curve corresponding to the original facial edge point is substituted into, and the corresponding vertical coordinate can be obtained, thereby obtaining the coordinate of each adjusted facial edge point, and the multiple adjusted facial edge points can be determined.

[0130] Figure 6 FIG. 1 is a schematic diagram of adjusting sampling feature points of an original face image according to an exemplary embodiment. Figure 6 As shown in the figure, the face is partially blocked. Figure 6 The white dots in are the adjustment sampling feature points. In order to reduce the amount of data processing, the number of adjustment face edge points can be appropriately reduced. Specifically, multiple adjustment face edge points can be downsampled according to a preset step size to obtain multiple adjustment sampling feature points. The preset step size can be set according to actual conditions. For example, the value of the preset step size can be 10 pixels, that is, one is taken from the multiple adjustment face edge points every 10 pixels to obtain multiple adjustment sampling feature points.

[0131] Then, a plurality of original sampling feature points corresponding to the plurality of adjusted sampling feature points are obtained from the plurality of original face edge points, and the original face image is adjusted according to the plurality of adjusted sampling feature points and the plurality of original sampling feature points to obtain a target face image.

[0132] Specifically, the specific method of adjusting the original face image according to the plurality of adjusted sampling feature points and the plurality of original sampling feature points to obtain the target face image may be:

[0133] Acquire first pixel points on both sides of the plurality of adjusted sampling feature points from the original face image;

[0134] Acquire second pixel points on both sides of the plurality of original sampling feature points from the original face image, wherein the number of the first pixel points is equal to the number of the second pixel points;

[0135] Constructing a first triangulated network according to the plurality of adjusted sampling feature points and the first pixel point;

[0136] Constructing a second triangulated network according to the plurality of original sampling feature points and the second pixel points;

[0137] The original face image is adjusted according to the first triangular network and the second triangular network to obtain a target face image.

[0138] In the above method, first pixel points on both sides of a plurality of adjusted sampling feature points are obtained from the original face image, wherein the first pixel point includes a plurality of pixel points, specifically, for each adjusted sampling feature point, a pixel point separated from the pixel by a preset pixel point distance is obtained, for example, pixel points separated by 30 pixels on both sides of the adjusted sampling feature point are respectively obtained as two pixel points in the first pixel point. A first triangulated network is constructed based on the plurality of adjusted sampling feature points and the first pixel point.

[0139] The same method is used to obtain second pixel points on both sides of the multiple original sampling feature points from the original face image, and the number of the first pixel points is equal to the number of the second pixel points, so as to facilitate image processing. A second triangular network is constructed based on the multiple original sampling feature points and the second pixel points, and then the original face image is adjusted based on the first triangular network and the second triangular network to obtain a target face image. Specifically, a warp operation is performed on the triangles in the first triangular mesh and the second triangular mesh to obtain the target face image.

[0140] The method of adjusting the original face image according to the first triangular network and the second triangular network to obtain a target face image may include:

[0141] Establishing an association relationship between a plurality of first triangular sub-meshes included in the first triangular network and a plurality of second triangular sub-meshes included in the second triangular network;

[0142] Acquire image contents within the plurality of second triangular sub-grids on the original face image;

[0143] For the first triangular network area on the original image, the image content in each second triangular sub-mesh is filled into the associated first triangular sub-mesh to obtain a target face image.

[0144] In this embodiment, when establishing an association relationship, the multiple first triangular sub-meshes contained in the first triangular network can be numbered in sequence, and then the multiple second triangular sub-meshes contained in the second triangular network can be numbered in sequence, and then, for the first triangular sub-meshes and the second triangular sub-meshes with the same number in the first triangular network and the second triangular network, an association relationship is established. Then, the image content in the multiple second triangular sub-meshes on the original face image is obtained, and it is filled into the first triangular sub-mesh according to the association relationship, and the target face image can be obtained. Through the above method, it is possible to avoid obtaining the target face image directly by changing the face contour line, and the target face image obtained can be made more natural.

[0145] The present invention extracts features from an original face image to obtain a plurality of original face feature points, then performs filtering processing on the original face image to obtain a face edge image, and then determines a plurality of original face edge points from the face edge image based on the plurality of original face feature points, and then obtains face adjustment information based on the plurality of original face edge points and adjustment parameters, and finally adjusts the original face image based on the face adjustment information to obtain a target face image. By determining a plurality of original face edge points from the face edge image based on a plurality of original features, it is possible to obtain original face edge points with high precision, so that the obtained plurality of original face edge points can accurately describe the face contour line, and then improve the precision of the obtained face adjustment information, so as to accurately process the face image and make the processed face image more natural. Moreover, the entire face area can be automatically processed, and there is no need to perform operations such as face thinning, cheekbone shrinking and chin shrinking in steps, so that there is no need for excessive user interaction, and the user's use cost is reduced.

[0146] Figure 7 FIG. 1 is a block diagram of a face image processing apparatus according to an exemplary embodiment. Figure 7 The device includes an acquisition module 71, a determination module 72, a fitting module 73 and an adjustment module 74.

[0147] The acquisition module 71 is configured to acquire a face edge image and a plurality of original face feature points corresponding to the original face image;

[0148] The determination module 72 is configured to determine a plurality of original face edge points from the face edge image based on the plurality of original face feature points;

[0149] The fitting module 73 is configured to obtain face adjustment information according to the plurality of original face edge points and the adjustment parameter, wherein the adjustment parameter represents the adjustment range of the face contour;

[0150] The adjustment module 74 is configured to adjust the facial contour of the original facial image according to the facial adjustment information to obtain a target facial image.

[0151] Optionally, the determining module 72 includes:

[0152] A first obtaining submodule is configured to connect multiple pairs of adjacent two original face feature points among the multiple original face feature points to obtain multiple connecting lines;

[0153] A selection submodule is configured to obtain, for each of the multiple connecting lines, a plurality of pixel points in the face edge image whose distance from the connecting line is within a preset distance, and select a plurality of pixel points with the largest edge strength from the plurality of pixel points;

[0154] The determination submodule is configured to determine the acquired multiple pixel points with the largest edge strength as the original face edge points.

[0155] Optionally, the fitting module 73 includes:

[0156] a grouping submodule, configured to divide the plurality of original face edge points into a plurality of groups of original face edge points according to positions of the plurality of original face edge points in the original face image;

[0157] A fitting submodule is configured to fit, according to a plurality of groups of original face edge points, edge function curves corresponding to the plurality of groups of original face edge points respectively;

[0158] The second obtaining submodule is configured to adjust the edge function curves corresponding to the plurality of groups of original face edge points respectively according to the adjustment parameters to obtain the face adjustment information.

[0159] Optionally, the fitting submodule includes:

[0160] an establishing subunit configured to establish, for each group of original face edge points, a coordinate system corresponding to the group of original face edge points by taking a line connecting two original face edge points at two ends of each group of original face edge points as an X-axis and a perpendicular bisector of a line connecting two original face edge points at two ends of the group of original face edge points as a Y-axis;

[0161] A determination subunit is configured to determine the coordinate points of each group of original face edge points in the corresponding coordinate system;

[0162] The fitting subunit is configured to fit each group of original face edge points according to the coordinate points of each group of original face edge points in the corresponding coordinate system to obtain the edge function curve corresponding to each group of original face edge points.

[0163] Optionally, the adjustment module 74 includes:

[0164] a third obtaining submodule, configured to obtain, according to the face adjustment information, a plurality of adjusted face edge points corresponding to the plurality of original face edge points;

[0165] A downsampling submodule is configured to downsample the plurality of adjusted face edge points according to a preset step length to obtain a plurality of adjusted sampling feature points;

[0166] An acquisition submodule is configured to acquire a plurality of original sampling feature points corresponding to the plurality of adjusted sampling feature points from the plurality of original face edge points;

[0167] The adjustment submodule is configured to adjust the original face image according to the multiple adjusted sampling feature points and the multiple original sampling feature points to obtain a target face image.

[0168] Optionally, the adjustment submodule includes:

[0169] A first acquisition subunit is configured to acquire first pixel points on both sides of the plurality of adjusted sampling feature points from the original face image;

[0170] A second acquisition subunit is configured to acquire second pixel points on both sides of the plurality of original sampling feature points from the original face image, wherein the number of the first pixel points is equal to the number of the second pixel points;

[0171] A first construction subunit is configured to construct a first triangulated network according to the plurality of adjusted sampling feature points and the first pixel point;

[0172] A second construction subunit is configured to construct a second triangulated network according to the plurality of original sampling feature points and the second pixel points;

[0173] The adjustment subunit is configured to adjust the original face image according to the first triangular network and the second triangular network to obtain a target face image.

[0174] Optionally, according to the adjustment subunit, including:

[0175] Establishing a subnode, configured to establish an association relationship between a plurality of first triangular sub-meshes included in the first triangular network and a plurality of second triangular sub-meshes included in the second triangular network;

[0176] An acquisition subnode is configured to acquire image contents within the plurality of second triangular sub-grids on the original face image;

[0177] The filling subnode is configured to fill the image content in each second triangular sub-mesh into the associated first triangular sub-mesh according to the association relationship for the first triangular network area on the original image, so as to obtain a target face image.

[0178] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0179] Figure 8 8 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0180] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0181] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0182] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0183] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0184] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0185] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0186] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0187] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0188] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 5G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0189] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned face image processing method.

[0190] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to complete the above-mentioned face image processing method. Optionally, the computer-readable storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0191] In an exemplary embodiment, a computer program product is also provided, the computer program product includes a computer program, the computer program can be executed by the processor 820 of the electronic device 800 to complete the above-mentioned face image processing method. Optionally, the computer program can be stored in a storage medium of the electronic device 800, and the computer-readable storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0192] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0193] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A facial image processing method, characterized in that: The method comprises: Obtaining a face edge image and a plurality of original face feature points corresponding to the original face image; Based on the multiple original face feature points, determining multiple original face edge points from the face edge image; Obtaining facial adjustment information according to the plurality of original facial edge points and the adjustment parameter, wherein the adjustment parameter represents an adjustment range of the facial contour; According to the facial adjustment information, adjusting the facial contour of the original facial image to obtain a target facial image; The step of determining a plurality of original face edge points from the face edge image based on the plurality of original face feature points comprises: Connecting a plurality of pairs of adjacent two original face feature points among the plurality of original face feature points to obtain a plurality of connecting lines; For each of the multiple connecting lines, obtaining a plurality of pixel points in the face edge image whose distance from the connecting line is within a preset distance, and selecting a plurality of pixel points with the largest edge strength from the plurality of pixel points; The acquired plurality of pixel points with the largest edge strength are determined as the original face edge points.

2. The method according to claim 1, characterized in that Obtaining face adjustment information according to the plurality of original face edge points and the adjustment parameters, including: According to the positions of the plurality of original face edge points in the original face image, the plurality of original face edge points are divided into a plurality of groups of original face edge points; According to the multiple groups of original face edge points, edge function curves corresponding to the multiple groups of original face edge points are respectively fitted; According to the adjustment parameters, the edge function curves corresponding to the plurality of groups of original face edge points are adjusted respectively to obtain the face adjustment information.

3. The method according to claim 2, characterized in that According to the multiple groups of original face edge points, edge function curves corresponding to the multiple groups of original face edge points are respectively fitted, including: For each group of original face edge points, a line connecting two original face edge points at both ends of each group of original face edge points is used as an X-axis, and a perpendicular bisector of a line connecting two original face edge points at both ends of the group of original face edge points is used as a Y-axis, to establish a coordinate system corresponding to the group of original face edge points; Determine the coordinate points of each group of original face edge points in the corresponding coordinate system; Each group of original face edge points is fitted according to the coordinate points of each group of original face edge points in the corresponding coordinate system to obtain the edge function curve corresponding to each group of original face edge points.

4. The method according to any one of claims 1 to 3, characterized in that: Adjusting the original face image according to the face adjustment information to obtain a target face image includes: According to the face adjustment information, a plurality of adjusted face edge points corresponding to the plurality of original face edge points are obtained; Downsampling the plurality of adjusted face edge points according to a preset step length to obtain a plurality of adjusted sampling feature points; Acquire a plurality of original sampling feature points corresponding to the plurality of adjusted sampling feature points from the plurality of original face edge points; The original face image is adjusted according to the plurality of adjusted sampling feature points and the plurality of original sampling feature points to obtain a target face image.

5. The method according to claim 4, characterized in that Adjusting the original face image according to the plurality of adjusted sampling feature points and the plurality of original sampling feature points to obtain a target face image includes: Acquire first pixel points on both sides of the plurality of adjusted sampling feature points from the original face image; Acquire second pixel points on both sides of the plurality of original sampling feature points from the original face image, wherein the number of the first pixel points is equal to the number of the second pixel points; Constructing a first triangulated network according to the plurality of adjusted sampling feature points and the first pixel point; Constructing a second triangulated network according to the plurality of original sampling feature points and the second pixel points; The original face image is adjusted according to the first triangular network and the second triangular network to obtain a target face image.

6. The method according to claim 5, characterized in that Adjusting the original face image according to the first triangular network and the second triangular network to obtain a target face image includes: Establishing an association relationship between a plurality of first triangular sub-meshes included in the first triangular network and a plurality of second triangular sub-meshes included in the second triangular network; Acquire image contents within the plurality of second triangular sub-grids on the original face image; For the first triangular network area on the original face image, the image content in each second triangular sub-mesh is filled into the associated first triangular sub-mesh according to the association relationship to obtain a target face image.

7. A facial image processing device, characterized in that: The device comprises: An acquisition module is configured to acquire a face edge image and a plurality of original face feature points corresponding to the original face image; a determination module configured to determine a plurality of original face edge points from the face edge image based on the plurality of original face feature points; A fitting module, configured to obtain face adjustment information according to the plurality of original face edge points and adjustment parameters, wherein the adjustment parameters represent an adjustment range of the face contour; an adjustment module, configured to adjust the facial contour of the original facial image according to the facial adjustment information to obtain a target facial image; The determining module comprises: A first obtaining submodule is configured to connect multiple pairs of adjacent two original face feature points among the multiple original face feature points to obtain multiple connecting lines; A selection submodule is configured to obtain, for each of the multiple connecting lines, a plurality of pixel points in the face edge image whose distance from the connecting line is within a preset distance, and select a plurality of pixel points with the largest edge strength from the plurality of pixel points; The determination submodule is configured to determine the acquired multiple pixel points with the largest edge strength as the original face edge points.

8. The device according to claim 7, characterized in that The fitting module comprises: a grouping submodule, configured to divide the plurality of original face edge points into a plurality of groups of original face edge points according to positions of the plurality of original face edge points in the original face image; A fitting submodule is configured to fit, according to a plurality of groups of original face edge points, edge function curves corresponding to the plurality of groups of original face edge points respectively; The second obtaining submodule is configured to adjust the edge function curves corresponding to the plurality of groups of original face edge points respectively according to the adjustment parameters to obtain the face adjustment information.

9. The device according to claim 8, characterized in that The fitting submodule comprises: an establishing subunit configured to establish, for each group of original face edge points, a coordinate system corresponding to the group of original face edge points by taking a line connecting two original face edge points at two ends of each group of original face edge points as an X-axis and a perpendicular bisector of a line connecting two original face edge points at two ends of the group of original face edge points as a Y-axis; A determination subunit is configured to determine the coordinate points of each group of original face edge points in the corresponding coordinate system; The fitting subunit is configured to fit each group of original face edge points according to the coordinate points of each group of original face edge points in the corresponding coordinate system to obtain the edge function curve corresponding to each group of original face edge points.

10. The device according to any one of claims 7 to 9, characterized in that: The adjustment module comprises: a third obtaining submodule, configured to obtain, according to the face adjustment information, a plurality of adjusted face edge points corresponding to the plurality of original face edge points; A downsampling submodule is configured to downsample the plurality of adjusted face edge points according to a preset step length to obtain a plurality of adjusted sampling feature points; An acquisition submodule is configured to acquire a plurality of original sampling feature points corresponding to the plurality of adjusted sampling feature points from the plurality of original face edge points; The adjustment submodule is configured to adjust the original face image according to the multiple adjusted sampling feature points and the multiple original sampling feature points to obtain a target face image.

11. The device according to claim 10, characterized in that The adjustment submodule comprises: A first acquisition subunit is configured to acquire first pixel points on both sides of the plurality of adjusted sampling feature points from the original face image; A second acquisition subunit is configured to acquire second pixel points on both sides of the plurality of original sampling feature points from the original face image, wherein the number of the first pixel points is equal to the number of the second pixel points; A first construction subunit is configured to construct a first triangulated network according to the plurality of adjusted sampling feature points and the first pixel point; A second construction subunit is configured to construct a second triangulated network according to the plurality of original sampling feature points and the second pixel points; The adjustment subunit is configured to adjust the original face image according to the first triangular network and the second triangular network to obtain a target face image.

12. The device according to claim 11, characterized in that According to the adjustment sub-unit, including: Establishing a subnode, configured to establish an association relationship between a plurality of first triangular sub-meshes included in the first triangular network and a plurality of second triangular sub-meshes included in the second triangular network; An acquisition subnode is configured to acquire image contents within the plurality of second triangular sub-grids on the original face image; The filling subnode is configured to fill the image content in each second triangular sub-mesh into the associated first triangular sub-mesh according to the association relationship for the first triangular network area on the original face image, so as to obtain a target face image.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the facial image processing method as described in any one of claims 1 to 7.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the facial image processing method as described in any one of claims 1 to 6.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the face image processing method according to any one of claims 1 to 6 is implemented.

Images