Image processing method and device

Abstract

The present application proposes an image processing method and device, which includes: obtaining a target face image of a user; according to the target face image, allocating the fine-tuning weights required for each virtual micro-plastic surgery treatment to the user, and the fine-tuning weights are allocated according to the part feature information of the target part targeted by the virtual micro-plastic surgery in the target face image; according to the target face image and the fine-tuning weights corresponding to each virtual micro-plastic surgery, generating a face fine-tuning image corresponding to the target face image. The present application allocates fine-tuning weights suitable for the user according to the actual situation of the target part targeted by the virtual micro-plastic surgery. According to the fine-tuning weights, the virtual micro-plastic surgery of the target face image can make the image obtained by the micro-plastic surgery more natural and beautiful, and meet the aesthetic needs of the user. And the order of each virtual micro-plastic surgery treatment is limited to ensure that the effects of each virtual micro-plastic surgery are superimposed on each other, improve the overall effect of face fine-tuning, and avoid mutual incompatibility between each virtual micro-plastic surgery.

Description

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 3, 2020, with application number 202011396257.5 and application name “A Method and Device for Image Processing”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of image processing technology, and specifically relates to an image processing method and device. Background Art

[0003] At present, many APPs (Applications) used to process videos or images need to perform virtual micro-plastic surgery on the user's facial images, such as whitening, slimming the face, enlarging the eyes, eliminating wrinkles and dark circles, and other fine-tuning processing on the user's facial images.

[0004] In the virtual micro plastic surgery solutions provided in the related art, a fine-tuning degree is usually set, and the micro plastic surgery processing of the fine-tuning degree is performed on the user's face image. For example, a face thinning degree is set, and the face contour fine-tuning processing of the face thinning degree is performed on the user's face image.

[0005] However, in the related art, all users are processed according to the same fine-tuning degree, which causes the user's face image to be over-processed and appear fake and unnatural. Or, the user's face image is not processed enough and the fine-tuning effect satisfactory to the user cannot be achieved. Summary of the invention

[0006] The present application proposes an image processing method and device. The present application allocates fine-tuning weights suitable for the user according to the actual conditions of the target parts of the virtual micro-plastic surgery. The virtual micro-plastic surgery of the target face image according to the fine-tuning weights can make the image obtained by the micro-plastic surgery more natural and beautiful, and meet the aesthetic needs of the user. The order of each virtual micro-plastic surgery treatment is limited to ensure that the effects of each virtual micro-plastic surgery are superimposed on each other, improve the overall effect of face fine-tuning, and avoid mutual incompatibility between each virtual micro-plastic surgery treatment.

[0007] The first embodiment of the present application provides an image processing method, including:

[0008] Obtain the target face image of the user;

[0009] According to the target facial image, the user is assigned a fine-tuning weight required for each virtual micro-plastic surgery process, wherein the fine-tuning weight is assigned based on part feature information of a target part in the target facial image targeted by the virtual micro-plastic surgery;

[0010] A face fine-tuning image corresponding to the target face image is generated according to the target face image and the fine-tuning weights corresponding to each virtual micro-plastic surgery process.

[0011] The second aspect of the present application provides an image processing device, including:

[0012] An acquisition module is used to acquire a target face image of a user;

[0013] A weight allocation module, used to allocate a fine-tuning weight required for each virtual micro-plastic surgery to the user according to the target face image, wherein the fine-tuning weight is allocated according to part feature information of a target part in the target face image targeted by the virtual micro-plastic surgery;

[0014] A generation module is used to generate a face fine-tuning image corresponding to the target face image according to the target face image and the fine-tuning weights corresponding to each virtual micro-plastic surgery process.

[0015] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:

[0016] In the embodiment of the present application, a fine-tuning weight suitable for the user is allocated according to the actual situation of the target part targeted by the virtual micro-plastic surgery. The virtual micro-plastic surgery is performed on the target face image of the user according to the fine-tuning weight, so that the fine-tuned face image obtained by micro-plastic surgery can be more natural and beautiful, and meet the aesthetic needs of the user. In addition, the order of each virtual micro-plastic surgery is limited to ensure that the effects of each virtual micro-plastic surgery are superimposed on each other, greatly improving the overall effect of face fine-tuning and avoiding the problem of incompatibility between each virtual micro-plastic surgery.

[0017] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0019] In the attached picture:

[0020] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application is shown;

[0021] Figure 2 and 3A schematic diagram showing the before and after effects of blurring a face image in a skin resurfacing process provided by an embodiment of the present application;

[0022] Figure 4-7 A schematic diagram showing horizontal blurring and vertical blurring of a first center point pixel provided by an embodiment of the present application is shown;

[0023] Figure 8 and 9 A schematic diagram showing the effect of image inversion and blurring provided by an embodiment of the present application is shown;

[0024] Fig.10 A schematic diagram of a drag control provided by an embodiment of the present application is shown;

[0025] Fig.11 A schematic diagram of a first lookup table provided in an embodiment of the present application is shown;

[0026] Fig.12 A grayscale schematic diagram of a second lookup table provided in an embodiment of the present application is shown;

[0027] Fig.13 A schematic diagram showing color search in a second lookup table provided in an embodiment of the present application is shown;

[0028] Fig.14 A schematic diagram of facial key points located on a standard face image provided by an embodiment of the present application is shown;

[0029] Fig.15 A standard eye key point image after interpolation and point filling provided by an embodiment of the present application is shown;

[0030] Fig.16 A double eyelid material image provided by an embodiment of the present application is shown;

[0031] Fig.17 A standard eye region image after triangulation provided by an embodiment of the present application is shown;

[0032] Fig.18 A schematic diagram showing a comparison between a target face provided by an embodiment of the present application and a target face after double eyelids are added;

[0033] Fig.19 A grayscale schematic diagram of a third lookup table provided in an embodiment of the present application is shown;

[0034] Fig. 20 A grayscale schematic diagram of a tooth whitening treatment provided by an embodiment of the present application after a part of the mouth is deducted and dyed is shown;

[0035] Fig.21A schematic diagram of a template material formed based on a standard face image provided by an embodiment of the present application is shown;

[0036] Fig. 22 is a grayscale schematic diagram of an undereye bag image shown by a template material provided in an embodiment of the present disclosure;

[0037] Fig.23 A schematic diagram of adjusting the rotation angle provided by an embodiment of the present application is shown;

[0038] Fig.24 A schematic diagram of a translation operation of a key point of a target part provided by an embodiment of the present application is shown;

[0039] Fig.25 A schematic diagram of the structure of an image processing device provided by an embodiment of the present application is shown;

[0040] Fig.26 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;

[0041] Fig. 27 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0042] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0043] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0044] The following describes an image processing method and device according to an embodiment of the present application in conjunction with the accompanying drawings.

[0045] In the current related art, when fine-tuning the user's facial image, such as skin smoothing, whitening, face thinning, eye enlargement, wrinkle and dark circles removal, etc., a fine-tuning degree is usually set, and the user's facial image is subjected to micro-plastic surgery of the fine-tuning degree. For example, a face thinning degree is set, and the user's facial image is subjected to facial contour fine-tuning of the face thinning degree. Since all users are processed according to the same fine-tuning degree, the user's facial image is over-processed and appears false and unnatural. Alternatively, the user's facial image is not processed enough, and the fine-tuning effect that satisfies the user cannot be achieved.

[0046] In order to solve the above-mentioned problems in the related art, an embodiment of the present application provides an image processing method, which allocates fine-tuning weights suitable for the user according to the actual situation of the target part targeted by the virtual micro-plastic surgery. Performing virtual micro-plastic surgery on the user's target facial image according to the fine-tuning weight can make the fine-tuned facial image obtained by micro-plastic surgery more natural and beautiful, and in line with the user's aesthetic needs. Moreover, for a variety of virtual micro-plastic surgery such as skin grinding, whitening, double eyelids, face slimming, eliminating wrinkles and dark circles, the method also defines the order of each virtual micro-plastic surgery to ensure that the effects of each virtual micro-plastic surgery are superimposed on each other, greatly improving the overall effect of facial fine-tuning and avoiding the problem of incompatibility between each virtual micro-plastic surgery.

[0047] See also Figure 1 , the method specifically comprises the following steps:

[0048] Step 101: Obtain a target face image of a user.

[0049] The execution subject of the embodiment of the present application is a terminal, and an APP capable of processing videos or images is installed on the terminal. When the user uses the APP, the terminal can call the camera to collect the user's target face image. The user can also directly upload a photo containing a face as the target face image, or the target face image can also be downloaded from a web page. The target face image can be an image containing a face area, or it can be multiple images containing a face area. The target face image can contain one face or multiple faces.

[0050] After obtaining the target facial image, the terminal can perform virtual micro-plastic surgery locally, or send the target facial image to the server corresponding to the APP for virtual micro-plastic surgery.

[0051] Step 102: According to the target facial image, the user is assigned the fine-tuning weight required for each virtual micro-plastic surgery process, and the fine-tuning weight is assigned based on the part feature information of the target part targeted by the virtual micro-plastic surgery in the target facial image.

[0052] In an embodiment of the present application, the virtual micro-plastic surgery process may include one or more of a skin grinding process, a whitening process, a double eyelid process, a local color fine-tuning process, and a local shape fine-tuning process. Among them, the local color fine-tuning process includes a teeth whitening process, a facial vegetation removal process, a blood bag process, etc., and the facial vegetation may be eye bags and / or wrinkles, and the wrinkles may include nasolabial folds, forehead wrinkles, wrinkles at the corners of the eyes, etc. The blood bag process is used to process a face image without blood bags or with unclear blood bags, so that the face image has obvious blood bags. The local shape fine-tuning process includes an eye zoom process, a first shape fine-tuning process for one or more target parts of the face contour, nose wing, nose tip, nose length, chin, and cheekbones, and a second shape fine-tuning process for fine-tuning one or more of the lip thickness, philtrum, inner corner of the eye, outer corner of the eye, and upper eyelid.

[0053] After the terminal obtains the target facial image of the user, for any of the above-mentioned virtual micro-plastic surgery processes, the part feature information of the target part targeted by the virtual micro-plastic surgery is extracted from the target facial image, and the part feature information includes the key point coordinates and texture information of the target part. In the embodiment of the present application, the part feature information of each part type corresponding to each part is pre-set. After obtaining the part feature information of the target part corresponding to the current user, the similarity between the part feature information of the target part and the pre-set part feature information of each part type corresponding to the target part is calculated respectively according to the part feature information of the target part and the pre-set part feature information of each part type, and the part type with the highest similarity is determined as the part type to which the target part belongs. Obtain the preset weight corresponding to the part type. The obtained preset weight is determined as the fine-tuning weight required for performing virtual micro-plastic surgery on the target facial image.

[0054] For example, a person's facial contour can be divided into multiple types, such as standard face, inverted triangle face, square face, mixed face, round face, long face, regular triangle face, etc. Similarly, eyes, chin, nose, cheekbones, etc. can all be divided into multiple types. For any part, the preset weight corresponding to each type included in the part is set separately, such as setting the preset weight corresponding to the standard face to 0.29, the preset weight corresponding to the round face to 0.57, etc.

[0055] According to the actual situation of the target part of the virtual micro-plastic surgery, a fine-tuning weight suitable for the user is allocated. Subsequently, the user's target face image is subjected to virtual micro-plastic surgery based on the fine-tuning weight, which can make the face fine-tuning image obtained by micro-plastic surgery more natural and beautiful, and fit the user's aesthetic needs.

[0056] The fine-tuning weights required for performing each virtual micro-plastic surgery on the target facial image of the user are determined in turn by the above method. Then, the target facial image of the user is respectively subjected to each virtual micro-plastic surgery through the following operation of step 103 .

[0057] Step 103: Generate a face fine-tuning image corresponding to the target face image according to the target face image and the fine-tuning weights corresponding to each virtual micro-plastic surgery process.

[0058] In order to avoid incompatibility between virtual micro-plastic surgery processes, such as enlarging the eyes first and then applying double eyelids, resulting in the problem of uneven double eyelid positions. In the embodiment of the present application, the execution order of various virtual micro-plastic surgery processes is limited, and the virtual micro-plastic surgery processes of the target face image are completed through the following steps S1-S5, including:

[0059] S1: Perform skin smoothing on the target face image according to the fine-tuning weight corresponding to the skin smoothing process to obtain a first image.

[0060] Because if virtual micro-plastic surgery operations such as whitening and double eyelid treatment are performed first and then skin grinding treatment is performed, the skin grinding treatment will eliminate the effects of virtual micro-plastic surgery operations such as whitening and double eyelid treatment. Therefore, the embodiment of the present application puts the skin grinding treatment first and first performs skin grinding treatment on the target face image to avoid the skin grinding treatment affecting other virtual micro-plastic surgery.

[0061] Specifically, the skin resurfacing treatment is performed through the following steps S11-S14, including:

[0062] S11: performing a first fuzzy processing of a first number of times on the face area in the target face image to obtain a first fuzzy map, wherein the first fuzzy processing includes a horizontal fuzzy processing and a vertical fuzzy processing.

[0063] Among them, the present application does not specifically limit the first number, for example, it can be once or multiple times. In another way, the specific value of the first number can be determined according to the number of face areas or the user attributes reflected. Specifically, a preset image recognition neural network can be used to detect the characteristic parameters of the face area in the target face image, determine the user attribute reflected by the face area, and the user attribute is used to characterize at least one of the gender, age and skin color of the person; determine the first number according to the user attribute. Among them, the present application does not specifically limit the image recognition neural network. For example, it can be a convolutional neural network (CNN). Convolutional neural networks are a type of feedforward neural networks (Feedforward Neural Networks) that include convolution calculations and have a deep structure, and are one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities and can perform translation-invariant classification of input information according to their hierarchical structure. Thanks to the powerful feature representation ability of CNN (convolutional neural network) for images, it has achieved remarkable results in image classification, target detection, semantic segmentation and other fields.

[0064] Furthermore, the present application can use the CNN neural network model to detect the characteristic parameters of the face area in the original image, and then perform feature recognition on the face area to determine the user attributes reflected by the face area. Among them, the face area image needs to be input into the preset convolutional neural network model, and the output of the last fully connected layer (FC) of the convolutional neural network model is used as the recognition result of the characteristic data corresponding to the face area image.

[0065] For example, when the user attribute reflected by the face region is female, the first number can be set to multiple times, and when the user attribute reflected by the face region is male, the first number can be set to once. Alternatively, when the user attribute reflected by the face region is young, the first number can be set to multiple times, and when the user attribute reflected by the face region is elderly, the first number can be set to once.

[0066] In some other embodiments of the present application, the number of facial areas in the target facial image can be determined first; if the number of facial areas is determined to be at least two, the first facial area in the target facial image is subjected to the first blurring processing for a fourth time, and the second facial area in the target facial image is subjected to the first blurring processing for a fifth time, wherein the fourth time is different from the fifth time.

[0067] For example, when the number of users reflected by the face region is multiple, the first number can be set to multiple times, and when the number of users reflected by the face region is one, the first number can be set to once, and so on.

[0068] Furthermore, blurring the face area in the target face image is a way of filtering the target face image. Specifically, image blurring is an operation in image processing and computer vision, which mainly removes noise in the image.

[0069] Furthermore, in the embodiment of the present application, the pixel value of the pixel where the noise is located can be processed as an approximation of the value of its horizontal (and vertical) adjacent pixel. Specifically, the present application needs to determine the average value of several horizontal and vertical pixels adjacent to the pixel to be processed. In order to calculate the average value of all pixels in an area with the same number of rows and columns centered on the current pixel, and use the calculated average value as the pixel value of the point.

[0070] During the first blurring process, a first number of first adjacent pixels that are horizontally adjacent to the first center point pixel are determined; and a first number of second adjacent pixels that are vertically adjacent to the first center point pixel are determined; based on the first center point pixel and the first adjacent pixels, a horizontal blurring process is performed on the face area in the target face image; after the horizontal blurring process is performed on each pixel in the face area, a vertical blurring process is performed on the face area based on the first center point pixel and the second adjacent pixels.

[0071] The first number may be 2, 4, or 6, etc. The first center point pixel may be any pixel point in the face area of ​​the target face image.

[0072] In the horizontal blurring process, the mean of the sum of the pixel values ​​of the first number of first adjacent pixels is first calculated, and the mean is called the first pixel mean. The pixel value of the first center point pixel is changed to the first pixel mean. After the horizontal blurring process is performed on each first center point pixel, the vertical blurring process is performed on each first center point pixel. In the vertical blurring process, the mean of the sum of the pixel values ​​of the first number of second adjacent pixels is calculated, and the mean is called the second pixel mean. The pixel value of the first center point pixel is changed to the second pixel mean.

[0073] In the process of performing horizontal blur processing and vertical blur processing on the target face image, the present application does not specifically limit the order of the horizontal blur processing and the vertical blur processing. For example, the image may be firstly subjected to horizontal blur processing, and then the vertical blur processing may be performed on the image subjected to horizontal blur processing. Alternatively, the image may be first subjected to vertical blur processing, and then the horizontal blur processing may be performed on the image subjected to horizontal blur processing.

[0074] like Figure 2 and 3 As shown, it is a before and after effect diagram of performing the first number of first blur processing on the face area of ​​the target face image, wherein in the embodiment of the present application, the pixel value of the first center point pixel can be processed as an approximate value of the value of its horizontally adjacent pixel. And, the pixel value of the first center point pixel is processed as an approximate value of the value of its vertically adjacent pixel. Specifically, the present application needs to determine the average value of a number of horizontal pixels adjacent to the pixel to be processed (i.e., the first number of first adjacent pixels adjacent to the first center point pixel horizontally) and a number of adjacent vertical pixels (i.e., the second number of second adjacent pixels adjacent to the first center point pixel vertically). So that the average value of all pixels in an area with the same number of horizontal rows and vertical columns, centered on the current pixel, can be calculated. And the calculated average value result is used as the pixel value of the point.

[0075] For example, the first number is 3, and horizontal blur processing is used as an example. Figure 4 and 5 As shown, the pixel with a pixel value of 2 can be taken as the first center point pixel, and three first adjacent pixels on the left and right that are horizontally adjacent to the first center point pixel are determined, that is, three pixels with 1, 1, 1 on the left and three pixels with 1, 1, 1 on the right. The pixel value of the first center point pixel with a pixel value of 2 is changed to the average of the sum of the pixel values ​​of the first adjacent pixels adjacent to it (that is, the pixel value 1).

[0076] Similarly, Figure 6 and 7 As shown, the pixel with a pixel value of 2 can be taken as the first center point pixel, and three second adjacent pixels vertically adjacent to the first center point pixel are determined, namely, three pixels with 1, 1, 1 on the upper side and three pixels with 1, 1, 1 on the lower side. The pixel value of the second center point pixel with a pixel value of 2 is changed to the average of the sum of the pixel values ​​of the second adjacent pixels adjacent to it (i.e., the pixel value is 1).

[0077] In a possible implementation of the present application, before S11 (performing the first blur processing of the face area for the first number of times), the target face image containing the face area may be scaled by a first ratio. The target face image is first scaled by the first ratio to reduce the resolution of the target face image, and then the first blur processing is performed, which can further improve the blur effect. The first ratio is not specifically limited in the present application, for example, it can be one-half, one-third, etc.

[0078] S12: performing fusion processing on the first fuzzy image and the target face image to obtain a first image to be processed.

[0079] Image fusion refers to the process of synthesizing multiple images to generate a target image. In this application, after the face area in the target face image is subjected to the first fuzzy processing for the first time, image processing and computer technology can be used to maximize the extraction of favorable information in the first fuzzy image and the original image, and finally integrated into a high-quality first image to be processed, so as to improve the utilization rate of image information, improve the accuracy and reliability of computer interpretation, and enhance the spatial resolution and spectral resolution of the original image.

[0080] In some embodiments of the present application, the first fuzzy image can be directly fused with the target face image to obtain the first image to be processed.

[0081] In some other embodiments of the present application, the first blurred image may also be firstly subjected to negation processing. Negation processing is to reverse the intensity of the pixels in the image, so that the foreground in the image becomes the background, and the background becomes the foreground. In the embodiment of the present application, the first blurred image may be subjected to negation processing by changing the pixel mapping of the first blurred image. For example, the pixel value of the first blurred image may be changed from 0 to 255, from 1 to 254, ..., from 254 to 1, and from 255 to 0, etc. Thus, the first blurred image after negation processing is obtained.

[0082] Then, edge extraction is performed on the first blurred image after the inversion process to obtain the second image to be processed. Specifically, it can be obtained using the following formula: Dst = src - src1;

[0083] Wherein, Dst is the second image to be processed, src is the first blurred image, and src1 is the first blurred image after inversion processing.

[0084] In addition, in order to make the mean and standard deviation local threshold segmentation better separate the target and the background, the present application can take the mean difference threshold (var) as Dst1.r+Dst1.g+Dst1.a*0.333. Among them, Dst1=Dst*0.7, and Dst3=src(first fuzzy image).rgb. Dst3.a=var.

[0085] In addition, in order to eliminate the noise points in the first blurred image after the inversion processing, as shown in FIG. Figure 8 and 9 As shown, the present application also needs to perform a second fuzzy processing of a second number of times on the second image to be processed to obtain a third fuzzy image, and the second fuzzy processing includes the above-mentioned horizontal fuzzy processing and vertical fuzzy processing. The first fuzzy image, the third fuzzy image and the target face image are fused to obtain the first image to be processed.

[0086] Among them, the present application does not specifically limit the second number, for example, it can be once or multiple times. In another way, the specific value of the second number can be determined according to the number of face areas or the user attributes reflected. Furthermore, the second number in the present application can be the same as the first number or different. It should be noted that the method of performing the second blurring process on the second image to be processed in the present application is consistent with that in step S11, and will not be repeated here.

[0087] In the present application, the first fuzzy image, the third fuzzy image and the target face image are fused to obtain the first image to be processed, which can be achieved by the following steps: the first fuzzy image, the third fuzzy image and the target face image are fused to obtain a fused image; in the fused image, a first image area whose brightness value is greater than a preset parameter and a second image area whose brightness value is not greater than the preset parameter are determined; and the first image area composed of the original image and the second image area composed of the non-original image are merged as the first image to be processed.

[0088] Furthermore, since the target face image usually includes a face area image and a non-face area image (for example, if a user takes a selfie with his back against a wall, the wall behind the user is the non-face area image), in the process of resurfacing the user image, usually only the face area needs to be processed, that is, there is no need to resurface the non-face area. Therefore, in the embodiment of the present application, after the first blurred image, the third blurred image and the target face image are fused to obtain a fused image, the face part and the non-face part in the fused image can be distinguished.

[0089] Furthermore, the embodiment of the present application can determine the face part and non-face part in the image based on the brightness value of each image area where the above-mentioned fused image is located. It can be understood that since the third fuzzy image is obtained by performing a second fuzzy process on the second image to be processed, and the second image to be processed is an image obtained after the first fuzzy image is inverted (that is, the brightness of the processed image in the user's face part is poor). Therefore, the present application can determine the first image area (that is, the non-face part) whose brightness value is greater than the preset threshold in the fused image, and the second image area (face part) whose brightness value is not greater than the preset threshold. And the first image area in the target face image (that is, the non-face part in the target face image is retained), and the second image area in the image after the first fuzzy image and the third fuzzy image are merged as the first image to be processed (that is, the face part in the image after the first fuzzy image and the third fuzzy image are retained).

[0090] It should be noted that the present application does not specifically limit the preset threshold value, for example, it can be 10 cd / m, or 100 cd / m, etc.

[0091] S13: Perform mean fuzzification on the first fuzzy image to obtain a second fuzzy image.

[0092] The first fuzzy image may be scaled to a second ratio. The second ratio is not specifically limited in the present application, for example, it may be one-half, one-third, etc. In addition, the second ratio may be the same as the first ratio, or may be different from the first ratio.

[0093] Then, the first blurred image scaled by the second ratio is average blurred to obtain a second blurred image. Specifically, the second center point pixel corresponding to the scaled first blurred image is determined; a third number of third adjacent pixels adjacent to the second center point pixel is determined; the pixel value of the second center point pixel is changed to the third pixel average to obtain the second blurred image, where the third pixel average is the average of the sum of the pixel values ​​of the third adjacent pixels.

[0094] The present application can first determine the second center point pixel corresponding to the first blurred image. The present application does not specifically limit the number of the second center point pixels. For example, it can be two pixel points of the first blurred image, or multiple pixel points of the first blurred image, or all pixel points of the first blurred image.

[0095] In the embodiment of the present application, the pixel value of the pixel where the second center point pixel is located can be processed as the average value of the values ​​of all the adjacent pixels around it. Specifically, the present application needs to determine the sum of several pixel values ​​adjacent to the pixel to be processed. In order to subsequently take the current pixel as the center, read all the pixel points in a surrounding adjacent area to calculate the average value. And the calculated average value result is used as the pixel value of the point.

[0096] Among them, the embodiment of the present application can implement the process of performing mean blurring on the first fuzzy image to obtain the second fuzzy image according to the following formula:

[0097]

[0098] Among them, u and v are the adjacent pixels around the second center point pixel, r is the blur radius, and σ is the standard deviation of the normal distribution.

[0099] S14: performing image fusion on the target face image, the first image to be processed and the second blurred image according to the fine-tuning weight corresponding to the skin resurfacing process.

[0100] The present application can also perform image fusion on the target face image, the first image to be processed and the second blur image according to the fine-tuning weight corresponding to the skin smoothing process, so as to obtain the final fine-tuned face image after the skin smoothing process.

[0101] Specifically, the target face image, the first image to be processed and the second fuzzy image are fused to obtain a first fused image; and the sharpness of the first fused image is adjusted based on a preset sharpness parameter.

[0102] The present application adaptively adjusts the image sharpness parameter of the first fused image according to the preset sharpness parameter, so as to achieve the purpose of improving the image recognition effect of the face area. Specifically, the present application can determine the sharpness parameter according to the following formula:

[0103]

[0104] Among them, M is the sharpness parameter, Imax is the maximum brightness of the image, and Imin is the minimum brightness of the image.

[0105] The sharpness parameter can be calculated by dividing the difference between the maximum brightness value and the minimum brightness value of the image by their sum. For example, in an image, when Imax=255 and Imin=10, the sharpness parameter M=0.9245.

[0106] In other embodiments of the present application, during the process of adjusting the sharpness of the first fused image, the image size of the first fused image may also be adaptively adjusted, thereby achieving the purpose of improving the object recognition effect while ensuring the clarity of the object in the depth image as much as possible.

[0107] After the sharpness of the first fused image is adjusted in the above manner, the first fused image after the sharpness adjustment is fused with the target face image according to the fine-tuning weight corresponding to the skin smoothing process to obtain a second fused image. The second fused image can be used as the face fine-tuning image obtained by the skin smoothing process, and the fine-tuning image can be displayed to the user.

[0108] When the second fused image is displayed to the user, a drag control for adjusting the fine-tuning weight is also displayed to the user. The drag control includes a drag block and a drag bar. The drag bar is used to indicate the range of the fine-tuning weight that can be adjusted. The user can set a custom fine-tuning weight by dragging the drag block. Fig.10 As shown, the dragging range of the drag bar is 0-1. When the graphics card displays the second fused image, the fine-tuning weight indicated by the drag block on the drag bar is the fine-tuning weight corresponding to the skin smoothing process. If the user is not satisfied with the skin smoothing effect after seeing the displayed image, he can submit a custom weight by dragging the drag block.

[0109] If it is determined that a modification instruction generated by the user for the second fused image is received, the user-submitted custom weight is determined based on the modification instruction; the first fused image after sharpness adjustment is fused with the target face image according to the custom weight to obtain a third fused image. The third fused image is then displayed to the user.

[0110] For example, if the fine-tuning weight corresponding to the skin resurfacing process is 50%, the second fused image shows 50% of the original image and 50% of the first fused image after sharpness adjustment for each pixel. If the user is not satisfied with the second fused image, he or she can submit a custom weight. It can be understood that the custom weight is used to characterize the degree of skin resurfacing expected by the user. If the user believes that the degree of skin resurfacing of the second fused image is not enough, he or she can submit a custom weight such as 80%, and then the first fused image can be fused with the target face image at a ratio of 80% to obtain a third fused image (that is, 20% of the original image and 80% of the first fused image after sharpness adjustment are displayed for each pixel).

[0111] By applying the above-mentioned skin smoothing processing scheme, the target face image can be subjected to multiple fusions of different degrees with the target face image and the blurred image obtained by including horizontal blur and vertical blur, thereby obtaining a target image after accurate skin smoothing. This avoids the problem of long processing time caused by only using the mean blur method to process images for users in the related art.

[0112] S2: Perform whitening processing on the first image according to the fine-tuning weight corresponding to the whitening processing to obtain a second image.

[0113] After the target face image is subjected to skin smoothing processing through the operation of step S1, facial image data is extracted from the first image obtained through skin smoothing processing and written into a storage device. Then, the color values ​​of the pixels of the image data are replaced through a lookup table. By replacing the color values, the facial image data can be whitened.

[0114] First, let's introduce the lookup table. The lookup table (LUT) is essentially a RAM. After the data is written into the RAM in advance, every time a signal is input, it is equivalent to inputting an address to look up the table, find the content corresponding to the address, and then output it.

[0115] LUT has a wide range of applications. For example, LUT can be applied to a pixel grayscale mapping table. It transforms the actual sampled pixel grayscale value into another corresponding grayscale value after certain transformations such as threshold, inversion, binarization, contrast adjustment, linear transformation, etc., which can highlight the useful information of the image and enhance the light contrast of the image. Many PC series cards have 8 / 10 / 12 / 16 or even 32-bit lookup tables. The specific transformation in the lookup table is defined by the software. The most important significance of the lookup table here is that it is compatible with the high-level display functions of ordinary monitors, so that those wide color gamuts (generally referring to those exceeding sRGB) that cannot be displayed by ordinary monitors can be roughly simulated on ordinary monitors as much as possible. However, the effect simulated by the lookup table can only be used as a reference and as a general look and feel when retouching. The best and most complete presentation is still the high consistency of various indicators of the monitor, graphics card and the material itself, the so-called "hard decoding".

[0116] The lookup table is equivalent to a discrete function. Given an input value, an output value is obtained through the lookup table. The lookup table can be used in many places. In the field of color adjustment, it is called a color lookup table. The color lookup table has three components R, G, and B. The input components are called Rin, Gin, and Bin in this article; the output components are called Rout, Gout, and Bout. The input and output are in a one-to-one correspondence. The system finds the output value for each input value according to the corresponding relationship in the lookup table, thereby completing the color conversion.

[0117] The lookup table can be divided into 1D lookup table and 3D lookup table based on the lookup method. The conversion formula is as follows:

[0118] 1D Lookup Table:

[0119] R out =F LUT (R in )

[0120] G out =F LUT (G in )

[0121] B out =F LUT (B in )

[0122] From the formula, we can see that the RGB data in the 1D lookup table are independent of each other, and can only affect the brightness of R, G, and B respectively. The Gamma value can be controlled to balance the RGB color.

[0123] The amount of data contained in a system with a color depth of 10 bits is relatively small.

[0124] 3D Lookup Table:

[0125]

[0126]

[0127]

[0128] Judging from the formula, the 3D lookup table has a more profound impact. Each RGB color component is related to the three input components. The 3D lookup table can affect the hue, saturation, brightness, etc., and can almost affect every aspect of the picture.

[0129] The data contained in a system with a color depth of 10 bits is huge, so usually the 3D lookup table is stored by listing nodes, and the data between two nodes is calculated by the difference method, so the number of nodes is a sign of the accuracy of the 3D lookup table. The maximum number of nodes currently supported by the color management system is 257.

[0130] In the embodiment of the present application, the facial image data may include a facial skin image. First, all facial key points in the first image are identified. Based on the identified facial key points, the facial skin image is extracted from the first image. Then, the facial skin image is replaced with light and dark using a preset first lookup table to obtain replaced image data. The first lookup table is a one-dimensional color lookup table with a gradual gradient from black to white. Specifically, Fig.11 shown.

[0131] Specifically, for each pixel in the facial skin image, a color value of the pixel is obtained from the facial skin image, where the color value of the pixel is a value between 0 and 255. The color value is determined as the one-dimensional coordinate to be searched. The color value at the one-dimensional coordinate is searched in a first lookup table. The color value of the pixel in the facial skin image is replaced with the color value at the one-dimensional coordinate found.

[0132] For example, if the color value of a pixel in the facial skin image is x, then the corresponding one-dimensional coordinate is (x, 1.0). Look up the color value y at the coordinate (x, 1.0) in the first lookup table. Replace the color value of the pixel in the facial skin image from x to y.

[0133] Then, the original color value of the pixel in the first image and the replaced color value are first fused. Specifically, the first fusion is performed according to the following formula:

[0134] C2=C*a+C1*(1-a)

[0135] Wherein, C2 represents the color value of the pixel after the first fusion, C1 represents the color value of the pixel after the facial skin image is replaced, and C represents the original color value of the pixel in the facial skin image. a is a weighting coefficient, and the value of a can be 0.5, 0.6, etc.

[0136] By the above method, the color value of each pixel in the facial skin image is replaced with the corresponding color value found in the first lookup table. Then, according to the above weighted fusion formula, the replaced facial skin image is fused with the original facial skin image, so that the color value of each pixel in the fused facial skin image is the weighted sum of the replaced color value and the color value before the replacement. The facial skin image is replaced with light and dark by the first lookup table, the brightness of the facial skin image is improved, and the effect of brightening the facial skin is achieved.

[0137] After the above fusion process, the fused facial skin image is fused with the weight coefficient. Specifically, the color values ​​of the pixels in the facial skin image in the first image after the first fusion process and the color values ​​of the pixels in the facial skin image in the original first image are fused for the second time according to the fine-tuning weights corresponding to the whitening process.

[0138] Specifically, the second fusion is performed according to the following formula:

[0139] C3=C*(1.0-D)+C2*D

[0140] Among them, D represents the fine-tuning weight corresponding to the whitening process, C2 represents the color value of the pixel after the first fusion, and C3 represents the color value of the pixel after the second fusion.

[0141] After brightening the facial skin image through the first lookup table, the color of the original facial skin image is merged with the brightened color according to the fine-tuning weight corresponding to the whitening process. In this way, the facial skin image can be whitened to a certain extent through the fine-tuning weight. The fine-tuning weight corresponding to the whitening process is equivalent to the whitening degree. The value range of the fine-tuning weight is 0-1, 0 means no whitening, and 1 means the maximum degree of whitening.

[0142] After the above-mentioned first fusion and second fusion processing, the color of the image after the second fusion processing is adjusted using a preset second lookup table. It should be noted that the second lookup table referred to in the present disclosure is a two-dimensional color lookup table composed of multiple color images, such as a lookup table composed of multiple color images arranged four by four. The upper left vertex of the color image located in the upper left corner of the second lookup table is black, and the lower right vertex of the color image located in the lower right corner is white. In the second lookup table, the color images are arranged in a gradient order from dark to light from left to right and from top to bottom, as shown in the following example. Fig.12 shown.

[0143] For each pixel in the facial skin image after the second fusion process, the RGB three-channel color value of the pixel is obtained from the facial skin image. According to the color value of the RGB three-channel of the pixel, the corresponding three-channel color value is queried in the second lookup table. That is, according to the value of the R channel of the pixel, the corresponding R channel value is queried in the second lookup table. According to the value of the G channel of the pixel, the corresponding G channel value is queried in the second lookup table. And, according to the value of the B channel of the pixel, the corresponding B channel value is queried in the second lookup table.

[0144] like Fig.13 As shown, RGB output = LUT (R input, G input, B input). If the input RGB three-channel values ​​are (50,50,50), the output three-channel values ​​are (70,70,70). If the input three-channel values ​​are (50,70,50), the output three-channel values ​​are (85,90,70). If the input three-channel values ​​are (50,70,60), the output three-channel values ​​are (90,95,80).

[0145] For each pixel in the facial skin image after the second fusion processing, the corresponding three-channel value is queried in the second lookup table in the above manner, and then the current three-channel color value of the pixel in the facial skin image after the second fusion processing is replaced by the queried three-channel color value.

[0146] The color values ​​of the pixels in the facial skin image after the second fusion process are replaced by the second lookup table to further whiten the facial skin image. The multiple color images arranged in the second lookup table can adjust the red and yellow colors in the facial skin image, and the whitening effect is better for facial skin images with reddish and yellowish colors.

[0147] The facial skin image is whitened by the operation of this step. The facial skin image obtained by the final processing is then merged with the facial features image segmented previously to form a face image, that is, a face image after face whitening is obtained. In this way, only the facial skin is whitened, and the color of the eyes, eyebrows, lips, hair and other areas will not be changed, making the facial whitening treatment more targeted and the whitening effect more natural.

[0148] This step uses the lookup table technology to adjust the brightness and replace the color of the facial image taken by the user, thereby providing a more realistic whitening effect.

[0149] S3: If it is detected that the eyes in the target face image are single eyelids, the eyes of the user are adjusted to double eyelids in the second image according to the fine-tuning weights corresponding to the double eyelid processing, so as to obtain a third image.

[0150] In the embodiment of the present application, after the target face image is subjected to skin grinding and whitening processing in steps S1 and S2, the eyelid type of the user's eyes in the second image obtained by the whitening processing is determined. If the user has double eyelids, the double eyelid processing of this step is not required, and the operation of the following step S4 is directly entered. If the user has single eyelids, the user's eyes are adjusted to double eyelids in the second image through the processing of this step.

[0151] First, obtain the standard eye key points, triangulation mesh and standard double eyelid material based on the standard face image. Among them, the standard face image is a face image with regular features, unobstructed face and eyes looking straight ahead. Fig.14 As shown, the facial key points located on the standard face image may include points around the face, around the eyes, around the nose triangle, around the mouth, around the eyebrows, etc., wherein the located facial key points include standard eye key points, including two key points at the left and right corners of the eyes, a key point at the highest point of the upper eyelid and two key points on the left and right sides of the key point, a key point at the lowest point of the lower eyelid and two key points on the left and right sides of the key point.

[0152] The above-mentioned standard eye key points are interpolated to generate new standard eye key points. Specifically, the standard eye key points can be first obtained from the located key points, and the preset interpolation algorithm is used to interpolate the standard eye key points, and then the interpolated standard eye key points are used as the starting point to move in the preset direction to generate new key points, and then the interpolated standard eye key points and the generated new key points are used as new standard eye key points, so that the eye points are denser, which is convenient for the subsequent integration of the standard double eyelid material into the eye of the target image. Among them, the preset interpolation algorithm can use a cubic Bezier curve, and the formula is as follows:

[0153] B3(t)=(1-t) 3 P0+3t(1-t) 2 P1+3t 2 (1-t)P2+t 3 P3,t∈[0,1]

[0154] Among them, P0, P1, P2, and P3 are four consecutive standard eye key points, and t is the interpolation coefficient. That is to say, four consecutive standard eye key points can be interpolated to obtain a new standard eye key point each time.

[0155] Move the interpolated standard eye key points to generate new key points. The moving formula is as follows:

[0156] P0'=P0+Dir*len

[0157] Among them, P0 is the standard eye key point (i.e., the original point), Dir is the unit vector of the moving direction, and len is the moving distance.

[0158] In one example, if Fig.15 The figure shows the standard eye key point image after interpolation. Specifically, the key point located on the upper eyelid is moved upward by a preset distance, the key point located on the lower eyelid is moved downward by a preset distance, and the key point located at the corner of the eye is moved laterally outward by a preset distance. It is worth noting that the upward and downward movement distances do not exceed the distance between the eyebrow and the eye.

[0159] The standard eye image corresponding to the new standard eye key point is intercepted from the standard face image. The standard double eyelid material is produced using the standard eye image as a template. In one embodiment, the standard eye image can be output and displayed, and then the standard double eyelid material drawn by the user on the standard eye image is received, and the standard double eyelid material is blurred and shaded gradually, so that the combination of the double eyelid material and the standard eye is more realistic, and then the double eyelid material is fused to the eye of the target image. The effect will be more fitting and followable, see Fig.16 Shown is the generated double eyelid material.

[0160] Among them, except for the pixels at the double eyelid position in the standard double eyelid material, the pixels at other positions are transparent, so that when merging with the eye area of ​​the target image, only the pixels at the double eyelid position are fused with the target image, and the pixels at other positions do not affect the pixels in the target image.

[0161] In the embodiment of the present application, a triangulation method may also be used to generate a triangulation mesh of a standard eye key point, which includes a plurality of triangles, each vertex of which is a standard eye key point, such as Fig.17 The figure shows a standard eye region image after triangulation.

[0162] After obtaining the standard eye key points, triangulation grid and standard double eyelid material based on the standard face image in the above manner, locate the face key points in the second image obtained by the above whitening process, and interpolate and fill in the eye key points included in the face key points in the second image to generate new eye key points. The interpolation and filling in here are the same as the interpolation and filling in the standard eye key points, which will not be repeated here.

[0163] Then, according to the fine-tuning weights corresponding to the double-eyelid processing and the new eye key points, the pre-made standard double-eyelid material is fused with the eye region in the second image to obtain a fused eye region image.

[0164] The fusion process may include: establishing a mapping relationship between a new standard eye key point and a new eye key point, and fusing the standard double eyelid material to the eye area in the second image according to the mapping relationship. Since the standard double eyelid material is made based on the standard eye image, the standard double eyelid material can be better fused to the eye of the second image according to the mapping relationship between the standard eye key point and the eye key point.

[0165] Among them, since the number of the new standard eye key points and the new eye key points is the same, a one-to-one mapping relationship can be established between the two, for example, the key points on the upper eyelid of the standard eye correspond to the key points on the upper eyelid in the second image. In one example, an index number can be assigned to each key point of the standard eye and each key point of the eye in the second image in a preset order, and a mapping relationship can be established between two key points with the same index number.

[0166] In some embodiments, the process of fusion according to the mapping relationship may include: according to the mapping relationship, obtaining the pixel points covered by the triangulated grid on the eye area in the second image, and then for each pixel point obtained, according to the fine-tuning weight corresponding to the double eyelid processing, the first pixel value of the pixel point and the second pixel value of the pixel point corresponding to the standard double eyelid material, calculating the target pixel value of the pixel point. Since the triangulated grid is established by the standard eye key points, and there is a one-to-one mapping relationship between the standard eye key points and the eye key points in the second image, the pixel points covered by the triangulated grid on the eye area of ​​the second image can be obtained according to the mapping relationship, and these pixel points are fused to ensure that the double eyelid effect obtained is more realistic.

[0167] The target pixel value of the pixel point is calculated according to the fine-tuning weight corresponding to the double-eyelid processing, the first pixel value and the second pixel value. The calculation formula is as follows:

[0168] P 目标 =α*(P 标准眼 *P 目标眼部 ) / 255

[0169] Among them, α is the fine-tuning weight corresponding to the double eyelid processing, P 标准眼 is the second pixel value, P 目标眼部 In one example, since the standard double eyelid material is made by using the standard eye as a template, the standard eye triangulation grid can also be pasted into the standard double eyelid material, and then the pixel points covered by the standard eye triangulation grid in the eye area of ​​the second image correspond to the pixel positions on the standard double eyelid material according to the triangles in the grid to which they belong.

[0170] In another example, the fine-tuning weight corresponding to the double eyelid processing can also be adjusted by the user to meet the user's actual adjustment needs, so it can also be adjusted by Fig.10 After receiving the custom weight input by the user, the drag control shown calculates the target pixel value of the pixel point according to the received custom weight, the first pixel value and the second pixel value.

[0171] During the pixel fusion process, since the pixels at the double eyelid position in the standard double eyelid material are transparent except for the pixels at the double eyelid position, the pixels at other positions are transparent. Therefore, the pixels at the double eyelid position can be fused to the eye pixels of the second image, and the pixels at other positions will not change the eye pixels of the second image.

[0172] In an exemplary scenario, see Fig.18 As shown, Figure (a) is a collected target image face, whose eyes are of single eyelid type. After being processed by the above embodiment scheme, the double eyelid effect image in Figure (b) is obtained, which is realistic and lifelike.

[0173] Through the above processing flow, on the basis of locating the eye key points of the face, the eye key points are further interpolated and supplemented to make the eye points more dense. Therefore, according to the newly generated eye key points, the pre-made standard double-eyelid material can be better integrated into the eyes of the face image, so that the generated double-eyelids are more fitting and followable, and the double-eyelid effect simulated by using the standard double-eyelid material will be more realistic and lifelike.

[0174] S4: If it is detected that the target part targeted by the local color fine-tuning processing in the target face image meets the preset color fine-tuning processing conditions, the target part targeted by the local color fine-tuning processing is adjusted in the third image according to the fine-tuning weight corresponding to the local color fine-tuning processing to obtain a fourth image.

[0175] If the user in the target face image has single eyelids, the third image is obtained by double eyelid processing in step S3, and then the operation in this step is performed. If the user in the target face image has double eyelids, the operation in step S3 is not performed, and the operation in this step is directly performed based on the second image obtained by whitening in step S2.

[0176] The local color fine-tuning process in this step may include teeth whitening, facial vegetation removal, eye bags, etc. The facial vegetation may be eye bags and / or wrinkles, and the wrinkles may include nasolabial folds, forehead wrinkles, eye wrinkles, etc. The eye bags are used to process a face image without eye bags or with inconspicuous eye bags, so that the face image has obvious eye bags. Among them, the local color fine-tuning processes such as teeth whitening, facial vegetation removal, eye bags, etc. may be processed in sequence in any order, or may be processed in parallel. The process of each local color fine-tuning process is described in sequence below.

[0177] The above-mentioned preset color fine-tuning processing conditions include trigger conditions corresponding to local color fine-tuning processing such as teeth whitening processing, facial vegetation removal processing, and eye bags processing. Among them, the trigger condition for teeth whitening processing can be that the lips are open in the face image to be processed, and the texture features of the lips can be identified by face recognition technology to determine whether the lips are open. The trigger condition for facial vegetation removal processing can be that the face image to be processed has facial vegetation such as bags under the eyes, nasolabial folds, forehead wrinkles, and wrinkles at the corners of the eyes, and face recognition technology can be used to identify whether these facial vegetation exist in the face image. The trigger condition for eye bags processing can be that there are no eye bags in the face image to be processed or the width of the eye bags is less than a preset value. Similarly, face recognition technology can be used to identify whether there are eye bags in the face image and the width of the eye bags.

[0178] The following is a detailed description of the processes of teeth whitening, facial growth removal, and under-eye bags treatment.

[0179] The teeth whitening process is similar to the facial skin whitening process described in step S2. First, according to the facial features of the human face, the mouth image is extracted from the image to be processed separately, and the extracted mouth image is stained. The image to be processed can be the second image obtained by the whitening process or the third image obtained by the double eyelid process.

[0180] Then, the mouth image is cut out, and the color of the mouth image data after the cut out is adjusted according to the preset third lookup table and the fine-tuning weight corresponding to the teeth whitening process. The third lookup table is arranged in the same way as the second lookup table, and both are lookup tables composed of multiple four-by-four color images. However, the third lookup table for each color image is different from the second lookup table. The grayscale image of the third lookup table is as follows: Fig.19 If the third lookup table is applied to the above-mentioned face whitening technical solution, the entire screen will appear white and the face of the person cannot be recognized. Since the teeth are white in color, the third lookup table, which is a specialized lookup table, needs to be used.

[0181] In this solution, the user extracts the mouth image from the face image according to the facial features of the face. After the mouth image is extracted, the extracted mouth image is dyed, such as pink or red. Fig. 20 The grayscale image is shown after the pink dyeing process. After the dyeing process, the mouth part image is cut out from the whole face image by using the image cutout, and the color is adjusted by the third lookup table.

[0182] Specifically, the teeth whitening process is to obtain a face image to be processed, identify and locate key point data in the face image. The key point data in the face image can be the face key points detected by a preset face key point detection model, including key points in areas such as face contour, eyes, nose, mouth, eyebrows, etc. According to the identified key point data, the mouth area is determined from the face image. A template material corresponding to the pre-made lips is obtained, and the template material is a mouth image of a preset color, and the preset color can be pink, red, etc. According to the pixel coordinates of the mouth area in the face image, the template material is attached to the blank texture by indexing, so that the position of the template material in the blank texture map is the same as the position of the above-mentioned mouth area in the face image, so that the mouth image is dyed by the template material.

[0183] Then, based on the texture data of the template material, teeth whitening is performed on the mouth area in the face image corresponding to the template material. First, based on the texture data of the template material, the color value corresponding to each pixel in the mouth area is queried in the third lookup table. Then, the current color value of each pixel in the mouth area in the face image is replaced by the color value corresponding to each pixel obtained by the query.

[0184] Specifically, a channel value of a preset color channel of a first pixel is obtained from a blank texture to which a template material is attached, and the first pixel is any pixel in the template material. The preset color channel may be any one of the three channels of R, G, and B. According to the obtained channel value, a first coordinate and a second coordinate corresponding to the first pixel are calculated. A first color value of the RGB three channels corresponding to the first coordinate and a second color value of the RGB three channels corresponding to the second coordinate are determined. A third color value corresponding to the first color value and a fourth color value corresponding to the second color value are respectively queried in a third lookup table. The average color value of the third color value and the fourth color value found is determined as the color value after whitening corresponding to the first pixel.

[0185] The above method is used to determine the color value of each pixel of the template material in the blank texture image after whitening. Then, for each pixel in the mouth area of ​​the face image, the current pixel value of the pixel is replaced with the color value of the pixel with the same coordinates in the template material after whitening.

[0186] In order to facilitate understanding of the above process of searching for the color value corresponding to the pixel point through the third lookup table, a specific example is used to illustrate. Assuming that the preset color channel is the B channel, the B channel value of the first pixel point is obtained from the blank texture map with the template material attached, and the first coordinate and the second coordinate corresponding to the first pixel point are calculated based on the B channel value. First, calculate B1=B / 255*15, then calculate Y=floor(B1)*0.2, X=floor(B1)-Y*4, V=(X, Y) as the first coordinate corresponding to the first pixel point. Calculate Y1=ceil(B1)*0.2, X1=ceil(B1)-Y*4, V1=(X1, Y1) as the second coordinate corresponding to the first pixel point. The three-channel color values ​​R, G, and B of the first pixel point are used as the color values ​​of the first coordinate (X, Y). Calculate R1=R*0.234375+0.0078125, G1=G*0.234375+0.0078125, and use R1, G1, and B1 as the color values ​​of the second coordinate (X1, Y1). Let RG(R1, G1), texPos1.XY=V*0.25+RG, and texPos2.X1Y1=V1*0.25+RG. Take texPos1.XY and texPos2.X1Y1 as the coordinates to be queried, query the color value COLOR=RGBA(texPos1.XY) at texPos1.XY in the third lookup table, and query the color value COLOR1=RGBA(texPos2.X1Y1) at texPos2.X1Y1. Calculate (COLOR+COLOR1) / 2, and use the calculated value as the color value after whitening corresponding to the first pixel. Then, the color value of the pixel point in the mouth area of ​​the face image whose coordinates are the same as the coordinates of the first pixel point is modified to (COLOR+COLOR1) / 2.

[0187] For the face image to be processed, the facial skin is whitened by the method of step S2, and the teeth whitening treatment is performed on the mouth area by the method of step S4, thereby improving the facial whitening effect of the face image. In the present application, the facial skin image is brightened by the first lookup table, and the color of the facial skin image is adjusted by the second lookup table, so that only the facial skin area is whitened, which will not affect the color of the entire image background, nor will it change the color of the eyes, hair, eyebrows, lips and other parts. And the mouth area is extracted from the face image, and the teeth whitening treatment is performed only on the mouth area by the third lookup table, which will not affect other areas of the face. The skin area and the teeth area are whitened separately, so that the whitened image has no filter feeling, the whitening effect is very good, and it is more natural visually.

[0188] For facial vegetation removal, facial vegetation includes eye bags and / or wrinkles, and wrinkles include nasolabial folds, forehead wrinkles, or wrinkles at the corners of the eyes. First, a first template material generated in advance is read, and the template material shows a target area of ​​a standard human face. The target area may include but is not limited to the area where the eye bags are located and / or the area where the nasolabial folds, forehead wrinkles, or wrinkles at the corners of the eyes are located.

[0189] Among them, in the process of generating the first template material, key point data can be extracted from the standard face image. And triangulation processing is performed based on the key point data to generate grid data. The method of obtaining the key point data and the grid data is the same as the method of obtaining the facial key point data and the triangulation grid from the standard face image in the double eyelid processing process mentioned above, and will not be repeated here. The first template material is made with the standard face image as a template. The first template material can be automatically extracted or manually drawn by a person; the first template material shows the location of wrinkles and / or eye bags in the standard face image.

[0190] The first template material is drawn in a blank texture by indexing to obtain texture data of the template material, so that the first template material corresponds to the target area for facial growth removal in the third image. Specifically, the grid data obtained based on the standard face image is drawn in the blank image, and then the first template material is drawn in the blank image according to the mapping relationship between the key points included in the first template material and the key points in the grid data, so that each key point in the first template material coincides with the corresponding key point in the grid data, and the texture data of the first template material is obtained.

[0191] Therefore, the texture data of the first template material obtained in the present application can indicate the location of the target area on the image to be processed, for example, it can indicate the location of the nasolabial folds and eye bags that need to be processed on the user's face image. It is understandable that the input texture in the present application can be a 2D texture. Of course, 3D textures or 1D textures can also be used under the guidance of the technical solution of the present application to express the details of target areas such as nasolabial folds and / or eye bags. The key point data as the coordinate data of the input texture can be used to indicate which part of the image to be processed is sampled from. The present application determines the vertex coordinates required by the rendering pipeline based on the key point data. The vertex coordinates are in the world coordinate system and are used to indicate the location of the pixel shading after sampling.

[0192] like Fig.21 As shown, in one or more embodiments of the present application, the target area in the image to be processed corresponding to the first template material may be the area where eye bags are located and / or the area where facial growths such as nasolabial folds are located.

[0193] Finally, according to the fine-tuning weight corresponding to the facial growth removal process, the target area targeted by the facial growth removal process in the background texture of the image to be processed corresponding to the texture data of the first template material is blurred. The background texture of the image to be processed is the image itself of the image to be processed. The image to be processed can be the third image obtained by the double eyelid process in step S3, or the second image obtained by the whitening process in step S2, or any face image that needs to remove facial growths. Specifically, the target area targeted by the facial growth removal process in the image data to be processed is indexed according to the texture data of the first template material to locate the target area from the image to be processed, and the target area is first blurred to obtain the first face image. Then, according to the fine-tuning weight corresponding to the facial growth removal process, the blur degree of the target area on the first face image is adjusted to obtain the second face image. The second face image is the final image obtained by the facial growth removal process, and the second face image is displayed.

[0194] The blurring processing method may include but is not limited to Gaussian blurring processing, so as to achieve "skin smoothing" processing of the target area. The image after facial growth removal processing can be directly presented to the user. If the user is not satisfied with the processing effect, the user can Fig.10 The drag control shown submits a custom weight, receives the custom weight submitted by the user, and adjusts the blurriness of the target area according to the custom weight to meet the user's aesthetic needs.

[0195] For the eye bags processing, firstly obtain the second template material, which can be an eye bags image drawn based on a standard face image. Fig. 22 The grayscale image of the second template material shown in the second template material has a beautifying effect of light and dark protrusions. In this way, the processed image obtained by performing image fusion processing on the under-eye bag image and the target face image is more natural, has a better beautifying effect, and improves the user experience.

[0196] In the process of generating the second template material, key point data can be extracted from the standard face image. And triangulation processing is performed based on the key point data to generate grid data. The method of obtaining the key point data and the grid data is the same as the method of obtaining the face key point data and the triangulation grid from the standard face image in the double eyelid processing process mentioned above, and will not be repeated here. The second template material is made with the standard face image as a template. The second template material can be drawn automatically or manually; the second template material reflects the convex effect of the eye bags through the combination of any two colors in the three RGB color channels, and the specific combination of the two colors is: the upper and lower edge lines of the eye bags are two colors respectively, and one color gradually changes from dark to light from the edge line where it is located to the edge line of another color, and the light and dark visual effect is achieved through this color gradient, thereby visually presenting a convex eye bags effect.

[0197] After obtaining the second template material, the second template material is drawn into the blank texture by indexing to obtain the texture data of the template material, so that the second template material corresponds to the target area for the eyebag treatment in the image to be processed, and the texture data of the template material is used to index the key point data of the image to be processed. The image to be processed can be the third image obtained by the double eyelid treatment in step S3, or the second image obtained by the whitening treatment in step S2, or any face image that needs to remove facial growths.

[0198] Specifically, the grid data obtained based on the standard face image is drawn in the blank image, and then the second template material is drawn in the blank image according to the mapping relationship between the key points included in the second template material and the key points in the grid data, so that each key point in the second template material coincides with the corresponding key point in the grid data, and the texture data of the second template material is obtained. Since the vertex of each grid in the grid data exists in both the standard face image and the image to be processed, the texture data of the second template material obtained above can index the key point data in the image to be processed.

[0199] According to the fine-tuning weight corresponding to the eyebag treatment, at least two color channels of the target area corresponding to the background texture of the image to be processed are weighted fused based on the texture data of the second template material to obtain and output the image after the eyebag treatment. The background texture of the image to be processed is the image to be processed itself. Specifically, the target area targeted by the eyebag treatment in the image data to be processed is indexed according to the texture data of the second template material to locate the target area from the image to be processed.

[0200] According to the fine-tuning weight corresponding to the undereye bag processing, the texture data of the second template material is fused with the target area in the background texture of the image to be processed in the first color channel and the second color channel respectively.

[0201] Among them, the first color channel can be a blue channel. For each pixel in the target area, the first color value of the blue channel of the pixel is obtained from the target area in the background texture of the image to be processed, and the second color value of the blue channel of the pixel is obtained from the texture data of the second template material in the target area. According to the fine-tuning weight, the first color value and the second color value corresponding to the eyebrow processing, the first target color value of the blue channel of the pixel after fusion is calculated. Specifically, the following formula is used to perform color fusion of the blue channel:

[0202] color2=color1 * n+color1 * m * B * Alp1;

[0203] m = 1-n;

[0204] Alp1=1.6;

[0205] Among them, color2 is the color value of the blue channel of the pixel point in the target area of ​​the image to be processed after the above color processing, n is the fine-tuning weight corresponding to the eyebag processing, color1 is the first color value of the blue channel of the pixel point in the target area of ​​the image to be processed, B is the second color value of the blue channel of the pixel point in the target area in the texture data of the second template material, and Alp1 is the color space.

[0206] The second color channel can be a green channel. For each pixel in the target area, the third color value of the green channel of the pixel is obtained from the background texture of the image to be processed in the target area, and the fourth color value of the green channel of the pixel is obtained from the texture data of the second template material in the target area. According to the fine-tuning weight, the third color value and the fourth color value corresponding to the eyebrow silkworm processing, the second target color value of the green channel of the pixel after fusion is calculated. Specifically, the following formula is used to perform color fusion of the green channel:

[0207] color3=color1 * n+color1 * m * G * Alp2;

[0208] m = 1-n;

[0209] Alp2=1.3;

[0210] Among them, color3 is the color value of the green channel of the pixel point in the target area of ​​the image to be processed after the above color processing, n is the fine-tuning weight corresponding to the eyebag processing, color1 is the third color value of the green channel of the pixel point in the target area of ​​the image to be processed, G is the fourth color value of the green channel of the pixel point in the target area in the texture data of the second template material, and Alp2 is the color space.

[0211] For each pixel in the target area, the first target color value of the blue channel and the second target color value of the green channel corresponding to each pixel after fusion are calculated in the above manner. Then, image rendering is performed based on the calculated pixel values ​​of the blue channel and the green channel of each pixel in the target area to obtain a face image after eyebag processing.

[0212] In the embodiment of the present application, the first color channel and the second color channel are not limited to blue and green, and can be a combination of any two color channels of the three primary colors of RGB.

[0213] After the above fusion process, the fused image is displayed to the user. If the user is not satisfied with the effect, the user can Fig.10 The drag control shown submits the custom weight. The custom weight of the user is received, and the texture data of the template material and the background texture of the image to be processed are re-colored in the above manner.

[0214] The eyebag processing process can accurately determine the target area of ​​eyebag processing in the texture data of the image to be processed and the second template material, and realize the light and dark visual effect through the gradual change of color in the second template material, thereby visually presenting a raised eyebag effect. The image after eyebag processing finally obtained has a better beauty effect, which improves the user experience.

[0215] An image obtained by any of the above local color fine-tuning processes may be referred to as a fourth image.

[0216] S5: According to the fine-tuning weight corresponding to the local shape fine-tuning process, shape adjustment is performed on the target part targeted by the local shape fine-tuning process in the fourth image to obtain a fine-tuned face image corresponding to the target face image.

[0217] In an embodiment of the present application, the local shape fine-tuning processing includes eye scaling processing, a first shape fine-tuning processing of one or more target parts including facial contour, nose wing, nose tip, nose length, chin, and cheekbones, and a second shape fine-tuning processing of fine-tuning one or more of lip thickness, philtrum, inner canthus, outer canthus, and upper eyelid.

[0218] If the user in the target face image has single eyelids, the third image is obtained by double eyelid processing in step S3, and then the operation in this step is performed. If the user in the target face image has double eyelids, the operation in step S3 is not performed, and the operation in this step is directly performed based on the second image obtained by whitening in step S2.

[0219] For the first shape fine-tuning process of one or more target parts of the facial contour, nose wing, nose tip, nose length, chin, and cheekbone, firstly, the coordinates of the adjustment points corresponding to the first shape fine-tuning process are obtained according to the image to be processed. The image to be processed may be the second image obtained by the whitening process, the third image obtained by the double eyelid process, and the fourth image after the local color fine-tuning process such as the above-mentioned teeth whitening process, facial growth removal process, and / or eye bags process.

[0220] The adjustment point coordinates include the control point coordinates corresponding to the first shape fine-tuning process and the moving point coordinates corresponding to the control point coordinates. Specifically, the facial key point coordinates are obtained from the image to be processed; the control point coordinates of the target part targeted by the first shape fine-tuning process are extracted from the facial key point coordinates; the zoom range and the straightening rotation angle corresponding to the image to be processed are determined; according to the zoom range, the straightening rotation angle, the fine-tuning weight corresponding to the first shape fine-tuning process and the preset moving distance, the control point coordinates are moved to obtain the moving point coordinates corresponding to the control point coordinates.

[0221] The zoom range indicates the zoom ratio of the face image. When determining the zoom range corresponding to the face image, first extract the coordinates of the center point of the user's left eye and the center point of the right eye from the coordinates of the key points of the face; determine the distance between the user's eyes based on the coordinates of the center point of the left eye and the center point of the right eye; and calculate the zoom range corresponding to the face image based on the distance between the eyes and the preset standard distance. The preset standard distance can be 230px or 235px, etc.

[0222] The distance between the eyes of the user is the length of the line between the coordinates of the center point of the left eye and the coordinates of the center point of the right eye. The ratio between the distance between the eyes and the preset standard distance is calculated, and the ratio is determined as the zoom range corresponding to the user's face image.

[0223] In the embodiment of the present application, in the face image of the user, the user may have his head tilted. If the fine-tuning operation is performed directly using the control point coordinates in the face image of the tilted head, the accuracy of the fine-tuning operation will be affected. Therefore, in the embodiment of the present application, it is necessary to first straighten the head of the user in the face image, determine the coordinates of the control points after the straightening, scale the coordinates of the control points after the straightening, and then restore the tilted head state. The above-mentioned straightening rotation angle means the angle by which the face image needs to be rotated to straighten the user's head.

[0224] Specifically, the coordinates of the center point of the user's left eye and the center point of the right eye are extracted from the coordinates of the key points of the face; based on the coordinates of the center point of the left eye and the center point of the right eye, a line between the two eyes is determined in the face image; the angle between the line between the two eyes and a first extension direction of a pixel row of the face image is determined as a straightening rotation angle corresponding to the face image, and the first extension direction is any extension direction of the pixel row.

[0225] The face image is a rectangular image including the user's face, and includes multiple pixel rows and multiple pixel columns in the face image, such as a 720*1280 face image including 720 pixel rows and 1280 pixel columns. The pixel rows are parallel to the upper and lower sides of the face image, and the pixel columns are parallel to the left and right sides of the face image. The pixel rows have two extension directions, left and right. The first extension direction can be a left extension direction or a right extension direction. Fig.23 As shown, assuming Fig.23 The line between the two black dots is the line between the two eyes of the user, the direction indicated by the arrow is the first extension direction of the pixel row, and the angle a between the line between the two eyes and the first extension direction is the straightening rotation angle corresponding to the face image.

[0226] In the embodiment of the present application, the straightening rotation angle can be calculated by the following formula.

[0227] a=Atan(Left(x,y)-right(x,y))*180 / PI

[0228] Among them, a is the rotation angle, Atan is the inverse tangent function, Left(x,y) is the coordinate of the center point of the left eye, right(x,y) is the coordinate of the center point of the right eye, and PI is the pi.

[0229] After the zoom range and the righting rotation angle are obtained in the above manner, the control point coordinates are moved according to the zoom range, the righting rotation angle, the fine-tuning weight corresponding to the first shape fine-tuning process and the preset moving distance to obtain the moving point coordinates corresponding to the control point coordinates.

[0230] In the embodiment of the present application, a preset moving distance corresponding to each fine-tuning task is also preset, and the preset moving distance may be 5 pixels or 6 pixels, etc. For example, the preset moving distance for fine-tuning the facial contour is 5 pixels, and the preset moving distance for fine-tuning the nose wing is 3 pixels.

[0231] For each control point coordinate, the control point coordinate is first rotated in the first extension direction by the determined correction rotation angle to obtain the coordinate corresponding to the control point after the user's head is corrected. Then, according to the moving direction corresponding to the current fine-tuning task, the rotated control point coordinate is moved by the preset moving distance corresponding to the current fine-tuning task. The moved control point coordinate is scaled according to the scaling range and the fine-tuning weight; the scaled control point coordinate is rotated in the opposite direction of the first extension direction by the correction rotation angle to obtain the coordinate corresponding to the scaled control point in the original tilted head state, which is the moving point coordinate corresponding to the control point coordinate.

[0232] The moving direction corresponding to the current fine-tuning task is related to whether the current fine-tuning task is to reduce or enlarge. If the current fine-tuning task is to slim the face, the moving direction of the control points on the left half of the face is to the right, while the moving direction of the control points on the right half of the face is to the left. If the current fine-tuning task is to fatten the face, it is just the opposite, that is, the moving direction of the control points on the left half of the face is to the left, while the moving direction of the control points on the right half of the face is to the right. Similarly, the moving directions corresponding to different fine-tuning tasks for other parts are also determined according to whether the fine-tuning task is to reduce or enlarge.

[0233] For scaling of the rotated control point coordinates, the scaled control point coordinates can be obtained by the formula (x+t*scale*smooth) or the formula (y+t*scale*smooth), where x is the horizontal coordinate of the control point coordinates after the rotation angle is corrected, y is the vertical coordinate of the control point coordinates after the rotation angle is corrected, t is the preset moving distance corresponding to the current fine-tuning task, scale is the scaling range, and smooth is the fine-tuning weight.

[0234] If the moving direction corresponding to the current fine-tuning task is to move left or right, the above formula (x+t*scale*smooth) is used to process the horizontal coordinate of the control point coordinate, and the vertical coordinate remains unchanged. If the moving direction corresponding to the current fine-tuning task is to move up or down, the above formula (y+t*scale*smooth) is used to process the vertical coordinate of the control point coordinate, and the horizontal coordinate remains unchanged.

[0235] For example, to fine-tune the facial contour, adjust points 9 and 23 on the facial contour, 9(x)+t*scale*smooth, 23(x)+t*scale*smooth; to fine-tune the nose wing, adjust points 82 and 83 on the nose wing, 82(x)+t*scale*smooth, 83(x)+t*scale*smooth; to fine-tune the nose tip, adjust points 107 and 108 on the nose tip, 107(y)+t*scale*smooth, 108(y )+t*scale*smooth; for chin fine-tuning, adjust points 15, 16, and 17 on the chin, 15(y)+t*scale*smooth, 16(y)+t*scale*smooth, 15(x)+t*scale*smooth, 16(x)+t*scale*smooth; for cheekbone fine-tuning, adjust points 3 and 29 on the cheekbones, 3(x)+t*scale*smooth, 29(x)+t*scale*smooth.

[0236] The embodiment of the present application also divides a triangular mesh of a preset size on the image to be processed. The preset size may be 21*21 or 22*22, etc. After obtaining the face image of the user, the terminal also evenly divides the entire rectangular image of the face image into a triangular mesh of a preset size, the triangular mesh including a plurality of rectangular sub-meshes. If the preset size is 22*22, the triangular mesh includes 22*22 rectangular sub-meshes.

[0237] After the triangular mesh is divided on the face image, the mesh information corresponding to the triangular mesh is also stored. The mesh information includes the coordinate information of each sub-mesh included in the triangular mesh. The sub-mesh coordinate information includes at least the coordinates of four vertices of the sub-mesh.

[0238] Then, the triangular patch mesh is adjusted according to the coordinates of the adjustment points to obtain the adjusted triangular patch mesh. Specifically, the coordinates of each sub-mesh within the overall mesh boundary line are obtained from the triangular patch mesh; the coordinates of each sub-mesh are moved according to the coordinates of the control point, the coordinates of the moving point and the coordinates of each sub-mesh; and the adjusted triangular patch mesh is obtained according to the coordinates of the overall mesh boundary line and the coordinates of each sub-mesh after the movement.

[0239] That is, the coordinates on the boundary line of the overall mesh of the triangular face mesh are kept unchanged. According to the coordinates of the control point, the coordinates of the moving point and the coordinates of each sub-mesh within the boundary line, each sub-mesh is moved and adjusted by the least square method. Then, according to the coordinates on the unchanged boundary line and the coordinates of each sub-mesh after the movement, the deformed and adjusted triangular face mesh is drawn.

[0240] Since the coordinates on the boundary line of the entire mesh are kept unchanged when the triangular mesh is deformed, when the deformed triangular mesh is used to perform overall adjustment on the face image, it can be ensured that there will be no blank space on the edge of the face image.

[0241] Finally, the image to be processed is subjected to the first shape fine-tuning process according to the triangular patch mesh before adjustment, the triangular patch mesh after adjustment and the fourth image. Specifically, the triangular patch mesh before adjustment, the triangular patch mesh after adjustment and the fourth image are input into the graphics card; in the graphics card, the pixels covered by the first sub-grid before adjustment in the fourth image are respectively rendered into the second sub-grid after adjustment through affine transformation, the first sub-grid is any sub-grid included in the triangular patch mesh before adjustment, and the second sub-grid is a sub-grid in the triangular patch mesh after adjustment corresponding to the first sub-grid; the image obtained by the radial transformation is determined as the face fine-tuning image corresponding to the first shape fine-tuning process.

[0242] That is, for each sub-grid in the adjusted triangular patch mesh, the color values ​​of each point in the image area covered by the corresponding sub-grid in the adjusted triangular patch mesh are obtained respectively, and the obtained color values ​​of each point are filled into the corresponding sub-grid in the adjusted triangular patch mesh. Each sub-grid completes the radial transformation of color, that is, the face fine-tuning image corresponding to the first shape fine-tuning processing is obtained. Finally, the face fine-tuning image is displayed through the graphics card.

[0243] Since most of the calculation processes in the embodiments of the present application are completed in the central processing unit of the terminal, the amount of data finally input into the graphics card is very small. The graphics card only needs to perform calculations on each sub-grid included in the triangular face mesh. That is, if the preset size of the triangular face mesh is 21*21, the graphics card only needs to perform calculations on 441 data. Compared with pixel-by-pixel calculations, the performance is greatly improved, and the efficiency of fine-tuning the facial image is very high.

[0244] In the embodiment of the present application, a drag control for adjusting the fine-tuning weight is also provided to the user. The drag control includes a drag block and a drag bar. The drag bar is used to indicate the range of the fine-tuning weight that can be adjusted. The user can set a custom fine-tuning weight by dragging the drag block. Fig.10 As shown, the dragging range of the drag bar is 0-1. When the graphics card displays the above-mentioned face fine-tuning image, the fine-tuning weight indicated by the drag block on the drag bar is the fine-tuning weight corresponding to the first shape fine-tuning process assigned to the user in step 102. If the user sees the displayed face fine-tuning image and is not satisfied with the fine-tuning effect, he can submit a custom weight by dragging the drag block.

[0245] The terminal receives the custom weight set by the user through the above-mentioned drag control; re-moves the control point coordinates according to the zoom range, the rotation angle, the custom weight and the preset moving distance corresponding to the current fine-tuning task, and obtains the new moving point coordinates corresponding to the control point coordinates; then, based on the control point coordinates, the new moving point coordinates and the triangular face mesh, the triangular face mesh is deformed and adjusted again; finally, a new face fine-tuning image is generated based on the triangular face mesh before adjustment, the re-adjusted triangular face mesh and the face image.

[0246] The terminal readjusts according to the user's customized weights in the above manner to generate a new fine-tuned face image, so that the final fine-tuned face image meets the user's own aesthetic requirements.

[0247] The eye scaling process includes the process of enlarging the eye or reducing the eye. The key points of the eye area in the image to be processed are determined, and the key points of the eye area include the outer contour key points and the pupil center key points. The size of the eye is calculated according to the key points of the eye area. Specifically, the image to be processed is divided into a plurality of rectangular grids. The division method of the rectangular grid is the same as the division method of the triangular facet grid of the preset size in the above-mentioned first shape fine-tuning process, which will not be repeated here.

[0248] According to the key points of the eye area, the eye grid is determined. That is, the eye area is determined according to the key points of the eye area, and the grid where each pixel of the eye area is located is determined. The eye grid is all the grids of the area occupied by the eye in the image to be processed; according to the key points of the eye area and the eye grid, the size of the eye is calculated. The image to be processed can be the second image obtained by whitening treatment, the third image obtained by double eyelid treatment, the fourth image after local color fine-tuning treatment such as the above-mentioned teeth whitening treatment, facial growth removal treatment and / or eye bags treatment, or the image after the above-mentioned first shape fine-tuning treatment.

[0249] Then, according to the fine-tuning weights corresponding to the eye scaling process, the size of the eye, and the key points of the eye area, the eye is scaled by moving the key points of the eye area. Specifically, according to the fine-tuning weights corresponding to the eye scaling process and the size of the eye, the moving distance of the vertices of each eye mesh along the direction of the line connecting itself and the corresponding pupil center key point is determined; according to the moving distance, the vertices of each eye mesh are moved in a direction away from the corresponding pupil center key point; the image is rendered according to the mesh obtained after the movement to obtain the image after eye plastic surgery. Specifically, the mesh obtained after the movement is rendered using a vertex shader.

[0250] The specific calculation method of the above-mentioned moving distance is to calculate the original distance between the vertices of each eye mesh and the corresponding pupil center key point; according to the fine-tuning weight corresponding to the eye scaling process, the original distance and the preset maximum moving distance of the vertices of the eye mesh, calculate the moving distance of the vertices of each eye mesh along the direction of the line connecting itself and the corresponding pupil center key point.

[0251] The above-mentioned original distance is determined in the following manner: a coordinate system is established on the image to be processed; the coordinates of the vertices of each eye mesh and the coordinates of the corresponding pupil center key point are determined; and the original distance between the vertices of each eye mesh and the corresponding pupil center key point is calculated based on the coordinates of the vertices of each eye mesh and the coordinates of the corresponding pupil center key point.

[0252] The eye mesh covering the left eye corresponds to the left eye pupil center keypoint, and the eye mesh covering the right eye corresponds to the right eye pupil center keypoint. The calculation formula for the original distance between the vertex of the eye mesh and its corresponding pupil center keypoint is l = [(x-x0) 2 +(y-y0) 2 ] 0.5 Among them, the vertex coordinates of the eye mesh are (x, y), and the coordinates of the corresponding pupil center key point are (x0, y0).

[0253] According to the calculated moving distance, the vertices of each eye mesh are moved in the direction away from the corresponding pupil center key point. After the vertices of the eye mesh corresponding to each eye are moved, the position and size of the eye mesh will change, thereby achieving the purpose of enlarging the eye. Moreover, since each part of the eye is enlarged in equal proportion, the enlarged eye will not be deformed.

[0254] The image is then rendered based on the mesh obtained after the movement to obtain an image after eye plastic surgery. Specifically, the mesh obtained after the movement can be rendered using a vertex shader. For example, a mesh s is moved to s', then the portion between s' and s is rendered, that is, the portion between the mesh obtained after the movement and the corresponding mesh before the movement is rendered.

[0255] In this embodiment, OpenGL (Open Graphics Library) may be used for image processing. OpenGL is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics.

[0256] In some embodiments, a vertex shader may be used for rendering, and the vertex shader performs rendering once for each vertex. A vertex shader is a set of instruction codes that are executed when a vertex is rendered. For example, when drawing a triangle, the triangle has 3 vertices, so the vertex shader will be executed 3 times.

[0257] Vertex rendering can replace the traditional transformation and lighting stages in terms of functionality. It is located after the tessellation stage and before the culling and clipping stages. After the tessellation stage, a vertex data is passed to the vertex renderer. The vertex may contain data such as position, texture coordinates, vertex color, normal, etc. The vertex renderer cannot create or delete vertices. After it is processed, it must at least output the position data in the vertex.

[0258] After the image after eye zoom processing is displayed to the user in the above manner, if the user is not satisfied with the effect, the user can Fig.10 The drag control shown submits the custom weight. The user's custom weight is received, and according to the custom weight, the key points of the eye area are re-moved in the above manner to scale the eye.

[0259] Through the eye zooming process of this embodiment, the eyes of the face image can be enlarged or reduced, and the edges of the eyes after adjustment are smooth and natural.

[0260] For the second shape fine-tuning process of fine-tuning one or more of the lip thickness, philtrum, inner eye corner, outer eye corner, and upper eyelid, firstly, the texture coordinates and the target face key points corresponding to the target part targeted by the second shape fine-tuning process are determined from the image to be processed. The image to be processed may be the second image obtained by whitening process, the third image obtained by double eyelid process, the fourth image after local color fine-tuning process such as the above-mentioned teeth whitening process, facial growth removal process and / or eye bags process, the image after the above-mentioned first shape fine-tuning process, or the image after the above-mentioned eye zoom process.

[0261] The above-mentioned target parts may be lips and / or eyes. The texture coordinates are the original pixel coordinates of the target parts. For the target facial key points corresponding to the target parts, facial key points within a preset range around the target parts may be selected from the new first facial key point set as the target facial key points corresponding to the target parts. In order to be closer to a real plastic surgery, for example, when shrinking the philtrum, the entire lip is not moved, but the upper lip is thickened by moving the key points around the upper lip. For another example, when the upper eyelid is lowered, only the key points around the outer corner of the eye are moved, and the outer corner of the eye is not moved. In the present application, the target part is reshaped by fine-tuning, and the plastic surgery effect is close to a real plastic surgery.

[0262] Then, the preset parameters are adjusted according to the fine-tuning weights corresponding to the second shape fine-tuning process, and the preset parameters include direction and distance. The preset parameters can be data determined in advance according to the plastic surgery parameters of a standard face. In the present application, the preset parameters can be adjusted according to the face shape of the current face image to be plastic-shaped, or the preset parameters can be adjusted according to the fine-tuning weights corresponding to the second shape fine-tuning process.

[0263] The target facial key points are moved according to the adjusted preset parameters to obtain new target facial key points. Facial graphics are rendered according to the image to be processed, the texture coordinates corresponding to the target part and the new target facial key points to obtain a fine-tuned image of the face after plastic surgery. First, all facial key points of the image to be processed are identified, and the facial triangular mesh data is determined based on all facial key points. Then, facial graphics are rendered according to the image to be processed, the facial triangular mesh data, the texture coordinates corresponding to the target part and the new target facial key points. Taking thick lips as an example, the key points on the lips are translated so that the points on the lips expand outward or shrink inward to achieve the effect of thick lips or thin lips. Fig.24 shown.

[0264] The fine-tuned face image after the above plastic surgery is displayed. If the user is not satisfied, the user can Fig.10 The drag control shown in the figure submits the custom weight. The custom weight submitted by the user is received; the preset parameters are adjusted according to the custom weight, and the target face key points are translated again according to the adjusted preset parameters. A face fine-tuning image is regenerated in the above manner.

[0265] Through the above-mentioned second shape fine-tuning processing, the effects of micro-plastic surgery such as lip thickness, philtrum length, inner canthus opening, outer canthus opening, and upper eyelid lowering can be made closer to real plastic surgery, thereby improving the user experience.

[0266] In the embodiment of the present application, a fine-tuning weight suitable for the user is allocated according to the actual situation of the target part targeted by the virtual micro-plastic surgery. The virtual micro-plastic surgery is performed on the target face image of the user according to the fine-tuning weight, so that the fine-tuned face image obtained by micro-plastic surgery can be more natural and beautiful, and meet the aesthetic needs of the user. In addition, the order of each virtual micro-plastic surgery is limited to ensure that the effects of each virtual micro-plastic surgery are superimposed on each other, greatly improving the overall effect of face fine-tuning and avoiding the problem of incompatibility between each virtual micro-plastic surgery.

[0267] The present application also provides an image processing device, which is used to execute an image processing method provided by any of the above embodiments. Fig.25 , the device comprises:

[0268] An acquisition module 201 is used to acquire a target face image of a user;

[0269] The weight allocation module 202 is used to allocate the fine-tuning weight required for each virtual micro-plastic surgery process to the user according to the target face image, and the fine-tuning weight is allocated according to the part feature information of the target part targeted by the virtual micro-plastic surgery in the target face image;

[0270] The generating module 203 is used to generate a face fine-tuning image corresponding to the target face image according to the target face image and the fine-tuning weights corresponding to each virtual micro-plastic surgery process.

[0271] The weight allocation module 202 is used to extract the part feature information of the target part for virtual micro plastic surgery from the target face image; determine the part type to which the target part belongs based on the part feature information; obtain the preset weight corresponding to the part type; and determine the preset weight as the fine-tuning weight required for virtual micro plastic surgery on the target face image.

[0272] In the embodiment of the present application, the virtual micro plastic surgery process includes one or more of a skin grinding process, a whitening process, a double eyelid process, a local color fine-tuning process, and a local shape fine-tuning process;

[0273] The generation module 203 includes:

[0274] A skin grinding processing unit, used for performing skin grinding processing on the target face image according to a fine-tuning weight corresponding to the skin grinding processing to obtain a first image;

[0275] A whitening processing unit, configured to perform whitening processing on the first image according to a fine-tuning weight corresponding to the whitening processing to obtain a second image;

[0276] A double eyelid processing unit, configured to adjust the eyes of the user to double eyelids in the second image according to a fine-tuning weight corresponding to the double eyelid processing if it is detected that the eyes in the target face image are single eyelids, so as to obtain a third image;

[0277] a local color fine-tuning processing unit, configured to adjust the target part targeted by the local color fine-tuning processing in the third image according to a fine-tuning weight corresponding to the local color fine-tuning processing, if it is detected that the target part targeted by the local color fine-tuning processing in the target face image meets the preset color fine-tuning processing condition, so as to obtain a fourth image;

[0278] The local shape fine-tuning processing unit is used to adjust the shape of the target part targeted by the local shape fine-tuning processing in the fourth image according to the fine-tuning weight corresponding to the local shape fine-tuning processing, so as to obtain a face fine-tuning image corresponding to the target face image.

[0279] The skin smoothing processing unit is used to perform a first blur processing of the face area in the target face image for the first time to obtain a first blur map, wherein the first blur processing includes horizontal blur processing and vertical blur processing; perform fusion processing with the target face image according to the first blur map to obtain a first image to be processed; perform mean blurring on the first blur map to obtain a second blur map; perform image fusion on the target face image, the first image to be processed and the second blur map according to the fine-tuning weight corresponding to the skin smoothing processing.

[0280] The skin resurfacing processing unit is used to determine a first number of first adjacent pixels that are horizontally adjacent to the first center point pixel; and determine a first number of second adjacent pixels that are vertically adjacent to the first center point pixel; the first center point pixel is any pixel in the face area of ​​the target face image; based on the first center point pixel and the first adjacent pixel, the face area in the target face image is horizontally blurred; after each pixel in the face area is horizontally blurred, the face area is vertically blurred based on the first center point pixel and the second adjacent pixel.

[0281] The skin grinding processing unit is used to perform an inversion process on the first blurred image; perform edge extraction on the first blurred image after the inversion process to obtain a second image to be processed; perform a second blur process on the second image to be processed for a second time to obtain a third blurred image, wherein the second blur process includes a horizontal blur process and a vertical blur process; perform image fusion on the first blurred image, the third blurred image and the target face image to obtain the first image to be processed.

[0282] The skin smoothing processing unit is used to fuse the target face image, the first image to be processed and the second blurred image to obtain a first fused image; based on a preset sharpness parameter, the first fused image is adjusted in sharpness; according to the fine-tuning weight corresponding to the skin smoothing processing, the first fused image after the sharpness adjustment is fused with the target face image to obtain a second fused image; if it is determined that a modification instruction generated by the user for the second fused image is received, based on the modification instruction, the user-submitted custom weight is determined; according to the custom weight, the first fused image after the sharpness adjustment is fused with the target face image to obtain a third fused image.

[0283] A whitening processing unit is used to search for corresponding color values ​​in a preset first lookup table according to the color values ​​of pixels in the first image, and replace the color values ​​of pixels in the first image with the searched color values; perform a first fusion of the original color values ​​of pixels in the first image and the replaced color values; perform a second fusion of the color values ​​of pixels in the first image after the first fusion processing and the color values ​​of pixels in the original first image according to the fine-tuning weights corresponding to the whitening processing; and use a preset second lookup table to perform color adjustment on the second fused image.

[0284] The double eyelid processing unit is used to locate the facial key points in the second image, interpolate the eye key points included in the facial key points to generate new eye key points; according to the fine-tuning weights corresponding to the double eyelid processing and the new eye key points, the pre-made standard double eyelid material is fused with the eye area in the second image to obtain a fused eye area image.

[0285] A double eyelid processing unit is used to obtain standard eye key points, triangulated grids and standard double eyelid materials based on a standard face image; establish a mapping relationship between standard eye key points and new eye key points, and fuse the standard double eyelid materials to the eye area in the second image according to the mapping relationship; obtain pixel points covered by the triangulated grid on the eye area in the second image according to the mapping relationship; for each pixel point obtained, calculate the target pixel value of the pixel point according to the fine-tuning weight corresponding to the double eyelid processing, the first pixel value of the pixel point and the second pixel value of the pixel point corresponding to the standard double eyelid material.

[0286] The local color fine-tuning processing includes teeth whitening processing. The local color fine-tuning processing unit is used to extract the mouth image separately from the third image and perform coloring on the mouth image; perform cutout processing on the mouth image, and perform color adjustment on the cutout mouth image according to the preset third lookup table and the fine-tuning weights corresponding to the teeth whitening processing.

[0287] The local color fine-tuning processing includes facial growth removal processing, and facial growths include bags under the eyes and / or wrinkles; the local color fine-tuning processing unit is used to read the first template material stored in advance, and draw the first template material in a blank texture by indexing, so that the first template material corresponds to the target area targeted by the facial growth removal processing in the third image; according to the fine-tuning weight corresponding to the facial growth removal processing, the target area on the third image corresponding to the first template material is blurred.

[0288] The local color fine-tuning processing includes eye bags processing; the local color fine-tuning processing unit is used to obtain a second template material, and draw the second template material into a blank texture by means of an index, so that the second template material corresponds to the target area in the third image targeted by the eye bags processing; according to the fine-tuning weight corresponding to the eye bags processing, at least two color channels of the target area in the third image are weightedly fused based on the second template material.

[0289] The local shape fine-tuning processing includes a first shape fine-tuning processing of one or more target parts including facial contour, nose wing, nose tip, nose length, chin, and cheekbone; a local shape fine-tuning processing unit is used to obtain the adjustment point coordinates corresponding to the first shape fine-tuning processing according to the fourth image; a triangular patch mesh of a preset size is divided on the fourth image; the triangular patch mesh is adjusted according to the adjustment point coordinates to obtain an adjusted triangular patch mesh; and the first shape fine-tuning processing is performed on the fourth image according to the triangular patch mesh before adjustment, the adjusted triangular patch mesh, and the fourth image.

[0290] The adjustment point coordinates include the control point coordinates corresponding to the first shape fine-tuning processing and the moving point coordinates corresponding to the control point coordinates; the local shape fine-tuning processing unit is used to obtain the coordinates of the key points of the face from the fourth image; extract the control point coordinates of the target part targeted by the first shape fine-tuning processing from the key point coordinates of the face; determine the zoom range and the straightening rotation angle corresponding to the fourth image; move the control point coordinates according to the zoom range, the straightening rotation angle, the fine-tuning weight corresponding to the first shape fine-tuning processing and the preset moving distance to obtain the moving point coordinates corresponding to the control point coordinates.

[0291] The local shape fine-tuning processing unit is used to obtain the coordinates of each sub-mesh within the overall mesh boundary line from the triangular face mesh; move the coordinates of each sub-mesh according to the control point coordinates, the moving point coordinates and the coordinates of each sub-mesh; and obtain the adjusted triangular face mesh according to the coordinates of the overall mesh boundary line and the coordinates of each sub-mesh after the move.

[0292] A local shape fine-tuning processing unit is used to input the triangular patch mesh before adjustment, the triangular patch mesh after adjustment and the fourth image into a graphics card; in the graphics card, the pixel points covered by the first sub-mesh before adjustment in the fourth image are respectively rendered into the second sub-mesh after adjustment through affine transformation, the first sub-mesh is any sub-mesh included in the triangular patch mesh before adjustment, and the second sub-mesh is a sub-mesh in the triangular patch mesh after adjustment corresponding to the first sub-mesh; and the image obtained by the radial transformation is determined as the face fine-tuning image corresponding to the first shape fine-tuning processing.

[0293] The local shape fine-tuning process includes eye scaling process; a local shape fine-tuning process unit is used to determine the pupil center key point of each eye from the fourth image; divide the fourth image into a plurality of rectangular grids; determine the eye grid; the eye grid is all grids of the area occupied by each eye in the fourth image; calculate the original distance between the vertices of each eye grid and the corresponding pupil center key point; calculate the moving distance of the vertices of each eye grid along the direction of the line connecting the vertices of the eye grid and the corresponding pupil center key point according to the fine-tuning weight corresponding to the eye scaling process, the original distance and the preset maximum moving distance of the vertices of the eye grid; according to the moving distance, move the vertices of each eye grid in the direction away from the corresponding pupil center key point; render the image according to the grid obtained after the movement to obtain the image after eye plastic surgery.

[0294] The local shape fine-tuning processing includes a second shape fine-tuning processing for fine-tuning one or more of the lip thickness, philtrum, inner canthus, outer canthus, and upper eyelid; a local shape fine-tuning processing unit is used to determine the texture coordinates and target facial key points corresponding to the target part targeted by the second shape fine-tuning processing from the fourth image; according to the fine-tuning weights corresponding to the second shape fine-tuning processing, the preset parameters are adjusted, and the preset parameters include direction and distance; the target facial key points are moved according to the adjusted preset parameters to obtain new target facial key points; facial graphics are rendered according to the fourth image, the texture coordinates corresponding to the target part and the new target facial key points to obtain a fine-tuned face image after plastic surgery.

[0295] The image processing device provided in the above-mentioned embodiment of the present application and the image processing method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0296] The present application also provides an electronic device to perform the above-mentioned image processing method. Fig.26 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. Fig.26 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802 and a communication interface 803, wherein the processor 800, the communication interface 803 and the memory 801 are connected via the bus 802; the memory 801 stores a computer program that can be run on the processor 800, and when the processor 800 runs the computer program, the image processing method provided in any of the aforementioned embodiments of the present application is executed.

[0297] The memory 801 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 803 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0298] The bus 802 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs, and the processor 800 executes the programs after receiving execution instructions. The image processing method disclosed in any implementation of the above-mentioned embodiment of the present application may be applied to the processor 800, or implemented by the processor 800.

[0299] The processor 800 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 800. The above processor 800 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 801, and the processor 800 reads the information in the memory 801 and completes the steps of the above method in combination with its hardware.

[0300] The electronic device provided in the embodiment of the present application and the image processing method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0301] The present application also provides a computer-readable storage medium corresponding to the image processing method provided in the above embodiment. Fig. 27 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the image processing method provided by any of the aforementioned embodiments will be executed.

[0302] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0303] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the image processing method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0304] It should be noted that:

[0305] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0306] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0307] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0308] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Obtain the target face image of the user; According to the target facial image, the user is assigned a fine-tuning weight required for each virtual micro-plastic surgery process, wherein the fine-tuning weight is assigned based on part feature information of a target part in the target facial image targeted by the virtual micro-plastic surgery; According to the target facial image and the fine-tuning weights corresponding to each virtual micro-plastic surgery process, a face fine-tuning image corresponding to the target facial image is generated, wherein the virtual micro-plastic surgery process includes a skin resurfacing process, and according to the fine-tuning weights corresponding to the skin resurfacing process, the skin resurfacing process is performed on the target facial image to obtain a first image; The step of performing skin smoothing on the target face image according to the fine-tuning weight corresponding to the skin smoothing to obtain the first image includes: Performing a first number of first fuzzy processing on the face area in the target face image to obtain a first fuzzy image, wherein the first fuzzy processing includes horizontal fuzzy processing and vertical fuzzy processing; Performing fusion processing on the first fuzzy image and the target face image to obtain a first image to be processed; Perform mean blurring on the first fuzzy image to obtain a second fuzzy image; According to the fine-tuning weight corresponding to the skin resurfacing process, the target face image, the first image to be processed and the second fuzzy image are fused.

2. The method according to claim 1, characterized in that: The step of allocating fine-tuning weights required for each virtual micro-plastic surgery process to the user according to the target face image includes: Extracting part feature information of a target part for virtual micro plastic surgery from the target face image; Determining the part type to which the target part belongs according to the part characteristic information; Obtaining a preset weight corresponding to the part type; The preset weight is determined as the fine-tuning weight required for performing the virtual micro-plastic surgery on the target facial image.

3. The method according to claim 1 or 2, characterized in that: The virtual micro plastic surgery process also includes one or more of whitening process, double eyelid process, local color fine-tuning process and local shape fine-tuning process; The step of generating a face fine-tuning image corresponding to the target face image according to the target face image and the fine-tuning weights corresponding to each virtual micro-plastic surgery process comprises: Performing whitening processing on the first image according to the fine-tuning weight corresponding to the whitening processing to obtain a second image; If it is detected that the eyes in the target face image are single eyelids, adjusting the eyes of the user to double eyelids in the second image according to the fine-tuning weights corresponding to the double eyelid processing to obtain a third image; If it is detected that the target part targeted by the local color fine-tuning process in the target face image meets the preset color fine-tuning process condition, adjusting the target part targeted by the local color fine-tuning process in the third image according to the fine-tuning weight corresponding to the local color fine-tuning process to obtain a fourth image; According to the fine-tuning weight corresponding to the local shape fine-tuning process, the shape of the target part targeted by the local shape fine-tuning process is adjusted in the fourth image to obtain a fine-tuned face image corresponding to the target face image.

4. The method according to claim 1, characterized in that The step of performing a first fuzzy process on the face area in the target face image comprises: Determine a first number of first adjacent pixels that are horizontally adjacent to the first center point pixel; and determine the first number of second adjacent pixels that are vertically adjacent to the first center point pixel; the first center point pixel is any pixel in the face area of ​​the target face image; Based on the first center point pixel and the first adjacent pixels, performing horizontal blur processing on the face area in the target face image; After the horizontal blurring process is performed on each pixel in the face area, the vertical blurring process is performed on the face area based on the first center point pixel and the second adjacent pixel.

5. The method according to claim 1, characterized in that The method of performing fusion processing on the first fuzzy image and the target face image to obtain a first image to be processed includes: Performing inversion processing on the first fuzzy image; Performing edge extraction on the first blurred image after the inversion process to obtain a second image to be processed; Performing a second fuzzy process for a second time on the second image to be processed to obtain a third fuzzy image, wherein the second fuzzy process includes the horizontal fuzzy process and the vertical fuzzy process; The first fuzzy image, the third fuzzy image and the target face image are fused to obtain a first image to be processed.

6. The method according to claim 1, characterized in that The step of fusing the target face image, the first image to be processed, and the second fuzzy image according to the fine-tuning weight corresponding to the skin resurfacing process comprises: Performing image fusion on the target face image, the first image to be processed, and the second fuzzy image to obtain a first fused image; Based on a preset sharpness parameter, adjusting the sharpness of the first fused image; According to the fine-tuning weight corresponding to the skin resurfacing process, the first fused image after the sharpness adjustment is fused with the target face image to obtain a second fused image.

7. The method according to claim 6, characterized in that The method further comprises: If it is determined that a modification instruction generated by the user for the second fused image is received, determining the user-submitted custom weight based on the modification instruction; According to the user-defined weight, the first fused image after the sharpness adjustment is fused with the target face image to obtain a third fused image.

8. The method according to claim 3, characterized in that The step of performing whitening processing on the first image according to the fine-tuning weight corresponding to the whitening processing includes: According to the color value of the pixel in the first image, searching for a corresponding color value in a preset first lookup table, and replacing the color value of the pixel in the first image with the searched color value; Performing a first fusion on the original color value and the replaced color value of the pixel in the first image: performing a second fusion on the color values ​​of the pixels in the first image after the first fusion process and the color values ​​of the pixels in the original first image according to the fine-tuning weights corresponding to the whitening process; The second fused image is color adjusted using a preset second lookup table.

9. The method according to claim 3, characterized in that: The adjusting the eyes of the user into double eyelids in the second image according to the fine-tuning weight corresponding to the double eyelid processing includes: Locating facial key points in the second image, and interpolating eye key points included in the facial key points to generate new eye key points; According to the fine-tuning weights corresponding to the double-eyelid processing and the new eye key points, the pre-made standard double-eyelid material is fused with the eye area in the second image to obtain a fused eye area image.

10. The method according to claim 9, characterized in that The step of fusing the pre-made standard double eyelid material with the eye region in the second image according to the fine-tuning weight corresponding to the double eyelid processing and the new eye key point to obtain a fused eye region image includes: Obtain standard eye key points, triangulation meshes and standard double eyelid materials based on standard face images; Establishing a mapping relationship between the standard eye key points and the new eye key points, and fusing the standard double-eyelid material to the eye region in the second image according to the mapping relationship; According to the mapping relationship, obtaining pixel points covered by the triangulated grid on the eye area in the second image; For each acquired pixel point, the target pixel value of the pixel point is calculated according to the fine-tuning weight corresponding to the double-eyelid processing, the first pixel value of the pixel point and the second pixel value of the pixel point corresponding to the standard double-eyelid material.

11. The method according to claim 8, characterized in that The local color fine-tuning process includes a teeth whitening process, and adjusting the target part targeted by the local color fine-tuning process in the third image according to the fine-tuning weight corresponding to the local color fine-tuning process includes: Extracting a mouth image separately from the third image, and performing coloring processing on the mouth image; The mouth image is subjected to a cutout process, and the color of the mouth image after the cutout process is adjusted according to a preset third lookup table and a fine-tuning weight corresponding to the teeth whitening process.

12. The method according to claim 3, characterized in that The local color fine-tuning process includes facial growth removal process, and the facial growth includes eye bags and / or wrinkles; and adjusting the target part targeted by the local color fine-tuning process in the third image according to the fine-tuning weight corresponding to the local color fine-tuning process includes: Reading the first template material generated earlier, and drawing the first template material in the blank texture by indexing, so that the first template material corresponds to the target area of ​​the facial growth removal process in the third image; According to the fine-tuning weight corresponding to the facial growth removal process, blurring is performed on the target area on the third image corresponding to the first template material.

13. The method according to claim 3, characterized in that The local color fine-tuning process includes eye bags processing; and adjusting the target part targeted by the local color fine-tuning process in the third image according to the fine-tuning weight corresponding to the local color fine-tuning process includes: Acquire a second template material, and draw the second template material into a blank texture by means of an index, so that the second template material corresponds to a target area for the eyebag treatment in the third image; According to the fine-tuning weight corresponding to the undereye bag processing, weighted fusion processing is performed on at least two color channels of the target area in the third image based on the second template material.

14. The method according to claim 3, characterized in that The local shape fine-tuning process includes a first shape fine-tuning process on one or more target parts of the facial contour, nose wing, nose tip, nose length, chin, and cheekbone; and the shape adjustment of the target part targeted by the local shape fine-tuning process in the fourth image according to the fine-tuning weight corresponding to the local shape fine-tuning process includes: According to the fourth image, obtaining the coordinates of adjustment points corresponding to the first shape fine-tuning process; Dividing the fourth image into a triangular mesh of a preset size; Adjust the triangular patch mesh according to the adjustment point coordinates to obtain an adjusted triangular patch mesh; The first shape fine-tuning process is performed on the fourth image according to the triangular patch mesh before adjustment, the triangular patch mesh after adjustment, and the fourth image.

15. The method according to claim 14, characterized in that The adjustment point coordinates include the control point coordinates corresponding to the first shape fine-tuning process and the moving point coordinates corresponding to the control point coordinates; The step of acquiring, according to the fourth image, coordinates of adjustment points corresponding to the first shape fine-tuning process includes: Acquire the coordinates of key points of the face from the fourth image; Extracting the control point coordinates of the target part targeted by the first shape fine-tuning process from the facial key point coordinates; Determine the zoom range and the alignment rotation angle corresponding to the fourth image; The control point coordinates are moved according to the zoom range, the straightening rotation angle, the fine-tuning weight corresponding to the first shape fine-tuning process, and a preset moving distance to obtain the moving point coordinates corresponding to the control point coordinates.

16. The method according to claim 15, characterized in that The step of adjusting the triangular patch mesh according to the adjustment point coordinates to obtain an adjusted triangular patch mesh comprises: Obtaining the coordinates of each sub-grid within the boundary line of the overall grid from the triangular patch grid; According to the control point coordinates, the moving point coordinates and the coordinates of each sub-grid, the coordinates of each sub-grid are moved; The adjusted triangular patch mesh is obtained according to the coordinates of the overall mesh boundary line and the coordinates of each sub-mesh after the movement.

17. The method according to claim 14, characterized in that The step of performing the first shape fine-tuning process on the fourth image according to the triangular patch mesh before adjustment, the triangular patch mesh after adjustment, and the fourth image includes: Inputting the triangular patch mesh before adjustment, the triangular patch mesh after adjustment and the fourth image into a graphics card; In the graphics card, pixel points covered by the first sub-grid before adjustment in the fourth image are respectively rendered into the second sub-grid after adjustment through affine transformation, the first sub-grid is any sub-grid included in the triangular patch mesh before adjustment, and the second sub-grid is a sub-grid in the triangular patch mesh after adjustment corresponding to the first sub-grid; The image obtained by the radial transformation is determined as the face fine-tuning image corresponding to the first shape fine-tuning process.

18. The method according to claim 3, characterized in that The local shape fine-tuning process includes an eye zoom process; and the shape adjustment of the target part targeted by the local shape fine-tuning process in the fourth image according to the fine-tuning weight corresponding to the local shape fine-tuning process includes: Determining eye region key points in the fourth image; Calculate the size of the eye according to the key points of the eye area; The eyes are scaled by moving the eye area key points according to the fine-tuning weights corresponding to the eye scaling process, the size of the eyes, and the eye area key points.

19. The method according to claim 3, characterized in that The local shape fine-tuning process includes a second shape fine-tuning process for fine-tuning one or more of lip thickness, philtrum, inner eye corner, outer eye corner, and upper eyelid; and the shape adjustment of the target part targeted by the local shape fine-tuning process in the fourth image according to the fine-tuning weight corresponding to the local shape fine-tuning process includes: Determining, from the fourth image, texture coordinates and target facial key points corresponding to the target part targeted by the second shape fine-tuning processing; Adjusting preset parameters according to the fine-tuning weight corresponding to the second shape fine-tuning process, where the preset parameters include direction and distance; The target face key points are moved according to the adjusted preset parameters to obtain new target face key points; The face graphics are rendered according to the fourth image, the texture coordinates corresponding to the target part and the new target face key points to obtain a fine-tuned face image after plastic surgery.

20. An image processing device, characterized in that: include: An acquisition module is used to acquire a target face image of a user; A weight allocation module, used to allocate a fine-tuning weight required for each virtual micro-plastic surgery to the user according to the target face image, wherein the fine-tuning weight is allocated according to part feature information of a target part in the target face image targeted by the virtual micro-plastic surgery; A generating module, configured to generate a face fine-tuning image corresponding to the target face image according to the target face image and the fine-tuning weights corresponding to each virtual micro-plastic surgery process, wherein the virtual micro-plastic surgery process includes a skin resurfacing process, and the skin resurfacing process is performed on the target face image according to the fine-tuning weights corresponding to the skin resurfacing process to obtain a first image; The step of performing skin smoothing on the target face image according to the fine-tuning weight corresponding to the skin smoothing to obtain the first image includes: Performing a first number of first fuzzy processing on the face area in the target face image to obtain a first fuzzy image, wherein the first fuzzy processing includes horizontal fuzzy processing and vertical fuzzy processing; Performing fusion processing on the first fuzzy image and the target face image to obtain a first image to be processed; Perform mean blurring on the first fuzzy image to obtain a second fuzzy image; According to the fine-tuning weight corresponding to the skin resurfacing process, the target face image, the first image to be processed and the second fuzzy image are fused.

Images