Image beautification processing method and apparatus, and device and storage medium

Abstract

Disclosed in the present application are an image beautification processing method and apparatus, and a device and a storage medium. The method comprises: acquiring a first facial image from a first image; executing a first beautification processing task on first facial information in the first facial image, so as to obtain a second facial image; executing a second beautification processing task on second facial information in the second facial image, so as to obtain a target facial image; and obtaining a target image on the basis of the target facial image and the first image. In this way, complex facial beautification processing is rationally split into a plurality of beautification processing tasks on the basis of the characteristics of different facial beautification processing tasks, and the plurality of beautification processing tasks are executed on a facial image in stages. The processing capability of different tasks can be improved, the facial beautification effect is improved, and the image quality of a non-facial region is not affected.

Description

Image beautification processing method, device, equipment, and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202311729226.0 and application date of December 14, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to image processing technology, and in particular to an image beautification processing method and apparatus, equipment, and storage medium. Background Art

[0004] Portrait beautification refers to the use of image processing technology to beautify portraits in images or videos to better meet the user's aesthetic needs.

[0005] Current beautification technology, while beautifying the portrait image, also beautifies the entire image, which can easily cause image distortion and poor image quality. Summary of the Invention

[0006] The embodiments of the present application hope to provide an image beautification processing method and apparatus, device, and storage medium.

[0007] The technical solution of this application is achieved as follows:

[0008] In a first aspect, a method for image beautification processing is provided, comprising:

[0009] Acquire a first face image in the first image;

[0010] performing a first beautification processing task on first facial information in the first facial image to obtain a second facial image;

[0011] performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image;

[0012] A target image is obtained according to the target face image and the first image.

[0013] In a second aspect, an image beautification processing device is provided, comprising:

[0014] an acquiring unit, configured to acquire a first face image from the first image;

[0015] a first processing unit, configured to perform a first beautification processing task on the first facial information in the first facial image to obtain a second facial image;

[0016] a second processing unit, configured to perform a second beautification processing task on the second facial information in the second facial image to obtain a target facial image;

[0017] The third processing unit is configured to obtain a target image based on the target face image and the first image.

[0018] According to a third aspect, an electronic device is provided, comprising: a processor and a memory configured to store a computer program capable of running on the processor,

[0019] Wherein, the processor is configured to execute the steps of the aforementioned method when running the computer program.

[0020] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program implements the steps of the aforementioned method when executed by a processor.

[0021] In an embodiment of the present application, a method and apparatus, device, and storage medium for image beautification processing are provided. The method comprises: obtaining a first facial image in a first image; performing a first beautification processing task on the first facial information in the first facial image to obtain a second facial image; performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image; and obtaining a target image based on the target facial image and the first image. In this way, according to the characteristics of different facial beautification processing tasks, the complex facial beautification processing is reasonably divided into multiple beautification processing tasks, and multiple beautification processing tasks are performed on the facial image in stages, which can improve the processing capabilities of different tasks, improve the beautification effect of the face, and do not affect the image quality of non-face areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a schematic diagram of a first process of an image beautification processing method according to an embodiment of the present application;

[0023] FIG2 is a schematic diagram of a first implementation flow of multiple facial beautification tasks according to an embodiment of the present application;

[0024] FIG3 is a schematic diagram of the edge transition processing flow of a face image in an embodiment of the present application;

[0025] FIG4 is a schematic diagram of a second process of the image beautification processing method according to an embodiment of the present application;

[0026] FIG5 is a schematic diagram of an image transformation process in an embodiment of the present application;

[0027] FIG6 is a schematic diagram of a second implementation flow of multiple beautification processing tasks according to an embodiment of the present application;

[0028] FIG7 is a schematic diagram of a third flow chart of the image beautification processing method according to an embodiment of the present application;

[0029] FIG8 is a schematic diagram of the structure of a neural network model according to an embodiment of the present application;

[0030] FIG9 is a schematic diagram of the implementation process of multiple neural network models in an embodiment of the present application;

[0031] FIG10 is a schematic diagram of the structure of an image beautification processing device according to an embodiment of the present application;

[0032] FIG11 is a schematic diagram of the composition structure of the electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The present invention provides an image beautification method, wherein the method includes:

[0034] Acquire a first face image in the first image;

[0035] performing a first beautification processing task on first facial information in the first facial image to obtain a second facial image;

[0036] performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image;

[0037] A target image is obtained according to the target face image and the first image.

[0038] In some embodiments, performing a first beautification task on the first facial information in the first facial image includes:

[0039] Performing a first transformation process on the first facial image to obtain a first target image having the same size as the first target image;

[0040] performing a first beautification processing task on the first facial information in the first target image to obtain the second facial image;

[0041] The performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image includes:

[0042] Performing a second transformation on the second facial image to obtain a second target image having the same size as the second target image;

[0043] A second beautification processing task is performed on the second facial information in the second target image to obtain the target facial image.

[0044] In some embodiments, performing a second beautification task on the second facial information in the second target image to obtain the target facial image includes:

[0045] performing a second beautification processing task on the second facial information in the second target image to obtain a third facial image;

[0046] The target facial image is obtained according to the first facial image, the second facial image and the third facial image.

[0047] In some embodiments, obtaining the target facial image based on the first facial image, the second facial image, and the third facial image includes:

[0048] subtracting the second face image from the first target image to obtain a first beauty difference image;

[0049] subtracting the third face image from the second target image to obtain a second beauty difference image;

[0050] Performing weighted fusion on the first beautification difference image and the second beautification difference image to obtain a third beautification difference image; wherein the third beautification difference image includes a beautification residual value of each pixel in the first facial image;

[0051] The target facial image is obtained according to the first facial image and the third beauty-difference image.

[0052] In some embodiments, performing weighted fusion on the first beauty-difference image and the second beauty-difference image to obtain a third beauty-difference image includes:

[0053] performing a second inverse transformation corresponding to the second transformation on the second beautification difference image to obtain an inversely transformed second beautification difference image;

[0054] performing weighted fusion on the first beautification difference image and the inverse-transformed second beautification difference image according to a target fusion weight to obtain a fused beautification difference image;

[0055] A first inverse transformation process corresponding to the first transformation process is performed on the fused beautification difference image to obtain the third beautification difference image.

[0056] In some embodiments, obtaining the target facial image based on the first facial image and the third beauty-difference image includes:

[0057] performing edge transition processing on the third beautification difference image to obtain a fourth beautification difference image;

[0058] The first facial image and the fourth beauty-difference image are added to obtain the target facial image.

[0059] In some embodiments, the first transformation process is used to transform the size of the first human face into an image size specified by the first beauty processing task;

[0060] The second transformation process is used to transform the size of the second facial image into an image size specified by the second beauty processing task.

[0061] In some embodiments, the first transformation process includes at least one of the following:

[0062] When a size relationship between a width and a height of the first facial image is inconsistent with a size relationship between a first width and a first height of the first target image, rotating the first facial image;

[0063] If the width of the first facial image is greater than the first width, or the height of the first facial image is greater than the first height, performing a scale transformation on the first facial image;

[0064] Perform edge padding on the first facial image according to the first width and the first height.

[0065] In some embodiments, the second transformation process includes image reduction.

[0066] In some embodiments, the method includes: performing a first beautification processing task on the first facial information in the first facial image through a first neural network model to obtain a second facial image; performing a second beautification processing task on the second facial information in the second facial image through a second neural network model to obtain a target facial image.

[0067] In some embodiments, the first neural network model specifically includes an input layer, at least one convolutional layer, at least one transposed convolutional layer, and an output layer, and the convolutional layers and transposed convolutional layers in the same layer are connected via a connection layer;

[0068] The first neural network model is configured as follows: performing pixel rearrangement processing on the first target image through the input layer to obtain multiple first feature maps that are smaller than the size of the first target image, where the first target image is an image obtained after the first transformation processing is performed on the first facial image and has the same size as the input image of the first neural network model; performing convolution processing on the multiple first feature maps in sequence through at least one convolution layer to obtain multiple second feature maps; performing transposed convolution processing on the multiple second feature maps in sequence through at least one transposed convolution layer to obtain multiple third feature maps; and performing pixel restoration processing on the multiple third feature maps through the output layer to obtain a second facial image.

[0069] In some embodiments, the first facial information is high-frequency information in the facial image, and the second facial information is low-frequency information in the facial image.

[0070] In some embodiments, obtaining the first facial image in the first image includes:

[0071] Performing face detection on the first image to determine at least one face frame;

[0072] Determine, from the at least one face frame, a first face frame whose face frame area is greater than or equal to an area threshold;

[0073] Enlarging the first face frame according to a target magnification factor;

[0074] The first facial image is acquired from the first image according to the enlarged first facial frame.

[0075] In some embodiments, acquiring the first facial image in the first image includes: acquiring the first facial image in the first image whose area is greater than or equal to an area threshold.

[0076] In some embodiments, obtaining the target image based on the target facial image and the first image includes: performing edge transition processing on the target facial image, and filling the transition-processed facial image into the corresponding facial position of the first image to obtain the target image.

[0077] In some embodiments, the method further includes: performing back-end beautification processing on the target facial image in response to a user's beautification processing operation on the target facial image.

[0078] The present application also provides an image beautification processing device, wherein the device includes:

[0079] an acquiring unit, configured to acquire a first face image from the first image;

[0080] a first processing unit, configured to perform a first beautification processing task on the first facial information in the first facial image to obtain a second facial image;

[0081] a second processing unit, configured to perform a second beautification processing task on the second facial information in the second facial image to obtain a target facial image;

[0082] The third processing unit is configured to obtain a target image based on the target face image and the first image.

[0083] An embodiment of the present application further provides an electronic device, wherein the electronic device includes: a processor and a memory configured to store a computer program that can be run on the processor,

[0084] Wherein, the processor is configured to execute the steps of any of the aforementioned methods when running the computer program.

[0085] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the aforementioned methods when executed by a processor.

[0086] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0087] FIG1 is a schematic diagram of a first process of an image beautification processing method according to an embodiment of the present application. As shown in FIG1 , the method may specifically include:

[0088] Step 101: Acquire a first face image in a first image;

[0089] It should be noted that the first image may be an image output by the imaging unit, or an image obtained by the image processing unit after performing certain processing on the image output by the imaging unit, or an image obtained from the storage unit.

[0090] In some embodiments, obtaining a first facial image in the first image includes: obtaining a first facial image in the first image whose area is greater than or equal to an area threshold.

[0091] Exemplarily, obtaining a first facial image from a first image includes: performing face detection on the first image to determine at least one face frame; determining a first face frame from the at least one face frame whose area is greater than or equal to an area threshold; enlarging the first face frame according to a target magnification factor; and obtaining the first facial image from the first image based on the enlarged first face frame. This allows filtering out faces smaller than the area threshold, improving face processing efficiency. Because such facial images contain too little information, beautification processing for such faces is meaningless or unnecessary.

[0092] Determining, from at least one face frame, a first face frame whose face frame area is greater than or equal to an area threshold, includes: sorting the face frames from largest to smallest based on their area; and determining the first N face frames whose face frame areas are greater than or equal to the area threshold. N is an integer greater than 1. Exemplarily, N is an integer such as 2, 4, 6, or 8. It should be noted that when the first image includes many faces greater than or equal to the area threshold, the number of faces processed may be limited to ensure face processing efficiency and avoid processing too many faces, which would affect the overall beautification effect.

[0093] By enlarging the first determined face frame, a larger facial region image can be captured, which helps minimize the impact on the main face during subsequent edge smoothing. For example, the face frame can be square or rectangular. For example, the target magnification factor can be a fixed value or flexibly selected based on the actual face frame size. In other words, the same magnification factor can be set for the width and height of the face frame, or different magnification factors can be set.

[0094] In some embodiments, the face frame may further include parts connected to the face, such as the neck and ears, that is, the face, neck and ears are all beautified.

[0095] Step 102: performing a first beautification processing task on the first facial information in the first facial image to obtain a second facial image;

[0096] Step 103: performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image;

[0097] According to the characteristics of different face beautification processing tasks, complex face beautification processing can be reasonably divided into multiple beautification processing tasks to improve the processing effect of each beautification processing task.

[0098] The first face information and the second face information may be different information to be beautified in the face. Beautifying the information to be beautified can improve the visual presentation effect of the face image.

[0099] In some embodiments, the first facial information is high-frequency information, and the second facial information is low-frequency information.

[0100] High-frequency information represents rapidly changing areas of the image, i.e., areas with drastic changes in color or grayscale values. High-frequency information primarily describes the image's microscopic details and noise. In some embodiments, high-frequency information can specifically represent local facial blemishes, including but not limited to pores, blackheads, fat particles, dead skin cells, lip lines, crow's feet, eye redness, wrinkles, acne, and local light and shadow differences. Accordingly, the first beauty processing task includes but is not limited to removing dirty pores, blackheads, fat particles, dead skin cells, lip lines, crow's feet, eye redness, wrinkles, acne, and smoothing local light and shadow.

[0101] Low-frequency information represents slowly changing areas of the image, i.e., areas where color or grayscale values ​​change smoothly, while high-frequency information primarily describes large, flat areas of the image. In some embodiments, low-frequency information can represent global facial flaws, including but not limited to eye bags, dark circles, nasolabial folds, jaw, mouth corners, teeth cleaning, and a wider range of light and shadow differences. Accordingly, the second beauty processing task includes but is not limited to removing eye bags, dark circles, nasolabial folds, jaw, mouth corners, teeth cleaning, and a wider range of light and shadow smoothing and reshaping.

[0102] In some embodiments, a first beautification processing task is performed on the first facial information in the first facial image, including: performing a first transformation processing on the first facial image to obtain a first target image having the same size as the first target image; performing the first beautification processing task on the first facial information in the first target image to obtain a second facial image.

[0103] That is to say, before performing the first beautification processing task on the first face image, image size conversion processing can be performed first to convert the size of the first face image to a fixed image size specified by the first beautification processing task, and the processing is performed on the fixed image size, which can ensure the processing effect of the first beautification processing task and shorten the processing time and reduce power consumption.

[0104] Correspondingly, a second beautification processing task is performed on the second facial information in the second facial image to obtain a target facial image, including: performing a second beautification processing task on the second facial information in the second facial image to obtain a third facial image; and obtaining the target facial image based on the first facial image and the third facial image.

[0105] In some embodiments, a second beautification processing task is performed on the second facial information in the second facial image to obtain a target facial image, including: performing a second transformation processing on the second facial image to obtain a second target image having the same size as the second target image; performing a second beautification processing task on the second facial information in the second target image to obtain the target facial image.

[0106] That is, before executing the second beautification task on the second facial image, an image resizing process can be performed to convert the second facial image to the fixed image size specified by the second beautification task. This fixed image size can be processed to ensure the effectiveness of the second beautification task while also reducing processing time and power consumption. It should be noted that when the fixed image size specified by the second beautification task matches the output image size of the first beautification task, the second resizing process is not required.

[0107] Correspondingly, a second beautification processing task is performed on the second facial information in the second facial image to obtain a target facial image, including: performing a second beautification processing task on the second facial information in the second facial image to obtain a third facial image; and obtaining the target facial image based on the first facial image, the second facial image and the third facial image.

[0108] FIG2 is a schematic diagram of a first implementation flow of multiple beautification processing tasks in an embodiment of the present application. As shown in FIG2 , the second facial image and the first target image are subtracted to obtain a first beautification difference image; the third facial image and the second target image are subtracted to obtain a second beautification difference image; the first beautification difference image and the second beautification difference image are weightedly fused to obtain a third beautification difference image; wherein the third beautification difference image includes the beautification residual value of each pixel in the first facial image; and the target facial image is obtained based on the first facial image and the third beautification difference image. The beautification difference image includes the residual information between the facial image before beautification and the facial image after beautification to obtain the beautification difference image corresponding to each beautification processing task. The beautification difference images obtained for each task are fused to obtain the third beautification difference image of the entire beautification processing process. The third beautification difference image includes the residual information between the first facial image before the entire beautification processing and the target facial image. The first facial image and the third beautification difference image are added to obtain the target facial image.

[0109] In some embodiments, the method further includes performing a third beautification task on the third facial information of the third facial image to obtain a fourth facial image. Accordingly, a target facial image is obtained based on the first facial image, the second facial image, the third facial image, and the fourth facial image. Exemplarily, the third beautification task may include even skin tone, optimized skin texture, and the like.

[0110] In some embodiments, the method further includes performing a fourth beautification task on fourth facial information of a fourth facial image to obtain a fifth facial image. Accordingly, a target facial image is obtained based on the first facial image, the second facial image, the third facial image, the fourth facial image, and the fifth facial image. Exemplarily, the fourth beautification task may include facial reshaping, etc.

[0111] Step 104: Obtain a target image based on the target face image and the first image.

[0112] In some embodiments, the target facial image is used to fill in the corresponding facial positions of the first image to obtain the target image after beautification. If multiple target facial images are included, the corresponding facial positions of the first image can be filled in order of the target facial images from largest to smallest in area.

[0113] Furthermore, the method may also include: performing edge transition processing on the target facial image, filling the facial image after transition processing into the corresponding facial position of the first image to obtain the target image after beautification, so that a smooth transition is achieved between the edge area of ​​the face frame and the original image. As shown in Figure 3, the edge bands around the face frame are gradually fused with the corresponding original image pixels. The embodiment of the present application does not impose specific restrictions on the width of the edge band. Exemplarily, the edge band width can be set to 10 pixels. The surrounding edge areas are gradually fused from the inside to the outside, and the edgemost position is 100% of the pixel value of the first image. The purpose of the transition processing is to avoid unnatural boundaries between the edge of the target facial image after beautification and the original image. The pixels subjected to edge transition processing may include only the surrounding edge pixels of the target facial image, or include the surrounding edge pixels of the target facial image and the surrounding edge pixels of the face frame in the first image.

[0114] In some embodiments, the method further includes: performing back-end beautification processing on the target facial image in response to a user's beautification processing operation on the target facial image.

[0115] In the embodiment of the present application, the sequential beautification of the first facial image through at least two beautification tasks can be understood as a pre-beautification process automatically implemented by the system, or as a built-in beautification function of the system, which removes local and global blemishes from the face in the first image to obtain a facial image that is close to the real face but without common blemishes. The user's beautification operations on the target facial image can be understood as user-defined beautification operations, which can be performed based on different beautification parameters set by the user online, including but not limited to whitening, deformation, and makeup, to obtain the final beautified portrait image of the original portrait image.

[0116] It should be noted that for the front-end beauty processing, users can also turn off all or part of the beauty processing tasks according to their own needs.

[0117] Specifically, the method may further include: in response to a user closing operation on the target beautification task, closing the target beautification task. Specifically, the first face image may be directly used as the target face image.

[0118] The method may further include: in response to a user's start operation on the target facial beautification task, starting the target facial beautification task. The target facial beautification task may be a part or all of the at least two facial beautification tasks.

[0119] FIG4 is a schematic diagram of a second process of the image beautification processing method according to an embodiment of the present application. As shown in FIG4 , the method may specifically include:

[0120] Step 401: Acquire a first face image in a first image;

[0121] Exemplarily, obtaining a first facial image from a first image includes: performing face detection on the first image to determine at least one face frame; determining a first face frame from the at least one face frame whose area is greater than or equal to an area threshold; enlarging the first face frame according to a target magnification factor; and obtaining the first facial image from the first image based on the enlarged first face frame. This allows filtering out faces smaller than the area threshold, improving face processing efficiency. Because such facial images contain too little information, beautification processing for such faces is meaningless or unnecessary.

[0122] Step 402: performing a first transformation process on the first face image to obtain a first target image having the same size as the first target image;

[0123] The first transformation processing is used to transform the facial image size determined from the first image into the image size specified by the first beautification processing task. For example, when the size relationship between the width and height of the first facial image is inconsistent with the size relationship between the first width and the first height of the first target image, the first facial image is rotated; when the width of the first facial image is greater than the first width, or the height of the first facial image is greater than the first height, the first facial image is scaled; and the first facial image is edge-filled according to the first width and the first height.

[0124] If rotation and / or scale transformation is performed before edge filling, edge filling is performed on the rotated and / or scale-transformed first face image to obtain a first input image.

[0125] It should be noted that, when the size of the second target image is smaller than that of the first target image, the second transformation process is performed on the second facial image to obtain the second target image.

[0126] It is understandable that different beauty processing tasks have different image resolution requirements. The minimum feasible fixed image size must be set according to the type of task being performed, the defect distribution characteristics, and the defect removal requirements. Processing is performed on the fixed image size, which can ensure the processing effect of the beauty task while shortening the processing time and reducing power consumption.

[0127] As shown in Figure 5, if the image size is set to 1408 x 896, and the length and short sides of the face image match the fixed input image size, no rotation is required. If not, a 90-degree rotation is performed. Specifically, when the input image size is 1408 x 896, the width is greater than the height. If the face image is also wider than the height, no rotation is required. If the height is greater than the width, a 90-degree rotation is performed to keep it consistent with the input size.

[0128] If the width and height of the rotated facial image are both less than or equal to those of the corresponding input image, no rescaling is required. If the width or height of the rotated facial image is greater than the width and height of the corresponding input image, rescaling is required. The aspect ratio of the rescaled facial image must be the same as before the rescaling, and the rescaled width and height must be less than or equal to those of the input image. The rescaling method should be as close to the width and height of the input image as possible. Scaling methods can include bilinear interpolation and nearest neighbor.

[0129] Fill the input image with the rotated and / or scaled face image. The unfilled portion can be assigned a value of 0 or no additional value, resulting in a face image that matches the input image size.

[0130] Step 403: performing a first beautification processing task on the first facial information in the first target image to obtain a second facial image;

[0131] Step 404: performing a second transformation process on the second facial image to obtain a second target image having the same size as the second target image;

[0132] The second transformation process is used to resize the second facial image to the image size specified by the second beautification task. The transformation rules of the second transformation process can be the same as or different from those of the first transformation process, but both aim to resize the image to the specified image size. Exemplarily, the second transformation process includes image reduction, which can be achieved through downsampling and / or upsampling.

[0133] In some embodiments, the image size specified by the second beautification task is less than or equal to the image size after processing by the first beautification task (i.e., the size of the second facial image). If so, the second beautification task can be directly executed on the second facial image. If less than, the second facial image needs to be scaled down before executing the second beautification task. For example, if the second facial image is set to 1408 width and 896 height, the image size specified by the second beautification task is set to 704 width and 448 height, which means that the second facial image is downsampled by 2 times to 704 width and 448 height.

[0134] Step 405: performing a second beautification processing task on the second facial information in the second target image to obtain a target facial image;

[0135] In some embodiments, a second beautification processing task is performed on the second facial information in the second facial image to obtain a target facial image, including: performing a second beautification processing task on the second facial information in the second facial image to obtain a third facial image; and obtaining the target facial image based on the first facial image, the second facial image, and the third facial image.

[0136] Exemplarily, obtaining a target facial image based on a first facial image, a second facial image, and a third facial image includes: subtracting the second facial image from the first target image to obtain a first beauty difference image; subtracting the third facial image from the second target image to obtain a second beauty difference image; performing weighted fusion on the first beauty difference image and the second beauty difference image to obtain a third beauty difference image; wherein the third beauty difference image includes the beauty residual value of each pixel in the first facial image; and obtaining the target facial image based on the first facial image and the third beauty difference image.

[0137] In some embodiments, weighted fusion is performed on the first beauty difference image and the second beauty difference image to obtain a third beauty difference image, including: performing a second inverse transformation corresponding to the second transformation on the second beauty difference image to obtain the inversely transformed second beauty difference image; weighted fusion is performed on the first beauty difference image and the inversely transformed second beauty difference image according to a target fusion weight to obtain a fused beauty difference image; and first inverse transformation is performed on the fused beauty difference image corresponding to the first transformation to obtain the third beauty difference image.

[0138] It should be noted that if the sizes of the beauty difference images at each stage are inconsistent, it is necessary to first perform the corresponding inverse transformation to make the sizes of the beauty difference images at each stage consistent before performing weighted fusion.

[0139] The calculation process of the target face image can be shown as formula (1):

[0140] The specific calculation steps can also be shown in Figure 6 and described as follows:

[0141] The third face image after the second beautification task output_img DNN2 Subtract the second target image input_img of the second beauty processing task DNN2 , get the second beauty difference image diff_img DNN2 ;

[0142] Second beautification difference image diff_img DNN2 , upsample to the size of the second face image after the first beautification task, and obtain the upsampled second beautification difference image

[0143] The second face image output_img after beautification of the first beautification task DNN1 Subtract the first target image input_img of the first beauty processing task DNN1 , get the first beauty difference image diff_img DNN1 ;

[0144] The first beauty difference image and the upsampled second beauty difference image are weightedly fused according to the strategy for different user groups or the degree of user customization, to obtain the fused beauty difference image all_diff_img DNN ;

[0145] According to the rotation, size scaling, filling and other transformation information saved in each face image, the fused beauty difference image all_diff_img DNN , inversely transformed to a third beauty difference image with the same size and orientation as the original first face image

[0146] Original first face image ori_img and third beauty difference image Add together, that is, the target face image beauty_img after two-stage beauty processing DNN .

[0147] In some embodiments, obtaining a target facial image based on the first facial image and the third beauty-difference image includes: performing edge transition processing on the third beauty-difference image to obtain a fourth beauty-difference image; and adding the first facial image and the fourth beauty-difference image to obtain the target facial image.

[0148] It can be understood that, compared with directly performing edge transition processing on the target face image, performing edge transition processing on the third beauty difference image is less difficult and has a higher processing efficiency.

[0149] Step 406: Obtain a target image based on the target face image and the first image.

[0150] In some embodiments, the target facial image is used to fill in the corresponding facial positions of the first image to obtain the target image after beautification. If multiple target facial images are included, the corresponding facial positions of the first image can be filled in order of the target facial images from largest to smallest in area.

[0151] If the target facial image has not undergone edge transition processing, the method may further include: performing edge transition processing on the target facial image, filling the transition-processed facial image into the corresponding facial position of the first image to obtain the target image after beautification, so that a smooth transition is achieved between the edge area of ​​the face frame and the original image.

[0152] In some embodiments, different neural network models can be used to implement different beautification tasks. For example, a first neural network model performs a first beautification task on first facial information in a first facial image to obtain a second facial image; and a second neural network model performs a second beautification task on second facial information in the second facial image to obtain a target facial image.

[0153] FIG7 is a schematic diagram of a third process of the image beautification processing method in an embodiment of the present application. As shown in FIG7 , the method may specifically include:

[0154] Step 701: Acquire a first face image in a first image.

[0155] In some embodiments, obtaining a first facial image in a first image includes: performing face detection on the first image to determine at least one face frame; determining a first face frame having an area greater than or equal to an area threshold from the at least one face frame; enlarging the first face frame according to a target magnification factor; and obtaining the first facial image from the first image based on the enlarged first face frame.

[0156] Step 702: performing a first beautification processing task on the first facial information in the first facial image using a first neural network model to obtain a second facial image;

[0157] In some embodiments, a first transformation process is performed on the first facial image through a first neural network model to obtain a first target image having the same size as the first target image; a first beauty processing task is performed on the first facial information in the first target image to obtain a second facial image.

[0158] In some embodiments, the first transformation processing includes: rotating the first facial image when the size relationship between the width and height of the first facial image is inconsistent with the size relationship between the first width and the first height of the first neural network model input image; scaling the first facial image when the width of the first facial image is greater than the first width, or the height of the first facial image is greater than the first height; and edge-filling the rotated and / or scaled first facial image according to the first width and the first height to obtain a first target image.

[0159] Exemplarily, the first transformation processing is used to transform the facial image size determined from the first image into the input image size specified by the first neural network model, and the first transformation processing includes at least one of the following: rotating the first facial image when the size relationship between the width and height of the first facial image is inconsistent with the size relationship between the first width and the first height of the input image of the first neural network model; performing scale transformation on the first facial image when the width of the first facial image is greater than the first width, or the height of the first facial image is greater than the first height; and edge filling the first facial image according to the first width and the first height to obtain a first target image.

[0160] If rotation and / or scale transformation is performed before edge filling, edge filling is performed on the rotated and / or scale-transformed first face image to obtain a first target image.

[0161] In some embodiments, the method may further include: obtaining rotation information, scale transformation information, and edge filling information of the first facial image to form first transformation information of the first facial image; and saving the first transformation information. The first transformation information is used to perform a first inverse transformation process corresponding to the first transformation process.

[0162] The first neural network model performs a first beautification processing task on the high-frequency information in the first target image. The high-frequency information can specifically be local blemishes on the face, including but not limited to pores, blackheads, fat particles, dead skin, lip lines, crow's feet, red bloodshot eyes, wrinkles, acne, local light and shadow flatness, etc.

[0163] The network structure characteristics of the first neural network model can specifically be a lightweight deep convolutional network model (Deep Convolution Neural Network, DNN), convolutional neural network model (Convolutional Neural Network, CNN), long short-term memory network (Long Short-Term Memory Network, LSTM), etc.

[0164] Exemplarily, the first neural network model can be a lightweight CNN that is easy to deploy on a mobile terminal. As shown in Figure 8, the first neural network model specifically includes an input layer, at least one convolutional layer, at least one transposed convolutional layer and an output layer. The convolutional layers and transposed convolutional layers in the same layer are connected through a connection layer; the embodiment of the present application does not impose any specific restrictions on the specific number of convolutional layers (and transposed convolutional layers) and the number of channels in each layer.

[0165] A first beautification processing task is performed on the first target image through a first neural network model to obtain a second facial image, including: performing pixel rearrangement processing on the first target image through an input layer to obtain multiple first feature maps that are smaller than the size of the first target image; performing convolution processing on the multiple first feature maps in sequence through at least one convolution layer to obtain multiple second feature maps; performing transposed convolution processing on the multiple second feature maps in sequence through at least one transposed convolution layer to obtain multiple third feature maps; and performing pixel restoration processing on the multiple third feature maps through an output layer to obtain a second facial image.

[0166] The input and output layers implement image pixel rearrangement and pixel restoration processing. Specifically, the space-to-depth and depth-to-space operations in TensorFlow can be used, or convolutional layers with a stride of 2 and transposed convolutional layers can be used. For example, the input layer reduces the size of the input image to 0.5 times the original size and the number of channels to 4 times the original pixel rearrangement, rearranging the adjacent 4-neighborhood pixels to the same position in different channels. The output layer processing is the opposite of the input layer processing. Using the input layer for pixel rearrangement can effectively reduce the processing time and memory usage of convolutional neural networks.

[0167] A convolutional layer can be a downsampling operation, while a transposed convolutional layer can be an upsampling operation. The image's height and width are reduced by a factor of 2 through the convolutional layer, and increased by a factor of 2 through the transposed convolutional layer, ensuring that the height, width, and number of channels of images in the same layer remain consistent. A direct connection operation occurs between the corresponding layers during the height and width reduction and increase processes.

[0168] Step 703: performing a second beautification processing task on the second facial information in the second facial image using a second neural network model to obtain a third facial image;

[0169] In some embodiments, a second transformation process is performed on the second facial image through a second neural network model to obtain a second target image having the same size as the second target image; and a second beauty processing task is performed on the second facial information in the second target image to obtain the target facial image.

[0170] In some embodiments, when the size of the input image of the second neural network model is smaller than the size of the first neural network model, a second transformation process is performed on the second facial image to obtain a second target image.

[0171] The second transformation processing is used to transform the output image size of the first neural network model into the input image size specified by the second neural network model. Exemplarily, the second transformation processing includes image reduction, which can be achieved by downsampling and / or upsampling.

[0172] The network structure of the second neural network model can specifically be a lightweight DNN, CNN, LSTM, etc. The network structure of the second neural network model may be the same as or different from the network structure of the first neural network model. For example, the network structure of the second neural network model may also be shown in FIG8 .

[0173] The second neural network model performs beautification processing on the low-frequency information in the second target image. This low-frequency information can include global facial blemishes such as eye bags, dark circles, nasolabial folds, jawline, mouth corners, teeth cleanliness, and more broadly, light and shadow smoothing and reshaping.

[0174] In some embodiments, when the size of the input image to the second neural network model is equal to that of the first neural network model, the second facial image is directly used as the second target image. The second target image retains more facial information, which can improve the processing effect of the second neural network model.

[0175] Step 704: Obtain a target facial image according to the first facial image, the second facial image, and the third facial image.

[0176] In some embodiments, obtaining a target facial image based on a first facial image, a second facial image, and a third facial image includes: subtracting the second facial image from the first facial image to obtain a first beauty-difference image; subtracting the third facial image from the second facial image to obtain a second beauty-difference image; performing weighted fusion on the first beauty-difference image and the second beauty-difference image to obtain a third beauty-difference image; wherein the third beauty-difference image includes a beauty residual value for each pixel in the first facial image; and obtaining the target facial image based on the first facial image and the third beauty-difference image.

[0177] The beautification difference image includes the residual information between the face image before and after beautification processing, so as to obtain the beautification difference image corresponding to each beautification processing task. The beautification difference images obtained for each task are fused to obtain the third beautification difference image of the entire beautification processing process. The third beautification difference image includes the residual information between the first face image before the entire beautification processing process and the target face image. The first face image and the third beautification difference image are added to obtain the target face image.

[0178] In other embodiments, the second facial image and the first target image are subtracted to obtain a first beauty difference image; the third facial image and the second target image are subtracted to obtain a second beauty difference image; the first beauty difference image and the second beauty difference image are weightedly fused to obtain a third beauty difference image; wherein the third beauty difference image includes the beauty residual value of each pixel in the first facial image; and the target facial image is obtained based on the first facial image and the third beauty difference image.

[0179] Correspondingly, the first beauty difference image and the second beauty difference image are fused to obtain a third beauty difference image, including: performing a second inverse transformation corresponding to the second transformation processing on the second beauty difference image to obtain the inversely transformed second beauty difference image; performing weighted fusion on the first beauty difference image and the inversely transformed second beauty difference image according to a target fusion weight to obtain a fused beauty difference image; and performing a first inverse transformation corresponding to the first transformation processing on the fused beauty difference image to obtain the third beauty difference image.

[0180] In some embodiments, the first neural network model further includes a subtraction layer; the method further includes: subtracting the second facial image from the first target image via the subtraction layer to obtain a first beauty-difference image. The second neural network model further includes a subtraction layer; the method further includes: subtracting the third facial image from the second target image via the subtraction layer to obtain a second beauty-difference image.

[0181] It is understood that the neural network model at each stage can also output a beauty difference image, and the beauty difference images at each stage are weighted fused to obtain the final beauty difference image. The specific calculation steps can also be shown in Figure 9.

[0182] It should be noted that one or more neural network models after the second neural network model can also obtain at least one beauty difference image corresponding to other models, and fuse it with the beauty difference images of the first two models to obtain the final third beauty difference image.

[0183] In some embodiments, obtaining a target facial image based on the first facial image and the third beauty-difference image includes: performing edge transition processing on the third beauty-difference image to obtain a fourth beauty-difference image; and adding the first facial image and the fourth beauty-difference image to obtain the target facial image.

[0184] It can be understood that compared with directly performing edge transition processing on the target face image, performing edge transition processing on the third beauty difference image is less difficult and is conducive to improving processing efficiency.

[0185] Step 705: Obtain a beautified target image based on the target face image and the first image.

[0186] In some embodiments, the corresponding face position of the first image is filled in according to the target face image to obtain the target image after beautification.

[0187] If the target facial image has not undergone edge transition processing, the method may further include: performing edge transition processing on the target facial image, filling the transition-processed facial image into the corresponding facial position of the first image to obtain the target image after beautification, so that a smooth transition is achieved between the edge area of ​​the face frame and the original image.

[0188] In some embodiments, the method further includes: performing back-end beautification processing on the target facial image in response to a user's beautification processing operation on the target facial image.

[0189] In some embodiments, the method further includes: obtaining a first input sample image of the first neural network model; performing beauty processing on the first input sample image according to the beauty processing task performed by the first neural network model to obtain a first target sample image; and training the first neural network model using the first input sample image and the first target sample image.

[0190] Compared with using a single model to implement all beautification processing tasks, in the embodiment of the present application, different neural network model structures are established according to the characteristics of different tasks, and corresponding training data can be established. Different training data can be established and iterated according to the characteristics of their respective tasks. The training data includes input sample images and target sample images that have been beautified. When constructing training data, the amount of data that needs to be beautify processing is relatively low. Different neural network models can also be trained independently, and training strategies can be established according to the characteristics of their respective tasks, so that the effect converges better.

[0191] In some embodiments, the method further includes: beautifying the sample image through the trained first neural network model to obtain a second input sample image of the second neural network model; performing beautification processing on the second input sample image according to the beautification processing task performed by the second neural network model to obtain a second target sample image; and training the second neural network model using the second input sample image and the second target sample image.

[0192] That is to say, when training the second-stage model, the input sample image can be first processed using the first neural network model, and then participate in the training of the second-stage model to conform to the actual reasoning process of multiple models.

[0193] The target sample image can be an image obtained by a professional photo retoucher using image editing tools for beautification. For example, if the first neural network model performs the beautification task of high-frequency information, the photo retoucher will perform the following operations on the input sample image, including but not limited to pores, blackheads, fat particles, dead skin, lip lines, crow's feet, eye redness, wrinkles, acne, and local light and shadow smoothing. If the second neural network model performs the beautification task of low-frequency information, the photo retoucher will perform the following operations on the input sample image, including but not limited to eye bags, dark circles, nasolabial folds, jaw, mouth corners, teeth cleaning, and a wider range of light and shadow smoothing and light and shadow reshaping.

[0194] For the target sample images corresponding to the first neural network model, users' perception of the beautification effect is relatively objective. This means that the differences in the retouching results for different faces by different retouchers are minimal. The retouching logic is simple, requiring only local reference areas, requiring relatively small amounts of training data and placing low demands on iterative updates to the first neural network model. Furthermore, for high-frequency information, retouching requires high precision and high input image quality. Therefore, high-frequency information beautification can be performed on facial images first, followed by beautification of other information, thus ensuring the beautification effect of the high-frequency information.

[0195] For the target sample images corresponding to the second neural network model, the user's perception of the beautification effect is relatively independent, that is, different photo retouchers produce quite different retouching results for different faces. The retouching logic refers to information about a larger facial area or the entire face, requiring a large amount of training data. Usually, the image quality / clarity of the original training face is not high. With the deepening of research and changes in user trend perception, the retouching targets may be frequently updated during the development of each generation of products or products of the same generation. However, the photo retoucher only needs to perform part of the beautification tasks, which reduces the amount of data and improves the model training efficiency compared to using a single model that needs to perform all the beautification tasks.

[0196] To implement the method of the embodiment of the present application, based on the same inventive concept, the embodiment of the present application further provides an image beautification processing device. As shown in FIG10 , the image beautification processing device 100 includes:

[0197] An acquiring unit 1001 is configured to acquire a first face image from a first image;

[0198] A first processing unit 1002 is configured to perform a first beautification processing task on the first facial information in the first facial image to obtain a second facial image;

[0199] The second processing unit 1003 is configured to perform a second beautification processing task on the second facial information in the second facial image to obtain a target facial image;

[0200] The third processing unit 1004 is configured to obtain a target image according to the target face image and the first image.

[0201] In some embodiments, the second processing unit 1003 is used to perform a second beautification processing task on the second facial information in the second facial image to obtain a third facial image; and to obtain a target facial image based on the first facial image, the second facial image and the third facial image.

[0202] In some embodiments, the first processing unit 1002 is configured to perform a first transformation process on the first facial image to obtain a first target image having the same size as the first target image; and perform a first beauty processing task on the first facial information in the first target image to obtain a second facial image.

[0203] The second processing unit 1003 is used to perform a second transformation process on the second facial image to obtain a second target image with the same size as the second target image; and perform a second beauty processing task on the second facial information in the second target image to obtain a target facial image.

[0204] In some embodiments, the second processing unit 1003 is used to perform a second beauty processing task on the second facial information in the second target image to obtain a third facial image; and to obtain the target facial image based on the first facial image, the second facial image and the third facial image.

[0205] In some embodiments, the second processing unit 1003 is used to subtract the second facial image from the first target image to obtain a first beauty difference image; subtract the third facial image from the second target image to obtain a second beauty difference image; perform weighted fusion on the first beauty difference image and the second beauty difference image to obtain a third beauty difference image; wherein the third beauty difference image includes the beauty residual value of each pixel in the first facial image; and obtain the target facial image based on the first facial image and the third beauty difference image.

[0206] In some embodiments, the second processing unit 1003 is used to perform weighted fusion on the first beauty difference image and the second beauty difference image to obtain a third beauty difference image, including: performing a second inverse transformation corresponding to the second transformation processing on the second beauty difference image to obtain the inversely transformed second beauty difference image; performing weighted fusion on the first beauty difference image and the inversely transformed second beauty difference image according to a target fusion weight to obtain a fused beauty difference image; and performing a first inverse transformation corresponding to the first transformation processing on the fused beauty difference image to obtain the third beauty difference image.

[0207] In some embodiments, the second processing unit 1003 is configured to perform edge transition processing on the third beauty difference image to obtain a fourth beauty difference image; and add the first face image and the fourth beauty difference image to obtain a target face image.

[0208] In some embodiments, the first transformation processing includes at least one of the following: rotating the first facial image when the size relationship between the width and height of the first facial image is inconsistent with the size relationship between the first width and the first height of the first target image; scaling the first facial image when the width of the first facial image is greater than the first width, or the height of the first facial image is greater than the first height; and edge filling the first facial image according to the first width and the first height.

[0209] In some embodiments, the first processing unit 1002 is further used to obtain rotation information, scale transformation information and edge filling information of the first facial image to form first transformation information; save the first transformation information for performing a first inverse transformation corresponding to the first transformation.

[0210] In some embodiments, the first processing unit 1002 is configured to perform the second transformation process including image reduction.

[0211] In some embodiments, a first beautification processing task is performed on the first facial information in the first facial image through a first neural network model to obtain a second facial image; and a second beautification processing task is performed on the second facial information in the second facial image through a second neural network model to obtain a target facial image.

[0212] In some embodiments, the first neural network model includes an input layer, at least one convolutional layer, at least one transposed convolutional layer, and an output layer, and the convolutional layers and transposed convolutional layers in the same layer are connected via a connection layer;

[0213] The first processing unit 1002 is used to perform pixel rearrangement processing on the first input image through the input layer to obtain multiple first feature maps smaller than the size of the first input image; perform convolution processing on the multiple first feature maps in sequence through at least one convolution layer to obtain multiple second feature maps; perform transposed convolution processing on the multiple second feature maps in sequence through at least one transposed convolution layer to obtain multiple third feature maps; and perform pixel restoration processing on the multiple third feature maps through the output layer to obtain a first output image.

[0214] In some embodiments, the first neural network model further includes a subtraction layer; the first processing unit 1002 is further configured to subtract the first output image from the first input image through the subtraction layer to obtain a first beauty difference image.

[0215] In some embodiments, the device also includes a training unit for obtaining a first input sample image of a first neural network model; performing beauty processing on the first input sample image according to the beauty processing task performed by the first neural network model to obtain a first target sample image; and training the first neural network model using the first input sample image and the first target sample image.

[0216] In some embodiments, the training unit is further used to beautify the sample image through the trained first neural network model to obtain a second input sample image of the second neural network model; perform beautification processing on the second input sample image according to the beautification processing task performed by the second neural network model to obtain a second target sample image; and train the second neural network model using the second input sample image and the second target sample image.

[0217] In some embodiments, the acquisition unit 1001 is used to perform face detection on the first image to determine at least one face frame; determine a first face frame whose face frame area is greater than or equal to an area threshold from the at least one face frame; enlarge the first face frame according to a target magnification factor; and obtain a first face image from the first image based on the enlarged first face frame.

[0218] In some embodiments, the device further includes: a fourth processing unit, configured to perform back-end beautification processing on the target facial image in response to a user's beautification processing operation on the target facial image.

[0219] In practical applications, the above-mentioned device can be an electronic device that implements image beautification processing, or it can be a chip applied to an electronic device. In the present application, the device can implement the functions of multiple units through software, hardware, or a combination of software and hardware, so that the device can execute the image beautification processing method provided in any of the above embodiments. The technical effects of each technical solution of the device can refer to the technical effects of the corresponding technical solutions in the image beautification processing method, and this application will not elaborate on them one by one.

[0220] Based on the hardware implementation of each unit in the above-mentioned image beautification processing device, an embodiment of the present application further provides an electronic device, as shown in FIG11 , the electronic device 110 includes: a processor 1101 and a memory 1102 configured to store a computer program that can be run on the processor;

[0221] The processor 1101 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0222] Of course, in actual application, as shown in Figure 11, the various components in the electronic device are coupled together via bus system 1103. It will be understood that bus system 1103 is used to enable communication between these components. In addition to the data bus, bus system 1103 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all of these buses are labeled as bus system 1103 in the figure.

[0223] In practical applications, the above-mentioned processor may be one or more processors selected from the group consisting of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), and a neural network processor (NPU). The neural network model in the embodiment of the present application may run on a GPU, a DSP, or an NPU processor.

[0224] In practical applications, the processor may also be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understood that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application do not specifically limit this.

[0225] The above-mentioned memory can be a volatile memory (volatile memory), such as a random-access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor.

[0226] Optionally, the electronic device 110 further includes an imaging unit 1104 that outputs the image being captured to a processor or memory.

[0227] Optionally, the electronic device 110 further includes an input / output unit 1105, including but not limited to a display, a touch screen, buttons, etc. The user interacts through the input / output unit, including displaying images, etc.

[0228] In an exemplary embodiment, the present application also provides a computer-readable storage medium, such as a memory including a computer program, which can be executed by a processor of an electronic device to complete the steps of the aforementioned method.

[0229] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0230] Optionally, the computer program product can be applied to the electronic device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0231] The embodiment of the present application also provides a computer program.

[0232] Optionally, the computer program can be applied to the electronic device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0233] It should be understood that in the embodiments of the present application, when user information and other related data are involved, when the embodiments of the present application are applied to specific products or technologies, user permission or consent must be obtained, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0234] It should be understood that the terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items. The expressions "having", "can have", "including" and "comprising", or "can include" and "can include" in this application can be used to indicate the presence of corresponding features (e.g., elements such as numerical values, functions, operations or components), but do not exclude the presence of additional features.

[0235] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various types of information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another and are not necessarily used to describe a specific order or precedence. For example, first information could be referred to as second information, and similarly, second information could be referred to as first information without departing from the scope of the present invention.

[0236] The technical solutions described in the embodiments of this application can be combined arbitrarily unless there is any conflict.

[0237] In the several embodiments provided in this application, it should be understood that the disclosed methods, devices and equipment can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0238] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0239] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0240] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application. Industrial Applicability

[0241] In an embodiment of the present application, a method and apparatus, device, and storage medium for image beautification processing are provided. The method comprises: obtaining a first facial image in a first image; performing a first beautification processing task on the first facial information in the first facial image to obtain a second facial image; performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image; and obtaining a target image based on the target facial image and the first image. In this way, according to the characteristics of different facial beautification processing tasks, the complex facial beautification processing is reasonably divided into multiple beautification processing tasks, and multiple beautification processing tasks are performed on the facial image in stages, which can improve the processing capabilities of different tasks, improve the beautification effect of the face, and do not affect the image quality of non-face areas.

Claims

1. A method for image beautification, wherein: The method comprises: Acquire a first face image in the first image; Performing a first beauty processing task on the first facial information in the first facial image to obtain a second facial image; performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image; A target image is obtained according to the target face image and the first image.

2. The method according to claim 1, wherein: The performing a first beautification processing task on the first face information in the first face image includes: Performing a first transformation process on the first face image to obtain a first target image having a size consistent with the first target image; Performing a first beauty processing task on the first face information in the first target image to obtain the second face image; The performing a second beautification processing task on the second facial information in the second facial image to obtain a target facial image includes: Performing a second transformation process on the second facial image to obtain a second target image having a size consistent with that of the second target image; A second beauty processing task is performed on the second facial information in the second target image to obtain the target facial image.

3. The method according to claim 2, wherein: The performing a second beautification task on the second face information in the second target image to obtain the target face image includes: performing a second beauty processing task on the second face information in the second target image to obtain a third face image; The target facial image is obtained according to the first facial image, the second facial image and the third facial image.

4. The method according to claim 3, wherein: The step of obtaining the target face image according to the first face image, the second face image and the third face image includes: Subtracting the second face image from the first target image to obtain a first beauty difference image; subtracting the third face image from the second target image to obtain a second beauty difference image; Performing weighted fusion on the first beautification difference image and the second beautification difference image to obtain a third beautification difference image; wherein the third beautification difference image includes a beautification residual value of each pixel in the first face image; The target face image is obtained according to the first face image and the third beauty difference image.

5. The method according to claim 4, wherein: The weighted fusion of the first beauty difference image and the second beauty difference image to obtain a third beauty difference image includes: performing a second inverse transformation process corresponding to the second transformation process on the second beautification difference image to obtain a second beautification difference image after inverse transformation; performing weighted fusion on the first beautification difference image and the inversely transformed second beautification difference image according to a target fusion weight, so as to obtain a fused beautification difference image; A first inverse transformation process corresponding to the first transformation process is performed on the fused beautification difference image to obtain the third beautification difference image.

6. The method according to claim 4, wherein: The step of obtaining the target face image according to the first face image and the third beauty difference image includes: performing edge transition processing on the third beautification difference image to obtain a fourth beautification difference image; The first facial image and the fourth beauty difference image are added to obtain the target facial image.

7. The method according to claim 2, wherein: The first transformation process is used to transform the size of the first face into an image size specified by the first beauty processing task; The second transformation process is used to transform the size of the second facial image into an image size specified by the second beauty processing task.

8. The method according to claim 2, wherein: The first transformation process includes at least one of the following: When the size relationship between the width and the height of the first facial image is inconsistent with the size relationship between the first width and the first height of the first target image, rotating the first facial image; When the width of the first facial image is greater than the first width, or the height of the first facial image is greater than the first height, performing a scale transformation on the first facial image; Perform edge padding on the first facial image according to the first width and the first height.

9. The method according to claim 2, wherein: The second transformation process includes: image reduction.

10. The method according to claim 1, wherein: The method comprises: Performing a first beauty processing task on the first facial information in the first facial image through a first neural network model to obtain a second facial image; A second beauty processing task is performed on the second facial information in the second facial image through a second neural network model to obtain a target facial image.

11. The method according to claim 10, wherein: The first neural network model includes an input layer, at least one convolutional layer, at least one transposed convolutional layer and an output layer, and the convolutional layer and the transposed convolutional layer in the same layer are connected through a connection layer; The first neural network model is configured as follows: Performing pixel rearrangement processing on the first target image through the input layer to obtain a plurality of first feature maps smaller than the size of the first target image, wherein the first target image is an image obtained after the first transformation processing is performed on the first face image and has the same size as the first neural network model input image; Convolution processing is performed on the multiple first feature maps in sequence through at least one convolution layer to obtain multiple second feature maps; transposition convolution processing is performed on the multiple second feature maps in sequence through at least one transposed convolution layer to obtain multiple third feature maps; pixel restoration processing is performed on the multiple third feature maps through the output layer to obtain a second face image.

12. The method according to claim 1, wherein: The first face information is high-frequency information in the face image, and the second face information is low-frequency information in the face image.

13. The method according to claim 1, wherein: The step of obtaining a first face image in the first image includes: Performing face detection on the first image to determine at least one face frame; Determine, from the at least one face frame, a first face frame whose face frame area is greater than or equal to an area threshold; Enlarging the first face frame according to a target enlargement factor; The first face image is acquired from the first image according to the enlarged first face frame.

14. The method according to claim 1, wherein: The acquiring of the first face image in the first image includes: acquiring the first face image in the first image whose area is greater than or equal to an area threshold.

15. The method according to claim 1, wherein: The step of obtaining a target image according to the target face image and the first image includes: The target face image is subjected to edge transition processing, and the face image after the transition processing is filled into the corresponding face position of the first image to obtain the target image.

16. The method according to claim 1, wherein: The method further comprises: In response to a user's beautification operation on the target facial image, back-end beautification processing is performed on the target facial image.

17. An image beautification processing device, wherein: The device comprises: An acquisition unit, used for acquiring a first face image in the first image; A first processing unit, configured to perform a first beauty processing task on the first face information in the first face image to obtain a second face image; A second processing unit, configured to perform a second beauty processing task on the second facial information in the second facial image to obtain a target facial image; The third processing unit is used to obtain a target image according to the target face image and the first image.

18. An electronic device, wherein: The electronic device comprises: a processor and a memory configured to store a computer program capable of running on the processor, Wherein, the processor is configured to execute the steps of the method according to any one of claims 1 to 16 when running the computer program.

19. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images