Image beautification method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image beautification method, device, electronic device and storage medium, the method comprising: obtaining a face image to be processed; performing blur processing on the face image to be processed to obtain a first low-frequency image; determining the residual between the face image to be processed and the first low-frequency image to obtain a high-frequency image to be processed of the face image to be processed; modifying the pixel values ​​in the high-frequency image to be processed that are less than a preset threshold to the preset threshold to obtain a target high-frequency image; fusing the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed. That is, according to the solution of the present disclosure, the texture information in the image can be retained while removing blackheads, thereby improving the realism of the beautified image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image beautification method, device, electronic device and storage medium. Background Art

[0002] In social networks, beautifying photos before sharing them has become the default social method for young people today. In order to attract users, various methods for beautifying facial images have emerged one after another. Usually, the scope of beautification of facial images includes skin quality, face shape, body shape, image quality, etc. Among them, skin resurfacing optimization for skin quality problems and removing blemishes to even out skin quality are the areas that users are more concerned about. Blackheads on the nose of the face are a common blemish that affects the evenness and refinement of the face. Therefore, removing blackheads can make the beautification effect of facial images better and further meet user needs.

[0003] However, there is currently a lack of methods to treat blackheads. Most methods directly smooth the nose by resurfacing the entire face. Although this method can remove blackheads, the texture details of the nose itself are also removed, resulting in a loss of realism and the effect is not ideal. Summary of the invention

[0004] The present disclosure provides an image beautification method, device, electronic device and storage medium to at least solve the problem in the related art that the nose is directly smoothed by whole-face skin resurfacing, resulting in the removal of the texture details of the nose tip itself, which loses the sense of reality and has an unsatisfactory effect. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image beautification method, the method comprising:

[0006] Obtain the face image to be processed;

[0007] Performing fuzzy processing on the face image to be processed to obtain a first low-frequency image;

[0008] Determine a residual between the face image to be processed and the first low-frequency image to obtain a high-frequency image to be processed of the face image to be processed;

[0009] Modify the pixel values ​​in the high-frequency image to be processed that are less than the preset threshold to the preset threshold, so as to obtain a target high-frequency image;

[0010] The target high-frequency image is fused with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed.

[0011] Optionally, obtaining the face image to be processed includes:

[0012] Acquire an original image, wherein the original image includes a human face;

[0013] Determine the face area in the original image based on the face key points and the face pitch angle in the original image;

[0014] Based on the position information and size information of the face region, the face region is drawn in an image window with a preset resolution to obtain a face image to be processed.

[0015] Optionally, the facial key points include eyebrow key points, mid-face key points, chin key points and cheek key points, the size information includes height information and width information, and determining the facial area in the original image based on the facial key points and the facial pitch angle includes:

[0016] Based on the eyebrow key point, the face key point and the preset expansion multiple, determining the forehead extension point corresponding to the face area in the original image;

[0017] Determining height information of a face region in the original image according to the forehead extension point, the chin key point and the face pitch angle;

[0018] Based on the cheek key points, width information of the face area in the original image is determined.

[0019] Optionally, determining a corresponding forehead extension point based on the eyebrow key point and the face key point includes:

[0020]

[0021] Among them, Xo represents the key point in the face, Xm represents the eyebrow key point, Xe represents the forehead extension point, and n represents the preset extension multiple.

[0022] Optionally, determining the height of the face area in the original image according to the forehead extension point, the chin key point and the face pitch angle includes:

[0023] H face =(face.top–face.bottom)*alpha*(beta-K pitch )

[0024] Among them, the H face represents the height information of the face area in the original image, alpha represents the first preset parameter, beta represents the second preset parameter, face.top represents the highest point in the forehead extension points, face.bottom represents the lowest point in the chin key points, and K pitch Represents the pitch angle of the face.

[0025] Optionally, fusing the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed includes:

[0026] Performing edge-preserving filtering on the face image to be processed to obtain a second low-frequency image;

[0027] fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image;

[0028] The target high-frequency image and the fused low-frequency image are superimposed to obtain a beautified image corresponding to the face image to be processed.

[0029] Optionally, the fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image includes:

[0030] Obtain a preset face mask image, wherein the mask image includes a mask value corresponding to each pixel in the face image to be processed, wherein the mask value corresponding to the pixel in the non-nose area of ​​the preset face mask image is 0, and the mask value corresponding to the pixel in the nose area is non-0;

[0031] The first low-frequency image and the second low-frequency image are fused according to the mask value to obtain a fused low-frequency image.

[0032] Optionally, the mask value is between 0 and 1, and the fusing the first low-frequency image and the second low-frequency image according to the mask value to obtain a fused low-frequency image includes:

[0033] For each pixel point, calculating a first product of the mask value and the pixel value in the first low-frequency image, and calculating a second product of a difference between 1 and the mask value and the pixel value in the second low-frequency image;

[0034] The sum of the first product and the second product is calculated, and the sum is used as the pixel value of the pixel point in the fused low-frequency image.

[0035] According to a second aspect of an embodiment of the present disclosure, there is provided an image beautification device, the device comprising:

[0036] An acquisition unit, configured to acquire a face image to be processed;

[0037] A blurring unit, configured to perform blurring processing on the face image to be processed to obtain a first low-frequency image;

[0038] A determining unit is configured to determine a residual between the face image to be processed and the first low-frequency image to obtain a high-frequency image to be processed of the face image to be processed;

[0039] A processing unit is configured to modify the pixel values ​​of the to-be-processed high-frequency image that are less than the preset threshold to the preset threshold, so as to obtain a target high-frequency image;

[0040] The fusion unit is configured to fuse the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed.

[0041] Optionally, the acquisition unit is configured to execute:

[0042] Acquire an original image, wherein the original image includes a human face;

[0043] Determine the face area in the original image based on the face key points and the face pitch angle in the original image;

[0044] Based on the position information and size information of the face region, the face region is drawn in an image window with a preset resolution to obtain a face image to be processed.

[0045] Optionally, the facial key points include eyebrow key points, mid-face key points, chin key points and cheek key points, the size information includes height information and width information, and the acquisition unit is configured to execute:

[0046] Determine the forehead extension point corresponding to the face area in the original image based on the eyebrow key point, the face key point and the preset extension multiple;

[0047] Determining height information of a face region in the original image according to the forehead extension point, the chin key point and the face pitch angle;

[0048] Based on the cheek key points, width information of the face area in the original image is determined.

[0049] Optionally, the acquisition unit is configured to execute:

[0050]

[0051] Among them, Xo represents the key point in the face, Xm represents the eyebrow key point, Xe represents the forehead extension point, and n represents the preset extension multiple.

[0052] Optionally, the acquisition unit is configured to execute:

[0053] H face =(face.top–face.bottom)*alpha*(beta-K pitch )

[0054] Among them, the Hface represents the height information of the face area in the original image, alpha represents the first preset parameter, beta represents the second preset parameter, face.top represents the highest point in the forehead extension points, face.bottom represents the lowest point in the chin key points, and K pitch Represents the pitch angle of the face.

[0055] Optionally, the fusion unit is configured to execute:

[0056] Performing edge-preserving filtering on the face image to be processed to obtain a second low-frequency image;

[0057] fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image;

[0058] The target high-frequency image and the fused low-frequency image are superimposed to obtain a beautified image corresponding to the face image to be processed.

[0059] Optionally, the fusion unit is configured to execute:

[0060] Obtain a preset face mask image, wherein the mask image includes a mask value corresponding to each pixel in the face image to be processed, wherein the mask value corresponding to the pixel in the non-nose area of ​​the preset face mask image is 0, and the mask value corresponding to the pixel in the nose area is non-0;

[0061] The first low-frequency image and the second low-frequency image are fused according to the mask value to obtain a fused low-frequency image.

[0062] Optionally, the mask value is between 0 and 1, and the fusion unit is configured to perform:

[0063] For each pixel point, calculating a first product of the mask value and the pixel value in the first low-frequency image, and calculating a second product of a difference between 1 and the mask value and the pixel value in the second low-frequency image;

[0064] The sum of the first product and the second product is calculated, and the sum is used as the pixel value of the pixel point in the fused low-frequency image.

[0065] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0066] processor;

[0067] a memory for storing instructions executable by the processor;

[0068] The processor is configured to execute the instructions to implement the image beautification method described in the first item above.

[0069] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any of the above-mentioned image beautification methods.

[0070] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the image beautification method described in the first item above is implemented.

[0071] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0072] Acquire a face image to be processed; perform fuzzy processing on the face image to be processed to obtain a first low-frequency image; determine the residual between the face image to be processed and the first low-frequency image to obtain a high-frequency image to be processed of the face image to be processed; modify the pixel values ​​in the high-frequency image to be processed that are less than the preset threshold to the preset threshold to obtain a target high-frequency image; fuse the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed.

[0073] That is to say, based on the idea of ​​separating high- and low-frequency information, the low-frequency information of the face image to be processed is extracted as the first low-frequency image, and then the corresponding high-frequency image to be processed is generated based on the first low-frequency image. Furthermore, by processing the high-frequency image to be processed, the blackheads are removed. At the same time, the texture information in the image is retained, thereby improving the realism of the beautified image.

[0074] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0076] Figure 1 The figure is a flow chart of an image beautification method according to an exemplary embodiment.

[0077] Figure 2 The figure is a block diagram of an image beautification device according to an exemplary embodiment.

[0078] Figure 3 The invention is a block diagram of an electronic device for image beautification according to an exemplary embodiment.

[0079] Figure 4The invention is a block diagram of a device for image beautification according to an exemplary embodiment. DETAILED DESCRIPTION

[0080] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0081] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0082] Figure 1 is a flow chart of an image beautification method according to an exemplary embodiment. Figure 1 As shown, the image beautification method specifically includes the following steps.

[0083] In step S11, a face image to be processed is obtained.

[0084] In this step, the face image to be processed may be image data containing a face area in any format, wherein the specific steps of obtaining the face image to be processed may include:

[0085] An original image is obtained, wherein the original image includes a face; a face region in the original image is determined based on face key points and face pitch angles in the original image; and the face region is drawn in an image window of a preset resolution based on position information and size information of the face region to obtain a face image to be processed.

[0086] Among them, the trained deep neural network can be used to identify the facial key points and facial pitch angles in the original image to obtain the facial key point position information K f And the face pitch angle K pitch The key points of the face may include eyebrow key points, mid-face key points, chin key points, cheek key points, lip key points, nose key points, etc., without specific limitation. The key points of the face and the face pitch angle can be used to estimate the face area, that is, the minimum bounding box of the face. The bounding box needs to cover the entire face information and as much neck area as possible.

[0087] The position information of the face area can indicate the position of the face area in the original image, and the size information includes the height information and width information of the face area. Then, after obtaining the position information and size information of the face area, the face area can be cut out and drawn on an image window with a fixed resolution k to obtain a face image S. In this way, the consistency of the face scale in subsequent calculations can be guaranteed, and faces of different sizes can be reduced. The size of blackheads is also variable relative to the image, which leads to subsequent misprocessing.

[0088] In one implementation, based on facial key points and facial pitch angle, determining a facial region in an original image includes:

[0089] Based on the eyebrow key points, the mid-face key points and the preset expansion multiples, the forehead extension points corresponding to the face area in the original image are determined; based on the forehead extension points, the chin key points and the face pitch angle, the height information of the face area in the original image is determined; based on the cheek key points, the width information of the face area in the original image is determined.

[0090] It can be understood that in the related art, the key points of the face usually identified do not include the forehead extension point. In this way, the recognition of the face area and the subsequent processing of blackheads are not perfect. Through this solution, the forehead extension point can be calculated, and then the height information and width information of the face area can be determined, so as to more accurately identify the face area and the nose area in the face area, which is also conducive to the subsequent beautification processing of blackheads.

[0091] For example, if the face key point is Xo and the eyebrow key point is Xm, the forehead extension point Xe is defined as the point on the extension line of points Xo and Xm, and the extension multiple is set to n. Based on this, the coordinates of the forehead extension point Xe can be calculated using the following formula:

[0092]

[0093] That is, based on the eyebrow key point, the face center key point and the preset expansion multiple, the forehead extension point corresponding to the face area in the original image is determined, including: calculating the difference between the coordinates of the eyebrow key point and the face center key point, adding the product of the difference and the preset expansion multiple to the coordinates of the face center key point, and obtaining the coordinates of the corresponding forehead extension point. After determining the forehead extension point, the height information of the face area can be further calculated, which is conducive to the subsequent beautification of blackheads.

[0094] Furthermore, according to the forehead extension point, the chin key point and the face pitch angle, the height of the face area in the original image can be determined using the following formula:

[0095] H face =(face.top–face.bottom)*alpha*(beta-Kpitch )

[0096] Among them, H face is the height of the face area. Alpha and beta are both adjustable preset parameters. face.top is the highest point among the forehead extension points, face.bottom is the lowest point among the chin key points, and K pitch The pitch angle of the face.

[0097] That is to say, the difference between the highest point among the forehead extension points and the lowest point among the chin key points is calculated as the first reference value; the difference between the first preset parameter and the pitch angle of the face is calculated as the second reference value; the product of the first reference value, the second reference value and the second preset parameter is calculated to obtain the height of the face area in the original image.

[0098] It can be understood that when a person's face is tilted back, the distance between the highest point and the lowest point of the face decreases, while the corresponding exposed area of ​​the neck increases. Therefore, introducing the face pitch angle can better cover the face range.

[0099] In step S12, the face image to be processed is blurred to obtain a first low-frequency image.

[0100] The low-frequency information of the face image to be processed can be calculated using a mean blur with a radius of r1 to obtain a first low-frequency image. Alternatively, the blur processing can also be Gaussian blur, median blur, etc., which are not specifically limited.

[0101] In step S13, a residual between the face image to be processed and the first low-frequency image is determined to obtain a high-frequency image to be processed of the face image to be processed.

[0102] In this step, the residual between the face image to be processed and the first low-frequency image is determined to generate a high-frequency image to be processed of the face image to be processed, including: calculating the difference between the face image to be processed and the first low-frequency image, scaling and compensating the difference, and obtaining the high-frequency image to be processed of the face image to be processed.

[0103] Among them, scaling means that the color level of each pixel is divided by the scaling value. In this step, the scaling value can be 2. Compensation means that the color level of each pixel is added or subtracted by the compensation value. In this step, the compensation value can be 128.

[0104] In step S14, pixel values ​​in the high-frequency image to be processed that are smaller than a preset threshold are modified to the preset threshold to obtain a target high-frequency image.

[0105] The preset threshold is the truncated empirical value T of the pixel value in the high-frequency image to be processed. The pixel points with values ​​higher than T in the high-frequency image to be processed represent blackheads. Then, the high-frequency image to be processed is truncated according to T, that is, the pixel values ​​corresponding to the positions less than the empirical value T in the high-frequency image to be processed are set to T. Because blackheads are local dark spots in the high-frequency image to be processed, removing the dark part information can achieve the removal of blackheads in the image to be processed.

[0106] In step S15, the target high-frequency image is fused with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed.

[0107] In this step, the target high-frequency image and the first low-frequency image are superimposed to obtain a beautified image corresponding to the face image to be processed, including:

[0108] The face image to be processed is subjected to edge-preserving filtering to obtain a second low-frequency image; the first low-frequency image and the second low-frequency image are fused to obtain a fused low-frequency image; and the target high-frequency image and the fused low-frequency image are superimposed to obtain a beautified image corresponding to the face image to be processed.

[0109] Among them, the edge-preserving filtering process can be a surface blurring process. The second low-frequency image processed by the edge-preserving filtering includes more structural information than the structural information included in the first low-frequency image. The surface blurring process is performed based on the radius r2 and the threshold Y, and r2=r1, which are all set based on the size of the normalized face. Y is the threshold set after parameter tuning. The superimposed target high-frequency image and the fused low-frequency image can be superimposed by linear light, because H only processes the dark part in the high frequency, and there will be no loss of texture information in the non-dark part, that is, the non-blackhead area. Since surface blurring is an edge-preserving filter, the fusion of the first low-frequency image L1 and the second low-frequency image L2 can retain the stereoscopic effect of the nose area to the greatest extent, and further retain the structural information on the basis of removing defects.

[0110] The step of fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image includes: obtaining a preset face mask image, wherein the mask image includes a mask value corresponding to each pixel in the face image to be processed, wherein the mask value corresponding to the pixel in the non-nose area in the preset face mask image is 0, and the mask value corresponding to the pixel in the nose area is non-0; and according to the mask value, fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image.

[0111] It can be understood that the preset face mask image can quickly distinguish the nose area and the non-nose area in the face area, which can improve the speed and efficiency of image beautification compared to the method of feature recognition of the face area.

[0112] Specifically, the mask value is between 0 and 1. According to the mask value, the first low-frequency image and the second low-frequency image are fused to obtain a fused low-frequency image, including: for each pixel point, calculating a first product of the mask value and the pixel value in the first low-frequency image, and calculating a second product of the difference between 1 and the mask value and the pixel value in the second low-frequency image; calculating the sum of the first product and the second product, and using the obtained sum as the pixel value of the pixel point in the fused low-frequency image.

[0113] For example, the mask image includes the entire face area, and the mask image is not a pure 1 or 0 binary mask value image, wherein the mask value of the non-nose area is 0, and the mask value of the nose covering area is between 0 and 1. The area where edge information needs to be retained has a mask value closer to 1, and vice versa.

[0114] The following formula can be used to fuse the first low-frequency image and the second low-frequency image:

[0115] S1=aL1+(1-a)L2

[0116] Among them, a represents the mask value, and S1 represents the fused low-frequency image, that is, a smooth and uniform nose result with all defects removed. In this way, according to the mask value of each pixel in the face mask, the first low-frequency image and the second low-frequency image can be processed to obtain an image beautification result and remove blackheads in the nose area.

[0117] As can be seen from the above, the technical solution provided by the embodiment of the present disclosure is based on the idea of ​​separating high- and low-frequency information, and extracts the low-frequency information of the face image to be processed as the first low-frequency image. Then, the corresponding high-frequency image to be processed is generated based on the first low-frequency image. Furthermore, by processing the high-frequency image to be processed, blackheads are removed. At the same time, the texture information in the image is retained, thereby improving the realism of the beautified image.

[0118] Figure 2 is a block diagram of an image beautification device according to an exemplary embodiment, the device comprising:

[0119] An acquisition unit 201 is configured to acquire a face image to be processed;

[0120] A blurring unit 202 is configured to perform blurring processing on the face image to be processed to obtain a first low-frequency image;

[0121] A determination unit 203 is configured to determine a residual between the face image to be processed and the first low-frequency image to obtain a high-frequency image to be processed of the face image to be processed;

[0122] The processing unit 204 is configured to modify the pixel values ​​in the high-frequency image to be processed that are less than the preset threshold to the preset threshold, so as to obtain a target high-frequency image;

[0123] The fusion unit 205 is configured to fuse the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed.

[0124] In one implementation, the acquisition unit is configured to execute:

[0125] Acquire an original image, wherein the original image includes a human face;

[0126] Determine the face area in the original image based on the face key points and the face pitch angle in the original image;

[0127] Based on the position information and size information of the face region, the face region is drawn in an image window with a preset resolution to obtain a face image to be processed.

[0128] In one implementation, the facial key points include eyebrow key points, mid-face key points, chin key points and cheek key points, the size information includes height information and width information, and the acquisition unit is configured to execute:

[0129] Determine the forehead extension point corresponding to the face area in the original image based on the eyebrow key point, the face key point and the preset extension multiple;

[0130] Determining height information of a face region in the original image according to the forehead extension point, the chin key point and the face pitch angle;

[0131] Based on the cheek key points, width information of the face area in the original image is determined.

[0132] In one implementation, the acquisition unit is configured to execute:

[0133]

[0134] Among them, Xo represents the key point in the face, Xm represents the eyebrow key point, Xe represents the forehead extension point, and n represents the preset extension multiple.

[0135] In one implementation, the acquisition unit is configured to execute:

[0136] H face =(face.top–face.bottom)*alpha*(beta-K pitch )

[0137] Among them, the H facerepresents the height information of the face area in the original image, alpha represents the first preset parameter, beta represents the second preset parameter, face.top represents the highest point in the forehead extension points, face.bottom represents the lowest point in the chin key points, and K pitch Represents the pitch angle of the face.

[0138] In one implementation, the fusion unit is configured to execute:

[0139] Performing edge-preserving filtering on the face image to be processed to obtain a second low-frequency image;

[0140] fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image;

[0141] The target high-frequency image and the fused low-frequency image are superimposed to obtain a beautified image corresponding to the face image to be processed.

[0142] In one implementation, the fusion unit is configured to execute:

[0143] Obtain a preset face mask image, wherein the mask image includes a mask value corresponding to each pixel in the face image to be processed, wherein the mask value corresponding to the pixel in the non-nose area of ​​the preset face mask image is 0, and the mask value corresponding to the pixel in the nose area is non-0;

[0144] The first low-frequency image and the second low-frequency image are fused according to the mask value to obtain a fused low-frequency image.

[0145] In one implementation, the mask value is between 0 and 1, and the fusion unit is configured to perform:

[0146] For each pixel point, calculating a first product of the mask value and the pixel value in the first low-frequency image, and calculating a second product of a difference between 1 and the mask value and the pixel value in the second low-frequency image;

[0147] The sum of the first product and the second product is calculated, and the sum is used as the pixel value of the pixel point in the fused low-frequency image.

[0148] As can be seen from the above, the technical solution provided by the embodiment of the present disclosure is based on the idea of ​​separating high- and low-frequency information, and extracts the low-frequency information of the face image to be processed as the first low-frequency image. Then, the corresponding high-frequency image to be processed is generated based on the first low-frequency image. Furthermore, by processing the high-frequency image to be processed, blackheads are removed. At the same time, the texture information in the image is retained, thereby improving the realism of the beautified image.

[0149] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0150] Figure 3 The invention is a block diagram of an electronic device for image beautification according to an exemplary embodiment.

[0151] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to perform the above method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0152] In an exemplary embodiment, a computer program product is also provided. When the computer program product is executed on a computer, the computer implements the above-mentioned image beautification method.

[0153] As can be seen from the above, the technical solution provided by the embodiment of the present disclosure is based on the idea of ​​separating high- and low-frequency information, and extracts the low-frequency information of the face image to be processed as the first low-frequency image. Then, the corresponding high-frequency image to be processed is generated based on the first low-frequency image. Furthermore, by processing the high-frequency image to be processed, blackheads are removed. At the same time, the texture information in the image is retained, thereby improving the realism of the beautified image.

[0154] Figure 4 is a block diagram of a device 800 for image beautification according to an exemplary embodiment.

[0155] For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast electronic device, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0156] Reference Figure 4 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0157] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0158] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0159] The power supply component 807 provides power to the various components of the device 800. The power supply component 807 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0160] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0161] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0162] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0163] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0164] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0165] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the methods described in the first and second aspects.

[0166] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to complete the above method. Optionally, for example, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0167] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to execute the image beautification method described in the above embodiment.

[0168] As can be seen from the above, the technical solution provided by the embodiment of the present disclosure is based on the idea of ​​separating high- and low-frequency information, and extracts the low-frequency information of the face image to be processed as the first low-frequency image. Then, the corresponding high-frequency image to be processed is generated based on the first low-frequency image. Furthermore, by processing the high-frequency image to be processed, blackheads are removed. At the same time, the texture information in the image is retained, thereby improving the realism of the beautified image.

[0169] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0170] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image beautification method, characterized in that: The method comprises: Obtain the face image to be processed; Performing fuzzy processing on the face image to be processed to obtain a first low-frequency image; Calculating the residual between the face image to be processed and the first low-frequency image to obtain a high-frequency image to be processed of the face image to be processed; Modify the pixel values ​​in the high-frequency image to be processed that are less than a preset threshold to the preset threshold, so as to obtain a target high-frequency image; fusing the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed; The step of fusing the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed includes: Performing edge-preserving filtering on the face image to be processed to obtain a second low-frequency image; Obtain a preset face mask image, wherein the mask image includes a mask value corresponding to each pixel in the face image to be processed, wherein the mask value corresponding to the pixel in the non-nose area of ​​the preset face mask image is 0, and the mask value corresponding to the pixel in the nose area is non-0; According to the mask value, fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image; The target high-frequency image and the fused low-frequency image are superimposed to obtain a beautified image corresponding to the face image to be processed.

2. The image beautification method according to claim 1, characterized in that: The step of obtaining a face image to be processed comprises: Acquire an original image, wherein the original image includes a human face; Determine the face area in the original image based on the face key points and the face pitch angle in the original image; Based on the position information and size information of the face region, the face region is drawn in an image window with a preset resolution to obtain a face image to be processed.

3. The image beautification method according to claim 2, characterized in that: The facial key points include eyebrow key points, mid-face key points, chin key points and cheek key points, the size information includes height information and width information, and determining the facial area in the original image based on the facial key points and the facial pitch angle in the original image includes: Determine the forehead extension point corresponding to the face area in the original image based on the eyebrow key point, the face key point and the preset extension multiple; Determining height information of a face region in the original image according to the forehead extension point, the chin key point and the face pitch angle; Based on the cheek key points, width information of the face area in the original image is determined.

4. The image beautification method according to claim 3, characterized in that: The determining of the corresponding forehead extension point based on the eyebrow key point, the face center key point and the preset extension multiple includes: Among them, Xo represents the key point in the face, Xm represents the eyebrow key point, Xe represents the forehead extension point, and n represents the preset extension multiple.

5. The image beautification method according to claim 3, characterized in that: The step of determining the height information of the face area in the original image according to the forehead extension point, the chin key point and the face pitch angle includes: H face =(face.top-face.bottom)*alpha*(beta-K pitch ) Among them, the H face represents the height information of the face area in the original image, alpha represents the first preset parameter, beta represents the second preset parameter, face.top represents the highest point in the forehead extension points, face.bottom represents the lowest point in the chin key points, and K pitch Represents the pitch angle of the face.

6. The image beautification method according to claim 1, characterized in that: The mask value is between 0 and 1, and according to the mask value, fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image includes: For each pixel point, calculating a first product of the mask value and the pixel value in the first low-frequency image, and calculating a second product of a difference between 1 and the mask value and the pixel value in the second low-frequency image; The sum of the first product and the second product is calculated, and the sum is used as the pixel value of the pixel point in the fused low-frequency image.

7. An image beautification device, characterized in that: The device comprises: An acquisition unit, configured to acquire a face image to be processed; A blurring unit, configured to perform blurring processing on the face image to be processed to obtain a first low-frequency image; A determining unit is configured to determine a residual between the face image to be processed and the first low-frequency image to obtain a high-frequency image to be processed of the face image to be processed; A processing unit is configured to modify pixel values ​​in the high-frequency image to be processed that are less than a preset threshold to the preset threshold, so as to obtain a target high-frequency image; a fusion unit configured to fuse the target high-frequency image with the first low-frequency image to obtain a beautified image corresponding to the face image to be processed; The fusion unit is configured to execute: Performing edge-preserving filtering on the face image to be processed to obtain a second low-frequency image; Obtain a preset face mask image, wherein the mask image includes a mask value corresponding to each pixel in the face image to be processed, wherein the mask value corresponding to the pixel in the non-nose area of ​​the preset face mask image is 0, and the mask value corresponding to the pixel in the nose area is non-0; According to the mask value, fusing the first low-frequency image and the second low-frequency image to obtain a fused low-frequency image; The target high-frequency image and the fused low-frequency image are superimposed to obtain a beautified image corresponding to the face image to be processed.

8. The image beautification device according to claim 7, characterized in that: The acquisition unit is configured to execute: Acquire an original image, wherein the original image includes a human face; Determine the face area in the original image based on the face key points and the face pitch angle in the original image; Based on the position information and size information of the face region, the face region is drawn in an image window with a preset resolution to obtain a face image to be processed.

9. The image beautification device according to claim 8, characterized in that: The facial key points include eyebrow key points, mid-face key points, chin key points and cheek key points, the size information includes height information and width information, and the acquisition unit is configured to execute: Determine the forehead extension point corresponding to the face area in the original image based on the eyebrow key point, the face key point and the preset extension multiple; Determining height information of a face region in the original image according to the forehead extension point, the chin key point and the face pitch angle; Based on the cheek key points, width information of the face area in the original image is determined.

10. The image beautification device according to claim 9, characterized in that: The acquisition unit is configured to execute: Among them, Xo represents the key point in the face, Xm represents the eyebrow key point, Xe represents the forehead extension point, and n represents the preset extension multiple.

11. The image beautification device according to claim 9, characterized in that: The acquisition unit is configured to execute: H face =(face.top-face.bottom)*alpha*(beta-K pitch ) Among them, the H face represents the height information of the face area in the original image, alpha represents the first preset parameter, beta represents the second preset parameter, face.top represents the highest point in the forehead extension points, face.bottom represents the lowest point in the chin key points, and K pitch Represents the pitch angle of the face.

12. The image beautification device according to claim 7, characterized in that: The mask value is between 0 and 1, and the fusion unit is configured to perform: For each pixel point, calculating a first product of the mask value and the pixel value in the first low-frequency image, and calculating a second product of a difference between 1 and the mask value and the pixel value in the second low-frequency image; The sum of the first product and the second product is calculated, and the sum is used as the pixel value of the pixel point in the fused low-frequency image.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image beautification method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image beautification method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image beautification method according to any one of claims 1 to 6 is implemented.

Images