Portrait beautifying processing method and device, equipment and storage medium

Abstract

The invention provides a portrait beautifying processing method and device, equipment and a storage medium, relates to the technical field of image processing, and is used for solving the problem of how to optimize a portrait beautifying processing process and improve the refinement degree of portrait beautifying processing. The method comprises the following steps: acquiring a contour feature image and a detail feature image of a to-be-beautified image; then, according to gradient values corresponding to a plurality of key point features in the detail feature image, a weighting coefficient of each key point feature is determined, and the key point features are features of junctions between different facial features; and then, according to the weighting coefficients of the plurality of key point features, performing weighting processing on each key point feature in the detail feature image to obtain a first image. And finally, fusing the first image and the contour feature image to obtain a target beautified image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a processing method, device, equipment and storage medium for portrait beautification. Background Art

[0002] With the development of communication technology, people have a higher demand for video call technology in daily communication. During video calls, users pay more and more attention to the appearance of portraits, and portrait beautification technology comes into being.

[0003] Traditional portrait beautification techniques are usually based on portrait beautification algorithms that automatically adjust and optimize images by analyzing facial features. They can smooth the skin and sharpen key parts such as eyes and lips while maintaining a natural appearance.

[0004] However, since image beautification processing has a low effect on details, portrait beautification has a poor effect on smoothing details, which affects the accuracy and naturalness of portrait beautification processing. Therefore, how to optimize the portrait beautification process and improve the refinement of portrait beautification processing has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a portrait beautification processing method, device, equipment and storage medium, which are used to solve the problem of how to optimize the portrait beautification processing process and improve the level of refinement of the portrait beautification processing.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a processing method for portrait beautification, the method comprising: a processing device for portrait beautification (hereinafter referred to as "processing device") obtains a contour feature image and a detail feature image of an image to be beautified. Then, according to the gradient values ​​corresponding to multiple key point features in the detail feature image, a weighting coefficient of each key point feature is determined, wherein the key point feature is a feature at the junction between different facial features. Then, according to the weighting coefficients of the multiple key point features, each key point feature in the detail feature image is weighted to obtain a first image. Finally, the first image and the contour feature image are fused to obtain a target beautified image.

[0008] The technical solution provided by the present application brings at least the following beneficial effects: in the process of processing the image to be beautified, the image to be beautified can be divided into a detail feature image and a contour feature image, and the detail feature image can be processed. First, the detail feature image can be processed based on the gradient value of the junction between multiple different facial features of the detail feature image to determine the weight corresponding to each junction. Then, according to the weight corresponding to each junction, each junction of the detail feature image can be weighted to obtain a processed detail feature image. Then, the processed detail feature image and the contour feature image are fused to obtain a beautified image to achieve the beautification of the image to be beautified. In this way, the detail layer image can be further processed to obtain a processed detail feature image to improve the beautification effect of the detail layer image, thereby improving the final effect of portrait beautification, better maintaining the original appearance of the face, and making the user's display in the video call more realistic.

[0009] Optionally, a key point feature corresponds to a preset gradient threshold. The above method of "determining the weighting coefficient of each key point feature according to the gradient values ​​corresponding to multiple key point features in the detail feature image" includes: calculating the difference between the gradient value corresponding to each key point feature and the preset gradient threshold, obtaining multiple first gradient differences, one first gradient difference corresponding to one key point feature. Determine the weighting coefficient of each key point feature according to each first gradient difference.

[0010] Optionally, the method of "obtaining a contour feature image and a detail feature image of the image to be beautified" includes: obtaining a high-frequency image and a low-frequency image of the image to be beautified. Filtering the high-frequency image to obtain the detail feature image. Filtering the low-frequency image to obtain the contour feature image.

[0011] Optionally, the method of “fusing the first image and the contour feature image to obtain a target beautified image” includes: obtaining a first transparency of the first image and a second transparency of the contour feature image. According to the first transparency and the second transparency, the first image and the contour feature image are fused to obtain the target beautified image.

[0012] Optionally, the above-mentioned method of "fusing the first image and the contour feature image according to the first transparency and the second transparency to obtain a target beautified image" includes: obtaining a first fusion coefficient of the first image and a second fusion coefficient of the contour feature image. Multiplying all pixel values ​​in the first image by the first fusion coefficient and the first transparency to obtain a second image. Multiplying all pixel values ​​in the contour feature image by the second fusion coefficient and the second transparency to obtain a third image. Adding the pixel values ​​at corresponding positions in the second image and the third image to obtain a target beautified image, each pixel value in the target beautified image is greater than a preset pixel threshold.

[0013] Optionally, before the above-mentioned method of "obtaining the contour feature image and detail feature image of the image to be beautified", the method also includes: segmenting the original image according to the target loss function to obtain multiple images to be beautified, and the target loss function includes: cross entropy loss function and Lovalz hinge loss function.

[0014] In a second aspect, the present application provides a device for portrait beautification processing, which includes: an acquisition module and a processing module.

[0015] The acquisition module is used to acquire the contour feature image and the detail feature image of the image to be beautified. The processing module is used to determine the weighting coefficient of each key point feature according to the gradient values ​​corresponding to the multiple key point features in the detail feature image, wherein the key point feature is the feature at the junction between different facial features. The processing module is also used to perform weighted processing on each key point feature in the detail feature image according to the weighting coefficients of the multiple key point features to obtain the first image. The processing module is also used to fuse the first image with the contour feature image to obtain the target beautified image.

[0016] Optionally, the processing module is specifically used to calculate the difference between the gradient value corresponding to each key point feature and the preset gradient threshold to obtain multiple first gradient differences, one first gradient difference corresponding to one key point feature. The processing module is also used to determine the weighting coefficient of each key point feature according to each first gradient difference.

[0017] Optionally, the acquisition module is used to acquire the high-frequency image and the low-frequency image of the image to be beautified. The processing module is specifically used to filter the high-frequency image to obtain the detail feature image. The processing module is used to filter the low-frequency image to obtain the contour feature image.

[0018] Optionally, the acquisition module is used to acquire the first transparency of the first image and the second transparency of the contour feature image. The processing module is used to fuse the first image and the contour feature image according to the first transparency and the second transparency to obtain a target beautified image.

[0019] Optionally, the acquisition module is used to acquire the first fusion coefficient of the first image and the second fusion coefficient of the contour feature image. The processing module is used to multiply all pixel values ​​in the first image by the first fusion coefficient and the first transparency to obtain the second image. The processing module is also used to multiply all pixel values ​​in the contour feature image by the second fusion coefficient and the second transparency to obtain the third image. The processing module is also used to add the pixel values ​​at corresponding positions in the second image and the third image to obtain the target beautified image, wherein each pixel value in the target beautified image is greater than a preset pixel threshold.

[0020] Optionally, the processing module is specifically used to segment the original image according to the target loss function to obtain multiple images to be beautified. The target loss function includes: a cross entropy loss function and a Lovalz hinge loss function.

[0021] In a third aspect, the present application provides a device for portrait beautification processing, the device comprising: a processor and a memory, the processor and the memory are coupled, the memory is used to store one or more programs, the one or more programs include computer execution instructions, when the portrait beautification processing device is running, the processor executes the computer execution instructions stored in the memory to implement the portrait beautification processing method described in the first aspect or any optional aspect of the first aspect.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method for portrait beautification described in the first aspect or any optional aspect of the first aspect.

[0023] In a fifth aspect, the present application provides a computer program product applied to a server, the computer program product comprising computer instructions. When the computer instructions are executed on the server, the server implements the method for portrait beautification described in the first aspect or any optional aspect of the first aspect.

[0024] In the above scheme, the technical problems that can be solved and the technical effects achieved by the portrait beautification processing device, equipment, computer storage medium or computer program product can be referred to the technical problems and technical effects solved by the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a communication system provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of a process flow of a portrait beautification processing method provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of a process flow of another method for beautifying a portrait provided in an embodiment of the present application;

[0028] Figure 4 An example flow chart of another method for beautifying a portrait provided in an embodiment of the present application;

[0029] Figure 5 This is a schematic diagram of an example of a portrait segmentation algorithm flow chart provided in an embodiment of the present application;

[0030] Figure 6A schematic diagram of an example of a portrait beautification processing method provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of an example of a rendering and transmission method provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of a portrait beautification processing device provided in an embodiment of the present application;

[0033] Fig. 9 A structural schematic diagram of a portrait beautification processing device provided in an embodiment of the present application;

[0034] Fig.10 A conceptual partial view of a computer program product provided for an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] In this article, the character " / " generally indicates that the objects before and after are in an "or" relationship. For example, A / B can be understood as A or B.

[0037] The terms “first” and “second” in the description and claims of the present application are used to distinguish different objects rather than to describe a specific order of the objects.

[0038] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include other steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.

[0039] In addition, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present concepts in a specific way.

[0040] Before introducing in detail the portrait beautification processing method provided by the embodiment of the present application, the implementation environment and application scenarios of the embodiment of the present application are first introduced.

[0041] First, the application scenarios of the embodiments of the present application are introduced.

[0042] With the development of communication technology, video calls have become one of the important ways for people to communicate in daily life, especially in the fields of remote office, distance education and social media. However, during video calls, the appearance of the portrait is crucial to the user experience. Many research and commercial solutions focus on providing real-time face beautification effects, especially in video calls and social media applications. In order to improve the appearance of the face and the environment in the video call, portrait beautification technology has emerged.

[0043] Existing portrait beautification technologies are usually based on image processing and computer vision methods. Currently, in terms of portrait beautification, common technologies include:

[0044] (1) Skin smoothing and blemish repair: By using image processing algorithms such as frequency domain filtering, mean shift, Gaussian blur, etc., the skin of portraits is smoothed and blemishes are repaired to improve problems such as uneven skin tone and acne.

[0045] (2) Whitening and skin tone adjustment: Whiten and adjust the skin tone of the portrait by adjusting parameters such as image brightness, contrast, and color balance to make the skin tone look brighter and more even.

[0046] (3) Face slimming and lengthening: By applying geometric transformation algorithms, such as affine transformation or nonlinear transformation, the shape of the face is fine-tuned to achieve the effect of face slimming and lengthening.

[0047] (4) Feature enhancement: By enhancing the brightness of the eyes, increasing the contrast of the eyes, improving the effect of eyelashes, etc., the features of the face are highlighted, making the eyes look brighter and more attractive.

[0048] (5) Portrait segmentation: Use traditional image segmentation technology or deep learning segmentation technology to identify the human body contour in the image and separate it from the background. The real-time performance of the segmentation algorithm on the server side cannot meet the real-time requirements of the low computing power of the web side. At the same time, it supports users to change the background, add filters to the background, and use depth of field mode and other personalized operations.

[0049] (6) Real-time rendering and transmission: In order to achieve portrait beautification and segmentation in real-time video calls, efficient image processing algorithms and real-time transmission technology are required to maintain low latency and high-quality video transmission.

[0050] However, current portrait beautification technology has some disadvantages. First, traditional portrait beautification algorithms may require a long processing time, resulting in poor real-time performance and significant delays in real-time video calls. This affects the user experience because users want to see the beautification and segmentation effects immediately in real-time calls. Secondly, traditional beautification algorithms may cause over-processing or unnatural effects when processing faces, such as over-smoothing of the skin, distorted facial proportions, etc. This may reduce the quality of portrait beautification, making it difficult to ensure accuracy and naturalness, and making users feel unreal or dissatisfied. In addition, the existing segmentation algorithm has low portrait segmentation accuracy, and the blurring effect of the edge of the portrait head is not very ideal. In particular, the lightweight algorithm has a fast detection speed, but the accuracy is not enough. It is necessary to design relevant network structures and functions to achieve fine segmentation of portraits, especially the segmentation of hair edges. In addition, different users may have different needs and preferences for portrait beautification. The existing technology usually provides some preset beautification effects, but cannot fully meet the personalized needs of users, such as specific skin beautification effects and face shape adjustment.

[0051] In other words, traditional portrait beautification and segmentation algorithms usually require a long processing time and cannot meet the needs of real-time video calls. Secondly, portrait beautification algorithms may cause over-processing or unnatural effects, affecting the authenticity and naturalness of the face. In addition, the segmentation accuracy is not high. Since the personalized needs of users are also a consideration, different users have different preferences for beautification effects. The segmentation effect will also affect subsequent user personalized operations such as changing the background, adding filters to the background, and depth of field mode.

[0052] At present, in view of the above existing problems, the real-time problem and the problem of personalized needs of users can be solved by existing technical means. For example, web real-time communication (WebRTC) technology can provide audio and video acquisition, transmission and rendering capabilities, and WebRTC technology can be used to realize real-time audio and video call functions. And the quality of service (quality of service, QOS) technology is used to improve the quality of audio and video, ensuring the real-time and smoothness of the call. In this way, support can be provided for the image processing process to ensure that fast beautification processing can be achieved in real-time video calls, reduce delays, and improve user experience. For another example, the general preset beautification effect is changed to make the user adjustable beautification parameter setting, allowing the user to customize the beautification effect according to personal needs and preferences. This personalized customization capability enables users to adjust the parameters of the beautification algorithm according to their preferences to achieve a beautification effect that meets personal aesthetic requirements. In this way, flexible parameter settings and adjustable beautification effects are provided. Users can adjust the parameters of the beautification algorithm according to their preferences and requirements to achieve personalized beautification effects to meet the needs of different users.

[0053] However, for video call portrait beautification, the current image beautification processing is not very effective in processing details, resulting in poor smoothing effect of portrait beautification on details, thereby affecting the accuracy and naturalness of portrait beautification processing.

[0054] In summary, how to optimize the portrait beautification process and improve the level of refinement of portrait beautification processing has become a technical problem that needs to be solved urgently.

[0055] In order to solve the above problems, the embodiment of the present application provides a processing method for portrait beautification, and the processing method for portrait beautification provided by the embodiment of the present application is applied to the scene of portrait beautification. In the process of processing the image to be beautified, the embodiment of the present application can divide the image to be beautified into a detail feature image and a contour feature image, and process the detail feature image. First, the detail feature image can be processed based on the gradient value of the intersection between multiple different facial features of the detail feature image to determine the weight corresponding to each intersection. Then, according to the weight corresponding to each intersection, each intersection of the detail feature image can be weighted to obtain a processed detail feature image. Then, the processed detail feature image and the contour feature image are fused to obtain a beautified image to achieve the beautification of the image to be beautified. In this way, the detail layer image can be further processed to obtain a processed detail feature image to improve the beautification effect of the detail layer image, thereby improving the final effect of portrait beautification, better maintaining the original appearance of the face, and making the user's display in the video call more realistic.

[0056] The implementation environment of the embodiments of the present application is introduced below.

[0057] like Figure 1 FIG. 1 is a schematic diagram of a communication system provided in an embodiment of the present application, and the communication system may include: a portrait beautification processing device 101 and a collection device 102. The processing device 101 may be connected to the collection device 102 by wire or wirelessly.

[0058] Specifically, the acquisition device 102 may acquire the contour feature image and the detail feature image in the image to be beautified, wherein the contour feature image is used to represent the contour information of the image, and the detail feature image is used to represent the detail information of the image. The acquisition device 102 may upload the acquired contour feature image and detail feature image to the processing device 101. Then, the processing device 101 may receive the contour feature image and detail feature image from the acquisition device 102, and process the detail feature image. The processing device 101 may determine the weight corresponding to each junction based on the gradient value of the junction between multiple different facial features of the detail feature image. Then, the processing device 101 may perform weighted processing on each junction of the detail feature image according to the weight corresponding to each junction, obtain the processed detail feature image, and fuse the processed detail feature image and the contour feature image to obtain the beautified image, so as to achieve the beautification processing of the image to be beautified.

[0059] It should be noted that the embodiment of the present application does not limit the acquisition device 102. The acquisition device 102 may be an infrared camera, or the acquisition device 102 may be a high-resolution smart phone, or the acquisition device 102 may be a microscope camera.

[0060] It should be noted that the embodiment of the present application does not limit the processing device 101. For example, the processing device 101 may be a terminal or a server.

[0061] The server may be a single physical server or a server cluster consisting of multiple servers. Alternatively, the server cluster may be a distributed cluster. Alternatively, the server may be a cloud server. The embodiments of the present application do not limit the specific implementation of the server.

[0062] The terminal may be a mobile phone, tablet computer, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, etc. with transceiver functions. The present application does not impose any special restrictions on the specific form of the terminal. It may interact with the user through one or more methods such as keyboard, touch pad, touch screen, remote control, voice interaction or handwriting device.

[0063] In other embodiments, the processing device 101 and the collection device 102 may be the same device.

[0064] After introducing the application scenario and implementation environment of the embodiment of the present application, the portrait beautification processing method provided by the embodiment of the present application is introduced in detail below in combination with the above-mentioned implementation environment.

[0065] The methods in the following embodiments can all be implemented in the above application scenarios and implementation environments. The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0066] Figure 2 A flowchart of a portrait beautification processing method provided in an embodiment of the present application.

[0067] like Figure 2 As shown, the method may include:

[0068] S201: Obtain a contour feature image and a detail feature image of an image to be beautified.

[0069] The contour feature image is an image used to represent contour features, and the detail feature image is an image used to represent detail features.

[0070] Exemplarily, the contour feature image may be a low-frequency image that has been filtered, and the detail feature image is used to indicate a high-frequency image that has been filtered.

[0071] It should be noted that the image to be beautified can be processed by Gaussian filtering to obtain two parts: a high-frequency image and a low-frequency image.

[0072] As a possible implementation, the processing device can filter the high-frequency image and the low-frequency image of the image to be beautified. The processing device filters the high-frequency image to obtain a detail feature image, and filters the low-frequency image to obtain a contour feature image.

[0073] It should be noted that high-frequency images can record the texture of the skin, and in high-frequency images, spots and blemishes on the skin can be eliminated. Low-frequency images can record the skin color, and in low-frequency images, the skin color can be adjusted.

[0074] It should be noted that the embodiments of the present application do not limit the filtering method. For example, the processing device may perform filtering by high-pass filtering, or by low-pass filtering, or by band-pass filtering.

[0075] As a possible design, the process of obtaining the high-frequency image and the low-frequency image of the image to be beautified can be expressed by the following formula 1.

[0076]

[0077] Among them, (x, y) represents the pixel coordinates, I σ represents the output image, I0 represents the input image, σ * Represents the optimal smoothing scale calculated based on local variance adaptive calculation.

[0078] For example, if (x, y) represents the pixel coordinates (2, 2), I0 represents the image to be beautified, I0(2, 2) is 100, σ * represents the optimal smoothing scale based on the local variance adaptive calculation, σ * (2, 2) is 1.5, I σ Represents low-frequency image, I σ (2, 2) is 16.9, that is, the pixel value of the pixel point in the low-frequency image is lower than that in the image to be beautified.

[0079] That is to say, in the process of image beautification, the image to be beautified can be decomposed into high-frequency image and low-frequency image, and the high-frequency image and low-frequency image are filtered respectively to obtain detail feature image and contour feature image, which can provide support for subsequent image beautification and ensure the final beautification effect.

[0080] In the embodiment of the present application, the image to be beautified includes a portrait image.

[0081] S202: Determine a weighting coefficient of each key point feature according to the gradient values ​​corresponding to the multiple key point features in the detail feature image.

[0082] The detail feature image includes images of multiple facial features in the portrait. The key point feature is the feature of the junction between different facial features.

[0083] Exemplarily, a facial feature image includes: key point feature A and key point feature B. Key point feature A is the junction between the eyes and the skin, and key point feature B is the junction between the mouth and the skin.

[0084] As a possible implementation, the processing device may pre-store a preset gradient threshold and determine a weighting coefficient of each key point feature based on the preset gradient threshold and the gradient value corresponding to each key point feature in the detail feature image.

[0085] As a possible implementation method, when the gradient value corresponding to the key point feature is greater than a preset gradient threshold, the key point feature corresponds to a first weighting coefficient, and when the gradient value corresponding to the key point feature is less than the preset gradient threshold, the key point feature corresponds to a second weighting coefficient.

[0086] Exemplarily, the detail feature image includes: key point feature A, key point feature B, key point feature C, key point feature D, and key point feature E. Among them, the gradient value corresponding to the key point feature A is 60, the gradient value corresponding to the key point feature B is 30, the gradient value corresponding to the key point feature C is 80, the gradient value corresponding to the key point feature D is 20, and the gradient value corresponding to the key point feature E is 70. If the preset gradient threshold is 50, the first weighting coefficient is 2, and the second weighting coefficient is 0.5. Then, the weighting coefficient of the key point feature A is 2, the weighting coefficient of the key point feature B is 0.5, the weighting coefficient of the key point feature C is 2, the weighting coefficient of the key point feature D is 0.5, and the weighting coefficient of the key point feature E is 2.

[0087] S203 . Perform weighted processing on each key point feature in the detail feature image according to the weighted coefficients of the multiple key point features to obtain a first image.

[0088] As a possible implementation manner, the processing device may perform weighted processing on each key point feature in the detail feature image according to weighted coefficients of the multiple key point features, and combine the multiple processed detail feature images to obtain the first image.

[0089] As a possible design, the process of obtaining the high-frequency image and the low-frequency image of the image to be beautified can be expressed by the following formula 2.

[0090] I enhanced =I detail *W gradient Formula 2.

[0091] Among them, I enhanced is the pixel value corresponding to the key point feature of the processed detail feature image, I detail is the pixel value corresponding to the key point feature of the unprocessed detail feature image, W gradient is the weighting coefficient of the key point feature.

[0092] Exemplarily, the detail feature image includes: key point feature A, key point feature B, and key point feature C. Among them, the pixel value I corresponding to the key point feature A detail is 30, and the weighting coefficient W of the key point feature A gradient is 2, the pixel value I corresponding to the key point feature B detail is 40, and the weighting coefficient W of the key point feature B gradient is 0.5, and the pixel value I corresponding to the key point feature C detail is 10, and the weighting coefficient W of the key point feature C gradient is 5. Then the pixel value I corresponding to the key point feature A after processing enhanced is 60, and the pixel value I corresponding to the key point feature B after processingenhanced is 20, and the pixel value I corresponding to the processed key point feature C enhanced is 50.

[0093] S204: Fusing the first image and the contour feature image to obtain a target beautified image.

[0094] As a possible implementation method, the processing device can obtain a first transparency of the first image and a second transparency of the contour feature image. Then, the processing device can fuse the first image and the contour feature image based on the first transparency and the second transparency to obtain a target beautified image.

[0095] It should be noted that the transparency of an image is usually represented by its Alpha channel (a transparency channel). The Alpha channel is an additional channel in the image that is used to define the transparency of each pixel. The Alpha channel is an additional channel in the image that represents transparency, and the transparency range is between 0 (completely transparent) and 255 (completely opaque).

[0096] Exemplarily, if the sum of the first transparency of the first image and the second transparency of the contour feature image is 225, wherein the first transparency of the first image is 128, then the second transparency of the contour feature image is 127.

[0097] That is to say, by adjusting the transparency, the contour feature image and the detail feature image after detail enhancement are fused, so that fine control of the image fusion process can be achieved, so that under different transparency settings, the two images can have a smooth transition, so as to achieve refined image processing.

[0098] The technical solution provided by the above embodiment brings at least the following beneficial effects: in the process of processing the image to be beautified, the image to be beautified can be divided into a detail feature image and a contour feature image, and the detail feature image can be processed. First, the detail feature image can be processed based on the gradient value of the intersection between multiple different facial features of the detail feature image to determine the weight corresponding to each intersection. Then, according to the weight corresponding to each intersection, each intersection of the detail feature image can be weighted to obtain a processed detail feature image. Then, the processed detail feature image and the contour feature image are fused to obtain a beautified image to achieve the beautification of the image to be beautified. In this way, the detail layer image can be further processed to obtain a processed detail feature image to improve the beautification effect of the detail layer image, thereby improving the final effect of portrait beautification, better maintaining the original appearance of the face, and making the user's display in the video call more realistic.

[0099] It should be noted that when the first image and the contour feature image are fused to obtain the target beautified image, a better fusion effect can be achieved by weighting the contour feature image and the detail feature image after detail enhancement.

[0100] In some embodiments, the processing device may obtain a first fusion coefficient and a first transparency of the first image and a second fusion coefficient and a second transparency of the contour feature image. All pixel values ​​in the first image are multiplied by the first fusion coefficient and the first transparency to obtain a second image. Then, all pixel values ​​in the contour feature image are multiplied by the second fusion coefficient and the second transparency to obtain a third image. Then, the pixel values ​​at corresponding positions in the second image and the third image are added to obtain a target beautified image.

[0101] Wherein, each pixel value in the target beautified image is greater than a preset pixel threshold.

[0102] It should be noted that the preset pixel threshold is used to indicate the minimum pixel standard that the target beautified image should reach during the image beautification process.

[0103] As a possible design, the process of obtaining the target beautified image can be expressed by the following formula three.

[0104] dst=(α*HighPixel*alpha+β*LowPixel*(255-alpha) Formula 3.

[0105] Among them, HighPixel is all pixel values ​​in the first image, LowPixel is all pixel values ​​in the contour feature image, alpha is the first transparency, (255-alpha) is the second transparency, α is the first fusion coefficient, β is the second fusion coefficient, and dst is the pixel value of the target beautified image.

[0106] For example, if all pixel values ​​in the first image are 200, all pixel values ​​in the contour feature image are 100, the first transparency is 128, the second transparency is 127, the first fusion coefficient is 0.7, the second fusion coefficient is 0.3, and the preset pixel threshold is 8, then the pixel value of the target beautified image is 89, and the target beautified image can be obtained.

[0107] It can be understood that by introducing weighting coefficients and transparency adjustment, the contour feature image and the detail feature image after detail enhancement can be accurately processed, and different weights can be assigned to the two parts respectively to achieve the best fusion effect.

[0108] It should be noted that the above embodiment determines the weighting coefficient of each key point feature when a preset gradient threshold is set for multiple key point features. However, since each key point feature needs to be processed to a different degree, a preset gradient threshold can be set for each key feature point to determine the corresponding weighting coefficient. When different key points have the same gradient value, different weighting coefficients can be obtained according to their specific positions to optimize the features of each key point.

[0109] In some embodiments, Figure 3 As shown, in the portrait beautification process, S202 may include:

[0110] S301. Calculate the difference between the gradient value corresponding to each key point feature and a preset gradient threshold to obtain a plurality of first gradient differences.

[0111] Among them, a first gradient difference corresponds to a key point feature.

[0112] As a possible implementation, the processing device may pre-store a preset gradient threshold, and one key point feature corresponds to one preset gradient threshold. The processing device may calculate the difference between the gradient value corresponding to each key point feature and the preset gradient threshold based on the preset gradient threshold to obtain multiple first gradient differences.

[0113] For example,

[0114] S302: Determine a weighting coefficient of each key point feature according to each first gradient difference.

[0115] As a possible design, the relationship between the gradient value corresponding to the key point feature, the preset gradient threshold and the weighting coefficient can be expressed by the following formula 4.

[0116] k=f(xy) Formula 4.

[0117] Among them, X represents the gradient value corresponding to the key point feature, y represents the preset gradient threshold, and k represents the weighting coefficient.

[0118] Exemplarily, the detail feature image includes: key point feature A, key point feature B, and key point feature C. Among them, the gradient value corresponding to the key point feature A is 60, and the preset gradient threshold corresponding to the key point feature A is 40, the gradient value corresponding to the key point feature B is 30, and the preset gradient threshold corresponding to the key point feature B is 40, and the gradient value corresponding to the key point feature C is 60, and the preset gradient threshold corresponding to the key point feature A is 50. If the weighting coefficient corresponding to the first gradient difference of -10 is 0.5, the weighting coefficient corresponding to the first gradient difference of 0 is 1, the weighting coefficient corresponding to the first gradient difference of 10 is 2, and the weighting coefficient corresponding to the first gradient difference of 20 is 5, then the key point feature A corresponds to the first gradient difference of 20, and the weighting coefficient of the key point feature A is 5, the key point feature B corresponds to the first gradient difference of -10, and the weighting coefficient of the key point feature B is 0.5, and the key point feature C corresponds to the first gradient difference of 10, and the weighting coefficient of the key point feature C is 2.

[0119] It is understandable that during the image beautification process, the gradient value of each key feature point will be compared with its preset gradient threshold to determine the corresponding weighting coefficient. When different key points have the same gradient value, different weighting coefficients can be obtained according to their specific positions. In this way, the features of each key point can be refined and personalized for optimization, thereby achieving a more refined and efficient beautification effect.

[0120] It should be noted that before obtaining the contour feature image and detail feature image of the image to be beautified, the processing device can perform segmentation processing on the original image, and optimize the segmentation processing method during the segmentation processing to improve the quality of the segmented image, obtain a better image to be beautified, provide support for subsequent beautification processing, and improve the effect of subsequent beautification processing.

[0121] In some embodiments, before obtaining the contour feature image and the detail feature image of the image to be beautified, the processing device may segment the original image. The processing device may segment the original image according to the target loss function to obtain multiple images to be beautified.

[0122] The original image is an initial image without any processing or modification, and the target loss functions include: a cross entropy loss function and a Lovasz hinge loss function.

[0123] As a possible implementation method, the processing device can determine the weighting coefficients corresponding to the cross entropy loss function and the Lovalz hinge loss function respectively according to the cross entropy loss function and the Lovalz hinge loss function, and perform segmentation processing on the original image to obtain multiple images to be beautified.

[0124] As a possible design, the process of determining the weighting coefficients corresponding to the cross entropy loss function and the Lovalz hinge loss function respectively can be expressed by the following formula 5.

[0125] L combined =m*L CE +(1-m)L lovasz Formula five.

[0126] Among them, L combined It is the weighted combination of the cross entropy loss function and the Lovalz hinge loss function, L CE is the cross entropy loss function, L lovasz is the Lovalz hinge loss function, m is the weighting coefficient corresponding to the cross entropy loss function, and (1-m) is the weighting coefficient corresponding to the Lovalz hinge loss function.

[0127] Optionally, before obtaining the contour feature image and detail feature image of the image to be beautified, the processing device can also integrate the spatial and channelsqueeze and excitation (scSE) attention mechanism into the convolution layer of ShuffleNetV2 (a lightweight convolutional neural network) when segmenting the original image to enhance the feature extraction capability of the convolution layer, and obtain ShuffleNet_scSE (ShuffleNetV2 combined with the scSE attention mechanism) to improve the segmentation processing accuracy and thereby obtain a better image to be beautified.

[0128] Among them, the scSE attention mechanism includes both spatial attention and channel attention. The weights obtained by different attention mechanisms can be multiplied with the feature map respectively and then added. Compared with the series type of the convolutional block attention module (CBAM), the scSE attention mechanism combines spatial attention and channel attention in parallel. The parallel combination of spatial attention and channel attention is shown in the following steps a-c:

[0129] Step a: Channel attention: First, the feature map is average-pooled to obtain a 1×1×C vector, the channel is scaled through two convolutional layers, and then restored and mapped to a decimal between 0 and 1 to obtain the channel attention weight, which is multiplied with the original feature map. This results in the channel squeeze and excitation (cSE) module.

[0130] Step b: Convolve the feature map with spatial attention, change the number of channels to 1, and obtain the 1xHxW position information weight, which is multiplied by the features at the corresponding positions. This results in the spatial and channel squeeze and excitation (sSE) module.

[0131] Step c: Combine the spatial attention cSE module and the channel attention sSE in parallel to obtain the scSE attention mechanism module.

[0132] It is understandable that in the process of segmenting the original image, the loss function is combined and optimized. The optimized loss function helps to capture the subtle features in the image, making the segmentation result closer to the real scene, and can significantly improve the accuracy and quality of the segmentation. In this way, it can lay the foundation for the subsequent image beautification processing, so that the original image can generate a more ideal beautified image after the segmentation processing, thereby ensuring a higher final beautification effect.

[0133] The following describes the portrait beautification processing method provided by the embodiment of the present application in combination with a specific example, including the following steps 1 to 7.

[0134] like Figure 4 As shown, first, the video stream is collected through the WebRTC client and transmitted to the server. Then, the server uses the first engine (such as the WebAssembly engine) to accelerate reasoning, and combines the toolkit for image segmentation tasks (such as the PaddleSeg development kit) to perform portrait segmentation and deep learning beauty algorithm processing (i.e. portrait beautification processing). Finally, the processed image is rendered in real time on the client through the web graphics library (WebGL) and the beautified result is displayed in canvas (a kind of canvas).

[0135] Step 1: Configure the client and server. The client can be a mobile phone, computer or other device that supports WebRTC technology, and the server is responsible for image beautification and data transmission. The client and server communicate through a network connection. In order to meet the processing power and bandwidth limitations of different devices, the server can use elastic computing and load balancing technology to ensure that the system can provide efficient portrait beautification services in different scenarios.

[0136] Step 2: Realize the real-time video call function. Through WebRTC technology, the client can establish a real-time video call connection. WebRTC provides audio and video acquisition, encoding and decoding, transmission and rendering functions, including recovering from packet loss through negative acknowledgment (NACK) and forward error correction (FEC) technology to solve the audio and video freezes caused by packet loss. The encoding and sending bit rates are adjusted through bandwidth assessment and congestion control technology to automatically adapt to changes in network bandwidth. Support vector classification (SVC) and multi-view track technology are used to ensure the video quality of users with different network qualities, ensuring the real-time and smoothness of the call.

[0137] In this way, WebRTC technology is used to achieve real-time video calls, and combined with optimized algorithms and data transmission mechanisms, the real-time and low-latency of subsequent processing is ensured. The beautification effect can be presented instantly in real-time video calls, reducing processing time and transmission delays, and improving user experience.

[0138] Step 3: Face detection and feature extraction. During the video call, the client uses the face detection algorithm to detect the face area in real time and extract key feature points, such as eyes and mouth. These feature points are used for subsequent beautification processing. In this process, in order to achieve real-time performance, you can choose a lightweight face key point detection network, such as Tiny FaceDetector (an efficient face detector), CenterFace (an efficient face detection model based on deep learning), etc. These networks have fewer parameters and computational complexity, and can perform key point detection faster.

[0139] In this way, real-time face detection and feature extraction functions can be realized, providing a basis for subsequent beautification processing.

[0140] Step 4: Real-time portrait segmentation algorithm. Figure 5 As shown in the figure, first input the portrait data, after data preprocessing, use the lightweight portrait segmentation model (such as PP-HumanSegV2-Lite model) in the existing PaddleSeg suite algorithm to perform real-time portrait segmentation on the video. The lightweight model is used to add the scSE attention mechanism to the backbone network ShuffleNetV2 of the feature extraction network to improve the accuracy. At the same time, the loss function is designed as a combination of the cross entropy loss function and the Lovaz hinge loss function. Through weighted processing, the segmented image is finally output. This solves the problem of category imbalance and improves the segmentation effect.

[0141] Step 5, real-time portrait beautification algorithm. According to the real-time video stream of the client and the extracted facial feature points, the server applies the real-time portrait beautification algorithm to beautify the portrait. The beautification algorithm may include skin smoothing, blemish repair, whitening, face thinning, eye brightening and other processing technologies. Based on the principle of high- and low-frequency skin grinding, the original image is divided into high- and low-frequency layers by adaptive Gaussian filtering. The high- and low-frequency layers are filtered to obtain a basic layer image (i.e., a contour feature image) and a detail layer image (i.e., a detail feature image). A detail enhancement algorithm is added to the detail layer image (i.e., according to the gradient values ​​corresponding to the multiple key point features in the detail feature image, the weighting coefficient of each key point feature is determined. Then, according to the weighting coefficients of the multiple key point features, each key point feature in the detail feature image is weighted to obtain a first image). The processed basic layer image and detail layer image are weightedly added to obtain the final skin-grinded image (i.e., the first image and the contour feature image are fused to obtain the target beautified image), so as to achieve real-time adjustment of skin color and whitening effects.

[0142] like Figure 6 As shown, firstly, the portrait data is input, then the high-frequency image (detail information) and the low-frequency image (contour information) are separated through adaptive Gaussian filtering, the detail enhancement algorithm is performed on the high-frequency image containing the detail information, and then the two parts are weighted fused, and finally the beautified image is output. In other words, by improving on the basis of the traditional high- and low-frequency skin resurfacing algorithms, the images are fused by using the improved adaptive weighted fusion and detail enhancement algorithms, and a result image with a strong sense of detail reality and skin resurfacing effect is obtained, providing a better and faster beautification effect, improving the appearance of the face in real time, and making the user look more beautiful and natural.

[0143] In this way, the improved portrait beautification algorithm can be used to focus on maintaining the naturalness and authenticity of the face. Through carefully designed algorithms and parameter adjustments, the use of improved adaptive weighted fusion high and low frequency skin grinding algorithms and detail enhancement algorithms can provide better and faster beautification effects, making the beautified portraits look more natural and avoiding over-processing or unrealistic effects. Compared with traditional beautification methods, it can better maintain the original appearance of the face, making users more realistic and confident in video calls.

[0144] Step 6: Real-time rendering and transmission. Figure 7As shown, real-time image data is input, and the trained deep learning model is encapsulated into a first format (such as Paddle.js format) for real-time reasoning on the WebRTC client. By calling the function in the first module (such as the WebAssembly module), the beautified portrait is passed into the WebAssembly module to accelerate the deep learning reasoning (i.e., WebAssembly engine acceleration). The obtained beautified portrait is rendered in real time through WebGL to generate a canvas and obtain a processed image (output the beautified segmented image). The server transmits the beautified portrait in real time to the other user participating in the video call.

[0145] In this way, through optimized data transmission and rendering mechanisms, low latency and high-quality image transmission and presentation effects can be guaranteed.

[0146] Step 7: User interaction and feedback. The client provides a user interaction interface, allowing users to adjust the parameters of the beautification effect according to their personal preferences. Users can adjust the parameters of the beautification algorithm according to their needs to achieve personalized beautification effects. In addition, the client also provides a user feedback and evaluation mechanism to improve the beautification algorithm and system performance.

[0147] In this way, the personalized customization capability enables users to adjust the parameters of the beautification algorithm according to their preferences to achieve a beautification effect that meets their personal aesthetic requirements. Compared with the general preset beautification effects, it can better meet the personalized needs of users.

[0148] And by providing real-time portrait beautification, the quality of the face appearance in video calls can be improved, which enhances the user's visual experience. Users can enjoy high-quality beautification and personalized background processing during video calls, making them more confident and satisfied during the call. This improved user experience helps to strengthen communication between people and improve communication effectiveness.

[0149] By combining the above technical solutions, WebRTC-based video calls can optimize portrait beautification and perform real-time beautification processing on portraits in real-time video calls. This solves the problems of real-time, naturalness, and personalization, improves the quality of facial appearance in video calls, and improves the user's visual experience.

[0150] It is understandable that, based on the use of conventional technical means, WebRTC technology, to achieve real-time video call functions, the portrait beautification process can be optimized, so that the portrait during the call can be beautified in real time. In this way, while ensuring real-time performance and low latency, the image processing process can be optimized to quickly process the video stream and transmit the beautified image in real time. This allows users to see the beautification effect immediately during the video call, improving the real-time call experience.

[0151] Secondly, in the image beautification process, the high- and low-frequency algorithms of improved adaptive weighted fusion can be used, and the high- and low-frequency images can be filtered separately, and the details of the filtered images can be enhanced. Then, the images are fused to obtain a result image with strong detail realism and skin smoothing effect. In this way, the detailed features of the face can be optimized, the authenticity of the face can be maintained, and over-processing or unnatural effects can be avoided in the beautification process. In this way, the naturalness and quality of the face can be ensured, making the beautified portrait look more natural and real, meeting the aesthetic requirements of the user.

[0152] In addition, before the image segmentation process is performed before the image beautification process, a lightweight backbone network can be used to increase the scSE attention mechanism to improve the accuracy while maintaining the high-precision segmentation effect. A loss function can be designed to solve the category imbalance problem and improve the segmentation effect by weighting the loss function. The images obtained on this basis can provide a high-quality image source for the subsequent image beautification process when the image is beautified.

[0153] In addition, considering the personalized needs of users, adjustable beautification parameter settings can be provided according to the existing technology. Users can adjust the parameters of the beautification effect according to their personal preferences and requirements to achieve personalized beautification effects. This personalized customized design increases user satisfaction and improves user experience.

[0154] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of a computer device. It is understandable that, in order to realize the above functions, the computer device includes a hardware structure and / or software module corresponding to each function. Those skilled in the art should easily realize that, in combination with the method steps of the portrait beautification processing of each example described in the embodiment disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0155] The embodiment of the present application also provides a portrait beautification processing device. The portrait beautification processing device can be a computer device, a CPU in the above computer device, a processing module for portrait beautification in the above computer device, or a client for portrait beautification in the above computer device.

[0156] The embodiment of the present application can divide the portrait beautification processing device into functional modules or functional units according to the above method example. For example, each functional module or functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units. Among them, the division of modules or units in the embodiment of the present application is schematic, which is only a logical function division, and there may be other division methods in actual implementation.

[0157] like Figure 8 FIG. 1 is a schematic diagram of a structure of a portrait beautification processing device provided in an embodiment of the present application. The portrait beautification processing device is used to perform Figure 2 , Figure 3 The portrait beautification processing method shown in the figure, the portrait beautification processing device 800 may include: an acquisition module 801 and a processing module 802.

[0158] The acquisition module 801 is used to acquire the contour feature image and the detail feature image of the image to be beautified. The processing module 802 is used to determine the weighting coefficient of each key point feature according to the gradient values ​​corresponding to the multiple key point features in the detail feature image, wherein the key point feature is the feature at the junction between different facial features. The processing module 802 is also used to perform weighted processing on each key point feature in the detail feature image according to the weighting coefficients of the multiple key point features to obtain the first image. The processing module 802 is also used to fuse the first image with the contour feature image to obtain the target beautified image.

[0159] Optionally, the processing module 802 is specifically used to calculate the difference between the gradient value corresponding to each key point feature and a preset gradient threshold value to obtain multiple first gradient differences, one first gradient difference corresponding to one key point feature. The processing module 802 is also used to determine the weighting coefficient of each key point feature according to each first gradient difference.

[0160] Optionally, the acquisition module 801 is used to acquire the high-frequency image and the low-frequency image of the image to be beautified. The processing module 802 is specifically used to filter the high-frequency image to obtain the detail feature image. The processing module is used to filter the low-frequency image to obtain the contour feature image.

[0161] Optionally, the acquisition module 801 is used to acquire a first transparency of the first image and a second transparency of the contour feature image. The processing module 802 is used to fuse the first image and the contour feature image according to the first transparency and the second transparency to obtain a target beautified image.

[0162] Optionally, the acquisition module 801 is used to obtain a first fusion coefficient of the first image and a second fusion coefficient of the contour feature image. The processing module 802 is used to multiply all pixel values ​​in the first image by the first fusion coefficient and the first transparency to obtain a second image. The processing module 802 is also used to multiply all pixel values ​​in the contour feature image by the second fusion coefficient and the second transparency to obtain a third image. The processing module 802 is also used to add pixel values ​​at corresponding positions in the second image and the third image to obtain a target beautified image, wherein each pixel value in the target beautified image is greater than a preset pixel threshold.

[0163] Optionally, the processing module 802 is specifically used to segment the original image according to a target loss function to obtain multiple images to be beautified, and the target loss function includes: a cross entropy loss function and a Lovalz hinge loss function.

[0164] Fig. 9 9 is a schematic diagram of a structure of a portrait beautification processing device according to an exemplary embodiment. The device may include a processor 902, and the processor 902 is used to execute application code to implement the portrait beautification processing method in this application.

[0165] The processor 902 may be a CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0166] like Fig. 9 As shown, the portrait beautification processing device may further include a memory 903. The memory 903 is used to store application program codes for executing the solution of the present application, and is controlled to be executed by the processor 902.

[0167] The memory 903 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 903 may exist independently and be connected to the processor 902 via the bus 904. The memory 903 may also be integrated with the processor 902.

[0168] like Fig. 9 As shown, the portrait beautification processing device may further include a communication interface 901, wherein the communication interface 901, the processor 902, and the memory 903 may be coupled to each other, for example, via a bus 904. The communication interface 901 is used to perform information interaction with other devices, for example, to support information interaction between the portrait beautification processing device and other devices.

[0169] It should be pointed out that Fig. 9 The device structure shown in the figure does not constitute a limitation on the portrait beautification processing device. Fig. 9 In addition to the components shown, the portrait beautification processing device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0170] In actual implementation, the functions implemented by the processing module 802 can be Fig. 9 The processor 902 shown calls the program code in the memory 903 to implement.

[0171] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer can execute the portrait beautification processing method provided in the above-mentioned embodiment. For example, the computer-readable storage medium may be a memory 903 including instructions, and the above instructions may be executed by a processor 902 of a computer device to complete the above method. Optionally, the computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, a non-temporary computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0172] Fig.10 A conceptual partial view of a computer program product provided by an embodiment of the present application is exemplarily shown, where the computer program product includes a computer program for executing a computer process on a computing device.

[0173] In one embodiment, the computer program product is provided using a signal bearing medium 1000. The signal bearing medium 1000 may include one or more program instructions that, when executed by one or more processors, may provide the above-described Figure 2 Thus, for example, reference to Figure 2 In the embodiment shown in , one or more features of S201 to S204 may be undertaken by one or more instructions associated with the signal bearing medium 1000. In addition, Fig.10 The program instructions in also describe example instructions.

[0174] In some examples, the signal bearing medium 1000 may include a computer readable medium 1001, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM) or a random access memory (RAM), and the like.

[0175] In some implementations, the signal bearing medium 1000 may include a computer recordable medium 1002 such as, but not limited to, a memory, a read / write (R / W) CD, R / W, DVD, or the like.

[0176] In some implementations, signal bearing medium 1000 may include communication medium 1003 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0177] The signal bearing medium 1000 may be communicated by a wireless form of communication medium 1003. The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0178] In some examples, such as for Figure 8 The described portrait beautification processing apparatus may be managed to provide various operations, functions, or actions in response to one or more program instructions in the computer-readable medium 1001 , the computer-recordable medium 1002 , and / or the communication medium 1003 .

[0179] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0181] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0182] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the full classification part or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute the full classification part or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes various media that can store program codes, such as-U disk, mobile hard disk, ROM, RAM, disk or CD.

[0184] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for beautifying a portrait, characterized in that: The method comprises: Obtaining a contour feature image and a detail feature image of the image to be beautified; Determining a weighting coefficient of each key point feature according to gradient values ​​corresponding to a plurality of key point features in the detail feature image, wherein the key point feature is a feature at a junction between different facial features; Performing weighted processing on each of the key point features in the detail feature image according to the weighted coefficients of the multiple key point features to obtain a first image; The first image and the contour feature image are fused to obtain a target beautified image.

2. The method according to claim 1, characterized in that One of the key point features corresponds to a preset gradient threshold; and determining a weighting coefficient of each of the key point features according to the gradient values ​​corresponding to the multiple key point features in the detail feature image includes: Calculating the difference between the gradient value corresponding to each of the key point features and the preset gradient threshold to obtain a plurality of first gradient differences, where one of the first gradient differences corresponds to one of the key point features; A weighting coefficient of each of the key point features is determined according to each of the first gradient differences.

3. The method according to claim 2, characterized in that The step of obtaining the contour feature image and the detail feature image of the image to be beautified includes: Acquire a high-frequency image and a low-frequency image of the image to be beautified; Performing filtering processing on the high-frequency image to obtain the detail feature image; The low-frequency image is filtered to obtain the contour feature image.

4. The method according to claim 2, characterized in that The step of fusing the first image and the contour feature image to obtain a target beautified image includes: Acquire a first transparency of the first image and a second transparency of the contour feature image; The first image and the contour feature image are fused according to the first transparency and the second transparency to obtain the target beautified image.

5. The method according to claim 4, characterized in that The step of fusing the first image and the contour feature image according to the first transparency and the second transparency to obtain the target beautified image includes: Acquire a first fusion coefficient of the first image and a second fusion coefficient of the contour feature image; Multiply all pixel values ​​in the first image by the first fusion coefficient and the first transparency to obtain a second image; Multiply all pixel values ​​in the contour feature image by the second fusion coefficient and the second transparency to obtain a third image; The pixel values ​​at corresponding positions in the second image and the third image are added to obtain the target beautified image, and each pixel value in the target beautified image is greater than a preset pixel threshold.

6. The method according to any one of claims 1 to 5, characterized in that Before acquiring the contour feature image and the detail feature image of the image to be beautified, the method further includes: According to the target loss function, the original image is segmented to obtain a plurality of images to be beautified, wherein the target loss function includes a cross entropy loss function and a Lovalz hinge loss function.

7. A processing device for beautifying a portrait, characterized in that: The device comprises: An acquisition module, used for acquiring a contour feature image and a detail feature image of an image to be beautified; A processing module, used to determine a weighting coefficient of each key point feature according to the gradient values ​​corresponding to the multiple key point features in the detail feature image; The processing module is further used to perform weighted processing on each of the key point features in the detail feature image according to the weighted coefficients of the multiple key point features to obtain a first image; The processing module is further used to fuse the first image and the contour feature image to obtain a target beautified image.

8. A processing device for beautifying a portrait, characterized in that: include: Processor and memory; The processor is coupled to the memory; The memory is used to store one or more programs, and the one or more programs include computer-executable instructions. When the portrait beautification processing device is running, the processor executes the computer-executable instructions stored in the memory to enable the portrait beautification processing device to perform the method described in any one of claims 1-6.

9. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, characterized in that: When a computer executes the instructions, the computer performs the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises computer program instructions, which implement the method according to any one of claims 1 to 6 when the computer program instructions are executed.

Images