A beauty system, method, computer device and medium

Abstract

The present disclosure provides a beautifying system, method, computer device and medium, the system comprising a DSP, a GPU, an NPU and a CPU; the DSP is configured to perform skin segmentation on an image region of an upper body bounding box; the NPU is configured to identify a plurality of candidate upper body bounding boxes and a plurality of candidate face bounding boxes of a target person in a preprocessed target image based on a target detection model; and identify a plurality of candidate face key points of the preprocessed target image through a first neural network; the CPU is configured to determine the upper body bounding box and the face bounding box from the plurality of candidate upper body bounding boxes and the plurality of candidate face bounding boxes; and determine a plurality of face key points from the plurality of candidate face key points; and the GPU is configured to preprocess an image region of the face bounding box. The above system improves processing efficiency by deploying each step of the algorithm on a compatible processor.

Description

A facial beauty and body shaping system, method, computer device and medium Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a facial beautification system, method, computer device, and medium. Background Art

[0002] Nowadays, live broadcasts, short videos, and social software need to provide beauty and styling functions to meet user needs. Beauty mainly refers to skin whitening, skin smoothing and other treatments, while styling mainly refers to face slimming, enlarging eyes and other treatments.

[0003] The facial beautification algorithm requires a large amount of computation. The solutions for deploying the facial beautification algorithm in related technologies do not utilize existing computing resources based on the principles and characteristics of the algorithm, resulting in long algorithm processing time and low processing efficiency.

[0004] Summary of the Invention

[0005] In view of this, the embodiments of the present disclosure propose a facial beautification system, method, computer device and medium to solve the technical problems in the related art.

[0006] According to a first aspect of an embodiment of the present disclosure, a facial beautification system is provided, the system comprising a DSP, a GPU, an NPU, and a CPU;

[0007] The DSP is used to acquire a target image and pre-process the target image; and perform skin segmentation on the image area of ​​the upper body detection frame to perform beauty processing;

[0008] The NPU is configured to identify, based on the target detection model, several candidate upper body detection frames and several candidate face detection frames of the target person in the preprocessed target image; and to identify, through the first neural network, several candidate face key points of the preprocessed target image;

[0009] The CPU is configured to determine an upper body detection frame and a face detection frame from the plurality of candidate upper body detection frames and the plurality of candidate face detection frames; and to determine a plurality of face key points from the plurality of candidate face key points;

[0010] The GPU is used to pre-process the image area of ​​the face detection frame; and perform beauty processing and beautification processing.

[0011] Optionally, the number of facial key points is 106, of which 98 key points are standard key points and 8 key points are custom key points located in the nose range; the custom key points are symmetrically distributed on the left and right sides of the nose, and intersect with the edge of the nose through a horizontal line on the nose.

[0012] Optionally, the skin segmentation of the image area of ​​the upper body detection frame is performed by a second neural network; the first neural network and the second neural network are lightweight neural networks.

[0013] Optionally, the target detection model is an improved YOLOv7 model, and the head network of the improved YOLOv7 model includes a feature fusion module, which is used to expand the receptive field, upsample and transform the number of channels of the low-resolution feature map, and fuse the processed low-resolution feature map with the high-resolution feature map.

[0014] According to a second aspect of the embodiments of the present disclosure, a facial beautification method is provided, the method comprising:

[0015] Acquire a target image through the DSP and pre-process the target image; and perform skin segmentation on the image area of ​​the upper body detection frame to perform beauty processing;

[0016] Identifying, through the NPU and based on the target detection model, several candidate upper body detection frames and several candidate face detection frames of the target person in the preprocessed target image; and identifying, through the first neural network, several candidate face key points in the preprocessed target image;

[0017] Determining, by the CPU, an upper body detection frame and a face detection frame from the plurality of candidate upper body detection frames and the plurality of candidate face detection frames; and determining a plurality of face key points from the plurality of candidate face key points;

[0018] The GPU is used to pre-process the image area of ​​the face detection frame, and perform beauty processing and beautification processing.

[0019] Optionally, the number of facial key points is 106, of which 98 key points are standard key points and 8 key points are custom key points located in the nose range; the custom key points are symmetrically distributed on the left and right sides of the nose, and intersect with the edge of the nose through a horizontal line on the nose.

[0020] Optionally, the skin segmentation of the image area of ​​the upper body detection frame is performed by a second neural network; the first neural network and the second neural network are lightweight neural networks.

[0021] Optionally, the target detection model is an improved YOLOv7 model, and the head network of the improved YOLOv7 model includes a feature fusion module, which is used to expand the receptive field, upsample and transform the number of channels of the low-resolution feature map, and fuse the processed low-resolution feature map with the high-resolution feature map.

[0022] According to a third aspect of an embodiment of the present disclosure, a computer device is provided, the computer device comprising a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor, the processor being prompted by the machine-executable instructions to: execute the method described in the second aspect above

[0023] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the second aspect is implemented.

[0024] It can be seen from the above technical solutions that the technical solution disclosed in the present invention deploys the various steps of the beautification algorithm on a corresponding processor based on the characteristics of different processors used and combined with the principles and characteristics of the beautification algorithm, thereby shortening the processing time and improving the processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] FIG1 is a flow chart illustrating a customized facial and body beautification algorithm according to an embodiment of the present disclosure;

[0027] FIG2 is a schematic diagram of a head network of an improved YOLOv7 model according to an embodiment of the present disclosure;

[0028] FIG3 is a schematic diagram showing the positions of key points on a human face according to an embodiment of the present disclosure;

[0029] FIG4 is a schematic diagram of a facial and body beautification system according to an embodiment of the present disclosure;

[0030] FIG5 is a flowchart of a facial beautification method according to an embodiment of the present disclosure;

[0031] FIG6 is a schematic diagram showing a hardware structure of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0033] The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present disclosure. The singular forms "a," "an," and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] For the purpose of brevity and ease of understanding, the terms "greater than," "less than," "higher than," and "lower than" are used herein to describe size relationships. However, those skilled in the art will understand that the term "greater than" also encompasses the meaning of "greater than or equal to," and "less than" also encompasses the meaning of "less than or equal to," and the term "higher than" also encompasses the meaning of "higher than or equal to," and "lower than" also encompasses the meaning of "lower than or equal to."

[0036] The solutions for deploying facial beautification algorithms in related technologies do not utilize existing computing resources according to the principles and characteristics of the algorithms, resulting in long algorithm processing time and low processing efficiency.

[0037] In response to the above technical problems, the inventors analyzed the various steps of the customized beauty and reshaping algorithm and the existing processor resources, and proposed a beauty and reshaping system. This system deploys the various steps of the beauty and reshaping algorithm on a compatible processor based on the characteristics of different processors and combines the principles and characteristics of the beauty algorithm, thereby shortening the processing time and improving processing efficiency.

[0038] The invention will be described in detail through one or more of the following embodiments.

[0039] In the facial beautification system of the present disclosure, each step of the customized facial beautification algorithm is deployed on different processors. For ease of understanding, the customized facial beautification algorithm is described below.

[0040] In the related art, the beautification and reshaping algorithm usually identifies the face detection frame of each person in the image, performs skin segmentation and key point detection operations on the image area of ​​the face detection frame, and then performs beautification and reshaping processing. The inventors found that people usually pay more attention to the beautification of the upper body area, but can only identify and segment the facial skin area in the image area of ​​the face detection frame, and cannot identify and segment other skin areas of the upper body in the image (such as the neck, shoulders, etc.). This will make the difference between the facial skin area and other skin areas of the upper body after the beautification processing obvious, and the viewing experience is poor. The inventors further found that when the image area of ​​the face detection frame includes other skin areas of the upper body in addition to the face, there will be too much noise when identifying facial key points in the image area of ​​the face detection frame, and the detection accuracy and precision will be low.

[0041] In response to the above technical problems, the inventor proposed a customized beautification algorithm, which identifies the upper body detection frame and face detection frame of the target person in the target image. The image area of ​​the upper body detection frame includes the facial skin area and other skin areas of the upper body, so that the transition between the facial skin area and other skin areas of the upper body after beautification processing is natural, thereby improving the viewing experience. In addition, the image area of ​​the face detection frame only includes the face area, which greatly reduces the noise when identifying facial key points in the image area of ​​the face detection frame, thereby improving the detection accuracy and precision.

[0042] FIG1 is a flowchart illustrating a customized facial beautification algorithm according to an embodiment of the present disclosure.

[0043] As shown in FIG1 , the facial beautification algorithm may include the following steps:

[0044] Step S101: Acquire a target image.

[0045] Step S102: Identify the upper body detection frame and face detection frame of the target person in the target image.

[0046] Step S103 , performing skin segmentation on the image area of ​​the upper body detection frame to perform beauty processing.

[0047] Step S104: identifying a number of facial key points in the image area of ​​the face detection frame to perform beautification processing.

[0048] In this embodiment, the upper body area of ​​the target person can be determined through the upper body detection frame. The upper body area can refer to the area above the chest, below the top of the head and between the shoulders; the face area of ​​the target person can be determined through the face detection frame. The face area can refer to the area enclosed by the left ear edge, right ear edge, chin and forehead as boundaries, and may not include hair to reduce noise when identifying facial key points in the image area of ​​the face detection frame.

[0049] In one embodiment, step S102, identifying the upper body detection frame and the face detection frame of the target person in the target image, may specifically include:

[0050] Preprocessing: Step S1021, preprocess the target image:

[0051] In the present disclosure, preprocessing may include resolution adjustment, format conversion, and normalization of the target image, and the present disclosure does not impose any restrictions on this.

[0052] Among them, resolution adjustment can be implemented based on image scaling algorithms such as bilinear interpolation and bicubic interpolation. The adjusted image resolution can be set according to actual conditions, for example, according to the input image size requirements of the target detection model used, or according to actual application scenarios. For example, when the target detection model is deployed on a processor with weaker performance, the resolution of the graphics can be reduced to reduce computational complexity and improve processing efficiency. Appropriately reducing the image resolution can take into account both image clarity and processing efficiency. The converted image format can be set according to the format requirements of the input image of the target detection model used, for example, it can be converted to RGB format, and then further normalized, such as scaling the value of each pixel in the image to a specified range (e.g., [0,1]) to reduce computational complexity and improve processing efficiency.

[0053] Reasoning: Step S1022 , identifying several candidate upper body detection frames and several candidate face detection frames of the target person in the preprocessed target image based on the target detection model;

[0054] In this embodiment, the target detection model can be a YOLO model, an R-CNN model, or an SSD model, etc., which is not limited in this disclosure.

[0055] In this embodiment, a target detection model can be used to identify a target image containing several target persons. For each target person, several candidate upper body detection frames can be identified to determine the target person's upper body region, as well as several candidate face detection frames to determine the target person's face region. The detection frame information can include the center coordinates, length, and width of the detection frame, which represent the size of the detection frame and its position in the target image. The detection frame information can also include a confidence level, which represents the probability that the target is present in the detection frame.

[0056] The YOLOv7 model is a target detection model. Its backbone network extracts features from the input image and outputs feature maps of different scales to the head network. The head network fuses these feature maps to obtain target detection results based on the fused feature maps.

[0057] For the YOLOv7 model, the present disclosure proposes a new multi-scale feature map fusion method, which performs two feature fusions on feature maps of three different scales. The scale of the feature maps fused each time is different. Each feature fusion first expands the receptive field of the low-resolution feature map, and then upsamples the processed feature map to improve the resolution of the low-resolution feature map and make it consistent with the resolution of the subsequently fused high-resolution feature map. Then, the number of channels of the upsampled feature map is transformed and made the same as the number of channels of the subsequently fused high-resolution feature map. Finally, the low-resolution feature map after a series of processing can be fused with the high-resolution feature map.

[0058] In one embodiment, the above-mentioned new multi-scale feature map fusion method can be embodied by an improved YOLOv7 model. As shown in the dotted box in Figure 2, the head network of the improved YOLOv7 model includes a feature fusion module. The feature fusion module can be used to expand the receptive field, upsample, and transform the number of channels of the low-resolution feature map and then fuse it with the high-resolution feature map.

[0059] Specifically, the feature fusion module may include several AC (Atrous Convolution) modules, several TC (Transposed Convolution) modules, several REP modules, and several add modules. The AC module can expand the receptive field by using a dilated convolution kernel with a specified dilation rate, so that the area corresponding to the pixel points of the feature map output in the subsequent processing process becomes larger on the input feature map, thereby improving the detection accuracy of large targets. The TC module can upsample the input feature map to improve the resolution of the low-resolution feature map. The REP module can be used to extract features from the feature map, perform feature fusion, and transform the number of channels of the feature map. The add module can fuse the processed low-resolution feature map with the high-resolution feature map through the add operation in the feature fusion method, so as to perform feature map fusion of the next level based on the fused feature map and obtain the target detection result. Compared with the concat operation that only increases the number of features, the use of the add operation fusion can increase the information content of each feature in the feature map while keeping the number of features unchanged, thereby more accurately identifying the position of the target in the image and improving the accuracy of target detection. In addition, the add operation requires less computation and generates less data after fusion. In scenarios where the neural network is deployed on devices with weaker performance, such as embedded devices in live broadcast scenarios, this can avoid the problem of low processing efficiency caused by excessive computation and improve user experience.

[0060] This embodiment proposes an improved YOLOv7 model. A feature fusion module is added to the head network of the model to perform two feature fusions on feature maps of three different scales using a new multi-scale feature map fusion method. This fusion method adds a new feature fusion channel from low resolution to high resolution in the deep layer of the head network, so that feature maps of different scales are further cross-fused, thereby increasing the depth of feature fusion and further improving the detection accuracy of large targets. The fused feature maps are more informative, thereby maintaining the detection accuracy of the YOLOv7 model for small targets while further improving the detection accuracy of large targets, thereby greatly improving the target detection accuracy of the YOLOv7 model.

[0061] In one embodiment, several candidate upper body detection frames and several candidate face detection frames of the target person in the preprocessed target image can be identified based on the improved YOLOv7 model.

[0062] Specifically, as shown in Figure 2, a target image of size 320*320 is input to the improved YOLOv7 model. The backbone network extracts features from the target image and outputs feature maps of sizes 512*10*10, 128*20*20, and 64*40*40 to the head network. The head network first performs channel number conversion and upsampling on the low-resolution feature map and then fuses it with the high-resolution feature map. Then, the high-resolution feature map is downsampled and fused with the low-resolution feature map to obtain feature maps of sizes 512*10*10, 128*20*20, and 64*40*40. The feature maps are 128*10*10, 64*20*20 and 32*40*40. The newly added feature fusion module performs two feature fusions on the feature maps of three different scales. After that, the CBM module extracts the features of the feature maps, performs feature fusion and transforms the number of channels of the feature maps to obtain several target detection result images with sizes of 21*10*10, 21*20*20 and 21*40*40 respectively. Among them, the detection frames identified in the target detection result images are divided into two categories: upper body detection frames and face detection frames.

[0063] Post-processing: Step S1023 , determining an upper body detection frame and a face detection frame from a plurality of candidate upper body detection frames and a plurality of candidate face detection frames.

[0064] In this embodiment, based on the NMS (Non-Maximum Suppression) algorithm, for each target person, an upper body detection frame and a face detection frame can be determined from several candidate upper body detection frames and several candidate face detection frames corresponding to the target person.

[0065] In one embodiment, step S103 of performing skin segmentation on the image area of ​​the upper body detection frame for beautification processing may specifically include:

[0066] Preprocessing: Step S1031 , preprocessing the image area of ​​the upper body detection frame.

[0067] In this embodiment, the image area of ​​the upper body detection frame can be determined in the target image based on the center coordinates, length and width of the detection frame in the upper body detection frame information, so as to cut out the upper body image in the target image and pre-process the upper body image.

[0068] Reasoning: Step S1032, using a second neural network to identify a skin probability distribution map and a non-skin probability distribution map for the preprocessed target image.

[0069] In this embodiment, the second neural network may be a neural network suitable for image segmentation, such as DeepLab or U-Net, and this disclosure does not impose any restrictions on this.

[0070] In this embodiment, the target image can be identified through the second neural network, and the probability that each pixel point in the target image belongs to the skin area of ​​the target person can be determined to obtain a skin probability distribution map. The probability that each pixel point in the target image does not belong to the skin area of ​​the target person can also be determined to obtain a non-skin probability distribution map.

[0071] Post-processing: Step S1033 , determining a skin mask image of the target image based on the skin probability distribution map and the non-skin probability distribution map.

[0072] In this embodiment, each pixel point in the target image can be traversed, and the probability value corresponding to the pixel point in the skin probability distribution map can be compared with the corresponding probability value in the non-skin probability distribution map to determine whether the pixel point belongs to the skin area, and then the set of pixel points belonging to the skin area of ​​the target person can be determined to obtain a skin mask image.

[0073] In one embodiment, the facial beautification method may further include:

[0074] Step S105 : performing beauty processing on the target image based on the skin mask image of the target image.

[0075] In this embodiment, the target image can be beautified based on the image processing algorithm. The specific image processing algorithm can be selected according to the actual beautification needs. For example, a 3D LUT (3D Lookup Table) algorithm can be used to achieve a whitening effect, and a Gaussian blur algorithm or a bilateral filtering algorithm can be used to achieve a skin smoothing effect. This disclosure does not impose any restrictions on this.

[0076] In one embodiment, step S104, identifying a number of facial key points in the image area of ​​the face detection frame to perform beautification processing, may specifically include:

[0077] Preprocessing: Step S1041, preprocessing the image area of ​​the face detection frame.

[0078] In this embodiment, the image area of ​​the face detection frame can be determined in the target image based on the center coordinates, length and width of the detection frame in the face detection frame information, so as to cut out the face image in the target image and pre-process the face image.

[0079] Reasoning: Step S1042, identifying several candidate facial key points of the preprocessed target image through the first neural network.

[0080] In this embodiment, the first neural network may be a neural network suitable for key point detection, such as MobileNet, ResNet, etc., and this disclosure does not impose any limitation on this.

[0081] In this embodiment, different key point regression methods based on the first neural network can be selected according to the actual application scenario. For example, a coordinate regression method can be used in scenarios with high real-time requirements, and a heat map regression method can be used in scenarios with high key point positioning accuracy. This disclosure does not impose any restrictions on this.

[0082] Post-processing: Step S1043, determining a number of facial key points from a number of candidate facial key points.

[0083] In the task of recognizing facial key points, several key points at designated locations on the face are usually predefined, and the corresponding locations on the face in the training set images are annotated. In this embodiment, based on the NMS algorithm, a facial key point can be determined for each target person from the candidate facial key points corresponding to each designated location on the target person's face, thereby determining the facial key points corresponding to the designated locations.

[0084] The inventors discovered that the standard 98 key points for the nose are located on the bridge and lower edge of the nose, resulting in low recognition accuracy. Therefore, the inventors came up with the idea of ​​adding custom key points to the nose area to improve recognition accuracy, thereby enhancing the effect of nose slimming during subsequent beauty treatments.

[0085] In one embodiment, the number of facial key points may be 106, of which 98 key points are standard key points and 8 key points are user-defined key points located in the nose area.

[0086] In this embodiment, the customized keypoints can be symmetrically distributed on the left and right sides of the nose and can intersect with the nose edge through a horizontal line on the nose. For example, as shown in Figure 3, points 0-97 are standard keypoints, and points 98-105 are customized keypoints. Points 98-101 and 102-105 are symmetrically distributed on the left and right sides of the nose and can intersect with the nose edge through the horizontal line at points 51-54, respectively.

[0087] In one embodiment, skin segmentation of the image area of ​​the upper body detection frame can be performed by a first neural network, and identification of several facial key points in the image area of ​​the face detection frame can be performed by a second neural network. The first neural network and the second neural network can be lightweight neural networks.

[0088] This embodiment takes into account the situation where the neural network is deployed on a device with weaker performance, such as an embedded device in a live broadcast scenario. By adopting a lightweight neural network, the problem of low processing efficiency caused by high computational complexity is avoided, thereby improving the user experience.

[0089] In one embodiment, the facial beautification method may further include:

[0090] Step S106: performing beautification processing on the target image based on a number of facial key points.

[0091] In this embodiment, beautification processing can be performed based on a deformation operation on the target image. Specifically, the image content of a specified area (such as the eye area, nose area, etc.) in the target image can be subjected to deformation operations such as translation, scaling or rotation according to actual beautification requirements. Then, the coordinates of each pixel point after deformation can be calculated corresponding to the coordinates of the original image, and the obtained original image coordinates can be interpolated to determine the pixel value corresponding to the pixel point. Algorithms such as bilinear interpolation, bicubic interpolation or nearest neighbor interpolation can be selected for interpolation operations, and the present disclosure does not impose any restrictions on this.

[0092] Next, the facial and body beautification system of the present disclosure will be described.

[0093] FIG4 is a schematic diagram of a facial beautification system according to an embodiment of the present disclosure.

[0094] As shown in FIG4 , the system includes a DSP (Digital Signal Processor), an NPU (Neural Network Processing Unit), a CPU (Central Processing Unit), and a GPU (Graphics Processing Unit).

[0095] The DSP is used to acquire a target image and pre-process the target image; and perform skin segmentation on the image area of ​​the upper body detection frame to perform beauty processing;

[0096] The NPU is configured to identify, based on the target detection model, several candidate upper body detection frames and several candidate face detection frames of the target person in the preprocessed target image; and to identify, through the first neural network, several candidate face key points of the preprocessed target image;

[0097] The CPU is configured to determine an upper body detection frame and a face detection frame from the plurality of candidate upper body detection frames and the plurality of candidate face detection frames; and to determine a plurality of face key points from the plurality of candidate face key points;

[0098] The GPU is used to pre-process the image area of ​​the face detection frame; and perform beauty processing and beautification processing.

[0099] In this embodiment, the inventors took into account that both DSP and GPU are suitable for image processing, both DSP and NPU are suitable for running neural networks, CPU is suitable for logical operations, and DSP is also suitable for logically simple data-intensive operations, and the preprocessing of skin segmentation and key point detection mainly involves operations such as resolution adjustment, format conversion or normalization of images, which are all suitable for deployment on DSP and GPU. The post-processing of skin segmentation mainly involves comparing the skin probability distribution map with the non-skin probability distribution map, which is suitable for deployment on DSP. The post-processing of key point detection mainly involves executing logical operations such as the NMS algorithm, which is suitable for deployment on the CPU. Therefore, it is thought that the relevant steps of skin segmentation can be performed by DSP.

[0100] At the same time, the GPU performs the preprocessing steps of key point detection, the NPU performs the inference steps of key point detection, and the CPU performs the post-processing steps of key point detection, so that the various steps of skin segmentation and key point detection are deployed on different processors respectively, thereby achieving the effect of executing the steps of skin segmentation and key point detection in parallel, further shortening the processing time, further improving the processing efficiency, and improving the utilization rate of these processors on the device to avoid resource waste.

[0101] Furthermore, the inventors took into account the situation where the performance of devices executing beauty and reshaping algorithms is relatively weak, such as embedded devices in live broadcast scenarios. Compared with GPUs, DSPs have lower power consumption and lower latency, and are more suitable for scenarios with higher real-time requirements. Therefore, the relevant steps of image processing can be deployed on DSPs in priority.

[0102] FIG5 is a flowchart of a facial beautification method according to an embodiment of the present disclosure, comprising:

[0103] Step 501: Obtain a target image through DSP and pre-process the target image; and perform skin segmentation on the image area of ​​the upper body detection frame to perform beauty processing;

[0104] Step 502: Using the NPU, based on the target detection model, identify several candidate upper body detection frames and several candidate face detection frames of the target person in the preprocessed target image; and using the first neural network, identify several candidate face key points in the preprocessed target image.

[0105] Step 503: determining, by the CPU, an upper body detection frame and a face detection frame from the plurality of candidate upper body detection frames and the plurality of candidate face detection frames; and determining a plurality of facial key points from the plurality of candidate facial key points;

[0106] Step 504: Pre-process the image area of ​​the face detection frame through the GPU; and perform beauty processing and beautification processing.

[0107] In one embodiment, the number of facial key points is 106, of which 98 are standard key points and 8 are custom key points located in the nose range; the custom key points are symmetrically distributed on the left and right sides of the nose, and intersect with the edge of the nose through a horizontal line on the nose.

[0108] In one embodiment, the skin segmentation of the image area of ​​the upper body detection frame is performed by a second neural network; the first neural network and the second neural network are lightweight neural networks.

[0109] In one embodiment, the target detection model is an improved YOLOv7 model, and the head network of the improved YOLOv7 model includes a feature fusion module, which is used to expand the receptive field, upsample and transform the number of channels of the low-resolution feature map, and fuse the processed low-resolution feature map with the high-resolution feature map.

[0110] Figure 6 is a schematic diagram illustrating the hardware structure of a computer device according to an embodiment of the present disclosure. The computer device may include a processor 601 and a machine-readable storage medium 602 storing machine-executable instructions. The processor 601 and the machine-readable storage medium 602 may communicate via a system bus 603. Furthermore, by reading and executing the machine-executable instructions corresponding to the beauty and reshaping display logic in the machine-readable storage medium 602, the processor 601 may execute the beauty and reshaping method described above.

[0111] The machine-readable storage medium 602 referred to herein can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium 602 can include at least one of the following types of storage media: volatile memory, non-volatile memory, or other types of storage media. Volatile memory can be RAM (Random Access Memory), and non-volatile memory can be flash memory, a storage drive (such as a hard disk drive), a solid-state drive, or a storage disk (such as a CD, DVD, etc.).

[0112] Based on the method described in any of the above embodiments, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can be used to execute the beautification method described in any of the above embodiments.

[0113] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0114] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0115] The foregoing description describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0116] The phrases "specific examples" or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present disclosure, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0117] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not claimed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0118] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0119] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A beauty and body shaping system, characterized in that, the system includes a DSP, a GPU, an NPU, and a CPU; the DSP is used to obtain a target image and preprocess the target image; and perform skin segmentation on the image area of the upper body detection frame for beauty treatment; the NPU is used to identify a plurality of candidate upper body detection frames and a plurality of candidate face detection frames of the target person in the preprocessed target image based on a target detection model; and identify a plurality of candidate face key points of the preprocessed target image through a first neural network; the CPU is used to determine an upper body detection frame and a face detection frame from the plurality of candidate upper body detection frames and the plurality of candidate face detection frames; and determine a plurality of face key points from the plurality of candidate face key points; the GPU is used to preprocess the image area of the face detection frame; and perform beauty treatment and body shaping treatment.

2. The system according to claim 1, characterized in that, the number of the face key points is 106, among which 98 key points are standard key points, and 8 key points are custom key points located in the nose range; the custom key points are symmetrically distributed on the left and right sides of the nose and intersect with the nose edge through a horizontal line on the nose.

3. The system according to claim 1, characterized in that, the skin segmentation of the image area of the upper body detection frame is performed by a second neural network; the first neural network and the second neural network are lightweight neural networks.

4. The system according to claim 1, characterized in that, the target detection model is an improved YOLOv7 model, and the head network of the improved YOLOv7 model includes a feature fusion module, and the feature fusion module is used to perform enlarged receptive field, upsampling, and channel number transformation processing on the low-resolution feature map, and fuse the processed low-resolution feature map with the high-resolution feature map.

5. A beauty and body shaping method, characterized in that, the method includes: obtaining a target image through a DSP and preprocessing the target image; and performing skin segmentation on the image area of the upper body detection frame for beauty treatment; identifying a plurality of candidate upper body detection frames and a plurality of candidate face detection frames of the target person in the preprocessed target image through an NPU based on a target detection model; and identifying a plurality of candidate face key points of the preprocessed target image through a first neural network; determining an upper body detection frame and a face detection frame from the plurality of candidate upper body detection frames and the plurality of candidate face detection frames through a CPU; and determining a plurality of face key points; preprocessing the image area of the face detection frame through a GPU; and performing beauty treatment and body shaping treatment.

6. The method according to claim 5, characterized in that, The number of the face key points is 106, among which 98 key points are standard key points, and 8 key points are custom key points located in the nose range; the custom key points are symmetrically distributed on the left and right sides of the nose and intersect with the nose edge through the horizontal line on the nose.

7. The method according to claim 5, wherein, the skin segmentation of the image region of the upper body detection frame is performed by a second neural network; the first neural network and the second neural network are lightweight neural networks.

8. The method according to claim 5, wherein, the target detection model is an improved YOLOv7 model, and the head network of the improved YOLOv7 model includes a feature fusion module, which is used for expanding the receptive field, upsampling and processing the number of channels of a low-resolution feature map, and fusing the processed low-resolution feature map with a high-resolution feature map.

9. A computer device, wherein, the computer device includes a processor and a machine-readable storage medium, the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is urged by the machine-executable instructions to execute the method according to any one of claims 5-8.

10. A computer-readable storage medium, wherein, a computer program is stored on the medium, and when the computer program is executed by a processor, the method according to any one of claims 5-8 is implemented.

Images