Makeup processing method and device based on face recognition, equipment and medium

Abstract

The present disclosure relates to a face recognition-based makeup processing method, device, equipment and medium, the method comprising: video capturing a face image to generate target video stream data, identifying and calibrating the position of the face region in the set recognition region in the target video stream data, determining a plurality of boundary positions of the face region corresponding bounding box on the last frame image of the target video stream data, generating a target face image corresponding to the face region according to the plurality of boundary positions, generating a face feature region image corresponding to each type of face feature according to the target face image, obtaining a target makeup effect corresponding to each face feature selected by a user, and performing makeup filling on the face feature region image corresponding to the face feature region of the corresponding type based on the target makeup effect. Thus, the real-time generated makeup filling is fitted with the currently captured face image, the recommended makeup effect is truly restored to the user, and the user's use satisfaction is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, specifically to a method, apparatus, device, and medium for beauty processing based on face recognition. Background Technology

[0002] With the widespread adoption of smart mobile devices and the improvement of their camera functions, more and more people are choosing smart mobile devices for daily photography, using beauty modes to enhance the results. Current mainstream technology beautifies the entire image through filters, merely enhancing the photo. While these photos can be shared, they don't provide true beauty. Some beauty apps on the market offer makeup functions, but they only display the effects of existing products, requiring users to manually select from a list with limited choices. Existing implementations typically rely on surveys to understand user habits or confirm skin condition using external devices, recommending makeup effects based on the survey results or skin condition. While existing beauty apps offer recommendations and show effects, the process is cumbersome, and the makeup effects often appear artificial, lacking a true match between the displayed effect and the user's experience, resulting in a poor user experience. Summary of the Invention

[0003] The purpose of this disclosure is to provide a method, apparatus, device, and medium for beauty processing based on facial recognition.

[0004] To achieve the above objectives, the first aspect of this disclosure provides a beauty processing method based on face recognition, the method comprising: In response to the detection of a face image within a defined recognition area, video capture is performed on the face image to generate target video stream data; The position of the face region in the target video stream data within the set recognition area is identified and calibrated to determine multiple boundary positions of each feature point on the bounding box corresponding to the face region on the last frame of the target video stream data. The face region within the set recognition area is cropped according to the multiple boundary positions to generate a target facial image corresponding to the face region. Based on the target facial image, various facial features of the face region are identified to generate facial feature region images corresponding to each type of facial feature. The user selects the target makeup effect corresponding to each facial feature from a variety of makeup effects for each facial feature corresponding to the target face model. Based on the target makeup effect, the corresponding type of facial region in the facial feature region image is filled with makeup, and a target makeup image matching the facial region is generated on the display device.

[0005] Optionally, in some embodiments, the step of cropping the face region within the defined recognition area based on the plurality of boundary positions to generate a target facial image corresponding to the face region includes: Based on the multiple boundary positions, a face bounding box that moves with the face region is generated on the display device corresponding to the set recognition area; Based on the face bounding box, the face region within the set recognition area is image acquired until multiple frames of face images of the face region at multiple preset acquisition angles are obtained. Based on multiple preset acquisition angles corresponding to the multiple frames of facial images, the multiple frames of facial images are fused to generate the target facial image of the facial region on the corresponding plane of the display device.

[0006] Optionally, in some embodiments, generating a face bounding box corresponding to the face region that moves with the face on the display device corresponding to the set recognition area based on the plurality of boundary positions includes: Based on the multiple boundary positions, an initial bounding box corresponding to the face region is generated on the last frame image; Obtain multiple first grayscale differences between the boundary pixel corresponding to the first boundary position and multiple adjacent pixels of the boundary pixel in the last frame image within a set area, wherein the first boundary position is any one of the multiple boundary positions; Continue to acquire images of the face region within the defined recognition area to obtain the next frame of face image; Determine multiple second grayscale differences between each pixel in the next frame of the face image and multiple adjacent pixels within the set area; Based on the plurality of second grayscale differences and the plurality of first grayscale differences, determine the first position of at least one boundary pixel in the next frame face image and the second position of the at least one boundary pixel relative to the initial bounding box. Based on the first position and the second position, the initial bounding box is moved to generate a face bounding box that moves with the face region.

[0007] Optionally, in some embodiments, determining the first position of at least one boundary pixel in the next frame face image and the second position of the at least one boundary pixel relative to the initial bounding box based on the plurality of second gray-level differences and the plurality of first gray-level differences includes: Based on the plurality of first grayscale differences and the plurality of second grayscale differences, the pixels with similar grayscale differences in the next frame face image are determined as target boundary pixels that match the boundary pixels corresponding to the first boundary position; The position of the target boundary pixel in the next frame of the face image is determined as the first position; Obtain the relative position between the first boundary position and the plurality of boundary positions; The relative position is taken as the second position of the target boundary pixel relative to the initial bounding box.

[0008] Optionally, in some embodiments, the step of performing region recognition on various facial features of the face region based on the target facial image to generate facial feature region images corresponding one-to-one for each type of facial feature includes: Based on the target facial image, the 3D contour of the face region in three-dimensional space is identified, and a three-dimensional facial contour model corresponding to the face region is generated. The target face model with the highest matching degree with the three-dimensional facial contour model is selected from the preset face model database. Based on the target face model, key point detection is performed on the contour key point features of various facial features in the target face image, and based on the position of the key points in the target face image, multiple target key points corresponding to various facial features are generated. Based on the positions of the multiple target key points in the target facial image, the facial features of the target facial image are divided into regions to generate facial feature region images corresponding to each type of facial feature.

[0009] Optionally, in some embodiments, the step of applying makeup to the corresponding type of facial region in the facial feature region image based on the target makeup effect, and generating a target makeup image matching the facial region on the display device, includes: Visually perceive the illumination intensity in the target area image to determine the illumination component and reflection component of the target area image; Based on the illumination component and the reflection component, the illumination distribution features are extracted on the L channel of the LAB color corresponding to the target region image to generate the illumination gradient map of the target region image. Based on the target makeup effect, the facial feature region image is filled with makeup to generate an initial makeup image; The illumination gradient map is fused with the initial makeup image using a Poisson fusion method to generate the target makeup image.

[0010] Optionally, in some embodiments, the step of capturing video of the face image in response to detecting a face image within a defined recognition area to generate target video stream data includes: In response to the detection of a face image within the defined recognition area, video stream within the defined recognition area is acquired to obtain initial video stream data within a defined duration range; If the duration of the face region corresponding to the face image in the initial video stream data being within the set recognition area is greater than a preset duration threshold, the initial video stream data will be used as the target video stream data.

[0011] According to a second aspect of this disclosure, a facial recognition-based beauty processing device is provided, the device comprising: The first generation module is configured to, in response to detecting a face image within a set recognition area, perform video capture on the face image to generate target video stream data; The determination module is configured to identify and calibrate the position of the face region in the target video stream data within the set recognition area, and determine multiple boundary positions of each feature point on the bounding box corresponding to the face region on the last frame of the target video stream data; The second generation module is configured to crop the face region within the set recognition area according to the multiple boundary positions, generate a target facial image corresponding to the face region, and perform region recognition on various facial features of the face region according to the target facial image, generating facial feature region images corresponding to each type of facial feature. The acquisition module is configured to acquire the target makeup effect corresponding to each facial feature selected by the user from a variety of makeup effects of each facial feature corresponding to the target face model. The execution module is configured to perform makeup filling on the corresponding type of facial region in the facial feature region image based on the target makeup effect, and generate a target makeup image matching the facial region on the display device.

[0012] According to a third aspect of this disclosure, an electronic device is provided, comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of the first aspects of this disclosure.

[0013] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the first aspects of this disclosure.

[0014] The above technical solution, in response to the detection of a face image within a designated recognition area, performs video capture of the face image to generate target video stream data. It then identifies and calibrates the position of the face region within the designated recognition area in the target video stream data, determining multiple boundary positions of various feature points on the bounding box corresponding to the face region in the last frame of the target video stream data. Based on these boundary positions, the face region within the designated recognition area is cropped to generate a target facial image corresponding to the face region. Furthermore, based on the target facial image, various facial features of the face region are identified, generating facial feature region images corresponding to each type of facial feature. The system then acquires the target makeup effect selected by the user from various makeup effects corresponding to the target face model, and applies makeup filler to the corresponding type of facial region in the facial feature region image based on the target makeup effect. Finally, it generates a target makeup image matching the face region on the display device. This improves the realism of the makeup filler effect, ensuring that the real-time generated makeup filler closely matches the currently captured face image, realistically restoring the recommended makeup effect to the user and increasing user satisfaction.

[0015] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a face recognition-based beauty processing method according to an exemplary embodiment.

[0017] Figure 2 This is a block diagram illustrating a face recognition-based beauty processing device according to an exemplary embodiment.

[0018] Figure 3 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0019] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0020] Figure 1 This is a schematic flowchart illustrating a face recognition-based beauty processing method according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps: Step S101: In response to detecting a face image within a set recognition area, video capture is performed on the face image to generate target video stream data.

[0021] For example, this embodiment is applied to a terminal device that can perform face image detection on images captured within a designated recognition area. When an image related to a face is detected, the face image is captured as video, generating a target video stream of a designated duration. It should be noted that in this embodiment, applying makeup filler to the face requires the detection of a face, and that the face must move within the designated recognition area to generate the target video stream of the designated duration. This distinguishes the actually detected face from the face image detected within the designated recognition area, allowing the terminal device to apply makeup filler to the real face detected within the designated recognition area.

[0022] In some embodiments, an infrared sensor can be configured in the terminal device to determine whether some or all human features exist within a currently defined recognition area. If some or all human features are present, a video stream is captured from the defined recognition area, and multiple frames of infrared images of the defined recognition area are generated based on the infrared sensor. By recognizing the contours of the multiple frames of infrared images, it is determined whether a face region exists within the defined recognition area. If a face contour appears in all multiple frames of infrared images, an image acquisition device is used to acquire images of the defined recognition area, generating target video stream data containing face images.

[0023] Optionally, in some embodiments, step S101 above includes: In response to the detection of a face image within the defined recognition area, video stream within the defined recognition area is acquired to obtain initial video stream data within a defined duration range; If the duration of the face region corresponding to the face image in the initial video stream data being within the set recognition area is greater than a preset duration threshold, the initial video stream data will be used as the target video stream data.

[0024] For example, in this embodiment, the duration of a face's stay within a defined recognition area is detected to determine whether a user within that area intends to apply makeup. In response to the detection of a face image within the defined recognition area, a video stream within that area is acquired, generating initial video stream data within a defined duration. The duration of the face image corresponding to a complete face region in the initial video stream data is detected. If the duration of the face region corresponding to the face image in the initial video stream data within the defined recognition area exceeds a preset duration threshold, the initial video stream data is used as the target video stream data. By recognizing and detecting the face image in the target video stream data, the face image is located, and makeup application corresponding to that face image is generated.

[0025] Step S102: Identify and calibrate the position of the face region in the target video stream data within the set recognition area, and determine the multiple boundary positions of each feature point on the bounding box corresponding to the face region in the last frame of the target video stream data.

[0026] For example, in this embodiment, the position of the face region in the defined recognition region captured in each frame of the target video stream data is identified, and the bounding box corresponding to the face region in each frame of the defined region captured image is determined. Based on the temporal continuity between the bounding boxes in the target video stream data, the bounding boxes in each frame of the defined region captured image are calibrated to generate a bounding box in the target video stream data that moves with the face region. This bounding box includes multiple boundary positions of various feature points corresponding to the face boundary contour.

[0027] Step S103: The face region within the set recognition area is cropped according to multiple boundary positions to generate the target face image corresponding to the face region. Based on the target face image, the face region is identified for various facial features to generate facial feature region images corresponding to various facial features.

[0028] For example, in this embodiment, the face region within a set recognition area is cropped based on multiple boundary positions corresponding to various contour feature points of the face region, generating a target facial image corresponding to the face region. Based on this target facial image, various facial features of the face region are identified, generating facial feature region images corresponding to various facial features.

[0029] It should be noted that in this embodiment, the type of facial feature to be identified can be determined based on the current refinement of the makeup effect on the terminal device. Alternatively, a makeup effect fill area option can be set in the terminal device to capture which parts of the face the user wants to fill with makeup. The facial feature type can include at least one of the following: eye area, eyebrow area, nose area, mouth area, cheek area, forehead area, jaw area, and philtrum area. This embodiment does not limit the type of facial feature that can be filled with makeup; various facial features can be combined based on the current makeup effect type or the user's selection to generate facial features that meet the user's current makeup needs. The mobile terminal identifies the region image of one or more facial features to determine the corresponding makeup fill type.

[0030] For example, in this embodiment, the face region within the defined recognition area is cropped at multiple boundary locations to generate a target facial image corresponding to the face region. Based on this target facial image, various facial features of the face region are identified to generate a facial feature region image corresponding to each facial feature.

[0031] Optionally, in some embodiments, step S103 is omitted, and includes: Based on the multiple boundary positions, a face bounding box that moves with the face region is generated on the display device corresponding to the set recognition area; Based on the face bounding box, the face region within the set recognition area is image acquired until multiple frames of face images of the face region at multiple preset acquisition angles are obtained. Based on multiple preset acquisition angles corresponding to the multiple frames of facial images, the multiple frames of facial images are fused to generate the target facial image of the facial region on the corresponding plane of the display device.

[0032] For example, in this embodiment, to ensure a more accurate fit between the makeup filler and the face, it is necessary to accurately identify the facial features within the designated recognition area. A facial bounding box that moves with the facial region is generated on the display device corresponding to the designated recognition area based on the multiple boundary positions. Based on the facial bounding box, image acquisition is performed on the facial region within the designated recognition area until multiple frames of facial images of the facial region are obtained at multiple preset acquisition angles. Based on the multiple preset acquisition angles corresponding to the multiple frames of facial images, the multiple frames of facial images are fused to generate the target facial image of the facial region on the plane corresponding to the display device.

[0033] Optionally, in some embodiments, the above step "generating a face bounding box corresponding to the face region that moves with the face on the display device corresponding to the set recognition area based on the plurality of boundary positions" includes: Based on the multiple boundary positions, an initial bounding box corresponding to the face region is generated on the last frame image; Obtain multiple first grayscale differences between the boundary pixel corresponding to the first boundary position and multiple adjacent pixels of the boundary pixel in the last frame image within a set area, wherein the first boundary position is any one of the multiple boundary positions; Continue to acquire images of the face region within the defined recognition area to obtain the next frame of face image; Determine multiple second grayscale differences between each pixel in the next frame of the face image and multiple adjacent pixels within the set area; Based on the plurality of second grayscale differences and the plurality of first grayscale differences, determine the first position of at least one boundary pixel in the next frame face image and the second position of the at least one boundary pixel relative to the initial bounding box. Based on the first position and the second position, the initial bounding box is moved to generate a face bounding box that moves with the face region.

[0034] For example, in this embodiment, to improve the user's experience using the terminal device, a bounding box that moves with the face is generated, thereby helping the user to better view the beauty image. For example, based on the multiple boundary positions, an initial bounding box corresponding to the face region is generated on the last frame image. Multiple first grayscale differences are obtained between the boundary pixel corresponding to the first boundary position and multiple adjacent pixels of the boundary pixel in the last frame image within a set area. The first boundary position is any one of the multiple boundary positions. Image acquisition continues for the face region within the set recognition area to obtain the next frame face image. Multiple second grayscale differences are determined between each pixel in the next frame face image and multiple adjacent pixels within the set area. Based on the multiple second grayscale differences and the multiple first grayscale differences, a first position of at least one boundary pixel in the next frame face image and a second position of the at least one boundary pixel relative to the initial bounding box are determined. Based on the first position and the second position, the initial bounding box is moved to generate the face bounding box that moves with the face region.

[0035] Optionally, in some embodiments, the above step: "determining the first position of at least one boundary pixel in the next frame face image and the second position of the at least one boundary pixel relative to the initial bounding box based on the plurality of second gray-level differences and the plurality of first gray-level differences", includes: Based on the plurality of first grayscale differences and the plurality of second grayscale differences, the pixels with similar grayscale differences in the next frame face image are determined as target boundary pixels that match the boundary pixels corresponding to the first boundary position; The position of the target boundary pixel in the next frame of the face image is determined as the first position; Obtain the relative position between the first boundary position and the plurality of boundary positions; The relative position is taken as the second position of the target boundary pixel relative to the initial bounding box.

[0036] For example, in this embodiment, the position of the bounding box in different frames is calibrated based on the same face boundary contour. By determining the position of the target boundary pixel in the next frame of the face image as the first position, the relative position between the first boundary position and the plurality of boundary positions is obtained, and the relative position is used as the second position of the target boundary pixel relative to the initial bounding box.

[0037] Optionally, in some embodiments, step S103 above includes: Based on the target facial image, the 3D contour of the face region in three-dimensional space is identified, and a three-dimensional facial contour model corresponding to the face region is generated. The target face model with the highest matching degree with the three-dimensional facial contour model is selected from the preset face model database. Based on the target face model, key point detection is performed on the contour key point features of various facial features in the target face image, and based on the position of the key points in the target face image, multiple target key points corresponding to various facial features are generated. Based on the positions of the multiple target key points in the target facial image, the facial features of the target facial image are divided into regions to generate facial feature region images corresponding to each type of facial feature.

[0038] For example, in this embodiment, a face model is used to transform and calibrate facial features. A 68-point model or a 106-point model is used to identify the 3D contour of the face region in three-dimensional space. Based on the target facial image, the 3D contour of the face region in three-dimensional space is identified, generating a corresponding 3D facial contour model. A target face model with the highest matching degree to the 3D facial contour model is selected from a preset face model database. Based on the target face model, key point detection is performed on the contour key points of various facial features in the target facial image. Based on the position of the key points in the target facial image, multiple target key points corresponding to each type of facial feature are generated. Based on the position of the multiple target key points in the target facial image, the facial features of the target facial image are divided into regions, generating facial feature region images corresponding to each type of facial feature.

[0039] Step S104: Obtain the target makeup effect corresponding to each facial feature selected by the user from various makeup effects of each facial feature corresponding to the target face model.

[0040] For example, in this embodiment, the user's intention to achieve the desired makeup effect can be collected through the terminal device. Based on the user's click operation on the display device, the target makeup effect selected by the user from the makeup effects corresponding to various facial features can be determined.

[0041] Step S105: Based on the target makeup effect, perform makeup filling on the corresponding type of facial region in the facial feature region image, and generate a target makeup image matching the facial region on the display device.

[0042] For example, in this embodiment, the corresponding type of facial region in the facial feature region image is filled with makeup using the target makeup effect, and a target makeup image matching the facial region is generated on the display device.

[0043] Optionally, in some embodiments, step S105 above includes: Visually perceive the illumination intensity in the target area image to determine the illumination component and reflection component of the target area image; Based on the illumination component and the reflection component, the illumination distribution features are extracted on the L channel of the LAB color corresponding to the target region image to generate the illumination gradient map of the target region image. Based on the target makeup effect, the facial feature region image is filled with makeup to generate an initial makeup image; The illumination gradient map is fused with the initial makeup image using a Poisson fusion method to generate the target makeup image.

[0044] For example, in this embodiment, to make the makeup effect fit the user's face and avoid the textured look of the makeup filler, the illumination intensity in the target area image is visually perceived to determine the illumination and reflection components of the target area image. Based on the illumination and reflection components, the illumination distribution features are extracted on the L channel of the LAB color corresponding to the target area image to generate an illumination gradient map of the target area image. Makeup filler is applied to the facial feature area image according to the target makeup effect to generate an initial makeup image. The illumination gradient map and the initial makeup image are then fused using a Poisson fusion to generate the target makeup image.

[0045] The above technical solution, in response to the detection of a face image within a designated recognition area, performs video capture of the face image to generate target video stream data. It then identifies and calibrates the position of the face region within the designated recognition area in the target video stream data, determining multiple boundary positions of various feature points on the bounding box corresponding to the face region in the last frame of the target video stream data. Based on these boundary positions, the face region within the designated recognition area is cropped to generate a target facial image corresponding to the face region. Furthermore, based on the target facial image, various facial features of the face region are identified, generating facial feature region images corresponding to each type of facial feature. The system then acquires the target makeup effect selected by the user from various makeup effects corresponding to the target face model, and applies makeup filler to the corresponding type of facial region in the facial feature region image based on the target makeup effect. Finally, it generates a target makeup image matching the face region on the display device. This improves the realism of the makeup filler effect, ensuring that the real-time generated makeup filler closely matches the currently captured face image, realistically restoring the recommended makeup effect to the user and increasing user satisfaction.

[0046] Figure 2 This is a schematic diagram illustrating a facial recognition-based beauty processing device according to an exemplary embodiment. Figure 2 As shown, the device 100 includes: The first generation module 110 is configured to, in response to detecting a face image within a set recognition area, perform video capture on the face image to generate target video stream data; The determination module 120 is configured to identify and calibrate the position of the face region in the target video stream data within a set recognition area, and determine multiple boundary positions of each feature point on the bounding box corresponding to the face region in the last frame of the target video stream data. The second generation module 130 is configured to crop the face region within the set recognition area according to multiple boundary positions, generate the target face image corresponding to the face region, and perform region recognition on various face features of the face region according to the target face image, and generate face feature region images corresponding to each of the various face features. The acquisition module 140 is configured to acquire the target makeup effect corresponding to each facial feature selected by the user from a variety of makeup effects of each facial feature corresponding to the target face model. The execution module 150 is configured to perform makeup filling on the corresponding type of facial region in the facial feature region image based on the target makeup effect, and generate a target makeup image that matches the facial region on the display device.

[0047] Optionally, in some embodiments, the second generation module 130 includes: The first generation submodule is used to generate a face bounding box that moves with the face region on the display device corresponding to the set recognition area according to the multiple boundary positions. The acquisition submodule is used to acquire images of the face region within the set recognition area based on the face bounding box, until multiple frames of face images of the face region at multiple preset acquisition angles are acquired. The second generation submodule is used to fuse the multiple frames of facial images according to multiple preset acquisition angles corresponding to the multiple frames of facial images, and generate the target facial image of the facial region on the corresponding plane of the display device.

[0048] Optionally, in some embodiments, the first generation submodule includes: The first generation unit is configured to generate an initial bounding box corresponding to the face region on the last frame image based on the multiple boundary positions. The acquisition unit is used to acquire multiple first grayscale differences between the boundary pixel corresponding to the first boundary position and multiple adjacent pixels of the boundary pixel on the last frame image within a set area, wherein the first boundary position is any one of the multiple boundary positions; The obtaining unit is used to continue to acquire images of the face region within the set recognition area to obtain the next frame of face image; The first determining unit is used to determine multiple second grayscale differences between each pixel in the next frame face image and multiple adjacent pixels within the set area. The second determining unit is configured to determine, based on the plurality of second grayscale differences and the plurality of first grayscale differences, a first position of at least one boundary pixel in the next frame face image, and a second position of the at least one boundary pixel relative to the initial bounding box. The second generation unit is used to move the initial bounding box according to the first position and the second position to generate the face bounding box that moves with the face region.

[0049] Optionally, in some embodiments, the second determining unit is used for: Based on the plurality of first grayscale differences and the plurality of second grayscale differences, the pixels with similar grayscale differences in the next frame face image are determined as target boundary pixels that match the boundary pixels corresponding to the first boundary position; The position of the target boundary pixel in the next frame of the face image is determined as the first position; Obtain the relative position between the first boundary position and the plurality of boundary positions; The relative position is taken as the second position of the target boundary pixel relative to the initial bounding box.

[0050] Optionally, in some embodiments, the second generation module is further configured to: Based on the target facial image, the 3D contour of the face region in three-dimensional space is identified, and a three-dimensional facial contour model corresponding to the face region is generated. The target face model with the highest matching degree with the three-dimensional facial contour model is selected from the preset face model database. Based on the target face model, key point detection is performed on the contour key point features of various facial features in the target face image, and based on the position of the key points in the target face image, multiple target key points corresponding to various facial features are generated. Based on the positions of the multiple target key points in the target facial image, the facial features of the target facial image are divided into regions to generate facial feature region images corresponding to each type of facial feature.

[0051] Optionally, in some embodiments, the execution module is further configured to: Visually perceive the illumination intensity in the target area image to determine the illumination component and reflection component of the target area image; Based on the illumination component and the reflection component, the illumination distribution features are extracted on the L channel of the LAB color corresponding to the target region image to generate the illumination gradient map of the target region image. Based on the target makeup effect, the facial feature region image is filled with makeup to generate an initial makeup image; The illumination gradient map is fused with the initial makeup image using a Poisson fusion method to generate the target makeup image.

[0052] Optionally, in some embodiments, the first generation module is further configured to: In response to the detection of a face image within the defined recognition area, video stream within the defined recognition area is acquired to obtain initial video stream data within a defined duration range; If the duration of the face region corresponding to the face image in the initial video stream data being within the set recognition area is greater than a preset duration threshold, the initial video stream data will be used as the target video stream data.

[0053] The above technical solution, in response to the detection of a face image within a designated recognition area, performs video capture of the face image to generate target video stream data. It then identifies and calibrates the position of the face region within the designated recognition area in the target video stream data, determining multiple boundary positions of various feature points on the bounding box corresponding to the face region in the last frame of the target video stream data. Based on these boundary positions, the face region within the designated recognition area is cropped to generate a target facial image corresponding to the face region. Furthermore, based on the target facial image, various facial features of the face region are identified, generating facial feature region images corresponding to each type of facial feature. The system then acquires the target makeup effect selected by the user from various makeup effects corresponding to the target face model, and applies makeup filler to the corresponding type of facial region in the facial feature region image based on the target makeup effect. Finally, it generates a target makeup image matching the face region on the display device. This improves the realism of the makeup filler effect, ensuring that the real-time generated makeup filler closely matches the currently captured face image, realistically restoring the recommended makeup effect to the user and increasing user satisfaction.

[0054] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0055] Figure 3 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 3 As shown, the electronic device 300 may include a processor 301 and a memory 302. The electronic device 300 may also include one or more of a multimedia component 303, an input / output (I / O) interface 304, and a communication component 305.

[0056] The processor 301 controls the overall operation of the electronic device 300 to complete all or part of the steps in the aforementioned face recognition-based beauty processing method. The memory 302 stores various types of data to support the operation of the electronic device 300. This data may include, for example, instructions for any application or method operating on the electronic device 300, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 302 or transmitted via communication component 305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 304 provides an interface between processor 301 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 305 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0057] In an exemplary embodiment, the electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described face recognition-based beauty processing method.

[0058] In another exemplary embodiment, a computer-readable storage medium is also provided, which includes program instructions that, when executed by a processor, implement the steps of the face recognition-based makeup processing method described above. For example, the computer-readable storage medium may be the memory 302 that includes the program instructions, which may be executed by the processor 301 of the electronic device 300 to complete the face recognition-based makeup processing method described above.

[0059] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the above-described face recognition-based beauty processing method.

[0060] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0061] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction.

[0062] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A facial recognition-based beauty processing method, characterized in that, The method includes: In response to the detection of a face image within a defined recognition area, video capture is performed on the face image to generate target video stream data; The position of the face region in the target video stream data within the set recognition area is identified and calibrated to determine multiple boundary positions of each feature point on the bounding box corresponding to the face region on the last frame of the target video stream data. The face region within the set recognition area is cropped according to the multiple boundary positions to generate a target facial image corresponding to the face region. Based on the target facial image, various facial features of the face region are identified to generate facial feature region images corresponding to each type of facial feature. The user selects the target makeup effect corresponding to each facial feature from a variety of makeup effects for each facial feature in the facial feature region image. Based on the target makeup effect, the corresponding type of facial region in the facial feature region image is filled with makeup, and a target makeup image matching the facial region is generated on the display device.

2. The beauty processing method based on face recognition according to claim 1, characterized in that, The step of cropping the face region within the defined recognition area based on the multiple boundary positions to generate a target facial image corresponding to the face region includes: Based on the multiple boundary positions, a face bounding box that moves with the face region is generated on the display device corresponding to the set recognition area; Based on the face bounding box, the face region within the set recognition area is image acquired until multiple frames of face images of the face region at multiple preset acquisition angles are obtained. Based on multiple preset acquisition angles corresponding to the multiple frames of facial images, the multiple frames of facial images are fused to generate the target facial image of the facial region on the corresponding plane of the display device.

3. The beauty processing method based on face recognition according to claim 2, characterized in that, The step of generating a face bounding box corresponding to the face region that moves with the face on the display device corresponding to the set recognition area based on the multiple boundary positions includes: Based on the multiple boundary positions, an initial bounding box corresponding to the face region is generated on the last frame image; Obtain multiple first grayscale differences between the boundary pixel corresponding to the first boundary position and multiple adjacent pixels of the boundary pixel in the last frame image within a set area, wherein the first boundary position is any one of the multiple boundary positions; Continue to acquire images of the face region within the defined recognition area to obtain the next frame of face image; Determine multiple second grayscale differences between each pixel in the next frame of the face image and multiple adjacent pixels within the set area; Based on the plurality of second grayscale differences and the plurality of first grayscale differences, determine the first position of at least one boundary pixel in the next frame face image and the second position of the at least one boundary pixel relative to the initial bounding box. Based on the first position and the second position, the initial bounding box is moved to generate a face bounding box that moves with the face region.

4. The facial recognition-based beauty processing method according to claim 3, characterized in that, The step of determining, based on the plurality of second grayscale differences and the plurality of first grayscale differences, a first position of at least one boundary pixel in the next frame face image, and a second position of the at least one boundary pixel relative to the initial bounding box, includes: Based on the plurality of first grayscale differences and the plurality of second grayscale differences, the pixels with similar grayscale differences in the next frame face image are determined as target boundary pixels that match the boundary pixels corresponding to the first boundary position; The position of the target boundary pixel in the next frame of the face image is determined as the first position; Obtain the relative position between the first boundary position and the plurality of boundary positions; The relative position is taken as the second position of the target boundary pixel relative to the initial bounding box.

5. The beauty processing method based on face recognition according to claim 1, characterized in that, The step of performing region recognition on various facial features of the face region based on the target facial image, and generating facial feature region images corresponding one-to-one for each type of facial feature, includes: Based on the target facial image, the 3D contour of the face region in three-dimensional space is identified, and a three-dimensional facial contour model corresponding to the face region is generated. The target face model with the highest matching degree with the three-dimensional facial contour model is selected from the preset face model database. Based on the target face model, key point detection is performed on the contour key point features of various facial features in the target face image, and based on the position of the key points in the target face image, multiple target key points corresponding to various facial features are generated. Based on the positions of the multiple target key points in the target facial image, the facial features of the target facial image are divided into regions to generate facial feature region images corresponding to each type of facial feature.

6. The facial recognition-based beauty processing method according to any one of claims 1-5, characterized in that, The step of applying makeup to the corresponding type of facial region in the facial feature region image based on the target makeup effect, and generating a target makeup image matching the facial region on the display device, includes: Visually perceive the illumination intensity in the target facial image to determine the illumination component and reflection component of the target region image; Based on the illumination component and the reflection component, the illumination distribution features are extracted on the L channel of the LAB color corresponding to the target region image to generate the illumination gradient map of the target region image. Based on the target makeup effect, the facial feature region image is filled with makeup to generate an initial makeup image; The illumination gradient map is fused with the initial makeup image using a Poisson fusion method to generate the target makeup image.

7. The beauty processing method based on face recognition according to claim 1, characterized in that, The step of detecting a face image within a defined recognition area and capturing video of the face image to generate target video stream data includes: In response to the detection of a face image within the defined recognition area, video stream within the defined recognition area is acquired to obtain initial video stream data within a defined duration range; If the duration of the face region corresponding to the face image in the initial video stream data being within the set recognition area is greater than a preset duration threshold, the initial video stream data will be used as the target video stream data.

8. A facial recognition-based beauty processing device, characterized in that, The device includes: The first generation module is configured to, in response to detecting a face image within a set recognition area, perform video capture on the face image to generate target video stream data; The determination module is configured to identify and calibrate the position of the face region in the target video stream data within the set recognition area, and determine multiple boundary positions of each feature point on the bounding box corresponding to the face region on the last frame of the target video stream data; The second generation module is configured to crop the face region within the set recognition area according to the multiple boundary positions, generate a target facial image corresponding to the face region, and perform region recognition on various facial features of the face region according to the target facial image, generating facial feature region images corresponding to each type of facial feature. The acquisition module is configured to acquire the target makeup effect corresponding to each facial feature selected by the user from a variety of makeup effects of each facial feature corresponding to the facial feature region image. The execution module is configured to perform makeup filling on the corresponding type of facial region in the facial feature region image based on the target makeup effect, and generate a target makeup image matching the facial region on the display device.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-7.

Images