Face image processing method and device and electronic equipment

Abstract

The embodiment of the invention provides a face image processing method. The face image processing method comprises the following steps of performing picture segmentation on a face image mask; according to a picture segmentation result, key feature points of the human face are recognized, and beauty makeup region segmentation is carried out; and performing personalized virtual beauty makeup recommendation for different beauty makeup areas through face feature analysis. According to the face image processing method provided by the embodiment of the invention, personalized virtual beauty makeup recommendation can be provided by automatically analyzing the face features of the user, and personalized requirements of the user are met. The embodiment of the invention further provides a face image processing device and electronic equipment.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and more specifically, to a method, device, and electronic device for processing facial images. Background Art

[0002] The beauty and makeup applications currently on the market have made significant progress in user facial feature recognition, skin color analysis, and filter application. However, most applications still have deficiencies in personalization, sophistication, and operability, especially in meeting professional beauty needs. Summary of the invention

[0003] In response to the problems existing in the above-mentioned prior art, the embodiments of the present application provide a facial image processing method, device, and electronic device, which can provide personalized virtual beauty recommendations to meet the personalized needs of users by automatically analyzing the user's facial features.

[0004] In a first aspect, an embodiment of the present application provides a face image processing method, comprising the following steps:

[0005] Perform screen segmentation on the face image mask;

[0006] Based on the image segmentation results, identify the key feature points of the face and perform beauty area segmentation; and

[0007] Through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas.

[0008] Furthermore, the personalized virtual makeup recommendations for different makeup areas are made through facial feature analysis, including:

[0009] By identifying facial skin color and color space conversion, virtual makeup recommendations that are adapted to facial skin color are made for different beauty areas.

[0010] Furthermore, after the face image mask is segmented, the method further includes:

[0011] Through image semantic segmentation, the key elements of the face image are identified and the image segmentation results are improved.

[0012] Furthermore, before the face image mask is segmented, the method further includes:

[0013] Create a face image mask.

[0014] Furthermore, the creating of the face image mask comprises:

[0015] By processing the depth information, a three-dimensional mask of the face image is generated.

[0016] Furthermore, after performing personalized virtual makeup recommendations for different makeup areas through facial feature analysis, the method further includes:

[0017] Through intelligent fill lighting, the personalized virtual beauty recommendation effects of different beauty areas are optimized.

[0018] Furthermore, after performing personalized virtual makeup recommendations for different makeup areas through facial feature analysis, the method further includes:

[0019] Export the results of personalized virtual beauty recommendations.

[0020] In a second aspect, the embodiment of the present application further provides a face image processing device, comprising:

[0021] A screen segmentation module is used to perform screen segmentation on the face image mask;

[0022] A region segmentation module, which is used to identify key feature points of the face and perform beauty region segmentation based on the image segmentation results; and

[0023] The beauty recommendation module is used to make personalized virtual beauty recommendations for different beauty areas through facial feature analysis.

[0024] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the facial image processing method according to the first aspect when executing the program.

[0025] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, wherein the computer program is used to implement the facial image processing method according to the first aspect above.

[0026] The embodiments of the present application bring the following beneficial effects:

[0027] In the face image processing method provided in the embodiment of the present application, the face image mask is first segmented, and based on the segmentation result, the key feature points of the face are identified and the beauty area is segmented. Then, through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas. The face image processing method provided in the embodiment of the present application can provide personalized virtual beauty recommendations by segmenting the face image mask into beauty areas and automatically analyzing the user's facial features, thereby meeting the user's personalized needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0029] Figure 1 A schematic diagram of a facial image processing method according to an embodiment of the present invention;

[0030] Figure 2 A structural block diagram of a face image processing device provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0032] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments described in the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0034] In the specification and claims of this application and the above-mentioned drawings, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "multiple" means two or more. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0035] Figure 1 FIG. 1 is a flowchart of a method for processing a face image according to an embodiment of the present application. Figure 1 As shown, the face image processing method of the embodiment of the present application includes the following steps:

[0036] S101: Segment the face image mask;

[0037] Mask is an image processing technology that is mainly used to block part of the image content and display the image content in a specific area. It is equivalent to a window, usually exists as a separate layer, and is usually an upper-to-lower blocking relationship. After the face image mask is created, the mask can be processed, for example, the user's facial and body features are calculated through a convolutional neural network to obtain a mask, and the screen is segmented based on this information.

[0038] S102: According to the image segmentation result, identify key feature points of the face and perform beauty area segmentation; and

[0039] That is, according to the picture segmentation results, the facial recognition algorithm is used to identify the key facial feature points and perform beauty area segmentation to achieve beauty effects for different areas.

[0040] S103: Through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas.

[0041] As described above, after segmenting the beauty makeup area based on the face image mask segmentation result, personalized virtual beauty makeup recommendations are made for different beauty makeup areas through facial feature analysis. The beauty makeup recommendations mentioned here include virtual try-on of beauty makeup, hairdressing, headwear, glasses and other accessories.

[0042] Therefore, in the face image processing method provided in the embodiment of the present application, the face image mask is first segmented, and based on the segmentation result, the key feature points of the face are identified and the beauty area is segmented. Then, through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas. The face image processing method provided in the embodiment of the present application can provide personalized virtual beauty recommendations by segmenting the face image mask into beauty areas and automatically analyzing the user's facial features, thereby meeting the user's personalized needs.

[0043] In some embodiments of the present application, the face image is segmented into 9 beauty areas, and the specific segmentation method is as follows:

[0044] Zone 1 – T-zone, forehead: As the largest area in the T-zone, the forehead is often the focus of facial blemishes. The algorithm will focus on the accumulation of oil and dirt to reduce the formation of acne. At the same time, for fine lines and wrinkles caused by facial expressions, the smoothing and enhancement algorithms will improve the uniformity of the texture and enhance the smoothness and glossiness of the forehead.

[0045] Zone 2 – T-zone, between the eyebrows: The area between the eyebrows is prone to acne and 11 lines. The beauty algorithm identifies and fades the lines in this area, thereby reducing the depth of fine lines formed by frowning and optimizing the smoothness and definition of the brow center.

[0046] Zones 3 and 4 – T-zone, eyes: The skin around the eyes is the most fragile area, and the algorithm will use intelligent de-wrinkle and enhancement modes to reduce the appearance of wrinkles, fine lines and crow's feet caused by dryness, prevent skin sagging and swelling, and ultimately present a brighter and more vibrant eye area.

[0047] Zone 5 – T-Zone, Nose: The nose is prone to oil accumulation and clogged pores. The algorithm optimizes skin texture, smooths pores, and reduces the visibility of broken capillaries for a clearer, more refined nose.

[0048] Zones 6 and 7 – Cheeks: The cheek area is more blemish-prone, so the beauty algorithm will focus on balancing oil, reducing acne and shine, while evening out pore texture and maintaining skin consistency, revealing a healthy and smooth complexion.

[0049] Zone 8 – T-Zone, Chin: The chin area is prone to clogged pores and blemishes, especially after long-term use of certain cosmetics. The algorithm automatically optimizes the skin texture here to give the chin an even and smooth effect, while avoiding the accumulation of dullness and signs of aging.

[0050] Zone 9 – Neck, Neckline: The neck area is often neglected, and the algorithm will specifically improve the texture and firmness of this area, reducing wrinkles and sagging effects to enhance the overall harmony of the neck with the face.

[0051] Furthermore, in some embodiments of the present application, the personalized virtual beauty recommendations for different beauty areas are performed through facial feature analysis, including:

[0052] By identifying facial skin color and color space conversion, virtual makeup recommendations that are adapted to facial skin color are made for different beauty areas.

[0053] Specifically, as described above, the face image mask is segmented, and based on the segmentation results, the key feature points of the face are identified and the beauty area is segmented. The face skin color recognition and color space conversion are performed on the segmented beauty area. For example, according to the segmentation of the above-mentioned areas, the skin color sampling is performed and converted into CMYK and Pantone color numbers. Virtual beauty recommendations that are adapted to the face skin color are made for different beauty areas, thereby realizing personalized virtual beauty recommendations to meet the personalized needs of users.

[0054] In addition, for users with dark skin, model training optimization is required, with an emphasis on significantly improving the accuracy of face recognition algorithms / face segmentation algorithms / beautification algorithms / beautification algorithms / makeup algorithms / body detection algorithms for users with dark skin under different environments / lighting / exposure conditions.

[0055] Furthermore, in some embodiments of the present application, after the face image mask is segmented, the method further includes:

[0056] Through image semantic segmentation, the key elements of the face image are identified and the image segmentation results are improved.

[0057] Specifically, through image semantic segmentation, key elements in the picture are identified and inaccurate segmentation is improved, especially for people with dark skin. For example, the segmentation structure can be improved through a hash algorithm, thereby further improving the accuracy of the picture segmentation results.

[0058] Furthermore, in some embodiments of the present application, before performing screen segmentation on the face image mask, the method further includes:

[0059] Create a face image mask.

[0060] Specifically, masking is an image processing technology that is mainly used to block part of the image content and display the image content in a specific area. It is equivalent to a window, usually exists as a separate layer, and is usually an upper-to-lower occlusion relationship. After the face image mask is created, the mask can be processed to facilitate subsequent image segmentation operations and personalized beauty recommendations.

[0061] Furthermore, in some embodiments of the present application, the step of creating a face image mask includes:

[0062] By processing the depth information, a three-dimensional mask of the face image is generated.

[0063] Specifically, deep information technology, especially stereoscopic vision and deep learning models, can efficiently measure the three-dimensional features of the user's face, such as the distance between the eyes, forehead width, cheekbone convexity and chin contour, and support the precise matching of personalized hairstyles and makeup solutions. By capturing and analyzing the depth information of each point on the face, a three-dimensional face model of the user is generated so that the system can accurately measure the head shape and face size, providing quantitative data support for recommending suitable makeup, hairstyles and accessories.

[0064] Furthermore, in some embodiments of the present application, after performing personalized virtual makeup recommendations for different makeup areas through facial feature analysis, the method further includes:

[0065] Through intelligent fill lighting, the personalized virtual beauty recommendation effects of different beauty areas are optimized.

[0066] Specifically, the AI ​​algorithm for facial fill light plays an important role in the image preprocessing stage, especially in environments with insufficient or uneven lighting, which can improve the clarity of facial features and color reproduction. Based on AI technology, the algorithm dynamically identifies facial shadow areas and intelligently adjusts brightness, contrast, and color temperature to simulate a uniform and natural light source effect. This process often uses convolutional neural networks (CNN), generative adversarial networks (GAN), and HDR imaging algorithms to achieve efficient light enhancement and softening effects.

[0067] In the specific implementation, the facial fill-light algorithm detects the shadow and highlight positions of the face and adjusts the local brightness of the image to ensure the lighting effect of the facial area. The "virtual fill-light" effect generated by the GAN algorithm can naturally enhance the details of the dark parts, make the facial colors more realistic, and adapt to the brightness requirements of different skin tones and skin qualities, thereby providing a more accurate image basis for beauty and color number analysis. This type of intelligent fill-light algorithm ensures that the system can capture clear skin tones and facial features under various lighting conditions, improve the accuracy of beauty and styling recommendations, and even simulate makeup effects under light sources of different color temperatures.

[0068] Furthermore, in some embodiments of the present application, after performing personalized virtual makeup recommendations for different makeup areas through facial feature analysis, the method further includes:

[0069] Export the results of personalized virtual beauty recommendations.

[0070] Specifically, after personalized virtual beauty recommendations are made for different beauty areas through facial feature analysis, users are supported to save customized beauty parameters and generate a report in PDF format containing recommended color numbers and a list of cosmetics, allowing users to easily reuse or share makeup plans in the future.

[0071] In addition, in some embodiments of the present application, a purchase link for related products can be provided based on the hairstyle or headwear selected by the user, and the beauty / makeup that the user is satisfied with can be exported to a physical purchase plan with one click, thereby achieving the connection between virtual makeup and actual consumption.

[0072] Figure 2 FIG. 2 is a structural block diagram of a face image processing device 200 provided in an embodiment of the present application. Figure 2 As shown, the face image processing device 200 of the embodiment of the present application includes: a screen segmentation module 210, a region segmentation module 220 and a beauty recommendation module 230, wherein:

[0073] The image segmentation module 210 is used to perform image segmentation on the face image mask;

[0074] A region segmentation module 220, for identifying key feature points of a face and performing beauty region segmentation according to the image segmentation result; and

[0075] The beauty recommendation module 230 is used to make personalized virtual beauty recommendations for different beauty areas through facial feature analysis.

[0076] In the face image processing device provided in the embodiment of the present application, the face image mask is first segmented, and based on the segmentation result, the key feature points of the face are identified and the beauty area is segmented. Then, through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas. The face image processing method provided in the embodiment of the present application can provide personalized virtual beauty recommendations by segmenting the face image mask into beauty areas and automatically analyzing the user's facial features, thereby meeting the user's personalized needs.

[0077] It should be noted that the specific implementation method of the face image processing device of the embodiment of the present application is similar to the specific implementation method of the face image processing method of the embodiment of the present application. Please refer to the description of the method part for details, and no further details will be given here.

[0078] Figure 3 Schematic diagram of the structure of an electronic device 300 according to an embodiment of the present application.

[0079] like Figure 3 As shown, the electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from the storage part 302 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0080] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage section 308 as needed.

[0081] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 309, and / or installed from a removable medium 311. When the computer program is executed by a central processing unit (CPU) 301, the above-mentioned functions defined in the electronic device of the present application are executed.

[0082] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electronic device, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0083] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction-executing electronic device, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction-executing electronic device, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0084] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the processing receiving device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the aforementioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based electronic device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] The units or modules involved in the embodiments of the present application may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the processor is used to implement the face image processing method when executing the program:

[0086] Perform screen segmentation on the face image mask;

[0087] Based on the image segmentation results, identify the key feature points of the face and perform beauty area segmentation; and

[0088] Through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas.

[0089] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently and not be assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the face image processing method described in the present application:

[0090] Perform screen segmentation on the face image mask;

[0091] Based on the image segmentation results, identify the key feature points of the face and perform beauty area segmentation; and

[0092] Through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas.

[0093] As another aspect, the present application further provides a computer program product, which may be included in the electronic device described in the above embodiment; or may exist independently without being installed in the electronic device. The above computer program product stores one or more programs, and when the above programs are used by one or more processors to execute the face image processing method described in the present application:

[0094] Perform screen segmentation on the face image mask;

[0095] Based on the image segmentation results, identify the key feature points of the face and perform beauty area segmentation; and

[0096] Through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas.

[0097] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. All equivalent structural changes made by using the contents of the present application specification and drawings under the application concept of the present application, or directly / indirectly used in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A facial image processing method, characterized in that: The following steps are involved: Perform screen segmentation on the face image mask; According to the image segmentation results, identify the key feature points of the face and perform beauty area segmentation; and Through facial feature analysis, personalized virtual beauty recommendations are made for different beauty areas.

2. The facial image processing method according to claim 1, characterized in that: The personalized virtual makeup recommendations are made for different makeup areas through facial feature analysis, including: By identifying facial skin color and color space conversion, virtual makeup recommendations that are adapted to facial skin color are made for different beauty areas.

3. The facial image processing method according to claim 1, characterized in that: After the face image mask is segmented, the method further includes: Through image semantic segmentation, the key elements of the face image are identified and the image segmentation results are improved.

4. The facial image processing method according to claim 1, characterized in that: Before the face image mask is segmented, the method further includes: Create a face image mask.

5. The facial image processing method according to claim 4, characterized in that: The step of creating a face image mask comprises: By processing the depth information, a three-dimensional mask of the face image is generated.

6. The facial image processing method according to claim 1, characterized in that: After performing personalized virtual makeup recommendations for different makeup areas through facial feature analysis, the method further includes: Through intelligent fill lighting, the personalized virtual beauty recommendation effects of different beauty areas are optimized.

7. The facial image processing method according to claim 1, characterized in that: After performing personalized virtual makeup recommendations for different makeup areas through facial feature analysis, the method further includes: Export the results of personalized virtual beauty recommendations.

8. A facial image processing device, characterized in that: include: A screen segmentation module is used to perform screen segmentation on the face image mask; A region segmentation module, which is used to identify key feature points of the face and perform beauty region segmentation based on the image segmentation results; and The beauty recommendation module is used to make personalized virtual beauty recommendations for different beauty areas through facial feature analysis.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the face image processing method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the face image processing method according to any one of claims 1-7.

Images