An AI hairstyle design method and system based on user facial features

By collecting and producing a three-dimensional model from the user's head from multiple angles and matching hairstyles based on facial features, the problem of poor simulation effect of hairstyle design in the existing technology is solved, and users' more intuitive hairstyle selection is achieved and design efficiency is improved.

CN118314298BInactive Publication Date: 2024-11-22ZHEJIANG TEXTILE & FASHION COLLEGE
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410485360.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-22
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing hair styling software cannot provide three-dimensional effects of hair styling design simulation, making it difficult for users to judge whether the hairstyle is suitable for them. The hairstyle images in the hairstyle library are simple and the matching effect is poor.

Method used

By collecting and producing a three-dimensional model from the user's head from multiple angles, matching the best hairstyle from the hairstyle library based on the user's face characteristics, generating a three-dimensional model and pushing it to the user terminal, users can refer to and adjust their hairstyles.

Benefits of technology

Improve the efficiency and matching effect of hairstyle design, users can more intuitively judge whether the hairstyle is suitable for them, and improve the diversity of hairstyle choices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118314298B_ABST
    Figure CN118314298B_ABST
Patent Text Reader

Abstract

The present invention discloses an AI hairstyle design method and system based on user facial features, and relates to the field of image processing technology. The present invention comprises the following steps: step S1: collecting hairstyle images and facial images of each type; step S2: collecting user facial images with forehead exposed from multiple angles; step S3: preprocessing the facial image of the current hairstyle to be designed; step S4: performing facial features positioning processing on the preprocessed facial image and obtaining the facial contour; step S5: generating a three-dimensional model of the facial image according to the three-dimensional point cloud data; step S6: determining the user's head shape features; step S7: matching the hairstyle model from the hairstyle library according to the head shape features, and generating a three-dimensional model after fusing the hairstyle with the user's head shape. The present invention makes a three-dimensional model by collecting the user's head from multiple angles, and matches any hairstyle from the hairstyle library according to the user's facial features, so that the user can refer to and judge whether the hairstyle is suitable for him or her.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an AI hairstyle design method and system based on user facial features. Background Art

[0002] Many people who love beauty often want to change their hairstyles. Nowadays, barbershops usually provide users with reference hairstyles in the form of manuals. Due to limitations, users can only evaluate whether the hairstyle looks good, but cannot determine whether it is suitable for them. In actual use, users often worry that they will be dissatisfied with the hairstyle after changing it, and it is not suitable for them. In addition, the hairstyle manual made by printing is not only expensive, but also difficult to keep up with the speed of hairstyle updates.

[0003] At present, all hair design software on the market have their own hair library, but the hair images in the hair library of most hair design software are relatively simple, or are hair images that have not been processed, or one or two positioning points are manually marked on the original hair image. When users use such hair design software to design hair, they can only make manual adjustments when selecting hair images to match with face images, resulting in low hair design efficiency. Even if one or two positioning points are relied on to make the selected hair image coincide with the face image to complete automatic matching, the matching effect is often poor.

[0004] Therefore, the problem that technicians in this field urgently need to solve is: to provide an AI hairstyle design method and system based on user facial features to solve the problem that the existing technology cannot provide a three-dimensional stereoscopic effect of hairstyle design simulation, so that users can refer to and judge whether the hairstyle is suitable for them. Summary of the invention

[0005] The purpose of the present invention is to provide an AI hairstyle design method and system based on user facial features, by collecting the user's head from multiple angles to create a three-dimensional model, and matching any hairstyle from a hairstyle library according to the user's facial features, thereby solving the problem that the existing hairstyle design software has poor matching effect and the user cannot judge whether the hairstyle is suitable for him.

[0006] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] The present invention is an AI hairstyle design method based on user facial features, comprising the following steps:

[0008] Step S1: collect each type of hairstyle image and face shape image, match each face shape with one or more optimal hairstyles to generate a combined image, and store it in the hairstyle library;

[0009] Step S2: collecting facial images of the user with the forehead exposed from multiple angles;

[0010] Step S3: pre-processing the facial image of the current hairstyle to be designed;

[0011] Step S4: performing facial feature location processing on the pre-processed facial image and obtaining the facial contour;

[0012] Step S5: generating a three-dimensional model of the facial image according to the three-dimensional point cloud data;

[0013] Step S6: Mapping the facial contour information and the facial features positioning information into the three-dimensional model to determine the user's head shape features;

[0014] Step S7: matching a hairstyle model from a hairstyle library according to the head shape feature, and fusing the hairstyle with the user's head shape to generate a three-dimensional model;

[0015] Step S8: Pushing the matched three-dimensional model to the user via the user's mobile terminal.

[0016] As a preferred technical solution, in step S1, the combined image stored in the hairstyle library needs to be preprocessed, specifically including:

[0017] Step S11: selecting a type of hairstyle image from the hairstyle library as the hairstyle image to be processed;

[0018] Step S12: Detect the collected different face images respectively to determine the facial contour line, eye position, nose tip position, eyebrow position, lip position and ear position of each face image;

[0019] Step S13: determining the center position of the hairstyle image to be processed, and setting a proportional enlargement or reduction parameter according to the center position of the hairstyle image;

[0020] Step S14: adjusting the proportion parameters of the hairstyle so that the enlarged or reduced hairstyle image and the inner contour lines of the left and right sideburns of the facial contour line on the face image overlap with each other;

[0021] Step S15: marking the facial contour line, the positions of the eyes, the nose tip, the eyebrows, the lips and the ears in each adjusted combined image to generate a hairstyle image;

[0022] Step S16: Combining the hairstyle images corresponding to different facial images one by one to generate a corresponding hairstyle combination image, and storing it in the hairstyle library.

[0023] As a preferred technical solution, in step S2, two cameras are used to collect the facial image of the user with the forehead exposed, and the two cameras are calibrated respectively to obtain the internal and external parameters of the two cameras respectively, and then a set of calibration points are used through coordinates to establish the positional relationship between the two cameras.

[0024] As a preferred technical solution, the calibration methods of the two cameras include direct linear variability, perspective transformation short array method and two-step method; the two cameras are located at the same horizontal height and are no more than 100 cm apart, and image capture of human facial images is performed from the front, back, left, right and top-down angles. The user sits on a chair in a barber shop, adjusts the height of the two cameras to the same height as the user's head, and turns the chair to complete multi-angle facial shooting of the user.

[0025] As a preferred technical solution, in step S3, the preprocessing process is as follows:

[0026] Step S31: determining whether there is face information in the acquired image information, and deleting images without face information;

[0027] Step S32: grayscale processing is performed on the remaining image information;

[0028] Step S33: binarizing the generated grayscale image;

[0029] Step S34: Perform image subtraction processing to obtain an image after background subtraction processing.

[0030] As a preferred technical solution, in step S4, performing facial features positioning processing on the pre-processed facial image and obtaining the facial contour specifically includes:

[0031] Step S41: extracting feature values ​​from the preprocessed facial image;

[0032] Step S42: Initialize the deep neural network;

[0033] Step S43: using the training data to train the model parameters of the deep neural network to obtain a facial feature template;

[0034] Step S44: input the facial image feature values ​​obtained in step S4 into the facial feature template for recognition.

[0035] As a preferred technical solution, in step S5, the multi-frame three-dimensional point cloud data corresponding to each viewpoint is subjected to posture correction by a principal component analysis method; the multi-frame three-dimensional point cloud data corresponding to each viewpoint after correction is subjected to coarse registration and fine registration; the multi-frame three-dimensional point cloud data corresponding to each viewpoint after registration is subjected to the fusion operation, and the fusion operation includes: projecting the point cloud of the corresponding viewpoint to the corresponding coordinate plane, rasterizing the area of ​​the coordinate plane, merging the points falling in the same grid into one point; interpolating and filtering the grid data in the coordinate plane; and mapping the grid data to three-dimensional space.

[0036] As a preferred technical solution, in step S6, the three-dimensional coordinates of each feature point of the user's head shape feature can be obtained according to the triangulation principle and the left and right cameras:

[0037]

[0038] Where b is the distance between the optical centers c1 and c2 of the left and right cameras, f is the focal length of the camera, the coordinates of the feature point P (X, Y, Z) in the left and right camera coordinate systems are (x1, y1, z1) and (x2, y2, z2), and the corresponding pixel coordinates on the left and right planes are (u1, v1) and (u2, v2); the spatial coordinates of the feature point P are expressed as:

[0039]

[0040] The present invention is an AI hairstyle design system based on user facial features, comprising two cameras, a user mobile terminal and a server;

[0041] The two cameras are installed in front of the barber shop chair, at the same level and no more than 100 cm apart; the two cameras are used to collect user head images at different angles;

[0042] The server includes a hairstyle library, a hairstyle preprocessing module, a facial image positioning processing module, a hairstyle selection and matching module, a facial features recognition module and a facial feature extraction module; the hairstyle library is used to collect and combine each type of hairstyle image and facial shape image one by one, and store the composed hairstyle images; the hairstyle preprocessing module is used to perform image processing on the images collected by the two cameras; the facial image positioning processing module is used to perform positioning processing on the facial image to be designed; the hairstyle selection and matching module is used to determine the facial type of the facial image to be designed, and retrieve the corresponding hairstyle in the hairstyle library according to the facial type for matching; the facial features recognition module is used to train a deep learning facial recognition network model, recognize the facial image to be designed, and determine the facial shape and the position and shape of the facial features; the facial feature extraction module is used to extract features from the preprocessed facial image;

[0043] The user mobile terminal is used to send the designed facial image output by the hairstyle selection and matching module of the server to the user mobile terminal for display.

[0044] The present invention has the following beneficial effects:

[0045] (1) The present invention creates a three-dimensional model by collecting data from multiple angles on the user's head, and matches any hairstyle from a hairstyle library according to the user's facial features, so that the user can refer to and judge whether the hairstyle is suitable for him or her;

[0046] (2) The present invention sends the hairstyle matching result to the user's mobile terminal. When the user is not satisfied with the hairstyle matched by the system, the user can directly access the hairstyle library and select a hairstyle based on the generated avatar model to generate the hairstyle he or she needs, thereby improving the diversity of the user's hairstyle selection.

[0047] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0049] Figure 1 This is a flow chart of an AI hairstyle design method based on user facial features of the present invention;

[0050] Figure 2 The figure is a schematic diagram of the structure of an AI hairstyle design system based on user facial features of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0053] In order to make the purpose, technical solution and advantages of this application clearer, the following Figure 1 The implementation methods of the present application are described in further detail.

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-2 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] Embodiment 1

[0056] See also Figure 1As shown, the present invention is an AI hairstyle design method based on user facial features, comprising the following steps:

[0057] Step S1: collect each type of hairstyle image and face shape image, match each face shape with one or more optimal hairstyles to generate a combined image, and store it in the hairstyle library;

[0058] The combined images stored in the hairstyle library need to be preprocessed, including:

[0059] Step S11: selecting a type of hairstyle image from the hairstyle library as the hairstyle image to be processed;

[0060] Step S12: Detect the collected different face images respectively to determine the facial contour line, eye position, nose tip position, eyebrow position, lip position and ear position of each face image;

[0061] Step S13: determining the center position of the hairstyle image to be processed, and setting a proportional enlargement or reduction parameter according to the center position of the hairstyle image;

[0062] Step S14: adjusting the proportion parameters of the hairstyle so that the enlarged or reduced hairstyle image and the inner contour lines of the left and right sideburns of the facial contour line on the face image overlap with each other;

[0063] Step S15: marking the facial contour line, the positions of the eyes, the nose tip, the eyebrows, the lips and the ears in each adjusted combined image to generate a hairstyle image;

[0064] Step S16: Combining the hairstyle images corresponding to different facial images one by one to generate a corresponding hairstyle combination image, and storing it in the hairstyle library.

[0065] Step S2: collecting facial images of the user with the forehead exposed from multiple angles;

[0066] In step S2, two cameras are used to collect the facial image of the user with the forehead exposed, and the two cameras are calibrated respectively to obtain the intrinsic and extrinsic parameters of the two cameras respectively, and then a set of calibration points are performed through coordinates to establish the positional relationship between the two cameras.

[0067] The calibration methods of the two cameras include direct linear variability, perspective transformation short array method and two-step method; the two cameras are located at the same horizontal height and the distance between them is no more than 100 cm, and the images of the human face are collected from the front, back, left, right and top view angles respectively. The user sits on a chair in a barber shop, the height of the two cameras is adjusted to be the same as the height of the user's head, and the chair is turned to complete the multi-angle facial shooting of the user;

[0068] Step S3: preprocessing the facial image of the current hairstyle to be designed, the preprocessing process is as follows:

[0069] Step S31: determining whether there is face information in the acquired image information, and deleting images without face information;

[0070] Step S32: grayscale processing is performed on the remaining image information;

[0071] Step S33: binarizing the generated grayscale image;

[0072] Step S34: Perform image subtraction processing to obtain an image after background subtraction processing.

[0073] Step S4: performing facial feature location processing on the pre-processed facial image and obtaining the facial contour;

[0074] The specific steps of locating the facial features of the pre-processed facial image and obtaining the facial contour include:

[0075] Step S41: extracting feature values ​​from the preprocessed facial image;

[0076] Step S42: Initialize the deep neural network;

[0077] Step S43: using the training data to train the model parameters of the deep neural network to obtain a facial feature template;

[0078] Step S44: input the facial image feature values ​​obtained in step S4 into the facial feature template for recognition.

[0079] Step S5: Generate a three-dimensional model of the facial image according to the three-dimensional point cloud data; perform posture correction on the multi-frame three-dimensional point cloud data corresponding to each viewpoint by using the principal component analysis method; perform coarse registration and fine registration operations on the multi-frame three-dimensional point cloud data corresponding to each viewpoint after correction; perform a fusion operation on the multi-frame three-dimensional point cloud data corresponding to each viewpoint after registration, and the fusion operation includes: projecting the point cloud of the corresponding viewpoint to the corresponding coordinate plane, rasterizing the area of ​​the coordinate plane, merging the points falling in the same grid into one point; interpolating and filtering the grid data in the coordinate plane; and mapping the grid data to three-dimensional space.

[0080] Step S6: Mapping the facial contour information and the facial features positioning information into the three-dimensional model to determine the user's head shape features;

[0081] Step S7: matching a hairstyle model from a hairstyle library according to the head shape feature, and fusing the hairstyle with the user's head shape to generate a three-dimensional model;

[0082] Step S8: Pushing the matched three-dimensional model to the user via the user's mobile terminal.

[0083] In step S6, the three-dimensional coordinates of each feature point of the user's head shape feature can be obtained according to the triangulation principle and the left and right cameras:

[0084]

[0085] Where b is the distance between the optical centers c1 and c2 of the left and right cameras, f is the focal length of the camera, the coordinates of the feature point P (X, Y, Z) in the left and right camera coordinate systems are (x1, y1, z1) and (x2, y2, z2), and the corresponding pixel coordinates on the left and right planes are (u1, v1) and (u2, v2); the spatial coordinates of the feature point P are expressed as:

[0086]

[0087] Embodiment 2

[0088] See also Figure 2 As shown, the present invention is an AI hairstyle design system based on user facial features, which can be used to execute the method content of Example 1 of the present invention, including: two cameras, a user mobile terminal and a server; the two cameras are installed in front of the barber shop seat, located at the same horizontal height, and no more than 100 cm apart; the two cameras are used to collect user head images at different angles; the server includes a hairstyle library, a hairstyle preprocessing module, a facial image positioning processing module, a hairstyle selection and matching module, a facial features recognition module and a facial feature extraction module; the hairstyle library is used to collect each type of hairstyle image and facial image and combine them one by one, and store the composed hairstyle images; the hairstyle preprocessing module The module is used to process the images collected by the two cameras; the facial image positioning and processing module is used to position the facial image to be designed; the hairstyle selection and matching module is used to determine the facial type of the facial image to be designed, and retrieve the corresponding hairstyle in the hairstyle library according to the facial type for matching; the facial features recognition module is used to train the deep learning facial recognition network model, recognize the facial image to be designed, and determine the face shape and the position and shape of the facial features; the facial feature extraction module is used to extract features from the pre-processed facial image; the user mobile terminal is used to send the designed facial image output by the hairstyle selection and matching module of the server to the user mobile terminal for display.

[0089] It is worth noting that in the above system embodiment, the various units included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0090] In addition, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing related hardware through a program, and the corresponding program can be stored in a computer-readable storage medium.

[0091] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An AI hairstyle design method based on user facial features, characterized in that: The steps include: Step S1: collect each type of hairstyle image and face shape image, match each face shape with one or more optimal hairstyles to generate a combined image, and store it in the hairstyle library; Step S2: collecting facial images of the user with the forehead exposed from multiple angles; Step S3: pre-processing the facial image of the current hairstyle to be designed; Step S4: performing facial feature location processing on the pre-processed facial image and obtaining the facial contour; Step S5: generating a three-dimensional model of the facial image according to the three-dimensional point cloud data; Step S6: Mapping the facial contour information and the facial features positioning information into the three-dimensional model to determine the user's head shape features; Step S7: matching a hairstyle model from a hairstyle library according to the head shape feature, and fusing the hairstyle with the user's head shape to generate a three-dimensional model; Step S8: Pushing the matched 3D model to the user via the user's mobile terminal; In the step S1, the combined image stored in the hairstyle library needs to be preprocessed, specifically including: step S11: selecting a type of hairstyle image from the hairstyle library as the hairstyle image to be processed; Step S12: Detect the collected different face images respectively to determine the facial contour line, eye position, nose tip position, eyebrow position, lip position and ear position of each face image; Step S13: determining the center position of the hairstyle image to be processed, and setting a proportional enlargement or reduction parameter according to the center position of the hairstyle image; Step S14: adjusting the proportion parameters of the hairstyle so that the enlarged or reduced hairstyle image and the inner contour lines of the left and right sideburns of the facial contour line on the face image overlap with each other; Step S15: marking the facial contour line, the positions of the eyes, the nose tip, the eyebrows, the lips and the ears in each adjusted combined image to generate a hairstyle image; Step S16: Combining the hairstyle images corresponding to different face images one by one to generate a corresponding hairstyle combination image, and storing it in the hairstyle library; In the step S4, the facial features positioning process is performed on the pre-processed facial image and the facial contour is obtained, specifically including: Step S41: extracting feature values ​​from the preprocessed facial image; Step S42: initializing the deep neural network; Step S43: using the training data to train the model parameters of the deep neural network to obtain a facial feature template; Step S44: inputting the facial image feature values ​​obtained in step S4 into the facial features template for recognition; In the step S5, the posture correction is performed on the multi-frame three-dimensional point cloud data corresponding to each viewpoint by the principal component analysis method; the coarse registration operation and the fine registration operation are performed on the multi-frame three-dimensional point cloud data corresponding to each viewpoint after the correction; the fusion operation is performed on the multi-frame three-dimensional point cloud data corresponding to each viewpoint after the registration, and the fusion operation includes: projecting the point cloud of the corresponding viewpoint to the corresponding coordinate plane, rasterizing the area of ​​the coordinate plane, and merging the points falling in the same grid into one point; interpolating and filtering the grid data in the coordinate plane; and mapping the grid data to the three-dimensional space; In step S6, the three-dimensional coordinates of each feature point of the user's head shape feature can be obtained according to the triangulation principle and the left and right cameras: ; Among them, b is the optical center of the left and right cameras c 1 and c The distance between 2, f is the focal length of the camera, and the coordinates of the feature point P (X, Y, Z) in the left and right camera coordinate systems are ( x 1, y 1, z 1 ) and ( x 2, y 2, z 2 ), the corresponding pixel coordinates on the left and right planes are ( u 1, v 1 ) and ( u 2, v 2); then the spatial coordinates of the feature point P are expressed as:

2. The AI ​​hairstyle design method based on user facial features according to claim 1, characterized in that: In step S2, two cameras are used to collect the facial image of the user with the forehead exposed, and the two cameras are calibrated to obtain the intrinsic and extrinsic parameters of the two cameras, and then a set of calibration points are used to establish the positional relationship between the two cameras.

3. The AI ​​hairstyle design method based on user facial features according to claim 2, characterized in that: The calibration methods for the two cameras include direct linear variability, perspective transformation short array method and two-step method; the two cameras are located at the same horizontal height and are no more than 100 cm apart, and the images of the human face are collected from the front, back, left, right and top-down angles respectively.

4. The AI ​​hairstyle design method based on user facial features according to claim 1, characterized in that: In step S3, the preprocessing process is as follows: Step S31: determining whether there is face information in the acquired image information, and deleting images without face information; Step S32: grayscale processing is performed on the remaining image information; Step S33: binarizing the generated grayscale image; Step S34: Perform image subtraction processing to obtain an image after background subtraction processing.

5. An AI hair style design system based on user facial features, using an AI hair style design method based on user facial features as claimed in any one of claims 1 to 4, characterized in that: Includes two cameras, user mobile terminal and server: The two cameras are installed in front of the barber shop chair, at the same level and no more than 100 cm apart; the two cameras are used to capture user head images at different angles; The server includes a hairstyle library, a hairstyle preprocessing module, a facial image positioning processing module, a hairstyle selection and matching module, a facial features recognition module and a facial feature extraction module; The hairstyle library is used to collect and combine each type of hairstyle image and face shape image one by one, and store the composed hairstyle images; the hairstyle preprocessing module is used to perform image processing on the images collected by the two cameras; the face image positioning processing module is used to perform positioning processing on the face image to be designed; the hairstyle selection and matching module is used to determine the face type of the face image to be designed, and retrieve the corresponding hairstyle in the hairstyle library according to the face type for matching; the facial features recognition module is used to train the deep learning facial recognition network model, recognize the face image to be designed, and determine the face shape and the position and shape of the facial features; the facial feature extraction module is used to extract features from the preprocessed facial image; The user mobile terminal is used to send the designed facial image output by the hairstyle selection and matching module of the server to the user mobile terminal for display.