A clothing style matching algorithm based on facial features

Abstract

The application discloses a clothes style matching algorithm based on facial features, and has the characteristics that the algorithm comprises the following steps: S1, constructing a convolutional neural network for gender recognition; S2, preprocessing an Adience image dataset and then inputting the convolutional neural network for gender recognition; S3, training the convolutional neural network for gender recognition; S4, recording a user's face video through a camera and extracting image frames from the video; S5, performing face key point detection on the image frames and inputting the convolutional neural network for gender recognition; S6, calculating face shape parameters according to the face key points; S7, constructing a relationship index table of the face shape parameters and the face shape; and S8, recommending clothes according to the index table and the gender recognition result. The technical scheme provided by the application can recommend clothes matching according to the face shape and gender of a user.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, and particularly to a clothing style matching algorithm based on facial features. BACKGROUND

[0002] With the development of neural network deep learning, providing intelligent services for users through image information has gradually become the focus of current technology companies. However, most of the behaviors in daily life are composed of multiple situations, and current image recognition deep learning is more used to complete the realization of a specific target, and ignores the realization of composite tasks.

[0003] For the common user demand of clothing style matching in daily life, the existing image recognition algorithm can only simply meet the face feature extraction and feature recognition, and there is no complete technical solution for completing the complete clothing recommendation process. The clothing recommendation process involves gender recognition, face feature extraction, face feature and clothing matching and other technologies. Although image recognition has proposed solutions for each single task, there is still a considerable technical gap from completing a mature clothing recommendation. SUMMARY

[0004] In view of the existing problems, the purpose of the present application is to provide a clothing style matching algorithm based on facial features, characterized in that it comprises the following steps: S1: constructing a convolutional neural network for gender recognition; S2: preprocessing the Adience image dataset, and then inputting the convolutional neural network for gender recognition; S3: training the convolutional neural network for gender recognition; S4: recording the user's face video through the camera, and extracting image frames from the video; S5: performing face key point detection on the image frames, and inputting the convolutional neural network for gender recognition; S6: calculating the face shape parameters according to the face key points; S7: constructing a relationship index table of face shape parameters and face shapes; S8: making clothing recommendations according to the index table and the gender recognition result.

[0005] Preferably, the convolutional neural network in step S1 comprises three convolutional layers, two fully connected layers, and one final output layer.

[0006] Preferably, the preprocessing of step S2 is specifically: scaling the images in the Adience image dataset to 256*256, and cropping the size of the images to 227*227 through center cropping.

[0007] Preferably, the constraint condition of training in step S3 is as follows: in the convolutional neural network for gender recognition, the weights of all layers are initialized with Gaussian random values with a standard deviation of 0.01 and a mean of 0; no pre-trained model is used during training, no data other than the benchmark available images and labels is used, and the network is trained from scratch; the target value of training is represented by a sparse binary vector corresponding to the real category; for each training image, the target label vector has a length of the number of categories, and the value at the index position of the real value is 1 and the values at other positions are 0; the training is trained using the stochastic gradient descent algorithm, with a batch size of 50, an initial learning rate of e-3, and a decrease to e-4 after 10K iterations.

[0008] Preferably, step S5 detects the face key points by the following method: reference the face key point detection model in dlib to detect the face key points, draw a face key point image including 68 face key points, record the two-dimensional coordinate values of each face key point, and number the 68 face key points 1-68, record the number of detection samples T, and each time the detection model detects the face key points from the user's face video, the value of T is increased by 1.

[0009] Preferably, step S6 is specifically: calculating the morphological facial index Is by the formula: Is = H S / W S *100%, wherein Is is the morphological facial index, H S is the morphological face height, and WS is the face width. M M / W S *100%, wherein I M is the zygomatic mandibular width index, W M is the mandibular width, and W S is the face width.

[0010] Preferably, step S6 calculates the following parameters from the face key point image of step S5: the morphological face height H S , the face width W S , and the mandibular width W M ; wherein the morphological face height H S is the Euclidean distance between point 28 and point 9 in the face key point image, the face width W S is the Euclidean distance between point 2 and point 16 in the face key point image, and the mandibular width W M is the Euclidean distance between point 5 and point 13 in the face key point image.

[0011] ​Preferably, step S7 is specifically: the morphological plane index greater than 100% is long face, the morphological plane index greater than 70% and less than 100% is medium-long face, and the morphological plane index less than 70% is short face; the zygomatic mandibular width index greater than 100% is triangular face, the zygomatic mandibular width index greater than 70% and less than 90% is pentagonal face, and the zygomatic mandibular width index greater than 40% and less than 70% is triangular face.

[0012] Preferably, step S8 is realized by the formula:

[0013]

[0014] Calculate the clothing recommendation parameter VEST, wherein G is a gender parameter, G=1 is male, G=-1 is female, T is the number of detection samples, I SK is the morphological plane index of the Kth detection sample, MALE is a male clothing adjustment parameter, FEM is a female clothing adjustment parameter, I MK is the zygomatic mandibular width index of the Kth detection sample, W SK is the face width of the Kth detection sample; the male clothing adjustment parameter MALE=1.8, and the female clothing adjustment parameter FEM=2.08; create a VEST recommendation label: when VEST>0.85, the VEST recommendation label records loose and vertical patterns; when 0.85>VEST>0.6, the VEST recommendation label records close-fitting and shaping; and when VEST<0.6, the VEST recommendation label records high-waisted and short; create a face shape recommendation label: add the following labels to a long face user: coat, wide-leg pants, V-neck, and long boots; add the following labels to a medium-long face user: T-shirt, sweatshirt, high-heeled shoes, and seven-eighths pants; add the following labels to a short face user: sweater, wide sleeve, short upper body, and jeans; add the following labels to a pentagonal face user: halter, backless, dress, and vest; add the following labels to a round face user: bubble sleeve, T-shirt, shorts, and sneakers; add the following labels to a triangular face user: shirt, jacket, short coat, and short skirt; and combine the user gender, the VEST recommendation label, and the face shape recommendation label to obtain a clothing matching recommendation.

[0015] The beneficial technical effects of the present application are that:

[0016] 1. The present application can guarantee high recognition accuracy and efficiency by recognizing the gender of the user through a convolutional neural network.

[0017] 2. The present application can provide a rational judgment for clothing matching by judging the face shape through face structure parameters.

[0018] 3. The present application can accurately reflect the face shape characteristics of the user by extracting face feature points based on dlib. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1is a flow chart of a clothing style matching algorithm based on facial features provided by the present application.

[0020] Figure 2 is a case diagram of extracting facial feature points in a preferred embodiment of the present application. DETAILED DESCRIPTION

[0021] The embodiments of the present application are described in detail below, and the following embodiments are implemented on the premise of the technical solutions of the present application, and detailed implementation modes and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0022] In this document, reference to“an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is explicitly contemplated that embodiments described herein can be combined with other embodiments in a non- conflicting manner.

[0023] As shown in Figure 1 A clothing style matching algorithm based on facial features, characterized in that it comprises the following steps: S1: constructing a convolutional neural network for gender recognition; S2: preprocessing the Adience image dataset, and then inputting the convolutional neural network for gender recognition; S3: training the convolutional neural network for gender recognition; S4: recording a user's face video through a camera, and extracting image frames from the video; S5: detecting facial key points on the image frames, and inputting the convolutional neural network for gender recognition; S6: calculating face shape parameters according to the facial key points; S7: constructing a relationship index table of face shape parameters and face shapes; and S8: making clothing recommendations according to the index table and the gender recognition result.

[0024] The convolutional neural network in step S1 comprises three convolutional layers, two fully connected layers, and a final output layer.

[0025] The preprocessing of step S2 is specifically: scaling the images in the Adience image dataset to 256*256, and cropping the size of the images to 227*227 through center cropping.

[0026] The constraint conditions of the training in step S3 are as follows: in the convolutional neural network for gender recognition, the weights of all layers are initialized with Gaussian random values with a standard deviation of 0.01 and a mean of 0; no pre-trained model is used during training, no data other than the benchmark available images and labels is used, and the network is trained from scratch; the target value of the training is represented by a sparse binary vector corresponding to the real category; for each training image, the target label vector has a length of the number of categories, and the value at the index corresponding to the real value is 1, and the values at other positions are 0; the training is performed using the stochastic gradient descent algorithm, with a batch size of 50, an initial learning rate of e-3, and a decrease to e-4 after 10K iterations.

[0027] As shown in Figure 2 , step S5 detects the face key points by the following method: referencing the face key point detection model in dlib to detect the face key points, drawing a face key point image including 68 face key points, recording the two-dimensional coordinate values of each face key point, and numbering the 68 face key points 1-68, recording the number of detection samples T, and each time the detection model detects the face key points from the user's face video, the value of T is increased by 1.

[0028] Step S6 is specifically: calculating the morphological facial index by the formula: Is = H S / W S * 100%, wherein Is is the morphological facial index, H S is the morphological facial height, and W S is the face width; calculating the subzygomatic mandibular width index by the formula: I M = W M / W S * 100%, wherein I M is the subzygomatic mandibular width index, W M is the mandibular width, and W S is the face width.

[0029] Step S6 calculates the following parameters from the face key point image of step S5: the morphological facial height H S , the face width W S , and the mandibular width W M ; wherein the morphological facial height H S is the Euclidean distance between point 28 and point 9 in the face key point image, the face width W S is the Euclidean distance between point 2 and point 16 in the face key point image, and the mandibular width W M is the Euclidean distance between point 5 and point 13 in the face key point image.

[0030] Step S7 is specifically: the morphological face index is greater than 100% for long face, the morphological face index is greater than 70% and less than 100% for medium-long face, and the morphological face index is less than 70% for short face; the zygomatic mandibular width index is greater than 100% for triangular face, the zygomatic mandibular width index is greater than 70% and less than 90% for pentagonal face, and the zygomatic mandibular width index is greater than 40% and less than 70% for triangular face.

[0031] Wherein, the relationship index table of face shape parameters and face shape is shown in Table 1:

[0032] Table 1: Relationship index table of face shape parameters and face shape

[0033] Is > 100% 70% < Is < 100% Is < 70% I M >100%]] Long triangular face Medium-long triangular face Short triangular face 90%>I M> 70%]]> Long pentagonal face Medium-long pentagonal face Short pentagonal face 40%>I M >70%]]> Long pear-shaped face Medium-long pear-shaped face Short pear-shaped face

[0034] Step S8 is calculated by the formula:

[0035]

[0036] The clothing recommendation parameter VEST is calculated, wherein G is the gender parameter, G = 1 for male and G = -1 for female, T is the number of detection samples, I SK is the morphological face index of the Kth detection sample, MALE is the male clothing adjustment parameter, FEM is the female clothing adjustment parameter, I MK is the zygomatic mandibular width index of the Kth detection sample, W SK is the face width of the Kth detection sample; the male clothing adjustment parameter MALE = 1.8, and the female clothing adjustment parameter FEM = 2.08; the VEST recommendation label is created: when VEST > 0.85, the VEST recommendation label records loose and vertical patterns; when 0.85 > VEST > 0.6, the VEST recommendation label records close-fitting and shaping; when VEST < 0.6, the VEST recommendation label records high-waisted and short; the face shape recommendation label is created: the following labels are added for long face users: overcoat, wide-leg pants, V-neck, and long boots; the following labels are added for medium-long face users: T-shirt, sweatshirt, high-heeled shoes, and seven-eighths pants; the following labels are added for short face users: sweater, wide sleeve, short upper body, and jeans; the following labels are added for pentagonal face users: halter, backless, dress, and vest; the following labels are added for oval face users: bubble sleeve, T-shirt, shorts, and sneakers; the following labels are added for triangular face users: shirt, jacket, short coat, and short skirt; the user gender, VEST recommendation label, and face shape recommendation label are combined to obtain clothing matching recommendation.

[0037] The preferred embodiments of the present application have been described above in detail. It should be understood that modifications and variations to the preferred embodiments could be made by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that there be included within the scope of the application, all such modifications and variations as would be apparent to those skilled in the art upon reading this disclosure. It is intended to obtain for the inventors such patent rights as are available for any patent granted on the present application.

Claims

1. A clothing style matching algorithm based on facial features, characterized in that, The method comprises the following steps: S1: constructing a convolutional neural network for gender recognition; S2: preprocessing the Adience image dataset and then inputting the convolutional neural network for gender recognition; S3: training the convolutional neural network for gender recognition; S4: recording a user's face video through a camera and extracting image frames from the video; S5: detecting facial key points of the image frames and inputting the convolutional neural network for gender recognition; S6: calculating face shape parameters according to the facial key points; S7: constructing a relationship index table of face shape parameters and face shapes; S8: recommending clothes according to the index table and the gender recognition result; The step S8 adds the following labels to the medium-long face users: T-shirt, sweatshirt, high heels, seven-eighths trousers; adds the following labels to the short face users: sweater, wide sleeve, short upper garment, jeans; adds the following labels to the five-angle face users: halter, backless, dress, vest; adds the following labels to the melon seed face users: bubble sleeve, T-shirt, shorts, sports shoes; and adds the following labels to the triangular face users: shirt, jacket, short coat, short skirt. calculating a garment recommendation parameter VEST, wherein G is a gender parameter, G=1 for male, G=-1 for female, T is a number of detection samples, I SK is a facial shape index of the Kth detection sample, MALE is a male garment adjustment parameter, FEM is a female garment adjustment parameter, I MK is a sub-zygomatic mandibular width index of the Kth detection sample, W SK is a face width of the Kth detection sample; the male garment adjustment parameter MALE=1.8, the female garment adjustment parameter FEM=2.08; creating a VEST recommendation label: when VEST>0.85, the VEST recommendation label records loose, vertical patterns; when 0.85>VEST>0.6, the VEST recommendation label records close-fitting, shaping; when VEST<0.6, the VEST recommendation label records high-waisted, short; creating a face shape recommendation label: adding the following label to a long face user: overcoat, wide-leg pants, V-neck, long boots; The user gender, the VEST recommendation label and the face shape recommendation label are combined to obtain a clothing matching recommendation. The step S5 detects the facial key points by the following method: detecting facial key points by referring to a facial key point detection model in dlib, drawing a facial key point image comprising 68 facial key points, recording two-dimensional coordinate values of each facial key point, numbering the 68 facial key points as 1-68, and recording a detection sample number T, which is increased by 1 each time the detection model detects the user's face video. The convolutional neural network in the step S1 comprises three convolutional layers, two fully connected layers and one final output layer. The preprocessing in the step S2 specifically comprises: scaling images in the Adience image dataset to 256*256 and cropping the size of the images to 227*227 through center cropping. The step S6 is specifically: calculating the morphological facial index by the formula: I s = H S / W S *100%; wherein, I s is the morphological facial index, H S is the morphological facial height, and W S is the face width; calculating the zygomatic mandibular width index by the formula: I M = W M / W S *100%; wherein, I M is the zygomatic mandibular width index, W M is the lower jaw width, and W S is the face width; the step S7 is specifically: the morphological facial index greater than 100% is long face, the morphological facial index greater than 70% and less than 100% is medium-long face, and the morphological facial index less than 70% is short face; the zygomatic mandibular width index greater than 100% is triangular face, the zygomatic mandibular width index greater than 70% and less than 90% is pentagonal face, and the zygomatic mandibular width index greater than 40% and less than 70% is triangular face. 2.The clothing style matching algorithm based on facial features according to claim 1, wherein, The constraint conditions for the training in the step S3 are as follows: in the convolutional neural network for gender recognition, the weights of all layers are initialized by using Gaussian random values with a standard deviation of 0.01 and a mean of 0; no pre-trained model is used during training, no data other than the benchmark available images and labels is used, and the network starts training from scratch; the target value for training is represented by a sparse binary vector corresponding to the true category; for each training image, the target label vector has a length of the number of categories, and the value at the index position of the true value is 1 and the values at other positions are 0; the training is performed by using a stochastic gradient descent algorithm, wherein the batch size is 50, the initial learning rate is e-3, and the learning rate is reduced to e-4 after 10K iterations. 3.The clothing style matching algorithm based on facial features according to claim 2, characterized in that, ​ 4.The clothing style matching algorithm based on facial features according to claim 3, characterized in that, ​

Images