Full-process fashion shopping auxiliary system integrating virtual beauty makeup, fitting and intelligent customer service

Abstract

The invention discloses a full-process fashion shopping auxiliary system integrating virtual beauty makeup, fitting and intelligent customer service, and relates to the technical field of electronic commerce. Comprising an image acquisition module, a face generation module, a beauty makeup simulation module, a human body generation module, a fitting simulation module, an image fusion module, an intelligent auxiliary module, a customer service module and a display module. According to the invention, user selection time can be saved and user satisfaction can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of e-commerce, and in particular to a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service. Background Art

[0002] With the popularization of the Internet and e-commerce, consumers can choose clothes and cosmetics for themselves online without leaving home. However, the flat display of clothes and the inaccurate clothing size cannot let consumers see the effect of wearing them intuitively, so they may buy clothes that are not suitable for their body shape, which affects consumers' enthusiasm for buying clothes online. In the prior art, online shopping is carried out through virtual reality, and the goods on the shopping platform are presented to users in three-dimensional form through virtual reality equipment for reference. However, the body data of the model corresponding to the goods is not consistent with that of the customer, and the user still cannot experience the effect of trying on, so it is impossible to determine whether the selected goods are suitable. In addition, there are many brand names, many product types and similar products in the cosmetics market, and there is a lot of information about the types of cosmetics products and the information about them. According to the skin condition of the user, the suitable cosmetic color and makeup tone may also be different, so it is impossible to determine the cosmetic tone that the user wants.

[0003] Therefore, it is an urgent problem for those skilled in the art to propose a full-process fashion shopping assistance system that integrates virtual makeup, fitting and intelligent customer service to solve the problems existing in the prior art. Summary of the invention

[0004] In view of this, the present invention provides a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service, which can save users' selection time and improve user satisfaction.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service, including image acquisition module, face generation module, makeup simulation module, human body generation module, fitting simulation module, image fusion module, intelligent assistance module, customer service module and display module:

[0007] An image acquisition module, including acquiring a user's recent full-body image and generating a customer image;

[0008] A face generation module, connected to the first output terminal of the image acquisition module, is used to extract face information from the customer image and generate a three-dimensional face model;

[0009] A makeup simulation module, a first input end of which is connected to the face generation module, and is used to generate a three-dimensional model after makeup according to the cosmetics selected and set by the user;

[0010] A human body generation module is connected to the second output terminal of the image acquisition module and is used to extract and generate human body information and generate a three-dimensional human body model;

[0011] A fitting simulation module, the first input end of which is connected to the human body generation module, and is used to generate a fitting three-dimensional model according to the clothing selected by the user;

[0012] The input end of the image fusion module is connected to the output end of the beauty simulation module and the fitting simulation module to generate an overall selection image;

[0013] A display module, connected to the first output terminal of the image fusion module, for displaying the overall selected image for user viewing;

[0014] A customer service module, connected to the second output terminal of the image fusion module, for recording the commodity information in the overall selected image;

[0015] The intelligent auxiliary module is connected to the second input end of the beauty simulation module and the second input end of the fitting simulation module respectively, and is used to select cosmetics and clothing according to the keywords set by the user.

[0016] In the above system, optionally, the face generation module includes:

[0017] S1. Recognize the facial image in the customer image;

[0018] S2, extracting three-dimensional vertex matrix and posture information based on face image;

[0019] S3. Generate a three-dimensional face model based on the three-dimensional vertex matrix and posture information.

[0020] The above system optionally includes a makeup simulation module that identifies different parts of a 3D facial model and the colors of cosmetics in a shopping mall, generates a marked area for the makeup parts, displays the selected cosmetic color in the marked area after the customer selects the cosmetics, and generates a frontal image of the 3D facial model after makeup.

[0021] The above system optionally includes a human body generation module, which includes obtaining customer images and image feature information, using the image feature information to detect human body key points through a human body key point detection neural network, and reconstructing a human body three-dimensional model through the human body key points.

[0022] The above system may optionally include a fitting simulation module that includes determining body parts based on body key points, constructing a three-dimensional model of a standard body and clothing, and aligning the standard body model with the three-dimensional body model after the customer selects clothing, thereby generating a frontal image of the three-dimensional body model after fitting.

[0023] In the above system, the intelligent auxiliary module may optionally include: establishing makeup plans for different facial contours, comparing the similarity of the facial contours based on the three-dimensional face model, and making the plan with the highest similarity to the facial contours of the three-dimensional face model as the recommended plan;

[0024] Establish dressing plans for different body types, compare body shape similarities based on the human body 3D model, and recommend the plan with the highest body shape similarity to the human body 3D model.

[0025] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service, which has the following beneficial effects: 1) The present invention can display the makeup effect in real time through makeup simulation, simplify the makeup steps, and help customers determine whether the color of the cosmetics is appropriate; 2) The present invention can display the dressing effect of clothes through the fitting simulation module, and can reduce the amount of calculation based on the human body model replacement method; 3) The present invention can recommend cosmetics and clothes based on the user's physical characteristics, which can save the user's selection time and improve the user's satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0027] Figure 1 This is a block diagram of a full-process fashion shopping assistance system that integrates virtual makeup, fitting and intelligent customer service disclosed by the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0030] Reference Figure 1 As shown, the present invention discloses a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service, including an image acquisition module, a face generation module, a makeup simulation module, a human body generation module, a fitting simulation module, an image fusion module, an intelligent assistance module, a customer service module and a display module:

[0031] An image acquisition module, including acquiring a user's recent full-body image and generating a customer image;

[0032] A face generation module, connected to the first output terminal of the image acquisition module, is used to extract face information from the customer image and generate a three-dimensional face model;

[0033] A makeup simulation module, a first input end of which is connected to the face generation module, and is used to generate a three-dimensional model after makeup according to the cosmetics selected and set by the user;

[0034] A human body generation module is connected to the second output terminal of the image acquisition module and is used to extract and generate human body information and generate a three-dimensional human body model;

[0035] A fitting simulation module, the first input end of which is connected to the human body generation module, and is used to generate a fitting three-dimensional model according to the clothing selected by the user;

[0036] The input end of the image fusion module is connected to the output end of the beauty simulation module and the fitting simulation module to generate an overall selection image;

[0037] A display module, connected to the first output terminal of the image fusion module, for displaying the overall selected image for user viewing;

[0038] A customer service module, connected to the second output terminal of the image fusion module, for recording the commodity information in the overall selected image;

[0039] The intelligent auxiliary module is connected to the second input end of the beauty simulation module and the second input end of the fitting simulation module respectively, and is used to select cosmetics and clothing according to the keywords set by the user.

[0040] Furthermore, the image acquisition step module: pre-processes the acquired image, and divides the user image into a human body image and the background normal image.

[0041] Furthermore, the face generation module includes:

[0042] S1. Recognize the facial image in the customer image;

[0043] S2, extracting three-dimensional vertex matrix and posture information based on face image;

[0044] S3. Generate a three-dimensional face model based on the three-dimensional vertex matrix and posture information.

[0045] Specifically, the three-dimensional vertex matrix and posture information in the original side face image are extracted through the 3DDFA-V2 algorithm, and each vertex in the three-dimensional image is mapped to the two-dimensional space through orthogonal projection, so as to obtain the texture features of each vertex in the face image, and generate a face image without shadows. The illumination information in the face image is used as the first illumination information, and the illumination information of the face image without shadows is used as the second illumination information. Based on the first illumination information and the second illumination information, an illumination loss value is generated, and a three-dimensional face model is generated with the goal of minimizing the illumination loss value.

[0046] Furthermore, the beauty simulation module includes: identifying different parts of the three-dimensional face model and the colors of cosmetics in the mall, generating marked areas for the makeup parts, displaying the selected cosmetic colors in the marked areas after the customer selects cosmetics, and generating a frontal image of the three-dimensional face model after makeup.

[0047] Specifically, the marked area is divided into i sub-areas, and the makeup process is divided into k steps. The customer selection area is marked as a i，k , fill the cosmetics into the corresponding area, record the marked area after the corresponding cosmetic color, such as filling area 3 with red cosmetics in the second step, then record it as red-a 3，2 .

[0048] Furthermore, the human body generation module includes obtaining customer images and image feature information, using the image feature information, detecting human body key points through a human body key point detection neural network, and reconstructing a human body three-dimensional model through the human body key points.

[0049] Specifically, the pixels of the customer image are adjusted to become an image of 960*512 pixels. After passing through two convolution layers, one maximum pooling layer, two convolution layers, one maximum pooling layer, two convolution layers, one maximum pooling layer, two convolution layers, one maximum pooling layer, two convolution layers, one maximum pooling layer, two convolution layers, one maximum pooling layer, and two fully connected layers, a feature vector is obtained as image feature information. Based on the image feature information, key points of the human body are determined, including the chin, left shoulder, left elbow, left wrist, right shoulder, right elbow, right wrist, left arm, left knee, left ankle, right arm, right knee and right ankle. The three-dimensional human body model is reconstructed in combination with the pre-stored three-dimensional human body model.

[0050] Furthermore, the fitting simulation module includes judging the body parts based on the key points of the body, constructing a three-dimensional model of the clothes including a standard human body and clothes, and after the customer selects the clothes, making the standard human body model consistent with the three-dimensional human body model, and generating a frontal image of the three-dimensional human body model after fitting.

[0051] Furthermore, the image fusion module includes obtaining a front image of a 3D human body model after fitting and a front image of a 3D face model after makeup, and splicing the images using the chin as a connecting part, based on the similarity between image features.

[0052] Furthermore, the intelligent auxiliary module includes: establishing makeup plans for different facial contours, comparing the similarity of the facial contours based on the three-dimensional face model, and making the plan with the highest similarity to the facial contours of the three-dimensional face model as the recommended plan;

[0053] Establish dressing plans for different body types, compare body shape similarities based on the human body 3D model, and recommend the plan with the highest body shape similarity to the human body 3D model.

[0054] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0055] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service, characterized in that: It includes image acquisition module, face generation module, beauty simulation module, human body generation module, fitting simulation module, image fusion module, intelligent assistance module, customer service module and display module: An image acquisition module, including acquiring a user's recent full-body image and generating a customer image; A face generation module, connected to the first output terminal of the image acquisition module, is used to extract face information from the customer image and generate a three-dimensional face model; A makeup simulation module, a first input end of which is connected to the face generation module, and is used to generate a three-dimensional model after makeup according to the cosmetics selected and set by the user; A human body generation module is connected to the second output terminal of the image acquisition module and is used to extract and generate human body information and generate a three-dimensional human body model; A fitting simulation module, the first input end of which is connected to the human body generation module, and is used to generate a fitting three-dimensional model according to the clothing selected by the user; The input end of the image fusion module is connected to the output end of the beauty simulation module and the fitting simulation module to generate an overall selection image; A display module, connected to the output end of the image fusion module, for displaying the overall selected image for user viewing; A display module, connected to the first output terminal of the image fusion module, for displaying the overall selected image for user viewing; A customer service module, connected to the second output terminal of the image fusion module, for recording the commodity information in the overall selected image; The intelligent auxiliary module is connected to the second input end of the beauty simulation module and the second input end of the fitting simulation module respectively, and is used to select cosmetics and clothing according to the keywords set by the user.

2. According to claim 1, a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service is characterized in that: The face generation module includes: S1. Recognize the facial image in the customer image; S2, extracting three-dimensional vertex matrix and posture information based on face image; S3. Generate a three-dimensional face model based on the three-dimensional vertex matrix and posture information.

3. According to claim 1, a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service is characterized in that: The beauty simulation module includes: identifying different parts of the 3D facial model and the colors of cosmetics in the mall, generating marked areas for the makeup parts, and after the customer selects the cosmetics, the marked area displays the selected cosmetic color, and generates a frontal image of the 3D facial model after makeup.

4. According to claim 1, a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service is characterized in that: The human body generation module includes obtaining customer images and image feature information, using the image feature information, detecting human body key points through a human body key point detection neural network, and reconstructing a human body three-dimensional model through the human body key points.

5. According to claim 1, a full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service is characterized in that: The fitting simulation module includes judging the body parts based on the key points of the body, constructing a three-dimensional model of the clothes including the standard human body and the clothes, and after the customer selects the clothes, making the standard human body model consistent with the three-dimensional human body model, and generating a frontal image of the three-dimensional human body model after fitting.

6. The full-process fashion shopping assistance system integrating virtual makeup, fitting and intelligent customer service according to claim 1, characterized in that: The intelligent auxiliary module includes: establishing makeup plans for different facial contours, comparing the similarity of facial contours based on the 3D face model, and recommending the plan with the highest similarity to the facial contours of the 3D face model; Establish dressing plans for different body types, compare body shape similarities based on the human body 3D model, and recommend the plan with the highest body shape similarity to the human body 3D model.

Images