A lipstick shopping guide method and device using a smart mirror

Abstract

The present invention discloses a lipstick shopping guide method for a smart mirror, the method comprising the following steps: obtaining a face image of a user when looking at a smart mirror; performing face recognition on the face image to obtain the user's feature data; calculating the recommendation degree of each lipstick in the lipstick product library to the user according to the user's feature data and the preset feature data of each lipstick in the lipstick product library; and displaying all lipsticks whose recommendation degree ranking is before a first threshold on the smart mirror for the user to select. Based on the advantages of the smart mirror itself, the user can obtain the user's face image when the user uses the smart mirror, and according to the recognition result of the user's face image, the user can be pushed suitable lipstick products in a targeted manner, which can promote the sales of the products and bring business opportunities. The present invention also proposes a shopping guide device for a smart mirror, which applies the above method and can have the advantages brought by the above method.

Description

Technical Field

[0001] The present disclosure relates to the technical field of smart mirrors, and in particular to a lipstick shopping guide method and device for a smart mirror. Background Art

[0002] Smart mirrors allow people to check news or do other interesting things while using the toilet or taking a shower, making full use of this time.

[0003] Smart mirrors in today's market are still in their infancy. Most manufacturers of smart mirrors have only integrated some classic functions and some innovative functions derived from the advantages of the smart mirrors themselves. For example, since users will inevitably see their own facial images when using smart mirrors, some women will naturally consider the choice of lipstick. If suitable lipstick products can be pushed to them intelligently, it will greatly promote the sales of lipstick. Summary of the invention

[0004] The present disclosure aims to solve at least one of the above problems, and provides a lipstick shopping guide method and device for a smart mirror.

[0005] In order to achieve the above object, according to one aspect of the present disclosure, a lipstick shopping guide method of a smart mirror is provided, the method comprising the following steps:

[0006] Obtain the face image of the user when looking at the smart mirror;

[0007] Performing face recognition on the face image to obtain feature data of the user;

[0008] Calculate the recommendation degree of each lipstick in the lipstick product library to the user based on the user's feature data and the preset feature data of each lipstick in the lipstick product library;

[0009] All lipsticks whose recommendation rankings are before the first threshold are displayed on the smart mirror for the user to select.

[0010] Furthermore, the above-mentioned face recognition of the face image to obtain the user's characteristic data specifically includes the following:

[0011] Extracting feature data from the face image to obtain relevant feature data of the user, and displaying an area of ​​interest of the face image of the user corresponding to each relevant feature data on the smart mirror;

[0012] Acquire the region of interest selected by the user on the smart mirror, and use the corresponding relevant feature data as the final feature data of the user;

[0013] Normalizing the final feature data by using a deep convolutional neural network to obtain normalized feature data;

[0014] The normalized feature data is used as the feature data of the user.

[0015] Furthermore, the lipstick shopping guide method further comprises:

[0016] The types of the user's feature data include one or more combinations of corresponding types of data of the user's eye contour data, the user's face contour data, the user's face skin color data, the user's hairstyle contour data, and the user's hair color data normalized by a deep convolutional neural network;

[0017] The corresponding preset feature data for each lipstick type is the corresponding type of data after the above data is normalized by the same deep convolutional neural network. If each type of data above is preset, the preset data is taken; if not, the data is left blank.

[0018] Furthermore, the method for calculating the recommendation degree of each lipstick in the lipstick product library to the user based on the user's characteristic data and the preset characteristic data of each lipstick in the lipstick product library comprises the following steps:

[0019] Define the user's feature data as S, and the preset feature data of each lipstick in the lipstick product library is S i , i = 1…n, n is the number of feature data preset for each lipstick in the lipstick product library, then through S and S i The feature distance between S and S is used to calculate the recommendation degree of lipstick to the user. i The recommendation degree between them is defined as: dist(S,S i )=1-Sim(S,S i ), where Sim(S,S i ) indicates S and S i The similarity of Among them, the characteristic similarity value w ij The value of is calculated as follows: let the feature type of S be feature, S i The feature type is feature j ,freq i,j Indicates feature type feature j In S i The weight in , j represents feature j The number of times each lipstick appears in the preset feature data is the feature similarity value w ij The value of is,

[0020]

[0021] Among them, m i,j is the expected weight value, which is determined by the following formula:

[0022]

[0023] Among them: freq is the weight of feature type feature in S, freq i,j Indicates feature type feature j In S i The weight in

[0024] Among them, Sim(S,S i )∈[0,1], so S and S i The similarity range is between [0,1], and S and S i The smaller the similarity, the smaller the dist(S,S i )=1-Sim(S,S i ) It can be seen that S and S i The greater the recommendation degree between them, the more recommended the lipstick is to users.

[0025] Furthermore, the method also includes, while pushing lipsticks to the user through the smart mirror, being able to use the lipstick brand selected by the user as a screening condition, and ultimately displaying lipsticks whose recommendation ranking is before a first threshold and that meet the lipstick brand selected by the user on the smart mirror for the user to select.

[0026] Furthermore, the lipstick product library also includes relevant detail page information of the type of lipstick, and when the lipstick is selected for push, it will be displayed in the smart mirror.

[0027] Furthermore, the method also includes that all users who have purchased the smart mirror will be regarded as a sub-node, and at the end of each month, the sales ranking of each type of lipstick in all sub-nodes will be counted, and a lipstick sales popularity list will be formed, which will be displayed on the smart mirror of each sub-node in the next month.

[0028] The present invention also provides a shopping guide device of a smart mirror, the device comprising:

[0029] A face image acquisition module is used to acquire a face image of a user when looking at the smart mirror;

[0030] A feature data extraction module performs face recognition on the face image to obtain feature data of the user;

[0031] A recommendation degree determination module calculates the recommendation degree of each lipstick in the lipstick product library to the user based on the user's feature data and the preset feature data of each lipstick in the lipstick product library;

[0032] The display product determination module displays all lipsticks whose recommendation rankings are before a first threshold on the smart mirror for users to select.

[0033] The present invention also proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a lipstick shopping guide method for a smart mirror as described in any one of claims 1-7 are implemented.

[0034] The beneficial effects of the present disclosure are as follows: the present invention provides a lipstick shopping guide method and device for a smart mirror, which can obtain the user's facial image when the user uses the smart mirror based on the advantages of the smart mirror itself, and push suitable lipstick products to the user based on the recognition result of the user's facial image, which can promote the sales of products and bring business opportunities. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0036] Figure 1 Shown is a flow chart of a lipstick shopping guide method using a smart mirror;

[0037] Figure 2 Shown is a diagram of a shopping guide device of a smart mirror. DETAILED DESCRIPTION

[0038] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present disclosure, so as to fully understand the purpose, scheme and effect of the present disclosure. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0039] like Figure 1 The flowchart of the lipstick shopping guide method of the smart mirror according to the present disclosure is shown below. Figure 1 To illustrate a lipstick shopping guide method of a smart mirror according to an embodiment of the present disclosure.

[0040] Embodiment 1, the present disclosure provides a lipstick shopping guide method using a smart mirror, the method comprising the following steps:

[0041] Step 110: Obtain a facial image of the user when looking at the smart mirror;

[0042] Step 120: Perform face recognition on the face image to obtain user feature data;

[0043] Step 130: Calculate the recommendation degree of each lipstick in the lipstick product library to the user based on the user's feature data and the preset feature data of each lipstick in the lipstick product library;

[0044] Step 140: Display all lipsticks whose recommendation rankings are before the first threshold on the smart mirror for the user to select.

[0045] In this embodiment 1, based on the advantages of the smart mirror itself, the user's facial image can be obtained when the user uses the smart mirror, and suitable lipstick products can be pushed to the user based on the recognition result of the user's facial image, which can promote the sales of the products and bring business opportunities.

[0046] As a preferred embodiment of the present invention, the above-mentioned face recognition of the face image to obtain the user's feature data specifically includes the following:

[0047] Extracting feature data from the face image to obtain relevant feature data of the user, and displaying an area of ​​interest of the face image of the user corresponding to each relevant feature data on the smart mirror;

[0048] Acquire the region of interest selected by the user on the smart mirror, and use the corresponding relevant feature data as the final feature data of the user;

[0049] Normalizing the final feature data by using a deep convolutional neural network to obtain normalized feature data;

[0050] The normalized feature data is used as the feature data of the user.

[0051] As a preferred embodiment of the present invention, the lipstick shopping guide method further comprises:

[0052] The types of the user's feature data include one or more combinations of corresponding types of data of the user's eye contour data, the user's face contour data, the user's face skin color data, the user's hairstyle contour data, and the user's hair color data normalized by a deep convolutional neural network;

[0053] The corresponding preset feature data for each lipstick type is the corresponding type of data after the above data is normalized by the same deep convolutional neural network. If each type of data above is preset, the preset data is taken; if not, the data is left blank.

[0054] As a preferred embodiment of the present invention, the method for calculating the recommendation degree of each lipstick in the lipstick product library to the user based on the user's feature data and the preset feature data of each lipstick in the lipstick product library comprises the following steps:

[0055] Define the user's feature data as S, and the preset feature data of each lipstick in the lipstick product library is S i , i = 1…n, n is the number of feature data preset for each lipstick in the lipstick product library, then through S and S i The feature distance between S and S is used to calculate the recommendation degree of lipstick to the user. i The recommendation degree between them is defined as: dist(S,S i )=1-Sim(S,S i ), where Sim(S,S i ) indicates S and S i The similarity of Among them, the characteristic similarity value w ij The value of is calculated as follows: let the feature type of S be feature, S i The feature type is feature j ,freq i,j Indicates feature type feature j In S i The weight in , j represents feature j The number of times each lipstick appears in the preset feature data is the feature similarity value w ij The value of is,

[0056]

[0057] Among them, m i,j is the expected weight value, which is determined by the following formula:

[0058]

[0059] Among them: freq is the weight of feature type feature in S, freq i,j Indicates feature type feature j In S i The weight in

[0060] Among them, Sim(S,S i )∈[0,1], so S and S i The similarity range is between [0,1], and S and S i The smaller the similarity, the smaller the dist(S,S i )=1-Sim(S,S i ) It can be seen that S and Si The greater the recommendation degree between them, the more recommended the lipstick is to users.

[0061] As a preferred embodiment of the present invention, the method also includes, while pushing lipstick to the user through the smart mirror, being able to use the lipstick brand selected by the user as a screening condition, and finally displaying the lipsticks whose recommendation ranking is before a first threshold and that meet the lipstick brand selected by the user on the smart mirror for the user to select.

[0062] As a preferred embodiment of the present invention, the lipstick product library also includes relevant detail page information of the type of lipstick, and when the lipstick is selected for push, it will be displayed in the smart mirror.

[0063] As a preferred embodiment of the present invention, the method also includes that all users who have purchased the smart mirror will be regarded as a sub-node, and at the end of each month, the sales ranking of each type of lipstick in all sub-nodes will be counted, and a lipstick sales popularity list will be formed, which will be displayed on the smart mirror of each sub-node in the next month.

[0064] Embodiment 2, the present invention further proposes a shopping guide device of a smart mirror, the device comprising:

[0065] The face image acquisition module 100 is used to acquire the face image of the user when looking at the smart mirror;

[0066] The feature data extraction module 200 performs face recognition on the face image to obtain feature data of the user;

[0067] The recommendation degree determination module 300 calculates the recommendation degree of each lipstick in the lipstick product library to the user based on the user's feature data and the preset feature data of each lipstick in the lipstick product library;

[0068] The display product determination module 400 displays all lipsticks whose recommendation rankings are before a first threshold on the smart mirror for the user to select.

[0069] In this embodiment 2, based on the advantages of the smart mirror itself, the user's facial image can be obtained when the user uses the smart mirror, and suitable lipstick products can be pushed to the user based on the recognition result of the user's facial image, which can promote the sales of the products and bring business opportunities.

[0070] The present invention also proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a lipstick shopping guide method for a smart mirror as described in any one of claims 1-7 are implemented.

[0071] The shopping guide device of the smart mirror can be run in computing devices such as desktop computers, notebooks, PDAs, and cloud servers. The shopping guide device of the smart mirror can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the example is only an example of a shopping guide device of a smart mirror, and does not constitute a limitation on a shopping guide device of a smart mirror. It can include more or less components than the example, or combine certain components, or different components. For example, the shopping guide device of the smart mirror can also include input and output devices, network access devices, buses, etc.

[0072] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the shopping guide device operating device of the smart mirror, and uses various interfaces and lines to connect the various parts of the entire shopping guide device operating device of the smart mirror.

[0073] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the shopping guide device of the smart mirror by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0074] Although the description of the present disclosure has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims by reference to the appended claims in view of the prior art, thereby effectively covering the intended scope of the present disclosure. In addition, the above description of the present disclosure is based on the embodiments foreseeable by the inventors, and its purpose is to provide a useful description, and those non-substantial changes to the present disclosure that have not yet been foreseen may still represent equivalent changes to the present disclosure.

Claims

1. A lipstick shopping guide method using a smart mirror, characterized in that: The method comprises the following steps: Obtain the face image of the user when looking at the smart mirror; Performing face recognition on the face image to obtain feature data of the user; Calculate the recommendation degree of each lipstick in the lipstick product library to the user based on the user's feature data and the preset feature data of each lipstick in the lipstick product library; Displaying all lipsticks whose recommendation rankings are before a first threshold on the smart mirror for selection by the user; The lipstick shopping guide method further comprises: The types of the user's feature data include one or more combinations of corresponding types of data of the user's eye contour data, the user's face contour data, the user's face skin color data, the user's hairstyle contour data, and the user's hair color data normalized by a deep convolutional neural network; The corresponding preset feature data for each lipstick type is the corresponding type of data after the above data is normalized by the same deep convolutional neural network. If each type of data above is preset, the preset data is taken; if not, the data is left blank; The method for calculating the recommendation degree of each lipstick in the lipstick product library to the user based on the user's characteristic data and the preset characteristic data of each lipstick in the lipstick product library comprises the following steps: Define the user's feature data as S, and the preset feature data of each lipstick in the lipstick product library is S i , i = 1…n, n is the number of feature data preset for each lipstick in the lipstick product library, then through S and S i The feature distance between S and S is used to calculate the recommendation degree of lipstick to the user. i The recommendation degree between is defined as: dist(S,S i )=1-Sim(S,S i ), where Sim(S,S i ) indicates S and S i The similarity of Among them, the characteristic similarity value w ij The value of is calculated as follows: let the feature type of S be feature, S i The feature type is feature j ,freq i,j Indicates feature type feature j In S i The weight in , j represents feature j The number of times each lipstick appears in the preset feature data is the feature similarity value w ij The value of is, Among them, m i,j is the expected weight value, which is determined by the following formula: Among them: freq is the weight of feature type feature in S, freq i,j Indicates feature type feature j In S i The weight in Among them, Sim(S,S i )∈[0,1], so S and S i The similarity range is between [0,1], and S and S i The smaller the similarity, the smaller the dist(S,S i )=1-Sim(S,S i ) It can be seen that S and S i The greater the recommendation degree between , the higher the recommendation degree of the lipstick to users; The above-mentioned face recognition of the face image to obtain the user's characteristic data specifically includes the following: Extracting feature data from the face image to obtain relevant feature data of the user, and displaying an area of ​​interest of the face image of the user corresponding to each relevant feature data on the smart mirror; Acquire the region of interest selected by the user on the smart mirror, and use the corresponding relevant feature data as the final feature data of the user; Normalizing the final feature data by using a deep convolutional neural network to obtain normalized feature data; The normalized feature data is used as the feature data of the user.

2. The lipstick shopping guide method of a smart mirror according to claim 1, characterized in that: The method also includes, while pushing lipsticks to the user through the smart mirror, being able to use the lipstick brand selected by the user as a screening condition, and finally displaying lipsticks whose recommendation ranking is before a first threshold and that meet the lipstick brand selected by the user on the smart mirror for the user to select.

3. The lipstick shopping guide method of a smart mirror according to claim 1, characterized in that: The lipstick product library also includes relevant detail page information of the type of lipstick, and when the lipstick is selected for push, it will be displayed in the smart mirror.

4. The lipstick shopping guide method of a smart mirror according to claim 1, characterized in that: The method also includes that all users who have purchased the smart mirror will be regarded as a sub-node, and at the end of each month, the sales ranking of each type of lipstick in all sub-nodes will be counted, and a lipstick sales popularity list will be formed, which will be displayed on the smart mirror of each sub-node in the next month.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a lipstick shopping guide method for a smart mirror as described in any one of claims 1-4 are implemented.

Images