Method, device and equipment for simulating makeup effect of base makeup product

The makeup effect prediction model established by the partial least squares method combines individual type angles and hue angles to solve the problem of inaccurate prediction of skin tone after makeup and unrealized simulation of makeup effect in the existing technology, achieving a more accurate and real Makeup effect simulation.

CN119068533AInactive Publication Date: 2024-12-03BEIJING TECH & BUSINESS UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411287256.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, when the facial makeup effect is displayed in real time through augmented reality technology, the skin color prediction after makeup is inaccurate and the makeup effect simulation is not real.

Method used

The makeup effect prediction model is established using the partial least squares method, and it is constructed based on individual type angles and hue angles. By obtaining the user's face image and the identification of the target makeup product, the skin color characteristics of the facial area are determined, and the color of the makeup product is combined with the makeup product. Features are superimposed and Gaussian fuzzy is processed to improve accuracy and authenticity.

Benefits of technology

It improves the accuracy and stability of skin tone prediction after makeup, provides makeup effects that are more in line with the real-world visual experience, and overcomes subjectivity and errors in traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068533A_ABST
    Figure CN119068533A_ABST
Patent Text Reader

Abstract

The invention provides a makeup effect simulation method, device and equipment for a base makeup product, and relates to the technical field of image processing, and the method comprises the steps: obtaining a face image of a user and an identifier of a target base makeup product selected by the user; based on a preset face division rule, determining a plurality of face regions, and determining skin color features of each face region; obtaining color features of the target base product from a pre-established base product information database based on the identifier of the target base product; based on the skin color features of all the face areas and the color features of the target base makeup product, superposition is carried out through a makeup effect prediction model established through a pre-established partial least square method, the simulated makeup effect is obtained, and the makeup effect prediction model is established based on an individual type angle and a hue angle. According to the method, the effect of the base makeup product can be simulated, the accuracy of skin color prediction is improved through image processing and a partial least square method, and a more vivid makeup effect is simulated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and equipment for simulating the makeup effect of a base makeup product. Background Art

[0002] As machine learning begins to be widely used in practical problems, computer equipment is used to accurately extract the skin color characteristics of the crowd and develop a foundation color prediction system to simulate realistic makeup effects. Currently, there are methods that use augmented reality (AR) technology to display facial makeup effects in real time, but such methods have problems such as inaccurate prediction of skin color after makeup and unrealistic simulation of makeup effects. Summary of the invention

[0003] The purpose of the present application is to provide a method, device and equipment for simulating the makeup effect of a base makeup product, so as to alleviate the above-mentioned technical problems existing in the prior art.

[0004] In a first aspect, the present application provides a method for simulating the makeup effect of a base makeup product. The method comprises:

[0005] Obtaining a user's facial image and an identifier of a target makeup product selected by the user;

[0006] Based on a preset facial segmentation rule, a plurality of facial regions are determined, and skin color characteristics of each facial region are determined;

[0007] Acquiring the color characteristics of the target base makeup product from a pre-established base makeup product information database;

[0008] Based on the skin color characteristics of each facial area and the color characteristics of the target foundation product, a makeup effect prediction model established by a pre-established partial least squares method is superimposed to obtain a simulated makeup effect. The makeup effect prediction model is constructed based on individual type angles and hue angles.

[0009] In some optional implementations, a makeup effect prediction model is established by partial least squares method, including:

[0010] Constructing a first variable matrix based on makeup influencing factors; wherein the makeup influencing factors include skin color brightness, skin color redness, and skin color yellowness before makeup;

[0011] The second variable matrix is ​​determined based on the after-makeup individual type angle ITA value and the after-makeup hue angle HAB;

[0012] Sequentially extracting the target component pairs corresponding to the first variable matrix and the second variable matrix; wherein the target component pairs are multiple groups;

[0013] Calculating the score vector of the target component pair, and calculating the correlation degree of the target component pair based on the inner product of the score vector; wherein the correlation degree satisfies a preset correlation threshold;

[0014] Establishing a regression model of the first variable and the second variable relative to the score vector, and determining the regression coefficient;

[0015] The least squares estimation is constructed based on the least squares estimation of the regression coefficient, and a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB is obtained.

[0016] In some optional implementations, a partial least squares equation is constructed based on the least squares estimate of the regression coefficient to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB, including:

[0017] The partial least squares equation constructed based on the least squares estimate of the regression coefficient is:

[0018]

[0019] Calculations are performed one by one based on the residual matrix until the absolute value of the elements in the residual matrix is ​​approximately 0, and the partial least squares equations related to multiple components are determined to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB.

[0020] In some optional implementations, determining skin color characteristics of each facial region includes:

[0021] Convert the RGB color space corresponding to the image of the facial area into the corresponding XYZ value in the CIEXYZ color space by a pre-selected matrix multiplication;

[0022] Divide the XYZ value by the standard value of the reference white point, calculate the values ​​of L, a and b to extract the skin color, calculate the individual type angle ITA according to ITA°={arctan[(L-50) / b]}*180 / π, and calculate the hue angle HAB° according to HAB°=arctan(b / a)*180 / π.

[0023] In some optional implementations, multiple facial regions are determined based on a preset facial segmentation rule, including:

[0024] The symmetrical area of ​​the cheek is taken as the first area, the T zone of the face is taken as the second area, the area around the eyes and lips is taken as the third area, and the rest is taken as the fourth area.

[0025] In some optional implementations, the method further includes:

[0026] The predicted simulated makeup effect is applied to each area of ​​the face region, and an overall Gaussian blur is performed based on a pre-selected Gaussian kernel.

[0027] In a second aspect, a makeup effect simulation device for a base makeup product is provided. The device comprises:

[0028] The first acquisition module is used to acquire the user's face image and the identifier of the target makeup product selected by the user;

[0029] A determination module, used to determine a plurality of facial regions based on a preset facial segmentation rule, and to determine a skin color feature of each facial region;

[0030] A second acquisition module, configured to acquire the color characteristics of the target base makeup product from a pre-established base makeup product information database based on the identifier of the target base makeup product;

[0031] The simulation module is used to obtain a simulated makeup effect by superimposing a makeup effect prediction model established by a pre-established partial least squares method based on the skin color characteristics of each facial area and the color characteristics of the target foundation product. The makeup effect prediction model is constructed based on individual type angles and hue angles.

[0032] In some optional implementations, the apparatus further includes an establishing module, specifically configured to:

[0033] Constructing a first variable matrix based on makeup influencing factors; wherein the makeup influencing factors include skin color brightness, skin color redness, and skin color yellowness before makeup;

[0034] Determine the second variable matrix based on the after-makeup individual type angle ITA value and the after-makeup hue angle HAB value;

[0035] Sequentially extracting the target component pairs corresponding to the first variable matrix and the second variable matrix; wherein the target component pairs are multiple groups;

[0036] Calculating the score vector of the target component pair, and calculating the correlation degree of the target component pair based on the inner product of the score vector; wherein the correlation degree satisfies a preset correlation threshold;

[0037] Establishing a regression model of the first variable and the second variable relative to the score vector, and determining the regression coefficient;

[0038] The least squares estimation is constructed based on the least squares estimation of the regression coefficient, and a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB is obtained.

[0039] In some optional implementations, the establishing module is specifically used to:

[0040] The partial least squares equation constructed based on the least squares estimate of the regression coefficient is:

[0041]

[0042] Calculations are performed one by one based on the residual matrix until the absolute value of the elements in the residual matrix is ​​approximately 0, and the partial least squares equations related to multiple components are determined to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB.

[0043] In some optional implementations, the second acquisition module is used to:

[0044] Convert the RGB color space corresponding to the image of the facial area into the corresponding XYZ value in the CIEXYZ color space by a pre-selected matrix multiplication;

[0045] Divide the XYZ value by the standard value of the reference white point, calculate the values ​​of L, a and b to extract the skin color, calculate the individual type angle ITA° according to ITA°={arctan[(L-50) / b]}*180 / π, and calculate the hue angle HAB° according to HAB°=arctan(b / a)*180 / π.

[0046] In some optional implementations, the determination module is used to:

[0047] The symmetrical area of ​​the cheek is taken as the first area, the T zone of the face is taken as the second area, the area around the eyes and lips is taken as the third area, and the rest is taken as the fourth area.

[0048] In some optional implementations, the method further includes:

[0049] The blur module is used to apply the predicted simulated makeup effect to various areas of the face area and perform overall Gaussian blur based on a pre-selected Gaussian kernel.

[0050] In a third aspect, the present application provides an electronic device including a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method for simulating the makeup effect of a foundation product according to any of the aforementioned embodiments.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method for simulating the makeup effect of a foundation product of any of the aforementioned implementations.

[0052] The beneficial effects of the makeup effect simulation method, device and equipment provided by the present application are as follows:

[0053] By obtaining the user's face image and the identification of the target makeup product selected by the user; based on the preset face division rules, multiple facial areas are determined, and the skin color characteristics of each facial area are determined; based on the identification of the target makeup product, the color characteristics of the target makeup product are obtained from a pre-established makeup product information database; based on the skin color characteristics of each facial area and the color characteristics of the target makeup product, a makeup effect prediction model established by a pre-established partial least squares method is superimposed to obtain a simulated makeup effect, and the makeup effect prediction model is constructed based on individual type angles and hue angles. The present application can simulate the effect of makeup products, and through image processing and partial least squares, the accuracy of skin color prediction is improved, and a more realistic makeup effect is simulated. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A flowchart of a method for simulating the makeup effect of a base makeup product provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of facial area division provided in an embodiment of the present application;

[0057] Figure 3 A flowchart of a specific method for simulating the makeup effect of a base makeup product provided in an embodiment of the present application;

[0058] Figure 4 An interactive connection relationship between execution subject devices provided in an embodiment of the present application;

[0059] Figure 5 A structural diagram of a makeup effect simulation device for a base makeup product provided in an embodiment of the present application;

[0060] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0063] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0064] Liquid foundation is a cosmetic that is usually used on the facial skin to create a uniform base to cover skin blemishes, unify skin tone, and lay the foundation for subsequent makeup. As an important category of cosmetics, the selection and use of liquid foundation is crucial to shaping a personal image. Color classification is usually designed based on the different characteristics of skin color to ensure a natural and uniform makeup. People choose the color and texture of liquid foundation according to their individual skin color, skin quality and needs, and make appropriate adjustments to achieve the best makeup effect. However, as people's demand for beauty and makeup becomes more refined, consumers' demand for cosmetics has also shown a personalized and diversified trend. However, traditional selection methods often rely on manual color testing, subjective calculations and judgments, which have certain subjectivity and errors, making it difficult to meet personalized needs or resulting in poor makeup effects.

[0065] As an important branch of artificial intelligence, machine learning has made significant progress in recent years. From the introduction of the basic concept of neural networks in the 1940s to today's improvement in computing power, machine learning has been widely used in practical problems such as image recognition and natural language processing. In the cosmetics industry, machine learning models are used to perform high-precision recognition and extraction of skin color features.

[0066] Therefore, accurately extracting the skin color characteristics of the crowd, studying the foundation color prediction system, and simulating realistic makeup effects not only improves the accuracy of skin color recognition, but also simulates realistic makeup effects of different skin colors.

[0067] Some existing mobile phone software, such as beauty cameras, can already display facial makeup effects in real time through augmented reality (AR) technology. However, due to the differences in skin color in different facial areas, the makeup effects simulated by current augmented reality technology also have problems such as unrealistic images and inaccurate angles.

[0068] Based on this, the embodiments of the present application provide a method, device and equipment for simulating the makeup effect of a foundation product, which can improve the accuracy and stability of skin color prediction after makeup, and provide a makeup effect that is more in line with the real-world visual experience.

[0069] The present application embodiment provides a method for simulating the makeup effect of a base makeup product. Figure 1 As shown, the method mainly includes the following steps:

[0070] Step S110, obtaining a user's facial image and an identifier of a target base makeup product selected by the user.

[0071] As an example, the user scans the target foundation product through a barcode scanning device, which can scan the barcode of the foundation product that the user wants to purchase, obtain the product number of the foundation product that the user wants to purchase, and trigger the camera device to take a photo after scanning the barcode to obtain the facial image of the current user.

[0072] As another example, the user can trigger the makeup effect simulation function of the base makeup product through the control interface provided by the terminal. The user can enter the product number of the base makeup product that the user wants to buy through the control interface, or trigger the scanning module through the control interface, and the scanning device can scan the barcode of the base makeup product that the user wants to buy through the scanning device; the local face image can also be obtained through the control interface, or the camera device can be called to take a photo to obtain the face image of the current user.

[0073] Step S120, based on a preset facial segmentation rule, determining a plurality of facial regions, and determining a skin color feature of each facial region.

[0074] The preset face segmentation rules may include multiple ones. As an example, see Figure 2 As shown, the face feature point model can be used to perform face detection on the face image, locate the key points, and intercept the symmetrical area of ​​the cheek as the first area, the facial T zone (i.e., forehead and chin) as the second area, the area around the eyes and lips as the third area, and the rest as the fourth area.

[0075] In one implementation, determining the skin color characteristics of each facial region may include the following steps 1-1 to 1-3:

[0076] Step 1-1, converting the RGB color space corresponding to the image of the facial area into the corresponding XYZ value in the CIE XYZ color space by a pre-selected matrix multiplication;

[0077] Step 1-2, divide the XYZ value by the standard value of the reference white point, calculate the values ​​of L, a and b to extract the skin color, and calculate the individual type angle ITA according to ITA°={arctan[(L-50) / b]}*180 / π;

[0078] Step 1-3, calculate the hue angle HAB° according to HAB°=arctan(b / a)*180 / π.

[0079] Step S130 , obtaining the color characteristics of the target base makeup product from a pre-established base makeup product information database based on the identifier of the target base makeup product.

[0080] Collect various liquid foundation products and record the color information of each product, including natural, fair and dark tones, and establish a base makeup product information database.

[0081] Step S140, based on the skin color characteristics of each facial area and the color characteristics of the target base makeup product, a makeup effect prediction model established by a pre-established partial least squares method is superimposed to obtain a simulated makeup effect, wherein the makeup effect prediction model is constructed based on the individual type angle and the hue angle. In practical applications, the individual type angle ITA is used to characterize the degree of skin whitening, and the hue angle HAB is used to characterize the degree of skin redness and yellowness.

[0082] In one embodiment, the makeup effect prediction model established by partial least squares method may include the following steps 2-1 to 2-6 during implementation:

[0083] Step 2-1, constructing a first variable matrix based on makeup influencing factors; wherein the makeup influencing factors include skin color brightness, skin color redness, and skin color yellowness before makeup.

[0084] Step 2-2, determining a second variable matrix using the after-makeup individual type angle ITA value and the after-makeup hue angle HAB value.

[0085] Step 2-3, sequentially extracting the target component pairs corresponding to the first variable matrix and the second variable matrix; wherein the target component pairs are multiple groups;

[0086] Step 2-4, calculating the score vector of the target component pair, and calculating the correlation degree of the target component pair based on the inner product of the score vector; wherein the correlation degree satisfies a preset correlation threshold;

[0087] Step 2-5, establishing a regression model of the relative score vector of the first variable and the second variable, and determining the regression coefficient;

[0088] Step 2-6, constructing a least squares estimate based on the least squares estimate of the regression coefficient to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB. Specifically, constructing a partial least squares equation based on the least squares estimate of the regression coefficient to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB includes:

[0089] The partial least squares equation constructed based on the least squares estimation of the regression coefficient is:

[0090]

[0091] Calculations are performed one by one based on the residual matrix until the absolute value of the elements in the residual matrix is ​​approximately 0, and the partial least squares equations related to multiple components are determined to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB.

[0092] For ease of understanding, a specific example is provided to illustrate the specific implementation of using the partial least squares method to establish a prediction model:

[0093] In one embodiment, first: recruit qualified volunteers. These volunteers need not smoke, drink, and have no allergic reaction to the experimental samples. At the same time, ensure that they participate in the same sampling position and light source conditions to ensure the accuracy of the data. Volunteers need to keep their faces clean before the experiment in order to obtain accurate data. Then: use the VISIA-CR system to collect facial images of volunteers under the condition of standard light source 2 to construct a face data set for skin color detection. Subsequently, the collected image data is carefully processed and the characteristics of various skin types in the skin image are marked. Next: based on the face picture, an advanced face feature point detection model is used to accurately detect the face and locate the key points. Afterwards, according to the face structure, the cheek symmetry area is defined as the first area, the facial T zone (including the forehead and chin) is the second area, the eye and lip area are the sensitive third area, and the rest are the fourth area. After completing the area division, add corresponding labels to the data of each area. Further: preprocess the data sets of the first, second, third and fourth areas. This process includes data cleaning, i.e. removing outliers, missing values ​​or erroneous data, and standardizing the original data, such as optimizing the data by subtracting the mean and scaling the variance, so that the weights can be reasonably allocated in subsequent analysis. Through the above process, training samples can be obtained.

[0094] According to the data in the training sample, the skin color extraction operation can be performed on the image of each region to obtain accurate skin color features.

[0095] When training the prediction model, you can continue to implement it through the following steps.

[0096] ① Analyze the factors xi (i.e. makeup influencing factors) that affect the ITA° data after makeup, and organize the data of n makeup influencing factors into an independent variable matrix X (i.e. the first variable matrix). Among them, x includes the brightness (L*), redness (A*), yellowness (B*) of the skin color before makeup, the individual type angle ITA° based on the L*a*b* color system, the hue angle (HAB), the brightness (L*), redness (A*), yellowness (B*), individual type angle ITA°, and hue angle (HAB) of the selected samples.

[0097] ② Define the after-makeup ITA value and the after-makeup HAB value as dependent variables Y1 and Y2, and obtain the second variable matrix through Y1 and Y2.

[0098] Extract the first pair of components u1 and v1 of the two sets of X and Y variables respectively to maximize the correlation. u1 is the independent variable set X = [x1, ... x m ] T A linear combination of, v1 is the independent variable set Y = [y1, y2] T Linear combinations of

[0099] ③ Set the parameters to determine the maximum number of principal components to 5.

[0100] ④Maximize the covariance matrix. Use the score vector The inner product calculation maximizes the correlation between u1 and v1.

[0101]

[0102]

[0103] ⑤Use the Lagrange multiplier method to find the unit vectors p1 and r1 so that r1 reaches the maximum;

[0104]

[0105] ⑥ From the standardized observation data matrices X and Y of the two sets of variables, the score vector of the first pair of components can be calculated.

[0106]

[0107] Establish the regression model of Yi on u1 and the regression model of Xi on u1, assuming that the regression model is:

[0108]

[0109] θ, σ are the parameter vectors in the many-to-one regression model, and A1, B1 are the residual matrices.

[0110] The least squares estimates of the regression coefficients θ and σ are:

[0111]

[0112] Replace A and B with the residuals A1 and B1, and repeat the above steps until the absolute value of the elements in the residual matrix is ​​approximately 0. Each time, a σ1 and θ1 are obtained.

[0113] ⑦ Repeat the above steps to get r components

[0114]

[0115] Substituting u1 into the partial least squares equation, we can get the prediction model of ITA and HAB.

[0116] In order to ensure that the makeup prediction effect is more realistic, in an optional implementation, after the makeup effect prediction is performed, the predicted simulated makeup effect can be applied to each area of ​​the face area, and the whole area is Gaussian blurred based on a pre-selected Gaussian kernel. This method selects a smaller Gaussian kernel size to retain the details in the image, making the makeup effect more natural.

[0117] In summary, the present application embodiment provides a specific solution implementation process, see Figure 3 As shown, the steps S1 to S8 are included:

[0118] Step S1: construct a face dataset;

[0119] Step S2: face recognition and key point positioning;

[0120] Step S3: extracting skin color indexes of various parts of the face;

[0121] Step S4: Establish a foundation color library and compile color attributes and numbers; collect various foundation products and record the color information of each product, such as natural, fair and dark color numbers;

[0122] Step S5: data preprocessing;

[0123] Step S6: predicting the skin color of each part of the face after superimposing the skin color with the foundation liquid, and establishing a skin color prediction model using partial least squares method;

[0124] Step S7: Gaussian blur processing;

[0125] Step S8: Preview the overall results.

[0126] To achieve the above method, Figure 4 It shows an interactive connection relationship between execution subject devices.

[0127] In summary, the method provided in the embodiment of the present application is combined with the field of image processing technology, and the face is divided into the first, second, third and fourth areas based on the basis, and the skin color features and skin color areas are extracted respectively, and then the foundation color number data is selected, and the skin color of each area of ​​the face is superimposed with the foundation color, and the partial least squares multi-dependent variable regression modeling method for multiple independent variables is utilized. Finally, through the overall Gaussian blur processing, while retaining the details and texture of the original skin color, the transition sense between the makeup effect and the original skin color is effectively eliminated, making the makeup effect smoother and more natural, and more accurately simulating the makeup effect.

[0128] The partial least squares method is used to establish a prediction model for the background pink and foundation skin color. By establishing a prediction regression equation, the skin color after makeup is predicted. This method can interact well with the existing cosmetics color trial system, overcome the problem of difficult to accurately judge the makeup effect of liquid foundation and the large workload, and avoid the waste of manpower and material resources. In addition, since this method has little impact on the transformation and upgrading of the existing system, the subsequent maintenance cost is also greatly reduced, thereby realizing the effective prediction of the makeup effect of liquid foundation, providing an efficient and accurate solution for the cosmetics field. The technology and algorithm based on this method have been widely used and verified in the fields of image processing and computer vision, and have high practicality and reliability.

[0129] Based on the above method embodiment, the present application embodiment also provides a makeup product makeup effect simulation device, see Figure 5 As shown, the device mainly includes the following parts:

[0130] The first acquisition module 501 is used to acquire a user's face image and an identifier of a target makeup product selected by the user;

[0131] A determination module 502, configured to determine a plurality of facial regions based on a preset facial segmentation rule, and to determine a skin color feature of each facial region;

[0132] The second acquisition module 503 is used to acquire the color characteristics of the target base makeup product from a pre-established base makeup product information database based on the identifier of the target base makeup product;

[0133] The simulation module 504 is used to obtain a simulated makeup effect by superimposing a makeup effect prediction model established by a pre-established partial least squares method based on the skin color characteristics of each facial area and the color characteristics of the target foundation product, wherein the makeup effect prediction model is constructed based on the individual type angle and the hue angle.

[0134] In some embodiments, the device further comprises an establishing module, specifically configured to:

[0135] Constructing a first variable matrix based on makeup influencing factors; wherein the makeup influencing factors include skin color brightness, skin color redness, and skin color yellowness before makeup;

[0136] Determine the second variable matrix based on the after-makeup individual type angle ITA value and the after-makeup hue angle HAB value;

[0137] Sequentially extracting the target component pairs corresponding to the first variable matrix and the second variable matrix; wherein the target component pairs are multiple groups;

[0138] Calculating the score vector of the target component pair, and calculating the correlation degree of the target component pair based on the inner product of the score vector; wherein the correlation degree satisfies a preset correlation threshold;

[0139] Establishing a regression model of the first variable and the second variable relative to the score vector, and determining the regression coefficient;

[0140] The least squares estimation is constructed based on the least squares estimation of the regression coefficient, and a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB is obtained.

[0141] In some embodiments, the establishment module is specifically used to:

[0142] The partial least squares equation constructed based on the least squares estimate of the regression coefficient is:

[0143]

[0144] Calculations are performed one by one based on the residual matrix until the absolute value of the elements in the residual matrix is ​​approximately 0, and the partial least squares equations related to multiple components are determined to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB.

[0145] In some embodiments, the second acquisition module 503 is used to:

[0146] Convert the RGB color space corresponding to the image of the facial area into the corresponding XYZ value in the CIEXYZ color space by a pre-selected matrix multiplication;

[0147] Divide the XYZ value by the standard value of the reference white point, calculate the values ​​of L, a and b to extract the skin color, calculate the individual type angle ITA° according to ITA°={arctan[(L-50) / b]}*180 / π, and calculate the hue angle HAB° according to HAB°=arctan(b / a)*180 / π.

[0148] In some embodiments, the determination module is configured to:

[0149] The symmetrical area of ​​the cheek is taken as the first area, the T zone of the face is taken as the second area, the area around the eyes and lips is taken as the third area, and the rest is taken as the fourth area.

[0150] In some embodiments, it also includes:

[0151] The blur module is used to apply the predicted simulated makeup effect to various areas of the face area and perform overall Gaussian blur based on a pre-selected Gaussian kernel.

[0152] The implementation principle and technical effects of the makeup effect simulation device for foundation products provided in the embodiment of the present application are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the embodiment of the makeup effect simulation device for foundation products, reference may be made to the corresponding contents in the aforementioned method embodiment of makeup effect simulation for foundation products.

[0153] The present application also provides an electronic device, such as Figure 6 As shown, it is a schematic diagram of the structure of the electronic device, wherein the electronic device 100 includes a processor 61 and a memory 60, the memory 60 stores computer executable instructions that can be executed by the processor 61, and the processor 61 executes the computer executable instructions to implement any of the above-mentioned methods for simulating the makeup effect of foundation products.

[0154] exist Figure 6 In the illustrated embodiment, the electronic device further includes a bus 62 and a communication interface 63 , wherein the processor 61 , the communication interface 63 and the memory 60 are connected via the bus 62 .

[0155] Among them, the memory 60 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 63 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 62 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 62 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0156] The processor 61 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 61. The above processor 61 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 61 reads the information in the memory and completes the steps of the makeup effect simulation method of the base makeup product of the above-mentioned embodiment in combination with its hardware.

[0157] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned method for simulating the makeup effect of foundation products. The specific implementation can be found in the aforementioned method embodiment, which will not be repeated here.

[0158] The computer program product of the method, device and equipment for simulating the makeup effect of foundation products provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0159] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0160] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0161] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the invented product is usually placed when in use. They are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0162] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for simulating the makeup effect of a base makeup product, It is characterized in that The method comprises: Obtaining a user's facial image and an identifier of a target makeup product selected by the user; Based on a preset facial segmentation rule, a plurality of facial regions are determined, and skin color characteristics of each facial region are determined; Acquiring the color characteristics of the target base makeup product from a pre-established base makeup product information database based on the identifier of the target base makeup product; Based on the skin color characteristics of each facial area and the color characteristics of the target foundation product, a makeup effect prediction model established by a pre-established partial least squares method is superimposed to obtain a simulated makeup effect. The makeup effect prediction model is constructed based on individual type angles and hue angles.

2. The method according to claim 1, It is characterized in that The method for constructing the makeup effect prediction model comprises: Constructing a first variable matrix based on makeup influencing factors; wherein the makeup influencing factors include skin color brightness, skin color redness, and skin color yellowness before makeup; Determine the second variable matrix based on the after-makeup individual type angle ITA value and the after-makeup hue angle HAB value; Sequentially extracting the target component pairs corresponding to the first variable matrix and the second variable matrix; wherein the target component pairs are multiple groups; Calculating the score vector of the target component pair, and calculating the correlation degree of the target component pair based on the inner product of the score vector; wherein the correlation degree satisfies a preset correlation threshold; Establishing a regression model of the first variable and the second variable relative to the score vector, and determining the regression coefficient; The least squares estimation is constructed based on the least squares estimation of the regression coefficient, and a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB is obtained.

3. The method according to claim 2, It is characterized in that A partial least squares equation is constructed based on the least squares estimation of the regression coefficient to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB, including: The partial least squares equation constructed based on the least squares estimate of the regression coefficient is: Calculations are performed one by one based on the residual matrix until the absolute value of the elements in the residual matrix is ​​approximately 0, and the partial least squares equations related to multiple components are determined to obtain a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB.

4. The method according to claim 1, It is characterized in that Determine skin tone characteristics for various facial areas, including: Convert the RGB color space corresponding to the image of the facial area into the corresponding XYZ value in the CIE XYZ color space by a pre-selected matrix multiplication; Divide the XYZ value by the standard value of the reference white point, calculate the values ​​of L, a and b to extract the skin color, calculate the individual type angle ITA° according to ITA°={arctan[(L-50) / b]}*180 / π, and calculate the hue angle HAB° according to HAB°=arctan(b / a)*180 / π.

5. The method according to claim 1, It is characterized in that Based on the preset facial segmentation rules, multiple facial regions are determined, including: The symmetrical area of ​​the cheek is taken as the first area, the T zone of the face is taken as the second area, the area around the eyes and lips is taken as the third area, and the rest is taken as the fourth area.

6. The method according to claim 5, It is characterized in that Also includes: The predicted simulated makeup effect is applied to each area of ​​the face region, and an overall Gaussian blur is performed based on a pre-selected Gaussian kernel.

7. A makeup effect simulation device for base makeup products, It is characterized in that The device comprises: The first acquisition module is used to acquire the user's face image and the identifier of the target makeup product selected by the user; A determination module, used to determine a plurality of facial regions based on a preset facial segmentation rule, and to determine a skin color feature of each facial region; A second acquisition module, configured to acquire the color characteristics of the target base makeup product from a pre-established base makeup product information database based on the identifier of the target base makeup product; The simulation module is used to obtain a simulated makeup effect by superimposing a makeup effect prediction model established by a pre-established partial least squares method based on the skin color characteristics of each facial area and the color characteristics of the target foundation product. The makeup effect prediction model is constructed based on individual type angles and hue angles.

8. The device according to claim 7, It is characterized in that The device also includes a building module, specifically used for: Constructing a first variable matrix based on makeup influencing factors; wherein the makeup influencing factors include skin color brightness, skin color redness, and skin color yellowness before makeup; Determine the second variable matrix based on the after-makeup individual type angle ITA value and the after-makeup hue angle HAB value; Sequentially extracting the target component pairs corresponding to the first variable matrix and the second variable matrix; wherein the target component pairs are multiple groups; Calculating the score vector of the target component pair, and calculating the correlation degree of the target component pair based on the inner product of the score vector; wherein the correlation degree satisfies a preset correlation threshold; Establishing a regression model of the first variable and the second variable relative to the score vector, and determining the regression coefficient; The least squares estimation is constructed based on the least squares estimation of the regression coefficient, and a makeup effect prediction model based on the individual type angle ITA and the hue angle HAB is obtained.

9. An electronic device, It is characterized in that It comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the makeup effect simulation method of the foundation product according to any one of claims 1 to 6.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the method for simulating the makeup effect of a foundation product according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Virtual make-up method, apparatus and device

    CN109409979A

  • Cosmetic simulation control method and device, computer equipment and storage medium

    CN109685713A

  • Cosmetic scheme recommendation method and system

    CN115577183A

  • Method for predicting makeup skin color

    JP1997178560A

  • Method for analyzing hair and predicting achievable hair dyeing ending colors

    WO2001087245A2