Makeup trying image generation method, makeup trying equipment and storage medium

Abstract

The invention discloses a makeup trying image generation method, makeup trying equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining a cosmetic image provided by a user to be subjected to makeup trying, determining the color value of cosmetics in the cosmetic image, then determining a standard cosmetic image matched with the color value, and obtaining a makeup trying image; and then compensating a standard color value of the standard cosmetic image based on the illumination intensity and color temperature data of the current makeup trying environment, generating a correction color adaptive to the current environment condition, and mapping the correction color to the face of the user to be subjected to makeup trying through a makeup mapping model. And generating an initial makeup image of the user to be subjected to makeup trying, and finally adjusting color transparency and hue offset of the initial makeup image to generate a target makeup image of the user to be subjected to makeup trying. Through color value extraction and dynamic ambient light compensation, the problem that the trial makeup effect of a traditional scheme is greatly different from the actual makeup is solved, so that the trial makeup effect is improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a method for generating try-on makeup images, a try-on makeup device, and a storage medium. Background Art

[0002] Currently, traditional try-on makeup methods usually rely on a pre-stored cosmetics image library. By selecting cosmetics images in the cosmetics image library by the user, the cosmetics images are simply superimposed on the user's facial image. However, in practical applications, the above method will result in a large difference between the try-on makeup effect and the actual makeup.

[0003] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] This application provides a method for generating try-on makeup images, a try-on makeup device, and a storage medium, aiming to solve the problem that there is a large difference between the try-on makeup effect of the traditional solution and the actual makeup.

[0005] To achieve the above object, a method for generating try-on makeup images provided by this application includes the following steps:

[0006] Obtain a cosmetics image provided by a user to be tried on makeup, and determine the color values of the cosmetics in the cosmetics image;

[0007] Determine a standard cosmetics image that matches the color values;

[0008] Based on the light intensity and color temperature data of the current try-on makeup environment, compensate the standard color values of the standard cosmetics image to generate a corrected color adapted to the current environmental conditions;

[0009] Map the corrected color to the face of the user to be tried on makeup through a makeup mapping model to generate an initial makeup image of the user to be tried on makeup;

[0010] According to the skin color data of the user to be tried on makeup, adjust the color transparency and hue offset of the initial makeup image to generate a target makeup image of the user to be tried on makeup.

[0011] In one embodiment, the step of determining the color values of the cosmetics in the cosmetics image includes:

[0012] Identify the cosmetics area in the cosmetics image through an edge detection algorithm, and determine the area ratio of each color in the cosmetics area;

[0013] Extract the main color and secondary color in the cosmetics area based on a color clustering algorithm;

[0014] Weight the primary color tone and the secondary color tone according to the corresponding area ratios to generate the color value.

[0015] In one embodiment, the method further includes:

[0016] Generate simulated lighting data in a multi-light source environment according to the first scenario label selected by the user to be tried on makeup.

[0017] Overlay the light intensity and the color temperature data in the current makeup trial environment with the simulated lighting data to generate a corrected color after dynamic compensation, and map the corrected color after dynamic compensation to the facial image of the user to be tried on makeup through a makeup mapping model to generate an initial makeup image of the user to be tried on makeup.

[0018] In one embodiment, after the step of mapping the corrected color to the face of the user to be tried on makeup through a makeup mapping model to generate an initial makeup image of the user to be tried on makeup, the method further includes:

[0019] Render the initial makeup image to the display screen of the makeup trial device in real time, and receive adjustment parameters input by the user to be tried on makeup based on the interaction interface of the makeup trial device, where the adjustment parameters include a transparency percentage and a hue offset.

[0020] Generate an offset matrix according to the transparency percentage and the hue offset in the adjustment parameters, and update the makeup mapping model based on the offset matrix to regenerate an initial makeup image of the user to be tried on makeup based on the updated makeup mapping model.

[0021] In one embodiment, after the step of adjusting the color transparency and the hue offset of the initial makeup image according to the skin color data of the user to be tried on makeup to generate a target makeup image of the user to be tried on makeup, the method further includes:

[0022] Obtain the virtual makeup corresponding to the target makeup image.

[0023] Package the virtual makeup and the cosmetic information associated with the virtual makeup to generate a shared makeup template, and publish the shared makeup template on a community platform.

[0024] When a control associated with the community platform is triggered, call the shared makeup template and fit it to the target makeup image of the user who triggered the control to generate a virtual makeup trial image.

[0025] In one embodiment, after the step of compensating the standard color value of the standard cosmetic image based on the light intensity and the color temperature data of the current makeup trial environment to generate a corrected color adapted to the current environmental conditions, the method further includes:

[0026] Obtain the historical makeup trial data of the user to be tried on makeup;

[0027] According to the historical makeup trial data, determine the colors with a preference matching degree higher than the threshold for the user to be tried on makeup through the AI recommendation algorithm;

[0028] Push the colors with a preference matching degree higher than the threshold to the display interface of the makeup trial device, so that the user to be tried on makeup selects the target color based on the colors displayed on the display interface of the makeup trial device, and determine the selected target color as the corrected color.

[0029] In one embodiment, the method further includes:

[0030] Obtain the second context label determined by the user to be tried on makeup based on the interaction interface of the makeup trial device;

[0031] According to the context label, determine at least one matching makeup plan;

[0032] Recommend at least one of the makeup plans to the user to be tried on makeup.

[0033] In one embodiment, after the step of recommending at least one of the makeup plans to the user to be tried on makeup, the method further includes:

[0034] Obtain the skin measurement data and the concerned makeup information of the user to be tried on makeup;

[0035] Based on the skin color level in the skin measurement data, screen the suitable color number information in the cosmetic information;

[0036] According to the color number information and the concerned makeup information, generate a preference priority ranking, and feedback the preference priority ranking to the interaction interface.

[0037] In addition, to achieve the above object, the present application further provides a makeup trial device, where the makeup trial device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the method for generating a makeup trial image as described above.

[0038] In addition, to achieve the above object, the present application further provides a storage medium, where the storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for generating a makeup trial image as described above are implemented.

[0039] The present application provides a method for generating a trial makeup image, a trial makeup device, and a storage medium. The method includes obtaining a makeup product image provided by a user to be tried on, determining the color value of the makeup product in the makeup product image, then determining a standard makeup product image that matches the color value, and then compensating the standard color value of the standard makeup product image based on the light intensity and color temperature data of the current trial makeup environment to generate a corrected color adapted to the current environmental conditions. The corrected color is mapped to the face of the user to be tried on through a makeup mapping model to generate an initial makeup image of the user to be tried on. Finally, according to the skin color data of the user to be tried on, the color transparency and hue offset of the initial makeup image are adjusted to generate a target makeup image of the user to be tried on. By accurately extracting color values, dynamically compensating for ambient light, and adaptively adjusting according to skin color, the present application solves the problem that there is a large difference between the trial makeup effect of the traditional solution and the actual makeup, thereby improving the trial makeup effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are incorporated herein and form a part of this specification, showing embodiments consistent with the present application and, together with the specification, are used to explain the principles of the present application.

[0041] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 It is a flowchart of the first embodiment of the method for generating a trial makeup image of the present application;

[0043] Figure 2 It is a flowchart of the second embodiment of the method for generating a trial makeup image of the present application;

[0044] Figure 3 It is a flowchart of the third embodiment of the method for generating a trial makeup image of the present application;

[0045] Figure 4 It is a flowchart of the fourth embodiment of the method for generating a trial makeup image of the present application;

[0046] Figure 5 It is an architecture diagram of the hardware operating environment of the trial makeup device involved in the embodiments of the present application.

[0047] The implementation, functional features, and advantages of the objectives of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] It should be understood that the specific embodiments described herein are merely for explaining the present application and are not intended to limit the present application.

[0049] To better understand the above technical solution, the exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0050] To better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0051] The main solution of the present application is: obtaining a cosmetic image provided by a user to be tried on makeup, and determining the color value of the cosmetics in the cosmetic image; determining the standard cosmetic image that matches the color value;

[0052] Based on the light intensity and color temperature data of the current makeup trial environment, compensating the standard color value of the standard cosmetic image to generate a corrected color adapted to the current environmental conditions; mapping the corrected color to the face of the user to be tried on makeup through a makeup mapping model to generate an initial makeup image of the user to be tried on makeup; adjusting the color transparency and hue offset of the initial makeup image according to the skin color data of the user to be tried on makeup to generate a target makeup image of the user to be tried on makeup.

[0053] Currently, traditional makeup trial methods usually rely on a pre-stored cosmetic image library, and simply superimpose the cosmetic image selected by the user in the cosmetic image library on the user's facial image. However, in practical applications, the above method will result in a large difference between the facial makeup trial effect and the actual makeup.

[0054] By obtaining a cosmetic image provided by a user to be tried on makeup, determining the color value of the cosmetics in the cosmetic image, then determining the standard cosmetic image that matches the color value, and then based on the light intensity and color temperature data of the current makeup trial environment, compensating the standard color value of the standard cosmetic image to generate a corrected color adapted to the current environmental conditions, and mapping the corrected color to the face of the user to be tried on makeup through a makeup mapping model to generate an initial makeup image of the user to be tried on makeup, and finally adjusting the color transparency and hue offset of the initial makeup image according to the skin color data of the user to be tried on makeup to generate a target makeup image of the user to be tried on makeup. The present application solves the problems of color distortion, ambient light interference and low skin color adaptability in virtual makeup trial through accurate color value extraction, dynamic ambient light compensation and skin color self-adaptation adjustment, thereby improving the makeup trial effect.

[0055] It should be noted that the execution entity of this embodiment can be a makeup trial system, or a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a makeup trial device capable of implementing the above functions. This embodiment does not make specific limitations in this regard. Taking the makeup trial system as the execution entity as an example, this embodiment and the following embodiments will be described below.

[0056] Embodiment 1

[0057] Based on this, the embodiment of the present application provides a method for generating a makeup trial image, referring to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the method for generating a makeup trial image of the present application. The method for generating a makeup trial image includes steps S10 to S40:

[0058] Step S10: Obtain a cosmetic image provided by a user to be made up, and determine the color value of the cosmetics in the cosmetic image.

[0059] In this embodiment, the processing action is performed by a makeup trial system, which is set in a makeup trial device. The makeup trial device can be a makeup mirror or other terminal devices. The makeup trial device includes a camera. The user captures the selected cosmetic image through the camera, that is, a physical image of the cosmetics taken by the built-in camera of the terminal device (such as an RGB camera or a high-resolution sensor). Then the makeup trial system obtains the cosmetic image and determines the color value of the cosmetic image. The color value refers to the quantified color parameters extracted from the cosmetic image through an image processing algorithm, usually represented in the RGB, HSV, or Lab color space.

[0060] Specifically, the makeup trial system calls the camera to collect in real time cosmetic images such as lipsticks and eyeshadows provided by the user. Through a color segmentation algorithm, specifically, K-means clustering can be used to extract the main color area, and the RGB pixel values are converted into the Lab color space, and the mean value is calculated as the color value.

[0061] As an optional implementation manner, the step of determining the color value of the cosmetics in the cosmetic image includes:

[0062] Step S11: Identify the cosmetic area in the cosmetic image through an edge detection algorithm, and determine the area proportion of each color in the cosmetic area.

[0063] Step S12: Extract the main color tone and secondary color tone in the cosmetic area based on a color clustering algorithm.

[0064] Step S13: Calculate the weighted sum of the primary color tone and the secondary color tone according to the corresponding area ratio to generate the color value.

[0065] In this embodiment, the edge detection algorithm refers to a contour extraction technique based on image gradient changes (such as the Canny algorithm or Sobel operator). The color region is a continuous and color-uniform closed region in the image, and the area ratio is the ratio of the number of pixels in each color region to the total number of pixels in the image. The color clustering algorithm refers to a technique for grouping pixels according to color similarity (such as K-means clustering or mean shift algorithm). The primary color tone is the center value of the color cluster with the largest proportion, and the secondary color tone is the center value of the color cluster with the second largest proportion. The weighted calculation refers to linearly superimposing the color values according to the area ratios of the primary and secondary color tones to generate a comprehensive color value, that is, the color value.

[0066] Specifically, the makeup trial system first preprocesses the cosmetic image captured by the camera by grayscale conversion and Gaussian filtering, uses the Canny algorithm to detect the edge contour, and combines the flood fill algorithm to segment each color region. Subsequently, the number of pixels in each region is counted, and its proportion to the total number of pixels is calculated. For example, if the total number of pixels in the image is 10,000 and the number of pixels in a certain red region is 6,000, the area ratio is 60%.

[0067] Then, the makeup trial system performs HSV color space conversion on the segmented color regions, uses K-means clustering (set K = 3) to group the pixels in the region according to hue (H) and saturation (S), and calculates the RGB mean value of the center point of each cluster as the primary and secondary color tones. For example, if a certain region is clustered into a red cluster (accounting for 70%), an orange cluster (accounting for 20%), and others (accounting for 10%) after clustering, the primary color tone is RGB(200, 30, 40) of the red cluster, and the secondary color tone is RGB(220, 80, 50) of the orange cluster.

[0068] Next, the makeup trial system multiplies the RGB value of the primary color tone by the primary area ratio, multiplies the RGB value of the secondary color tone by the secondary area ratio, and adds the two to obtain the final color value. For example, if the primary color tone RGB(200, 30, 40) accounts for 70% and the secondary color tone RGB(220, 80, 50) accounts for 20%, then the color value R = 200×0.7 + 220×0.2 = 140 + 44 = 184, G = 30×0.7 + 80×0.2 = 21 + 16 = 37, B = 40×0.7 + 50×0.2 = 28 + 10 = 38, that is, RGB(184, 37, 38).

[0069] Exemplarily, when there are red (pixel 5000), pink (pixel 3000), and packaging black (pixel 2000) regions in the lipstick image, the system calculates that the proportion of red is 50%, the proportion of pink is 30%, and the proportion of black is 20%. The purple regions in the eyeshadow image are clustered to obtain the main color tone RGB(150, 50, 180) (proportion 65%) and the secondary color tone RGB(170, 70, 190) (proportion 25%), and the rest are noise. When the main color tone of the lipstick RGB(210, 50, 90) accounts for 60% and the secondary color tone RGB(190, 40, 80) accounts for 30%, then the color values are R = 210×0.6 + 190×0.3 = 126 + 57 = 183, G = 50×0.6 + 40×0.3 = 30 + 12 = 42, B = 90×0.6 + 80×0.3 = 54 + 24 = 78, and RGB(183, 42, 78) is output.

[0070] Step S20: Determine the standard cosmetic image that matches the color value.

[0071] In this embodiment, the preset color library can be loaded first, so as to determine the matching standard cosmetic image in the preset color library. Among them, the preset color library is a database containing the standard color numbers of brand cosmetics, which stores the Lab values of each color number and the corresponding virtual texture images. Through the color similarity algorithm, specifically, the Euclidean distance or cosine similarity can be used to screen the standard cosmetic image corresponding to the closest color number in the preset color library.

[0072] Specifically, the color value obtained by the system is compared with the Lab data of each color number in the color library, and the color difference ΔE = √(ΔL2 + Δa 2 + Δb 2 ) is calculated, and the color number with the smallest ΔE is selected as the matching result. For example, if the user inputs Lab as (45, 70, 15), and a certain color number in the library is (48, 68, 18), then ΔE = √(3 2 + 22 + 3 2 ) = 4.69. If this value is the smallest in the library, the cosmetic corresponding to this color number is selected, and the corresponding standard cosmetic image is determined.

[0073] Step S30: Based on the light intensity and color temperature data of the current makeup trial environment, compensate the standard color value of the standard cosmetic image to generate a corrected color adapted to the current environmental conditions.

[0074] In this embodiment, dynamic compensation means that according to the illuminance (unit: lux) and color temperature (unit: K) collected by the ambient light sensor, the color value of the original color number is adjusted through the color temperature-color mapping matrix, and the corrected color is the virtual color data adapted to the ambient light after compensation.

[0075] Specifically, the makeup trial system obtains the ambient light intensity (such as 800 lux) and color temperature (such as 5500 K) through a light sensor, and adjusts the original color number Lab value based on a pre-trained color temperature compensation model (such as polynomial regression or 3×3 color temperature matrix). For example, if the original color number Lab is (50, 60, 20), under warm light (3000 K), the compensation model will increase the a value (red channel) by 5 units to obtain the corrected color Lab (50, 65, 20).

[0076] Exemplarily, when it is detected that the ambient light is low illuminance (200 lux) and cold light (6500 K), the makeup trial system calls the compensation model to perform a gain adjustment on the color number Lab (40, 70, 30) in the library, and outputs the corrected color Lab (42, 75, 28) to enhance the color saturation.

[0077] As an alternative implementation, in this embodiment, the method for compensating the standard color value of the standard cosmetic image further includes:

[0078] Generating simulated lighting data in a multi-light source environment according to the first scenario label selected by the user to be tried on; superimposing the lighting intensity and the color temperature data in the current environment with the simulated lighting data to generate a corrected color after dynamic compensation.

[0079] Specifically, the first scenario label refers to a preset lighting scene classification, such as "indoor warm light", "outdoor sunlight", "night party". The simulated lighting data is a set of lighting parameter sets including multi-light source color temperature, illuminance, and direction angle, which is generated by an ambient light rendering model. Superimposing refers to performing weighted fusion on the spectral energy distributions of the actual ambient light and the simulated light to generate equivalent mixed light source parameters; the corrected color after dynamic compensation is the virtual color Lab value adjusted based on the mixed light source.

[0080] The makeup trial system calls the corresponding light source parameter combination from the preset scenario library according to the first scenario label selected by the user to be tried on, such as "café warm light". For example, the label "night party" corresponds to a main light source (color temperature 3000 K, intensity 200 lux, direction angle 45°) and an auxiliary light source (color temperature 4500 K, intensity 100 lux, direction angle 120°), and generates simulated lighting data through a lighting mixing algorithm. Then the makeup trial system obtains the color temperature T1 (unit: K) and illuminance L1 (unit: lux) of the current ambient light through a light sensor, and performs superimposition calculation with the color temperature T2 and illuminance L2 of the simulated light source. For example, if the original corrected color Lab is (50, 60, 20), after superimposition, T_mix = 4000 K, L_mix = 800 lux, and the compensation model will increase the a value by 3 units to obtain the final Lab (50, 63, 20).

[0081] In this embodiment, through the simulation of multiple light sources driven by the first scenario label and the superposition with the ambient light, the problem that a single ambient light compensation cannot restore a complex lighting scene is solved, so that the virtual try-on makeup color still maintains a real visual effect under mixed lighting conditions.

[0082] Step S40: Map the corrected color to the face of the user to be tried on makeup through the makeup mapping model to generate the initial makeup image of the user to be tried on makeup.

[0083] Step S50: Adjust the color transparency and hue offset of the initial makeup image according to the skin color data of the user to be tried on makeup to generate the target makeup image of the user to be tried on makeup.

[0084] In this embodiment, the makeup mapping model refers to a pre-trained AR mapping model, which includes a texture fitting algorithm based on facial feature point detection (such as the dlib library), and can map the makeup to the human face image based on the above algorithm; the skin color data includes skin color Lab values, skin texture roughness, etc. The color transparency adjustment is achieved through Alpha channel blending, and the hue offset is calculated through the hue offset in the HSV space.

[0085] Specifically, the try-on makeup system uses face key point detection to locate the user's lip / eyelid area, maps the corrected color texture to the corresponding position, and adjusts the transparency according to the skin color brightness (L value) in the skin color data (for example, the transparency is reduced by 20% for dark skin), and at the same time offsets the HSV hue value of the virtual color based on the skin color hue (H value).

[0086] Exemplarily, when the user's skin color Lab is (65, 10, 20), indicating that the skin color is relatively fair, the system superimposes the corrected lipstick color Lab (50, 65, 20) with 80% transparency and fine-tunes its hue from 355° to 350° in the HSV space to make the try-on makeup effect more natural.

[0087] Optionally, in this embodiment, after the step of adjusting the color transparency and hue offset of the initial makeup image according to the skin color data of the user to be tried on makeup to generate the target makeup image of the user to be tried on makeup, it further includes:

[0088] Obtain the virtual makeup corresponding to the try-on makeup image; package the virtual makeup and the cosmetic information associated with the virtual makeup to generate a shared makeup template, and publish the shared makeup template on the community platform; when the control associated with the community platform is triggered, call the shared makeup template and fit it with the target makeup image of the user who triggers the control to generate a virtual try-on makeup image.

[0089] Specifically, after the user selects the trial makeup image in the interaction interface, corresponding instructions can be fed back to the trial makeup system, enabling the trial makeup system to obtain the virtual makeup corresponding to the trial makeup image. Then, the generated virtual makeup is associated with the information of the used cosmetics, including brand, model, color number, etc. The virtual makeup and the associated cosmetics information are packaged into a shared makeup template file, and the template file is compressed and optimized to reduce the file size and improve the loading speed. Then, the generated shared makeup template is uploaded to the community platform for other users to browse and download. When uploading, relevant description information, such as makeup style, applicable occasions, etc., can be added to enable users to better understand the content of the template. Relevant controls, such as buttons, links, etc., are set in the community platform for users to trigger and call the shared makeup template. When the user clicks on the control, the system matches and fits the selected template with the current user image to generate the corresponding virtual trial makeup image.

[0090] In the technical solution provided in this embodiment, by obtaining the cosmetic image provided by the user to be tried on makeup, determining the color value of the cosmetics in the cosmetic image, then determining the standard cosmetic image that matches the color value, and then compensating the standard color value of the standard cosmetic image based on the light intensity and color temperature data of the current trial makeup environment to generate a corrected color adapted to the current environmental conditions, and mapping the corrected color to the face of the user to be tried on makeup through a makeup mapping model to generate the initial makeup image of the user to be tried on makeup. Finally, according to the skin color data of the user to be tried on makeup, the color transparency and hue offset of the initial makeup image are adjusted to generate the target makeup image of the user to be tried on makeup. This embodiment solves the problems of color distortion, environmental light interference, and low skin color adaptability in virtual trial makeup through accurate color value extraction, dynamic ambient light compensation, and skin color self-adaptation adjustment, thereby improving the trial makeup effect.

[0091] Embodiment Two

[0092] Based on the same inventive concept, the present application also provides a second embodiment. Refer to Figure 2 , Figure 2 which is a schematic flowchart of the control method of the trial makeup system according to the second embodiment of the present application. In this embodiment, after step S40, steps S41 to S45 are further included:

[0093] Step S41: Render the trial makeup image to the display screen of the trial makeup device in real time, and receive the adjustment parameters input by the user to be tried on makeup based on the interaction interface of the trial makeup device, where the adjustment parameters include the transparency percentage and the hue offset.

[0094] In this embodiment, the interaction interface is the front-end interface of the trial makeup system, the display screen is the display screen of the trial makeup device, and the adjustment parameters include numerical values such as the transparency percentage and the hue offset.

[0095] Specifically, the makeup trial system superimposes the makeup trial image onto the real-time camera footage through the GPU rendering engine and displays transparency and hue adjustment controls on the side of the screen. The user adjusts and determines the adjustment parameters through the controls in the interaction interface, and the makeup trial system converts the adjustment parameters into a parameter matrix (for example, transparency α = 0.5, corresponding to the Alpha channel value 128).

[0096] Step S42: Generate an offset matrix based on the transparency percentage and the hue offset in the adjustment parameters, and update the makeup mapping model based on the offset matrix to regenerate the initial makeup image of the user to be tried on based on the updated makeup mapping model.

[0097] In this embodiment, updating the makeup mapping model means adjusting the blending weights of the texture maps, the hue offset matrix, and the transparency channel according to the new parameters.

[0098] Specifically, the makeup trial system generates a new HSV offset matrix (such as H + 5°, S + 10%) and an Alpha value (α = 0.6) according to the adjustment parameters input through the interaction interface, and modifies the rendering parameters of the AR model in real time through the Shader script. For example, the original lip color HSV (350°, 80%, 90%) is updated to HSV (345°, 90%, 90%), and the transparency is adjusted from 0.7 to 0.6.

[0099] In this embodiment, the target makeup trial image refers to the final rendering result after fusing the user interaction parameters, and its color fit and dynamic effects are synchronized in real time. The system calls the updated AR model parameters, recalculates the texture mapping coordinates of the facial feature points, and performs secondary mixing on the corrected color and the user skin color data (such as Lab values). For example, for the updated HSV (345°, 90%, 90%), combined with the user skin color L = 65, brightness adaptation calculation is performed in the Lab space to generate a natural transition lip rendering effect.

[0100] In addition, it is also possible to receive the adjustment requirements of the user to be tried on for the makeup style through voice commands, then parse the adjustment requirements in the voice commands, and map them to the corresponding hue parameters and transparency parameters to determine the mapping result.

[0101] Specifically, the user can also input the adjustment requirements for the makeup style based on voice commands. The voice commands refer to the voice signals input by the user through the microphone. After receiving the adjustment requirements, they are parsed into semantic tags through the natural language processing (NLP) model. The makeup trial system calls the speech recognition API to convert the voice into text, and uses the pre-trained keyword matching model to extract the operation type and object in the command. The mapping result refers to the rule library that converts the semantic command into specific color parameters.

[0102] Specifically, based on a predefined semantic-parameter mapping table, the system converts the parsed instructions into numerical change amounts. For example, if the instruction is "more red", it is mapped to a decrease in the HSV hue H by ΔH = -5° (in the direction of red); if the instruction is "semi-transparent", then α = 0.5 is set.

[0103] In the technical solution provided in this embodiment, by adjusting the makeup parameters in real time through multimodal interaction, the problems of single interaction and low adjustment efficiency in the traditional makeup trial system are solved, and the user's personalized control ability and immersive experience of the virtual makeup are improved.

[0104] Embodiment III

[0105] Based on the same inventive concept, the present application also provides a third embodiment. Refer to Figure 3 , Figure 3 which is a schematic flowchart of the control method for the makeup trial system of the third embodiment of the present application. In this embodiment, the method includes steps S50 to S70:

[0106] Step S50: Obtain the historical makeup trial data of the user to be tried on.

[0107] In this embodiment, the user's historical makeup trial data refers to structured data stored locally or in the cloud, including the color system (such as the Lab value range) selected by the user during multiple makeup trials, the score (1-5 points) for the makeup texture (matte / pearl), and the saved virtual makeup ID and parameters, etc.

[0108] Specifically, the makeup trial system extracts the historical data associated with the user ID from the database, and parses the preferred color system, texture scores (such as a matte score of 4.2 points and a pearl score of 3.5 points), and the top 3 makeup color numbers with the most saves. The data is implemented through SQL queries or NoSQL document reads.

[0109] Step S60: According to the historical makeup trial data, determine the color with a matching degree higher than the threshold with the user's preference through an AI recommendation algorithm.

[0110] Step S70: Push the color with a preference matching degree higher than the threshold to the display interface of the makeup trial device, so that the user to be tried on selects the target color based on the color displayed on the display interface of the makeup trial device, and determines the selected target color as the corrected color.

[0111] In this embodiment, the AI recommendation algorithm refers to a collaborative filtering model or a deep learning recommendation model based on user behavior data; the matching degree is obtained by calculating the similarity between the corrected color and the historical preferred color system and the texture score weight. The threshold can be set as a screening condition with a matching degree greater than or equal to 85%, which can be adjusted according to actual needs and is not limited here.

[0112] Calculate the cosine similarity between the corrected color Lab value after dynamic compensation (e.g., Lab(48, 68, 18)) and the historical preferred color system. If the historical score for the texture corresponding to the corrected color (such as matte) is 4.5 points, the matching degree increases the weight coefficient (such as score / 5×0.2). Filter the colors with a matching degree ≥ 85%, sort them in descending order of the matching degree, and push them to the interface. After the user clicks to select, the system will overwrite the original corrected color with the Lab value of this color.

[0113] In the technical solution provided in this embodiment, through the AI recommendation algorithm driven by historical data, the problem that the color after ambient light compensation may deviate from the long-term preferences of users is solved, the efficient combination of personalized makeup trial recommendation and user selection is realized, and the accuracy of makeup trial and user satisfaction are improved.

[0114] Embodiment Four

[0115] Based on the same inventive concept, the present application also provides a fourth embodiment, referring to Figure 4 , Figure 4 is a schematic flowchart of the control method of the makeup trial system according to the sixth embodiment of the present application. In this embodiment, the method includes steps S80 to S100:

[0116] Step S80: Obtain the second context label determined by the user to be made up based on the interaction interface of the makeup trial device.

[0117] Step S90: Determine at least one matching makeup plan according to the context label.

[0118] Step S100: Recommend at least one of the makeup plans to the user to be made up.

[0119] In this embodiment, the second context label refers to the scene classification selected by the user on the interaction interface. Among them, the context label includes a dating label, a business label, etc., and each label is associated with a predefined makeup style rule library. The recommended makeup plan is a set of color combinations and texture parameters of sub-makeups including eyeshadow, lip gloss, blush, etc., generated through a label-scheme mapping table or a collaborative filtering algorithm. The user interface feedback refers to displaying the recommended plan in the form of a thumbnail list or 3D preview, supporting click selection or swipe browsing.

[0120] Specifically, the makeup trial system receives user input through UI controls and binds the selected label ID to the metadata in the preset context library. For example, when the user clicks the "business" label, the makeup trial system calls the light intensity, color temperature preference, and makeup style parameters corresponding to the label ID.

[0121] In addition, recommendations can also be made based on the selection records of other users under the same tag, sorted by popularity. Finally, the try-on system renders the recommended solutions into virtual makeup preview images (such as the Lab values of lip colors and eye shadow texture maps), and arranges them in descending order of matching degree in the interface. After the user clicks on a certain solution, the system calls the parameters of this solution (lip gloss Lab (50, 65, 20), transparency α = 0.7) to update the makeup mapping model.

[0122] Exemplarily, the user selects the "business meeting" tag in the interface. The system obtains the tag ID as "tag_102", and the associated parameters are "natural makeup effect, matte texture, low saturation". The interface displays the preview image and text description of the solution. After the user selects it, the system loads the corresponding makeup parameters into the try-on system.

[0123] Optionally, in this embodiment, after the step of recommending at least one of the makeup solutions to the user to be tried on, the method further includes:

[0124] Obtaining the skin measurement data and the concerned makeup information of the user to be tried on; screening the suitable color number information in the cosmetic information based on the skin color level in the skin measurement data; generating a preference priority ranking according to the color number information and the concerned makeup information, and feeding back the preference priority ranking to the interaction interface.

[0125] Specifically, the skin measurement data is the Lab value of the skin color, color level partition, etc. collected by a multi-light source camera; the concerned makeup information is the makeup information marked by the user in the interface. The suitable color number information refers to the cosmetic color number that satisfies the preset matching rule (such as ΔE ≤ 5) with the user's skin color level, and is screened by a color compatibility algorithm. The preference priority ranking refers to the result of comprehensively ranking the suitable color numbers by weighting the user's attention and historical selection frequency.

[0126] Specifically, the skin measurement module is called to collect the Lab values of the skin color in multiple areas of the user's face and the Lab of the cheek, calculate the average color level and classify it. At the same time, the concerned parts are extracted from the user's concerned makeup information to generate tagged data. According to the user's skin color Lab value, the color difference between each color number Lab value in the cosmetic color number library and the skin color is calculated, and the corresponding color numbers are screened. If the concerned makeup information includes texture requirements, the color numbers with non-matching textures are further excluded. The color numbers are sorted in descending order of scores to generate a starred list and pushed to the interface. Through the double screening of the skin color level and the concerned information and the dynamic weight ranking, the problem that the traditional recommendation system ignores the user's real-time preferences is solved, the high-precision adaptation of cosmetic color numbers and the personalized priority push are realized, and the user's decision-making efficiency is significantly improved.

[0127] In addition, makeup recommendations can also be made based on the user's dressing information and hair accessory information. The dressing information includes the top and pants currently paired by the user, and the hair accessory information includes the hairband and earrings worn by the user, etc. Specifically, first obtain the user image provided by the user, and identify the dressing information and hair accessory information in the user image; then, based on the dressing information and hair accessory information, combine the recommendation algorithm to generate suitable makeup information, and feedback the makeup information to the interface of the makeup trial device for the user to select, so as to improve the effect of the user's makeup matching.

[0128] In the technical solution provided in this embodiment, through the intelligent recommendation driven by the context label, the problems of low efficiency and mismatched styles in the user's manual adjustment of makeup are solved, the rapid adaptation and accurate push of the scene makeup are realized, and the practicability and user experience of the makeup trial system are significantly improved.

[0129] Since the system introduced in the embodiments of the present application is the system adopted for implementing the method in the embodiments of the present application, based on the method introduced in the embodiments of the present application, those skilled in the art can understand the specific structure and deformation of the system, so it will not be elaborated here. Any system adopted by the method in the embodiments of the present application belongs to the scope to be protected by the present application.

[0130] The present application provides a makeup trial device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for generating a makeup trial image in the first embodiment above.

[0131] Next, refer to Figure 5 , which shows a schematic structural diagram of a makeup trial device suitable for implementing the embodiments of the present application. The makeup trial device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The shown makeup trial device is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0132] As Figure 5As shown, the makeup trial device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. In the random access memory 1004, various programs and data required for the operation of the makeup trial device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the makeup trial device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a makeup trial device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or had alternatively.

[0133] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0134] The makeup trial device provided by the present application adopts the method for generating a makeup trial image in the above embodiment, and can solve the technical problem that there is a large difference between the makeup trial effect and the actual makeup in the traditional solution. Compared with the prior art, the beneficial effects of the makeup trial device provided by the present application are the same as those of the method for generating a makeup trial image provided by the above embodiment, and other technical features in the makeup trial device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0135] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0136] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0137] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for generating a trial makeup image in the above embodiments.

[0138] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination of the above.

[0139] The above computer-readable storage medium can be included in the trial makeup device; it can also exist separately without being assembled into the trial makeup device.

[0140] The above computer-readable storage medium carries one or more programs, which, when executed by the makeup trial device, cause the makeup trial device to: obtain a cosmetic image provided by a user to be tried on with makeup, and determine the color value of the cosmetics in the cosmetic image; determine a standard cosmetic image that matches the color value; based on the light intensity and color temperature data of the current makeup trial environment, compensate the standard color value of the standard cosmetic image to generate a corrected color adapted to the current environmental conditions; map the corrected color to the face of the user to be tried on with makeup through a makeup mapping model to generate an initial makeup image of the user to be tried on with makeup; and adjust the color transparency and hue offset of the initial makeup image according to the skin color data of the user to be tried on with makeup to generate a target makeup image of the user to be tried on with makeup.

[0141] Computer program code for performing the operations of the present application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0143] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0144] The readable storage medium provided by the present application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-described method for generating trial makeup images, and can solve the technical problem that there is a large difference between the trial makeup effect of the traditional solution and the actual makeup. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the method for generating trial makeup images provided in the above embodiments, and will not be elaborated here.

[0145] The embodiments of the present application provide a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method for generating trial makeup images as described above are implemented.

[0146] The computer program product provided by the present application can solve the technical problem that there is a large difference between the trial makeup effect of the traditional solution and the actual makeup. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of the present application are the same as those of the method for generating trial makeup images provided in the above embodiments, and will not be elaborated here.

[0147] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent scope of the present application.

Claims

1. A method for generating a trial makeup image, characterized in that, The method for generating the trial makeup image includes the following steps: Obtain the makeup product image provided by the user to be tried on, and determine the color value of the makeup product in the makeup product image; Determine the standard makeup product image that matches the color value; Based on the light intensity and color temperature data of the current trial makeup environment, compensate the standard color value of the standard makeup product image to generate a corrected color adapted to the current environmental conditions; Map the corrected color to the facial image of the user to be tried on through a makeup mapping model to generate the initial makeup image of the user to be tried on; According to the skin color data of the user to be tried on, adjust the color transparency and hue offset of the initial makeup image to generate the target makeup image of the user to be tried on.

2. The method according to claim 1, characterized in that, The step of determining the color value of the makeup product in the makeup product image includes: Identify the makeup product area in the makeup product image through an edge detection algorithm, and determine the area proportion of each color in the makeup product area; Extract the main color and secondary color in the makeup product area based on a color clustering algorithm; Perform weighted calculation on the main color and secondary color according to the corresponding area proportion to generate the color value.

3. The method according to claim 1, characterized in that The method further includes: Generate simulated lighting data in a multi-light source environment according to the first scenario label selected by the user to be tried on; Overlay the light intensity, the color temperature data in the current trial makeup environment and the simulated lighting data to generate a corrected color after dynamic compensation, so as to map the corrected color after dynamic compensation to the facial image of the user to be tried on through a makeup mapping model to generate the initial makeup image of the user to be tried on.

4. The method according to claim 1, wherein After the step of mapping the corrected color to the facial image of the user to be tried on through a makeup mapping model to generate the initial makeup image of the user to be tried on, it further includes: Render the initial makeup image to the display screen of the trial makeup device in real time, and receive the adjustment parameters input by the user to be tried on based on the interaction interface of the trial makeup device, where the adjustment parameters include the transparency percentage and the hue offset; Generate an offset matrix according to the transparency percentage and the hue offset in the adjustment parameters, and update the makeup mapping model based on the offset matrix to regenerate the initial makeup image of the user to be tried on based on the updated makeup mapping model.

5. The method according to claim 1, wherein After the step of adjusting the color transparency and hue offset of the initial makeup image according to the skin color data of the user to be tried on to generate the target makeup image of the user to be tried on, it further includes: Obtain the virtual makeup corresponding to the target makeup image; Package the virtual makeup and the cosmetic information associated with the virtual makeup to generate a shared makeup template, and publish the shared makeup template on a community platform; When the control associated on the community platform is triggered, call the shared makeup template and fit it with the target makeup image of the user who triggers the control to generate a virtual trial makeup image.

6. The method according to claim 1, wherein The method further includes: Obtain the historical trial makeup data of the user to be tried on; According to the historical trial makeup data, determine the color with a preference matching degree higher than the threshold for the user to be tried on through an AI recommendation algorithm; Push the color with the preference matching degree higher than the threshold to the display interface of the makeup trial device, so that the user to be made up can select a target color based on the color displayed on the display interface of the makeup trial device, and determine the selected target color as the corrected color.

7. The method according to claim 1, wherein The method further includes: Obtain a second context label determined by the user to be made up based on the interaction interface of the makeup trial device; Determine at least one matching makeup plan according to the second context label; Recommend at least one of the makeup plans to the user to be made up.

8. The method according to claim 7, wherein After the step of recommending at least one of the makeup plans to the user to be made up, it further includes: Obtain the skin measurement data and the concerned makeup information of the user to be made up; Based on the skin color level in the skin measurement data, filter the suitable color number information in the cosmetic information; Generate a preference priority ranking according to the color number information and the concerned makeup information, and feedback the preference priority ranking to the interaction interface.

9. A makeup trial device, characterized in that, The makeup trial device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the method for generating a makeup trial image according to any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements the steps of the method for generating a makeup trial image according to any one of claims 1 to 8.

Images