Makeup and skin care guidance method, system and device based on ai multi-modal perception

Abstract

The application relates to a makeup and skin care guidance method, system and device based on AI multi-modal perception, and relates to the technical field of intelligent cosmetic mirrors. The method comprises the following steps: recognizing the identity of a user by acquiring image information, interaction information and environment information of the current user, acquiring user living habit information based on the user identity, determining scene state information according to the information and generating target makeup recommendation information; matching the target makeup recommendation information with preset cosmetic storage information and preset skin care product storage information to obtain cosmetic matching results and skin care product matching results; optimizing based on the cosmetic matching results and a preset optimization algorithm to obtain cosmetic parameter information and generate and display makeup effects and a makeup guidance method; and generating and displaying a skin care guidance method based on the skin care product matching results, the identity information, the environment information and the living habit information. The matching degree of makeup effects and actual user needs is improved, and the personalized makeup needs of users are met.

Description

Technical Field

[0001] This invention relates to the field of smart makeup mirror technology, and in particular to a beauty and skincare guidance method, system and device based on AI multimodal perception. Background Technology

[0002] The continuous development of smart home, computer vision, and artificial intelligence technologies has brought numerous conveniences to people's daily lives. In the beauty and skincare field, advancements in related technologies have also led to a shift in consumer demand towards more personalized and intelligent products and services. Users no longer require makeup mirrors to merely offer basic reflections; they expect to receive more personalized beauty and skincare guidance tailored to their individual needs.

[0003] In traditional beauty and skincare settings, people primarily use traditional makeup mirrors. These mirrors only offer basic reflection, fulfilling the user's most basic need to see their face. However, with technological advancements, smart mirrors have emerged. Some smart mirrors feature lighting, providing better makeup application conditions in various lighting environments; others offer basic skin analysis capabilities, allowing for preliminary assessment and analysis of the user's skin.

[0004] However, traditional makeup mirrors cannot meet users' personalized makeup needs, while some smart mirrors with certain functions offer limited analysis dimensions and rigid recommendations. Therefore, there is an urgent need for a makeup and skincare guidance technology based on multimodal perception. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a beauty and skin care guidance method, system and device based on AI multimodal perception, aiming to solve at least one of the above-mentioned technical problems.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0007] Firstly, this application provides a beauty and skincare guidance method based on AI multimodal perception, employing the following technical solution:

[0008] A beauty and skincare guidance method based on AI multimodal perception includes:

[0009] The system acquires the current user's image information, interaction information, and environmental information, and identifies the current user's identity based on the image information. The user identity includes the user's age, occupation, basic skin information, and preference information. The preference information includes dietary preferences and makeup preferences. The interaction information represents the data generated when the current user interacts with the makeup mirror. The environmental information includes humidity and light.

[0010] Based on the current user's identity information, obtain the current user's lifestyle information, which includes sleep time, exercise information, and endocrine cycle change information.

[0011] Based on the interaction information, environmental information, and lifestyle information, scene status information is determined, and target beauty recommendation information is generated based on the scene status information. The scene status information includes a scene state and a sceneless state. The scene state includes a scene type and the scene corresponding to the scene type.

[0012] The target beauty recommendation information is matched with the preset cosmetics storage information and the preset skin care product storage information to obtain cosmetics matching results and skin care product matching results.

[0013] Based on the cosmetic matching results and the preset optimization algorithm, the parameters of the cosmetics in the cosmetic storage information are optimized to obtain cosmetic parameter information;

[0014] Based on makeup parameter information, generate and display makeup effects and makeup guidance methods;

[0015] Furthermore, a skincare guidance method is generated and displayed based on the skincare product matching results, the current user's identity information, the environmental information, and the lifestyle information.

[0016] The beneficial effects of this invention are as follows: By comprehensively utilizing multi-dimensional data such as user image information, interaction information, environmental information, and lifestyle information, a comprehensive perception and accurate modeling of the user's state is achieved; by intelligently identifying scene state information and combining it with the attribute characteristics of cosmetics and skincare products, highly personalized makeup and skincare solutions are generated; furthermore, by optimizing algorithms to finely adjust the parameters for using cosmetics, the matching degree between makeup effects and the user's actual needs is significantly improved, solving the problems of traditional makeup mirrors having single analysis dimensions, rigid recommendations, and weak interaction, thus meeting the user's personalized makeup needs.

[0017] Based on the above technical solution, the present invention can be further improved as follows.

[0018] Furthermore, based on the current user's identity information, the current user's lifestyle information is obtained. This lifestyle information includes sleep duration, exercise information, and endocrine cycle change information, including:

[0019] Based on the current user's identity information, the wearable device associated with the identity information and the basic lifestyle habit profile associated with the user's identity are determined. The basic lifestyle habit profile includes historical data on the user's self-reported sleep cycle, exercise frequency, and manually entered historical data on endocrine cycles.

[0020] The wearable device retrieves the current user's current lifestyle profile.

[0021] The basic lifestyle habit profile, the current lifestyle habit profile, and quantifiable indicators are fused from multiple data sources, and the current user's lifestyle habit information is generated through a preset weight allocation algorithm.

[0022] The beneficial effects of adopting the above-mentioned further solutions are as follows: by linking user identity information with wearable devices and basic lifestyle habit profiles, the automated collection of multi-source lifestyle data is realized; by weighting and integrating the real-time monitored current lifestyle habit data with the historical data, the accuracy and comprehensiveness of lifestyle habit information acquisition are improved, providing a reliable data foundation for the accurate generation of subsequent personalized beauty and skincare solutions.

[0023] Furthermore, the scenario type includes a first scenario and a second scenario. The first scenario represents a scenario defined by user intent or a specific event, and the second scenario represents a potential demand scenario triggered by the environment perceived by the makeup mirror in conjunction with the user's physiological state.

[0024] The step of determining scene state information based on the interaction information, environmental information, and lifestyle habit information, and generating target beauty recommendation information based on the scene state information, includes:

[0025] Based on natural language processing technology, keywords are extracted from the interactive information to obtain keyword information;

[0026] Based on the environmental information, the UV index level and dryness level are determined;

[0027] Based on the aforementioned lifestyle information, the physiological state level is determined;

[0028] Based on the keyword information, determine whether a first scenario exists in the interaction information, and based on the UV index level, dryness level, and physiological state level, determine whether a second scenario exists.

[0029] If a first scenario or a second scenario exists, the scenario status information is determined to be in the state of having a scenario.

[0030] If neither the first scene nor the second scene exists, the scene status information is determined to be "no scene".

[0031] When the scene status information is a scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, historical beauty and skincare information corresponding to the user identity, scene type, scene and hot topic information corresponding to the scene type. The hot topic information represents real-time popular trends, seasonal demands and sudden events related to beauty and skincare.

[0032] When the scene status information is no scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, and the historical beauty and skincare information corresponding to the user identity.

[0033] The beneficial effects of adopting the above-mentioned further solutions are as follows: Through multimodal perception technologies such as natural language processing, environmental sensor data analysis, and physiological state analysis, collaborative recognition and intelligent judgment of users' explicit intentions (first scenario) and implicit needs (second scenario) are achieved. By constructing a dual-path recommendation mechanism with and without specific scenarios, it can respond to users' precise beauty needs in specific scenarios and provide personalized solutions that conform to users' habits in daily life. When a specific scenario exists, target beauty recommendations are generated by combining the current user's image information, user identity information, historical beauty and skincare information, scenario type, corresponding scenario, and trending information, ensuring that the recommended beauty information conforms to specific scenarios and real-time trends. When a specific scenario does not exist, target beauty recommendations are generated based on the current user's image information, user identity information, and historical beauty and skincare information, satisfying users' daily beauty needs without specific scenarios and achieving personalized beauty recommendations across multiple scenarios.

[0034] Furthermore, when the scene status information indicates a scene exists, the step of generating target beauty recommendation information based on the current user's image information, user identity information, historical beauty and skincare information corresponding to the user identity, scene type, and scene and hotspot information corresponding to the scene type includes:

[0035] Based on the current user's image information and a preset deep learning facial analysis model, the real-time multidimensional skin texture features of the current user are determined. The real-time multidimensional skin texture features include skin color value, pore size, wrinkle depth, and the distribution and severity of oil and blemishes.

[0036] Based on the scene type and the scene corresponding to the scene type, scene demand feature information is determined from the scene demand mapping relationship. The scene demand feature information includes makeup longevity index, waterproof index, concealing index and style orientation index.

[0037] Based on trend information, the popular features of makeup styles are determined, including popular color schemes and the popularity of makeup techniques.

[0038] Based on the current user's real-time multidimensional skin features, preference information, user age, scenario requirements, makeup trends, and a preset makeup scheme generation model, multiple candidate makeup schemes are generated. The makeup scheme generation model is a deep learning model trained on user skin data, makeup effect data, and user feedback data.

[0039] Obtain historical recommendation data for each candidate beauty scheme, and based on the historical recommendation data for each candidate beauty scheme, determine the user feedback rating and selection frequency for each candidate beauty scheme;

[0040] Based on user feedback ratings and selection frequency for each candidate beauty solution, the first priority of each candidate beauty solution is determined.

[0041] Based on the scenario demand characteristics, the second priority of each candidate beauty solution is determined in multiple dimensions, including makeup longevity, water resistance, coverage and style orientation.

[0042] Based on the second priority of each candidate beauty solution in multiple dimensions and the real-time multi-dimensional skin texture feature information, the third priority of each candidate beauty solution is determined.

[0043] Based on the user's historical beauty and skincare information, the fourth priority of each candidate beauty solution is determined.

[0044] Based on the makeup difficulty of each candidate makeup scheme and the current user's makeup skill level, multiple matching values ​​are determined.

[0045] Based on multiple matching values, the fifth priority of each candidate beauty solution is determined;

[0046] Based on the first, third, fourth, and fifth priorities of each candidate beauty solution, the target beauty recommendation information is determined.

[0047] The beneficial effects of adopting the above-mentioned further solutions are as follows: By determining real-time multi-dimensional skin texture features based on deep learning facial analysis models, the user's current skin condition can be accurately grasped; by determining scene requirement feature information from scene requirement mapping relationships, the recommended makeup solutions can be tailored to specific scene requirements; by determining makeup trend feature information based on hot topic information, the recommended makeup solutions can keep up with fashion trends; by generating multiple candidate makeup solutions using a makeup solution generation model, a diverse range of choices can be provided; by determining the priority of each candidate makeup solution in different dimensions, and comprehensively considering historical recommendation data, scene requirements, real-time skin texture, historical makeup and skincare information, and the matching degree between makeup difficulty and user makeup skills, the target makeup recommendation information can be finally determined, generating makeup recommendation solutions that better meet the user's personalized needs, scene requirements, and popular trends.

[0048] Furthermore, when the scene status information is a no-scene state, the step of generating target beauty recommendation information based on the current user's image information, user identity information, and the historical beauty and skincare information corresponding to the user identity includes:

[0049] Based on environmental information, calculate the stress index of the environment on the skin and determine the target environmental impact factors;

[0050] Based on the product usage frequency and satisfaction rating in the historical beauty and skincare information corresponding to the user's identity, a user daily preference vector is generated.

[0051] The real-time skin condition assessment results, user identity information, environmental stress index and target environmental impact factors, and user daily preference vector are input into the daily beauty recommendation model to generate a basic daily beauty and skincare plan; the daily beauty recommendation model is a generative model trained based on user behavior data without specific scenario constraints.

[0052] Based on the time-limited usage patterns in the historical beauty and skincare information corresponding to the user's identity, the basic daily beauty and skincare routine is adjusted to adapt to the time, generating target beauty recommendation information.

[0053] The beneficial effects of adopting the above-mentioned further solutions are as follows: By introducing quantitative analysis of environmental stress index and target environmental impact factors, combined with precise modeling of user historical preference vectors, a deep perception of users' beauty needs is achieved in daily life without specific scenarios; through a generative recommendation model specifically designed for scenario-free environments, real-time skin condition, environmental influencing factors, and long-term user preferences are effectively integrated to generate basic beauty solutions that fit users' daily needs; and by dynamically adjusting usage patterns over time periods, the temporal adaptability of daily recommendation solutions is improved, ensuring that beauty suggestions maintain optimal applicability and user satisfaction at different times.

[0054] Furthermore, before matching the target beauty product recommendations with the preset cosmetics storage information and the preset skincare product storage information to obtain the cosmetics matching results and skincare product matching results, the process also includes:

[0055] Based on the target beauty recommendation information, multiple first beauty recommendation information that have a similarity threshold greater than the target beauty recommendation information are matched from the recommendation information database;

[0056] Multiple first beauty recommendation information are sorted according to the similarity threshold with the target beauty recommendation information to generate a candidate beauty recommendation information set;

[0057] Based on the target beauty recommendation information, the alternative beauty recommendation information set, and the preset augmented reality technology, the makeup effect corresponding to the target beauty recommendation information and the makeup effect corresponding to each of the first beauty recommendation information in the alternative beauty recommendation information set are displayed on the makeup mirror.

[0058] In response to the current user's trigger action on makeup, operation information is obtained, including makeup effect selection operation, makeup intensity component adjustment operation, makeup position component adjustment operation, and style component selection operation;

[0059] Based on the operation information, target beauty recommendation information, and multiple first beauty recommendation information, new target beauty recommendation information is generated.

[0060] The beneficial effects of adopting the above-mentioned further solution are as follows: by matching and providing multiple highly similar alternative makeup schemes from the recommendation information database, and combining augmented reality technology to simultaneously present the main recommendation and alternative makeup effects on the makeup mirror, the user's choice range and visual experience are greatly enriched; by responding to the user's multi-dimensional interactive operations on makeup effects, intensity, position and style in real time, the system intelligently captures the user's personalized preference details and dynamically generates optimized new recommendation schemes, effectively solving the problem of insufficient flexibility in traditional recommendation systems.

[0061] Furthermore, the cosmetic matching result includes a cosmetic list and basic attribute parameters of the cosmetics. These basic attribute parameters include coverage, spreadability, wear time, and ingredient characteristics. The parameters of the cosmetics in the stored cosmetic information are optimized based on the cosmetic matching result and a preset optimization algorithm to obtain cosmetic parameter information, including:

[0062] Based on the user's makeup skill level and historical operation preferences in the user's identity information, determine the constraints of makeup techniques;

[0063] Based on the real-time image information of the current user, facial partition features are determined. The optimization objective functions are naturalness of makeup effect, blemish coverage, makeup durability, and resource consumption. The optimization objective functions are solved by a multi-objective optimization algorithm, which is constrained by the basic attribute parameters of cosmetics, facial partition feature data, and personalized makeup techniques. The makeup parameter information for each facial region is obtained. The makeup parameter information includes product dosage parameters, application range parameters, concentration ratio parameters, technique intensity parameters, and time series parameters. The time series parameters represent the optimal time interval and duration of each makeup step.

[0064] Based on the makeup parameter information of each facial region, a set of makeup parameter information is generated, and the feasibility of each parameter is verified and conflict detection is performed.

[0065] The verified makeup parameter information is bound to the corresponding cosmetic list to obtain the makeup parameter information.

[0066] The beneficial effects of adopting the above-mentioned further solutions are as follows: Under the optimization objectives of comprehensively considering the naturalness of the makeup effect, the coverage of blemishes, the longevity of the makeup, and the consumption of resources, the refined customization of makeup parameters is achieved by combining the user's personalized makeup technique constraints, facial zoning feature data, and basic cosmetic attribute parameters for collaborative optimization; by generating multi-dimensional parameter information including product dosage, application range, concentration ratio, technique intensity, and time series, and by performing feasibility verification and conflict detection on the parameters, the scientific nature and operability of the makeup solution are improved, ultimately ensuring that the generated makeup parameter information can not only meet the personalized makeup effect requirements but also conform to the user's actual operational capabilities.

[0067] Secondly, this application provides a beauty and skincare guidance system based on AI multimodal perception, which adopts the following technical solution:

[0068] A beauty and skincare guidance system based on AI multimodal perception includes:

[0069] The image acquisition module is used to acquire the image information of the current user and send the image information of the current user to the generation module;

[0070] The environmental perception module is used to detect the brightness and humidity of the environment around the makeup mirror and send the brightness and humidity around the makeup mirror to the generation module.

[0071] The interaction module is used to collect the current user's interaction information and send the interaction information to the generation module;

[0072] The generation module is used to acquire the current user's image information, interaction information, and environmental information, and to identify the user's identity based on the image information. The user's identity includes the user's age, occupation, basic skin information, and preference information. The preference information includes dietary preferences and makeup preferences. The interaction information represents the data generated when the current user interacts with the makeup mirror. The environmental information includes humidity and light.

[0073] Based on the current user's identity information, obtain the current user's lifestyle information, which includes sleep time, exercise information, and endocrine cycle change information.

[0074] Based on the interaction information, environmental information, and lifestyle information, scene status information is determined, including scene type and scene.

[0075] Based on the scene state information, target beauty product recommendation information is generated;

[0076] The target beauty recommendation information is matched with the preset cosmetics storage information and the preset skin care product storage information to obtain cosmetics matching results and skin care product matching results.

[0077] Based on the cosmetic matching results and the preset optimization algorithm, the parameters of the cosmetics in the cosmetic storage information are optimized to obtain cosmetic parameter information;

[0078] Based on makeup parameter information, generate and display makeup effects and makeup guidance methods;

[0079] Furthermore, a skincare guidance method is generated and displayed based on the skincare product matching results, the current user's identity information, the environmental information, and the lifestyle information.

[0080] Thirdly, this application provides an electronic device that adopts the following technical solution:

[0081] An electronic device includes a memory and a processor, wherein the memory stores a computer program capable of being loaded by the processor and executing the AI-based multimodal perception-based beauty and skincare guidance method as described in any of the first aspects.

[0082] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution:

[0083] A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing the AI-based multimodal perception-based beauty and skincare guidance method as described in any one of the first aspects.

[0084] Additional aspects and advantages of this application will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of this application. Attached Figure Description

[0085] Figure 1 This is a flowchart illustrating a beauty and skincare guidance method based on AI multimodal perception, provided as an embodiment of the present invention.

[0086] Figure 2 This is a schematic diagram of the structure of a beauty and skincare guidance system based on AI multimodal perception, provided in one embodiment of the present invention.

[0087] Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0089] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0090] like Figure 1 As shown, a beauty and skincare guidance method based on AI multimodal perception is applied to a makeup mirror. The makeup mirror includes two 1080P high-definition wide-angle cameras, a multispectral imaging sensor, a temperature and humidity sensor, an ambient light sensor, a processor, a touch screen, and a voice microphone. The two 1080P high-definition wide-angle cameras, the multispectral imaging sensor, the temperature and humidity sensor, the ambient light sensor, the touch screen, and the voice microphone are all connected to the processor.

[0091] This method mainly includes:

[0092] S1, acquire the current user's image information, interaction information and environmental information, and identify the current user's identity based on the image information. The user identity includes the user's age, occupation, basic skin information and preference information. The preference information includes dietary preferences and makeup preferences. The interaction information represents the data generated when the current user interacts with the makeup mirror. The environmental information includes humidity and light.

[0093] In this embodiment, image information can be acquired using a high-definition camera. High-definition cameras offer high resolution and sensitivity, enabling them to clearly capture facial details. Alternatively, multiple cameras can be combined to acquire images from different angles. Interaction information refers to data generated during user interaction with the makeup mirror, such as user clicks, swipes, and voice recordings on the mirror screen. Environmental information includes humidity and light intensity. Humidity and light sensors can be used to detect ambient humidity and light levels, respectively. A capacitive humidity sensor can be used, offering advantages such as high measurement accuracy and fast response. A photoresistive light sensor can be used, which is cost-effective and offers stable performance.

[0094] After receiving environmental information, the processor can adjust the brightness of the makeup lights in the makeup mirror according to the environmental information.

[0095] The illumination compensation formula is as follows:

[0096] ;

[0097] in, The corrected target illumination intensity, The ambient light intensity is the original light intensity measured by the ambient light sensor. Ambient light influencing factors This refers to the mirror brightness.

[0098] S2, based on the current user's identity information, obtain the current user's lifestyle information, including sleep time, exercise information, and endocrine cycle change information;

[0099] In this embodiment of the application, S2 specifically includes the following sub-steps:

[0100] S21, based on the current user's identity information, determine the wearable device associated with the identity information and the basic lifestyle habit file associated with the user's identity. The basic lifestyle habit file includes historical user-reported sleep cycles, exercise frequency, and manually entered historical data of endocrine cycles.

[0101] S22, Obtain the current user's current lifestyle profile from the wearable device;

[0102] S23, the basic lifestyle habit profile, the current lifestyle habit profile, and quantifiable indicators are fused from multiple sources, and the current user's lifestyle habit information is generated through a preset weight allocation algorithm.

[0103] In the above embodiments, the wearable device can be a smart bracelet. The basic lifestyle profile includes the user's average weekly sleep duration, weekly exercise frequency, endocrine cycle, heart rate changes, etc., over a period of time. The current lifestyle profile, collected in real time by the wearable device, records data on the user's current lifestyle habits and reflects the user's current actual lifestyle status.

[0104] S3. Based on the interaction information, environmental information, and lifestyle information, determine the scene status information, and generate target beauty recommendation information based on the scene status information. The scene status information includes a scene state and a sceneless state. The scene state includes a scene type and the scene corresponding to the scene type.

[0105] In this application embodiment, the scenario type includes a first scenario and a second scenario. The first scenario represents a scenario defined by user intent or a specific event, and the second scenario represents a potential demand scenario triggered by the environment perceived by the makeup mirror and the user's physiological state.

[0106] The step of determining scene state information based on the interaction information, environmental information, and lifestyle habit information, and generating target beauty recommendation information based on the scene state information, includes:

[0107] Based on natural language processing technology, keywords are extracted from the interactive information to obtain keyword information;

[0108] Based on the environmental information, the UV index level and dryness level are determined;

[0109] Based on the aforementioned lifestyle information, the physiological state level is determined;

[0110] Based on the keyword information, determine whether a first scenario exists in the interaction information, and based on the UV index level, dryness level, and physiological state level, determine whether a second scenario exists.

[0111] If a first scenario or a second scenario exists, the scenario status information is determined to be in the state of having a scenario.

[0112] If the first or second scenario does not exist, the scenario status information is determined to be "no scenario".

[0113] When the scene status information is a scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, historical beauty and skincare information corresponding to the user identity, scene type, scene and hot topic information corresponding to the scene type. The hot topic information represents real-time popular trends, seasonal demands and sudden events related to beauty and skincare.

[0114] When the scene status information is no scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, and the historical beauty and skincare information corresponding to the user identity.

[0115] In the above embodiments, environmental information is classified according to pre-set standards to determine the UV index level and dryness level. For example, a UV index of 0-2 is level one, 3-4 is level two, 5-6 is level three, 7-9 is level four, and 10 and above is level five. Air humidity below 30% is highly dry, 30%-50% is moderately dry, 50%-70% is slightly dry, and above 70% is humid.

[0116] If a user gets enough sleep and has a normal endocrine cycle, their physiological state is rated as good; if they don't get enough sleep but their endocrine cycle is normal, their rating is average; if they have severe sleep deprivation or endocrine cycle disorders, their rating is poor.

[0117] Next, based on the keywords, the user's primary scenario is determined. For example, if a user says in the chat window, "I'm going to an outdoor wedding tomorrow," then the keyword is "wedding," the primary scenario is determined to be a wedding scenario, and the scenario status information is determined to be "there is a scenario."

[0118] The second scenario is determined based on the UV index level, dryness level, and physiological state level. For example, if the UV index level is level 1, the air humidity is humid, and the physiological state level is good, then it is determined that there is no second scenario.

[0119] If a first scenario or a second scenario exists, the scenario status information is determined to be "scenario present"; if a first scenario or a second scenario does not exist, the scenario status information is determined to be "scenario absent".

[0120] In this embodiment of the application, when the scene status information indicates a scene exists, target beauty recommendation information is generated based on the current user's image information, user identity information, historical beauty and skincare information corresponding to the user identity, scene type, and scene and hotspot information corresponding to the scene type. This includes:

[0121] Based on the current user's image information and a preset deep learning facial analysis model, the real-time multidimensional skin texture features of the current user are determined. The real-time multidimensional skin texture features include skin color value, pore size, wrinkle depth, and the distribution and severity of oil and blemishes.

[0122] Based on the scene type and the scene corresponding to the scene type, scene demand feature information is determined from the scene demand mapping relationship. The scene demand feature information includes makeup longevity index, waterproof index, concealing index and style orientation index.

[0123] Based on trend information, the popular features of makeup styles are determined, including popular color schemes and the popularity of makeup techniques.

[0124] Based on the current user's real-time multidimensional skin features, preference information, user age, scenario requirements, makeup trends, and a preset makeup scheme generation model, multiple candidate makeup schemes are generated. The makeup scheme generation model is a deep learning model trained on user skin data, makeup effect data, and user feedback data.

[0125] Obtain historical recommendation data for each candidate beauty scheme, and based on the historical recommendation data for each candidate beauty scheme, determine the user feedback rating and selection frequency for each candidate beauty scheme;

[0126] Based on user feedback ratings and selection frequency for each candidate beauty solution, the first priority of each candidate beauty solution is determined.

[0127] Based on the scenario demand characteristics, the second priority of each candidate beauty solution is determined in multiple dimensions, including makeup longevity, water resistance, coverage and style orientation.

[0128] Based on the second priority of each candidate beauty solution in multiple dimensions and the real-time multi-dimensional skin texture feature information, the third priority of each candidate beauty solution is determined.

[0129] Based on the user's historical beauty and skincare information, the fourth priority of each candidate beauty solution is determined.

[0130] Based on the makeup difficulty of each candidate makeup scheme and the current user's makeup skill level, multiple matching values ​​are determined.

[0131] Based on multiple matching values, the fifth priority of each candidate beauty solution is determined;

[0132] Based on the first, third, fourth, and fifth priorities of each candidate beauty solution, the target beauty recommendation information is determined.

[0133] In the above embodiments, the makeup mirror analyzes the collected images of the current user based on a preset deep learning facial analysis model to determine the user's real-time multi-dimensional skin features, including skin tone values ​​(e.g., using RGB values ​​to represent skin tone), pore size (e.g., categorized as mild, moderate, or severe), wrinkle depth, oil distribution and severity (e.g., dividing the face into different areas and assessing oil secretion), and blemish distribution and severity (e.g., the location, size, and number of pimples and age spots). This model has been trained on a large number of photos with different skin types, skin tones, and facial features, enabling it to accurately identify facial features.

[0134] The makeup mirror stores a scene requirement mapping table. Based on the determined "outdoor wedding scene", the corresponding scene requirement feature information is retrieved from the table.

[0135] Extract trending makeup features from the collected trending information. For example, the currently popular color scheme is the warm peach tone, and the popular makeup technique is the "dewy skin effect creation technique," etc.

[0136] Subsequently, based on the user's real-time multi-dimensional skin characteristics, preferences entered during registration, age, scenario-specific needs, makeup trends, and a pre-defined makeup solution generation model, multiple candidate makeup solutions are generated. The makeup solution generation model is a deep learning model trained on a large amount of user skin data, makeup effect data, and user feedback data. This model comprehensively considers various factors to generate personalized makeup solutions that meet the user's needs. Each solution includes detailed product recommendations and application steps from base makeup to color makeup.

[0137] The makeup mirror is equipped with a makeup skill level assessment system. Users can determine their makeup skill level through a simple test during registration. The system can also capture the actual makeup application status and steps in real time during the application process, compare the actual makeup status and steps with makeup guidance methods, and assess the user's makeup skill level.

[0138] By using a deep learning-based facial analysis model to determine real-time multi-dimensional skin features, the system can accurately grasp the user's current skin condition. Determining scene-specific needs through scene-based mapping allows recommended makeup solutions to fit specific scene requirements. Identifying trending makeup features based on trending topics ensures recommended makeup solutions stay on trend. Generating multiple candidate makeup solutions using a makeup solution generation model provides diverse choices. By prioritizing each candidate makeup solution across different dimensions, and comprehensively considering historical recommendation data, scene requirements, real-time skin condition, historical makeup and skincare information, and the match between makeup difficulty and the user's makeup skill level, the system ultimately determines the target makeup recommendation information, generating makeup recommendations that better meet the user's personalized needs, scene requirements, and current trends.

[0139] In this embodiment of the application, when the scene state information is a no-scene state, target beauty recommendation information is generated based on the current user's image information, user identity information, and historical beauty and skincare information corresponding to the user identity, including:

[0140] Based on environmental information, calculate the stress index of the environment on the skin and determine the target environmental impact factors;

[0141] Based on the product usage frequency and satisfaction rating in the historical beauty and skincare information corresponding to the user's identity, a user daily preference vector is generated.

[0142] The real-time skin condition assessment results, user identity information, environmental stress index and target environmental impact factors, and user daily preference vector are input into the daily beauty recommendation model to generate a basic daily beauty and skincare plan; the daily beauty recommendation model is a generative model trained based on user behavior data without specific scenario constraints.

[0143] Based on the time-limited usage patterns in the historical beauty and skincare information corresponding to the user's identity, the basic daily beauty and skincare routine is adjusted to adapt to the time, generating target beauty recommendation information.

[0144] In this embodiment, different factors are assigned corresponding weights based on their degree of harm to the skin, and the environmental stress index on the skin is calculated by weighted average.

[0145] The system analyzes user usage patterns across different time periods based on their historical beauty and skincare information. For example, users typically perform basic skincare and simple makeup in the morning and deep cleansing and full skincare before bed. By dynamically adjusting these usage patterns, the system improves the time-sensory adaptability of daily recommendations, ensuring that beauty suggestions remain optimally applicable and satisfying for users at different times of the day.

[0146] S4, match the target beauty recommendation information with the preset cosmetics storage information and the preset skin care product storage information to obtain cosmetics matching results and skin care product matching results;

[0147] In this embodiment of the application, the target beauty recommendation information is analyzed in detail and divided into two parts: cosmetics and skin care products. For example, if the recommendation information includes foundation, it is then determined whether the foundation is matched with a corresponding skin care product.

[0148] Before matching the target beauty recommendations with the preset cosmetics and skincare products to obtain the cosmetics matching results and skincare product matching results, the process includes:

[0149] Based on the target beauty recommendation information, multiple first beauty recommendation information that have a similarity threshold greater than the target beauty recommendation information are matched from the recommendation information database;

[0150] Multiple first beauty recommendation information are sorted according to the similarity threshold with the target beauty recommendation information to generate a candidate beauty recommendation information set;

[0151] Based on the target beauty recommendation information, the alternative beauty recommendation information set, and the preset augmented reality technology, the makeup effect corresponding to the target beauty recommendation information and the makeup effect corresponding to each of the first beauty recommendation information in the alternative beauty recommendation information set are displayed on the makeup mirror.

[0152] In response to the current user's trigger action on makeup, operation information is obtained, including makeup effect selection operation, makeup intensity component adjustment operation, makeup position component adjustment operation, and style component selection operation;

[0153] Based on the operation information, target beauty recommendation information, and multiple first beauty recommendation information, new target beauty recommendation information is generated.

[0154] By matching and providing multiple highly similar alternative makeup schemes from the recommendation database, and combining augmented reality technology to simultaneously present the main recommendation and alternative makeup effects on the makeup mirror, the user's choice range and visual experience are greatly enriched. By responding to the user's multi-dimensional interactive operations on makeup effects, intensity, position and style in real time, the system intelligently captures the user's personalized preference details and dynamically generates optimized new target makeup recommendation schemes, improving the flexibility of target makeup recommendation scheme generation.

[0155] S5. Based on the cosmetic matching results and the preset optimization algorithm, optimize the parameters of the cosmetics in the cosmetic storage information to obtain cosmetic parameter information.

[0156] In this embodiment of the application, the cosmetic matching result includes a cosmetic list and basic attribute parameters of the cosmetics, including coverage, spreadability, makeup duration and ingredient characteristics.

[0157] The cosmetic parameter information is obtained by optimizing the parameters of the cosmetics in the cosmetic storage information based on the cosmetic matching results and a preset optimization algorithm, including:

[0158] Based on the user's makeup skill level and historical operation preferences in the user's identity information, determine the constraints of makeup techniques;

[0159] Based on the real-time image information of the current user, facial partition features are determined. The optimization objective functions are naturalness of makeup effect, blemish coverage, makeup durability, and resource consumption. The optimization objective functions are solved by a multi-objective optimization algorithm, which is constrained by the basic attribute parameters of cosmetics, facial partition feature data, and personalized makeup techniques. The makeup parameter information for each facial region is obtained. The makeup parameter information includes product dosage parameters, application range parameters, concentration ratio parameters, technique intensity parameters, and time series parameters. The time series parameters represent the optimal time interval and duration of each makeup step.

[0160] Based on the makeup parameter information of each facial region, a set of makeup parameter information is generated, and the feasibility of each parameter is verified and conflict detection is performed.

[0161] The verified makeup parameter information is bound to the corresponding cosmetic list to obtain the makeup parameter information.

[0162] In this embodiment of the application, for example, if the user is a novice and has a history of preferring to apply makeup with their fingers, then the constraints on the makeup technique may be that the pressure applied should be gentle and the makeup steps should be as simple as possible.

[0163] Using real-time image information of the current user, image recognition technology is used to determine facial features such as the size, shape, and skin texture of areas like the forehead, cheeks, nose, and chin.

[0164] The optimization objective functions are naturalness of makeup effect, blemish coverage, makeup longevity, and resource consumption. The constraints are the basic attribute parameters of cosmetics, facial partition feature data, and personalized makeup techniques. Multi-objective optimization algorithms, such as particle swarm optimization, are used to solve the optimization objective functions.

[0165] S6 generates and displays makeup effects and makeup guidance methods based on makeup parameter information;

[0166] In this embodiment, a makeup effect image is generated using virtual makeup trial technology based on makeup parameter information and displayed on the screen for the user to view. Simultaneously, detailed makeup guidance methods are presented to the user in the form of text, images, or videos, including specific operations for each makeup step, product usage tips, and technique demonstrations.

[0167] S7. Based on the skincare product matching results, the current user's identity information, the environmental information, and the lifestyle information, generate and display a skincare guidance method.

[0168] In this embodiment, based on the skincare product matching results, the current user's identity information (age, gender, skin type, etc.), environmental information (such as season, temperature, humidity, etc.), and lifestyle information (such as work and rest time, dietary preferences, etc.), a personalized skincare guidance method is generated using a preset skincare knowledge base and algorithm, and displayed on the makeup mirror interface in the form of text, pictures, or videos.

[0169] This method comprehensively utilizes multi-dimensional data, including user image information, interaction information, environmental information, and lifestyle information, to achieve a complete perception and accurate modeling of the user's state. By intelligently identifying scene state information and combining it with the attribute characteristics of cosmetics and skincare products, it generates highly personalized makeup and skincare solutions. Furthermore, by optimizing algorithms to finely adjust the parameters for using cosmetics, it significantly improves the matching degree between makeup effects and users' actual needs, solving the problems of traditional makeup mirrors such as single analysis dimensions, rigid recommendations, and weak interaction, and meeting users' personalized makeup needs.

[0170] Figure 2 A schematic diagram of the structure of a beauty and skincare guidance system 200 based on AI multimodal perception is shown.

[0171] like Figure 2 As shown, a beauty and skincare guidance system 200 based on AI multimodal perception mainly includes:

[0172] Image acquisition module 201 is used to acquire the image information of the current user and send the image information of the current user to the generation module;

[0173] The environmental perception module 202 is used to detect the brightness and humidity of the environment around the makeup mirror and send the brightness and humidity around the makeup mirror to the generation module.

[0174] Interaction module 203 is used to collect the current user's interaction information and send the interaction information to the generation module;

[0175] The generation module 204 is used to acquire the current user's image information, interaction information and environmental information, and identify the user's identity based on the image information. The user's identity includes the user's age, occupation, basic skin information and preference information. The preference information includes dietary preferences and makeup preferences. The interaction information represents the data generated when the current user interacts with the makeup mirror. The environmental information includes humidity and light.

[0176] Based on the current user's identity information, obtain the current user's lifestyle information, which includes sleep time, exercise information, and endocrine cycle change information.

[0177] Based on the interaction information, environmental information, and lifestyle information, scene status information is determined, including scene type and scene.

[0178] Based on the scene state information, target beauty product recommendation information is generated;

[0179] The target beauty recommendation information is matched with the preset cosmetics storage information and the preset skin care product storage information to obtain cosmetics matching results and skin care product matching results.

[0180] Based on the cosmetic matching results and the preset optimization algorithm, the parameters of the cosmetics in the cosmetic storage information are optimized to obtain cosmetic parameter information;

[0181] Based on makeup parameter information, generate and display makeup effects and makeup guidance methods;

[0182] Furthermore, a skincare guidance method is generated and displayed based on the skincare product matching results, the current user's identity information, the environmental information, and the lifestyle information.

[0183] In one example, the module in any of the above devices may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0184] For example, when modules in a device can be implemented via a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these modules can be integrated together as a system-on-a-chip (SOC).

[0185] In this application, various objects such as messages / information / devices / network elements / systems / apparatus / actions / operations / processes / concepts may be named. It is understood that these specific names do not constitute a limitation on the relevant objects. The names may be changed depending on the scenario, context, or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from their functions and technical effects embodied / performed in the technical solution.

[0186] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0187] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0188] Figure 3 This is a structural block diagram of an electronic device 300 according to an embodiment of this application.

[0189] like Figure 3As shown, the electronic device 300 includes a processor 301 and a memory 302, and may further include one or more of an information input / output (I / O) interface 303, a communication component 304, and a communication bus 305.

[0190] The processor 301 controls the overall operation of the makeup mirror 300 to complete all or part of the steps in the AI-based multimodal perception-based beauty and skincare guidance method described above. The memory 302 stores various types of data to support the operation of the makeup mirror 300. This data may include, for example, instructions for any application or method operating on the makeup mirror 300, as well as application-related data. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as one or more of Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0191] I / O interface 303 provides an interface between processor 301 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 304 is used to test wired or wireless communication between the cosmetic mirror 300 and other devices. Wireless communication includes Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof. Therefore, the corresponding communication component 304 may include a Wi-Fi component, a Bluetooth component, and an NFC component.

[0192] The communication bus 305 may include a path for transmitting information between the aforementioned components. The communication bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 305 may be divided into an address bus, a data bus, a control bus, etc.

[0193] The electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the AI-based multimodal perception-based beauty and skincare guidance method given in the above embodiments.

[0194] The following describes the computer-readable storage medium provided in the embodiments of this application. The computer-readable storage medium described below can be referred to in correspondence with the AI-based multimodal perception-based beauty and skincare guidance method described above.

[0195] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described AI-based multimodal perception-based beauty and skincare guidance method.

[0196] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0197] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0198] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions claimed in this application.

Claims

1. A beauty and skincare guidance method based on AI multimodal perception, characterized in that, include: The system acquires the current user's image information, interaction information, and environmental information, and identifies the current user's identity based on the image information. The user identity includes the user's age, occupation, basic skin information, and preference information. The preference information includes dietary preferences and makeup preferences. The interaction information represents the data generated when the current user interacts with the makeup mirror. The environmental information includes humidity and light. Based on the current user's identity information, obtain the current user's lifestyle information, which includes sleep time, exercise information, and endocrine cycle change information. Based on the interaction information, environmental information, and lifestyle information, scene status information is determined, and target beauty recommendation information is generated based on the scene status information. The scene status information includes a scene state and a sceneless state. The scene state includes a scene type and the scene corresponding to the scene type. The target beauty recommendation information is matched with the preset cosmetics storage information and the preset skin care product storage information to obtain cosmetics matching results and skin care product matching results. Based on the cosmetic matching results and the preset optimization algorithm, the parameters of the cosmetics in the cosmetic storage information are optimized to obtain cosmetic parameter information; Based on makeup parameter information, generate and display makeup effects and makeup guidance methods; Furthermore, a skincare guidance method is generated and displayed based on the skincare product matching results, the current user's identity information, the environmental information, and the lifestyle information; The scenario types include a first scenario and a second scenario. The first scenario represents a scenario defined by user intent or a specific event, and the second scenario represents a potential demand scenario triggered by the environment perceived by the makeup mirror in conjunction with the user's physiological state. The step of determining scene state information based on the interaction information, environmental information, and lifestyle habit information, and generating target beauty recommendation information based on the scene state information, includes: Based on natural language processing technology, keywords are extracted from the interactive information to obtain keyword information; Based on the environmental information, the UV index level and dryness level are determined; Based on the aforementioned lifestyle information, the physiological state level is determined; Based on the keyword information, determine whether a first scenario exists in the interaction information, and based on the UV index level, dryness level, and physiological state level, determine whether a second scenario exists. If a first scenario or a second scenario exists, the scenario status information is determined to be in the state of having a scenario. If the first or second scenario does not exist, the scenario status information is determined to be "no scenario". When the scene status information is a scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, historical beauty and skincare information corresponding to the user identity, scene type, scene and hot topic information corresponding to the scene type. The hot topic information represents real-time popular trends, seasonal demands and sudden events related to beauty and skincare. When the scene status information is no scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, and the historical beauty and skincare information corresponding to the user identity.

2. The beauty and skincare guidance method based on AI multimodal perception according to claim 1, characterized in that, Based on the current user's identity information, the system obtains the current user's lifestyle information, which includes sleep duration, exercise information, and endocrine cycle change information, including: Based on the current user's identity information, the wearable device associated with the identity information and the basic lifestyle habit profile associated with the user's identity are determined. The basic lifestyle habit profile includes historical data on the user's self-reported sleep cycle, exercise frequency, and manually entered historical data on endocrine cycles. The wearable device retrieves the current user's current lifestyle profile. The basic lifestyle habit profile, the current lifestyle habit profile, and quantifiable indicators are fused from multiple data sources, and the current user's lifestyle habit information is generated through a preset weight allocation algorithm.

3. The beauty and skincare guidance method based on AI multimodal perception according to claim 1, characterized in that, When the scene status information indicates a scene exists, the process of generating target beauty recommendation information based on the current user's image information, user identity information, historical beauty and skincare information corresponding to the user identity, scene type, and scene and hotspot information corresponding to the scene type includes: Based on the current user's image information and a preset deep learning facial analysis model, the real-time multidimensional skin texture features of the current user are determined. The real-time multidimensional skin texture features include skin color value, pore size, wrinkle depth, and the distribution and severity of oil and blemishes. Based on the scene type and the scene corresponding to the scene type, scene demand feature information is determined from the scene demand mapping relationship. The scene demand feature information includes makeup longevity index, waterproof index, concealing index and style orientation index. Based on trend information, the popular features of makeup styles are determined, including popular color schemes and the popularity of makeup techniques. Based on the current user's real-time multidimensional skin features, preference information, user age, scenario requirements, makeup trends, and a preset makeup scheme generation model, multiple candidate makeup schemes are generated. The makeup scheme generation model is a deep learning model trained on user skin data, makeup effect data, and user feedback data. Obtain historical recommendation data for each candidate beauty scheme, and based on the historical recommendation data for each candidate beauty scheme, determine the user feedback rating and selection frequency for each candidate beauty scheme; Based on user feedback ratings and selection frequency for each candidate beauty solution, the first priority of each candidate beauty solution is determined. Based on the scenario demand characteristics, the second priority of each candidate beauty solution is determined in multiple dimensions, including makeup longevity, water resistance, coverage and style orientation. Based on the second priority of each candidate beauty solution in multiple dimensions and the real-time multi-dimensional skin texture feature information, the third priority of each candidate beauty solution is determined. Based on the user's historical beauty and skincare information, the fourth priority of each candidate beauty solution is determined. Based on the makeup difficulty of each candidate makeup scheme and the current user's makeup skill level, multiple matching values ​​are determined. Based on multiple matching values, the fifth priority of each candidate beauty solution is determined; Based on the first, third, fourth, and fifth priorities of each candidate beauty solution, the target beauty recommendation information is determined.

4. The beauty and skincare guidance method based on AI multimodal perception according to claim 3, characterized in that, When the scene status information is "no scene," the step of generating target beauty recommendation information based on the current user's image information, user identity information, and the user's corresponding historical beauty and skincare information includes: Based on environmental information, calculate the stress index of the environment on the skin and determine the target environmental impact factors; Based on the product usage frequency and satisfaction rating in the historical beauty and skincare information corresponding to the user's identity, a user daily preference vector is generated. The real-time skin condition assessment results, user identity information, environmental stress index and target environmental impact factors, and user daily preference vector are input into the daily beauty recommendation model to generate a basic daily beauty and skincare plan; the daily beauty recommendation model is a generative model trained based on user behavior data without specific scenario constraints. Based on the time-limited usage patterns in the historical beauty and skincare information corresponding to the user's identity, the basic daily beauty and skincare routine is adjusted to adapt to the time, generating target beauty recommendation information.

5. The beauty and skincare guidance method based on AI multimodal perception according to claim 1, characterized in that, Before matching the target beauty recommendations with the preset cosmetics and skincare products stored in the database to obtain the cosmetics matching results and skincare product matching results, the process also includes: Based on the target beauty recommendation information, multiple first beauty recommendation information that have a similarity threshold greater than the target beauty recommendation information are matched from the recommendation information database; Multiple first beauty recommendation information are sorted according to the similarity threshold with the target beauty recommendation information to generate a candidate beauty recommendation information set; Based on the target beauty recommendation information, the alternative beauty recommendation information set, and the preset augmented reality technology, the makeup effect corresponding to the target beauty recommendation information and the makeup effect corresponding to each of the first beauty recommendation information in the alternative beauty recommendation information set are displayed on the makeup mirror. In response to the current user's trigger action on makeup, operation information is obtained, including makeup effect selection operation, makeup intensity component adjustment operation, makeup position component adjustment operation, and style component selection operation; Based on the operation information, target beauty recommendation information, and multiple first beauty recommendation information, new target beauty recommendation information is generated.

6. The beauty and skincare guidance method based on AI multimodal perception according to claim 1, characterized in that, The cosmetic matching results include a list of cosmetics and basic attribute parameters of the cosmetics. These basic attribute parameters include coverage, spreadability, wear time, and ingredient characteristics. Based on the cosmetic matching results and a preset optimization algorithm, the parameters of the cosmetics in the stored cosmetic information are optimized to obtain cosmetic parameter information, including: Based on the user's makeup skill level and historical operation preferences in the user's identity information, determine the constraints of makeup techniques; Based on the real-time image information of the current user, facial partition features are determined. The optimization objective functions are naturalness of makeup effect, blemish coverage, makeup durability, and resource consumption. The optimization objective functions are solved by a multi-objective optimization algorithm, which is constrained by the basic attribute parameters of cosmetics, facial partition feature data, and personalized makeup techniques. The makeup parameter information for each facial region is obtained. The makeup parameter information includes product dosage parameters, application range parameters, concentration ratio parameters, technique intensity parameters, and time series parameters. The time series parameters represent the optimal time interval and duration of each makeup step. Based on the makeup parameter information of each facial region, a set of makeup parameter information is generated, and the feasibility of each parameter is verified and conflict detection is performed. The verified makeup parameter information is bound to the corresponding cosmetic list to obtain the makeup parameter information.

7. A beauty and skincare guidance system based on AI multimodal perception, characterized in that, include: The image acquisition module is used to acquire the image information of the current user and send the image information of the current user to the generation module; The environmental perception module is used to detect the brightness and humidity of the environment around the makeup mirror and send the brightness and humidity around the makeup mirror to the generation module. The interaction module is used to collect the current user's interaction information and send the interaction information to the generation module; The generation module is used to acquire the current user's image information, interaction information, and environmental information, and to identify the user's identity based on the image information. The user's identity includes the user's age, occupation, basic skin information, and preference information. The preference information includes dietary preferences and makeup preferences. The interaction information represents the data generated when the current user interacts with the makeup mirror. The environmental information includes humidity and light. Based on the current user's identity information, obtain the current user's lifestyle information, which includes sleep time, exercise information, and endocrine cycle change information. Based on the interaction information, environmental information, and lifestyle information, scene status information is determined, including scene type and scene. Based on the scene state information, target beauty product recommendation information is generated; The target beauty recommendation information is matched with the preset cosmetics storage information and the preset skin care product storage information to obtain cosmetics matching results and skin care product matching results. Based on the cosmetic matching results and the preset optimization algorithm, the parameters of the cosmetics in the cosmetic storage information are optimized to obtain cosmetic parameter information; Based on makeup parameter information, generate and display makeup effects and makeup guidance methods; Furthermore, a skincare guidance method is generated and displayed based on the skincare product matching results, the current user's identity information, the environmental information, and the lifestyle information; The scenario types include a first scenario and a second scenario. The first scenario represents a scenario defined by user intent or a specific event, and the second scenario represents a potential demand scenario triggered by the environment perceived by the makeup mirror in conjunction with the user's physiological state. The generation module is specifically used for: Based on natural language processing technology, keywords are extracted from the interactive information to obtain keyword information; Based on the environmental information, the UV index level and dryness level are determined; Based on the aforementioned lifestyle information, the physiological state level is determined; Based on the keyword information, determine whether a first scenario exists in the interaction information, and based on the UV index level, dryness level, and physiological state level, determine whether a second scenario exists. If a first scenario or a second scenario exists, the scenario status information is determined to be in the state of having a scenario. If the first or second scenario does not exist, the scenario status information is determined to be "no scenario". When the scene status information is a scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, historical beauty and skincare information corresponding to the user identity, scene type, scene and hot topic information corresponding to the scene type. The hot topic information represents real-time popular trends, seasonal demands and sudden events related to beauty and skincare. When the scene status information is no scene status, target beauty recommendation information is generated based on the current user's image information, user identity information, and the historical beauty and skincare information corresponding to the user identity.

8. An electronic device, characterized in that, Includes a processor, which is coupled to a memory; The processor is configured to execute a computer program stored in the memory, such that the cosmetic mirror performs the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Includes a computer program or instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1-6.

Images