Artificial intelligence-based systems and methods for providing personalized skin product recommendations

Abstract

Artificial intelligence-based systems and methods for providing personalized skin product recommendations are described. Natural language data of a user is received using a dialog engine such as a generative pre-trained transformer AI model. The natural language data is input into a synthetic image generation model to generate a digital twin image depicting a skin condition. A product recommendation model inputs the digital twin image and / or phenotyping and / or demographic classification of the user and outputs a skin care product recommendation for the user. An image simulation model inputs the product recommendation, the digital twin image of the user, and phenotyping and / or demographic classification to output a simulated image of the user with one or more graphical enhancements. The simulated image can be displayed to the user on a display screen along with natural language data describing the simulated image, the simulated image depicting how the skin is expected to appear after treatment with the skin care product.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202480072494.6 (22) Application Date 2024.11.19 (30) Priority Data 18 / 515,427 2023.11.21 US (85) PCT International Application Entering National Phase Date 2026.05.14 (86) PCT International Application Application Data PCT / US2024 / 056468 2024.11.19 (87) PCT International Application Publication Data WO2025 / 111239 EN 2025.05.30 (71) Applicant Procter & Gamble Company Address Ohio, USA (72) Inventor D.M.M.B. Disanayak (74) Patent Agency Shanghai Patent & Trademark Agency Co., Ltd. 31100 Patent Attorney Gao Jian Huang Songquan (51) Int.Cl. G06Q 30 / 0601 (2006.01) G06N 3 / 045 (2006.01) G16H 50 / 20 (2006.01) (54) Invention Title: Artificial Intelligence-Based System and Method for Providing Personalized Skin Product Recommendations (57) Abstract: This paper describes an artificial intelligence-based system and method for providing personalized skin product recommendations. A dialogue engine, such as a generative pre-trained transformer AI model, is used to receive natural language data from a user. This natural language data is input into a synthetic image generation model to generate a digital twin image depicting the user's skin condition. A product recommendation model takes the user's digital twin image and / or phenotypic and / or demographic classification as input and outputs skin care product recommendations for the user. An image simulation model takes the product recommendations, the user's digital twin image, and the phenotypic and / or demographic classification as input and outputs a simulated image of the user with one or more graphic enhancements. The simulated image, along with natural language data describing the simulated image, can be displayed to the user on a screen, depicting the expected appearance of the skin after treatment with the skin care product. Claims 3 pages, Description 33 pages, Drawings 8 pages, CN 122207052 A 2026.06.12 CN 1 22 20 70 52 A 1. An artificial intelligence (AI) based system configured to provide personalized skin product recommendations, the AI ​​based system comprising: one or more processors; one or more memories communicatively coupled to the one or more processors; an application (app) stored in the one or more memories and including computational instructions configured to execute on the one or more processors; a natural language model,The natural language model is accessible by the application and trained using the natural language data and demographic data of the corresponding user. The natural language model is configured to output one or more phenotypic classifications corresponding to one or more phenotypic traits of the corresponding user, and is further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user. A synthetic image generation model is accessible by the application and trained on multiple images corresponding to the one or more phenotypic classifications and the one or more demographic classifications output by the natural language model. The synthetic image generation model is configured to output a digital twin image of the corresponding user based on the multiple images corresponding to the one or more phenotypic classifications and the one or more demographic classifications of the corresponding user as input. A product recommendation model is accessible by the application and trained on the digital twin image of the corresponding user output by the synthetic image generation model. The product recommendation model is also trained on the one or more phenotypic classifications and the one or more demographic classifications. The product recommendation model is configured to output product recommendations for the corresponding user based on the digital twin image, the one or more phenotypic classifications, and the one or more demographic classifications as input. An image simulation model, accessible by the application and trained on the digital twin image output by the synthetic image generation model, and also trained on the product recommendation output by the product recommendation model, which is further trained on one or more phenotypic classifications and one or more demographic classifications, is configured to generate a simulated image based on the digital twin image, the simulated image having one or more graphic enhancements based on the product recommendation; and wherein the computational instructions of the application, when executed by the one or more processors, cause the one or more processors to: (a) receive user-specific natural language data of a user, the natural language data defining user-specific phenotypic information and user-specific demographic data of the user; (b) input the user-specific natural language data into the natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user; and (c) input the one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into the synthetic image generation model, wherein the synthetic image generation model generates a user-specific digital twin image of the user.(d) Inputting the user-specific digital twin image of the user, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user into the product recommendation model, wherein the product recommendation model generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem identified in the user-specific digital twin image or the one or more user-specific phenotypic classifications, Claim 1 / 3 page 2 CN 122207052 A (e) Inputting the user-specific digital twin image, the user-specific product recommendations, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user into the image simulation model, wherein the image simulation model generates a user-specific simulated image based on the user-specific digital twin image and the user-specific product recommendations, the user-specific simulated image graphically depicting the user-specific digital twin image graphically enhanced based on the user-specific product recommendations, and (f) Outputting natural language data describing the user-specific product recommendations to the user; and (g) Displaying the user-specific simulated image on a display screen. 2. The AI-based system of claim 1, wherein the one or more phenotypic or demographic classifications include one or more of the following: (a) gender type classification; (b) age type classification; (c) ethnic type classification; or (d) skin distress type classification. 3. The AI-based system of any preceding claim, wherein the natural language model comprises a large language model. 4. The AI-based system of any preceding claim, further comprising an imaging model trained on one or more corresponding images of the corresponding user, wherein the imaging model is further configured to output the one or more phenotypic classifications corresponding to one or more phenotypes of the corresponding user. 5. The AI-based system of any preceding claim, wherein the output of the natural language model and the output of the imaging model are combined to determine the one or more phenotypes of the corresponding user. 6. The AI-based system of any preceding claim, wherein the synthetic image generation model comprises an AI model configured to generate photorealistic images, wherein the digital twin images comprise photorealistic images, and wherein the user-specific digital twin images comprise photorealistic images of the user. 7. The AI-based system according to any preceding claim, wherein the product recommendation model comprises a model trained to determine the efficacy of one or more skin products corresponding to the one or more phenotypes of the corresponding user. 8. The AI-based system according to any preceding claim,The image simulation model includes a model trained to determine the influence of one or more product attributes corresponding to the product recommendation. 9. The AI-based system according to any preceding claim, further comprising: providing the user-specific digital twin image to the user; receiving additional user-specific natural language data from the user to manipulate the user-specific digital twin image; and updating the user-specific digital twin image based on the additional user-specific natural language data. 10. The AI-based system according to any preceding claim, wherein the user-specific simulation image is presented on the display screen in real-time or near real-time during or after receiving the user's natural language data. 11. The AI-based system according to any preceding claim, wherein the user-specific product recommendation includes a product recommendation for a manufactured product. 12. The AI-based system of claim 11, wherein the user-specific product recommendation is displayed on the display screen, accompanied by instructions to treat at least one feature identifiable in pixel data including a skin region of the user using the manufactured product. 13. The AI-based system of claims 11 to 12, wherein the computing instructions further cause the one or more processors to: initiate the delivery of the manufactured product to the user based on the user-specific product recommendation. 14. The AI-based system of any preceding claim, wherein at least one of the one or more processors comprises a processor of a mobile device. 15. The AI-based system of any preceding claim, wherein the one or more processors comprises a server processor of a server, wherein the server is communicatively coupled to the computing device via a computer network, and wherein the application comprises a server application portion configured to execute on the one or more processors of the server and a computing device application portion configured to execute on the one or more processors of the computing device, the server application portion being configured to communicate with the computing device application portion, wherein the server application portion is configured to implement one or more instructions of instructions a to g according to claim 1. Claims 3 / 3 Page 4 CN 122207052 A Artificial Intelligence-Based System and Method for Providing Personalized Skin Product Recommendations Technical Field

[0001] This disclosure relates generally to artificial intelligence (AI)-based systems and methods, and more particularly to AI-based systems and methods for providing personalized skin product recommendations. Background Art

[0002] Human skin is unique to a given individual,Skin changes can be based on factors such as race, age, and sun exposure. Individuals may have one or more skin problems, conditions, or concerns, including but not limited to, pigmentation, wrinkles and / or fine lines, acne, enlarged pores, sagging and / or loss of elasticity, uneven texture, thinning skin, dryness, oiliness, sensitivity, uneven skin tone, eczema, and / or dermatitis. This uniqueness and diversity of skin makes it difficult for the corresponding individual to identify products that can treat or otherwise alleviate their unique combination of concerns. Furthermore, even after identifying a product, it may be difficult to determine what differences or effects such a product might have on a given individual's skin.

[0003] Users may encounter problems when attempting to identify the causes of these skin conditions, problems, or concerns themselves. This can lead to problems involving incorrect identification. Incorrect identification, in turn, can lead to ineffective treatment. For example, a product designed to treat one skin condition, problem, or concern may be incorrectly applied in an attempt to treat a different skin condition, problem, or concern, which may be at least ineffective and potentially dangerous (e.g., application of a prescription drug that is not intended for use).

[0004] These problems can be exacerbated by the complexity of skin types, especially when considered across different users (each of whom may be associated with different demographic characteristics, ethnicities, and / or ethnic groups). This creates problems in the analysis and treatment of a wide range of human skin conditions and characteristics. For example, existing methods, including trials of personal consumer products, can be time-consuming or error-prone (and potentially negative). Furthermore, a user may attempt empirical experiments with various products or technologies without achieving satisfactory results and / or resulting in possible negative side effects that could affect the health of his or her skin or its other visual appearance.

[0005] Further, to obtain certain skin care products, a user may need to visit a skin care specialist, such as a dermatologist. However, this approach can be problematic, time-consuming, and unavailable if the user cannot access such a specialist's services outside of a given healthcare coverage plan. Additionally, various conventional computer-related techniques are known to be used to identify specific skin problems, but such conventional computer-related techniques cannot capture the specific needs of a given user and cannot address specific skin concerns in a manner similar to a user's in-person visit to a skin specialist, such as a dermatologist. Furthermore, such conventional computer-related technologies often require the public disclosure and / or transmission of personally identifiable information (PII), such as sensitive health information and / or images of users, which may require close-up images of the user's skin. The public disclosure or transmission of such PII may pose security risks to the underlying computing system, where such information may be at risk of being intercepted and / or exploited by malicious actors.

[0006] For the foregoing reasons, there is a need for AI-based systems and methods for providing personalized skin product recommendations.As described herein. Summary of the Invention

[0007] Generally speaking, as described herein, AI-based systems and methods for providing personalized skin product recommendations. Specification 1 / 33 pages 5 CN 122207052 A Such artificial intelligence-based systems provide artificial intelligence (AI)-based solutions that include large language models (LLMs) for engaging in dialogue with a specific user, such as conversational generative pre-trained transformers (GPTs), i.e., national languages ​​or dialogue engines, which can be used to overcome specific user problems arising from the difficulty in identifying and processing various endogenous and / or exogenous factors or attributes that affect human skin health. For example, this may include generating or otherwise determining user-specific recommendations for skin products that have therapeutic and / or dermatological efficacy for predicted user-specific skin diseases and / or other skin conditions.

[0008] The AI-based systems and methods described herein allow users to engage in natural language dialogues, similar to how users talk to skin experts, such as dermatologists. In various aspects, AI-based systems and methods utilize multiple AI models to provide natural language processing (NLP), user-specific skin problem identification, and / or user-specific and user-identified skin problem product recommendations. For example, a first AI model may include a dialogue engine, where the AI ​​model communicates with the user to ask questions and receive information (e.g., this may be included in an iterative implementation) to receive sufficient information for use as input to a second AI model. For example, the first AI model may initiate a dialogue with the user via a GPT-type interface. The first AI model may pose questions that a skin specialist (e.g., a dermatologist) would ask the user, where the first AI model is trained on dermatologist-specific information, including, for example, dermatologist-specific information, including text-based information and / or voice information specific to the field of dermatology. Natural language data (e.g., the user's text and / or voice data) may be received from the user and fed into a second AI model, which is trained on such data and configured to output user-specific demographic classifications and / or define user-specific demographic classifications (e.g., the user's gender, age, skin color, severity of skin condition, etc.). For example, one type of classification may include the severity of acne, and such information may be provided to a second AI model as natural language data received from the user regarding a severity range (e.g., mild, moderate, or severe acne). Such a range may be an ordinal range based on the class itself, and / or may be converted into a numerical scale (e.g., 1-10) for use in the training and / or output of the second AI model.

[0009] Then,The third AI model can use such user-specific demographic classifications and / or user-specific demographic classifications as input to generate synthetic images representing the user. Such synthetic images may include or be based on digital twin images generated for a specific user, representing that user, for example, having the same skin color, hair color, etc. Specific issues identified for the user (e.g., specific skin problems) can then be applied (e.g., overlaid or masked) to the digital twin image. In this way, skin problems identified by natural language data provided by the user can be used to generate images depicting the skin problems in user-specific digital twin images. In some aspects, the AI ​​model can be trained using pixel data from a potential pool of tens of thousands (or more) images depicting the skin or skin regions of a corresponding individual with various skin problems. For example, these images may include sample images of the corresponding user's skin, including pixels or pixel data indicating spots (e.g., hemoglobin and / or melanin-related spots) or other attributes / conditions of human skin. The AI ​​model can generate or update the digital twin image to include, for example, pixel data associated with skin problems identified for the user. Importantly, user-specific digital twin images can be generated for the user without requiring the user to provide an actual image of himself or her. Conversely, natural language data (e.g., text / voice information) is received from the user via GPT-type prompts, enabling the digital twin image to be generated solely from the natural language data.

[0010] The digital twin image may be provided to a fourth AI model, which is trained to recommend products for addressing skin problems or other conditions (e.g., acne). In some aspects, the fourth AI model may generate a user-specific simulated image based on a user-specific digital twin image and user-specific product recommendations. The user-specific simulated image may graphically depict the user-specific digital twin image enhanced based on user-specific product recommendations. In various aspects, the graphically enhanced simulated image may depict the predicted effect and / or efficacy of the recommended skin product on the digital twin image. That is, such predicted effect and / or efficacy may include an estimated skin depiction superimposed or otherwise presented on or as part of the digital twin image to show the effect of the product on the skin. For example, in various specific implementations, such presentation may include graphical representations, overlays, annotations, etc., for addressing skin issues and / or features in pixel data. In various aspects, user-specific digital twin images and / or analog-enhanced images may be output or displayed on a display screen, for example, for user viewing. Additionally or alternatively, natural language data may be output to the user, for example, via a display screen, to describe user-specific product recommendations.

[0011] More specifically, as described herein,A system based on artificial intelligence (AI) configured to provide personalized skin product recommendations is disclosed. The AI-based system may include one or more processors and one or more memories communicatively coupled to the processors. The AI-based system may also include an application (app) stored in the one or more memories and including computational instructions configured to execute on the one or more processors. The AI-based system may also include a natural language model accessible by the application and trained using natural language data and demographic data of a corresponding user. The natural language model may be configured to output one or more phenotypic classifications corresponding to one or more phenotypes of the corresponding user. The natural language model may be further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user. The AI-based system may also include a synthetic image generation model accessible by the application and trained on multiple images corresponding to the one or more phenotypic classifications and one or more demographic classifications output by the natural language model. The synthetic image generation model may be configured to output a digital twin image of the corresponding user based on multiple images corresponding to the one or more phenotypic classifications and one or more demographic classifications of the corresponding user as input. The AI-based system may also include a product recommendation model, accessible by the application and trained on a digital twin image of the corresponding user, output by a synthetic image generation model. The product recommendation model may also be trained on one or more phenotypic classifications and one or more demographic classifications. The product recommendation model can be configured to output product recommendations to the corresponding user based on the digital twin image, one or more phenotypic classifications, and one or more demographic classifications as input. The AI-based system may also include an image simulation model, accessible by the application and trained on a digital twin image output by a synthetic image generation model, and also trained on product recommendations output by the product recommendation model. The product recommendation model may also be trained on one or more phenotypic classifications and one or more demographic classifications. The image simulation model can be configured to generate simulated images based on the digital twin image, the simulated images having one or more graphic enhancements based on product recommendations.

[0012] When executed by one or more processors, the computational instructions of the application cause one or more processors to: (a) receive user-specific natural language data defining user-specific phenotypic information and user-specific demographic data, and (b) input the user-specific natural language data into a natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user.(c) Inputting one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into a synthetic image generation model, wherein the synthetic image generation model generates a user-specific digital twin image of the user; (d) Inputting the user-specific digital twin image of the user, one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into a product recommendation model, wherein the product recommendation model generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem identified in the user-specific digital twin image or one or more user-specific phenotypic classifications; (e) Inputting the user-specific digital twin image, user-specific product recommendations, one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into an image simulation model, wherein the image simulation model generates a user-specific simulated image based on the user-specific digital twin image and the user-specific product recommendations, the user-specific simulated image graphically depicting the user-specific digital twin image graphically enhanced based on the user-specific product recommendations; and (f) Outputting natural language data describing the user-specific product recommendations to the user on page 3 / 33 of the specification (CN 122207052 A); and (g) Displaying the user-specific simulated image on a display screen.

[0013] In addition, as described herein, an artificial intelligence (AI)-based method for providing personalized skin product recommendations is disclosed. The AI-based approach includes a multi-AI model training phase, comprising: training a natural language model by one or more processors using natural language data and demographic data of a corresponding user; the natural language model being configured to output one or more phenotypic classifications corresponding to one or more phenotypes of the corresponding user, and further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user; training a synthetic image generation model by one or more processors on multiple images corresponding to the one or more phenotypic classifications and one or more demographic classifications output by the natural language model; the synthetic image generation model being configured to output a digital twin image of the corresponding user based on multiple images corresponding to the one or more phenotypic classifications and one or more demographic classifications of the corresponding user as input; training a product recommendation model by one or more processors on the digital twin image of the corresponding user output by the synthetic image generation model; the product recommendation model being trained on one or more phenotypic classifications and one or more demographic classifications; the product recommendation model being configured to output product recommendations for the corresponding user based on the digital twin image, one or more phenotypic classifications, and one or more demographic classifications as input; and training an image simulation model by one or more processors on the digital twin image output by the synthetic image generation model.Furthermore, the image simulation model is also trained on product recommendations output by the product recommendation model, which is also trained on one or more phenotypic classifications and one or more demographic classifications. The image simulation model is configured to generate simulated images based on digital twin images, which have one or more graphical enhancements based on product recommendations. Each of the natural language model, the synthetic image generation model, the product recommendation model, and the image simulation model is stored in one or more memories accessible by an application (app), which includes computational instructions stored in the one or more memories, which are communicatively coupled to the one or more processors.

[0014] The AI-based approach also includes a multi-AI model execution phase, which comprises: (a) the application receiving user-specific natural language data defining user-specific phenotypic information and user-specific demographic data; (b) the application inputting the user-specific natural language data into a natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user; (c) the application inputting one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into a synthetic image generation model, wherein the synthetic image generation model generates a user-specific digital twin image of the user; and (d) the application combining the user-specific digital twin image, one or more phenotypic classifications, and one or more demographic classifications of the user. The input is fed into a product recommendation model, which generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem of identification in a user-specific digital twin image or one or more user-specific phenotypic classifications; (e) the application inputs the user-specific digital twin image, the user-specific product recommendations, one or more phenotypic classifications of the user, and one or more demographic classifications of the user into an image simulation model, wherein the image simulation model generates a user-specific simulated image based on the user-specific digital twin image and the user-specific product recommendations, the user-specific simulated image being graphically depicted as a user-specific digital twin image graphically enhanced based on the user-specific product recommendations; (f) the application outputs natural language data describing the user-specific product recommendations to the user; and (g) the application displays the user-specific simulated image on a display screen.

[0015] Furthermore, as described herein, a tangible non-transitory computer-readable medium is disclosed, the tangible non-transitory specification 4 / 33 pages 8 CN 122207052 A computer-readable medium storing instructions for providing personalized skin product recommendations. When executed by one or more processors, this instruction enables one or more processors to perform multiple AI model training phases.The multi-AI model training phase includes: training a natural language model by one or more processors using natural language data and demographic data of the corresponding user; the natural language model being configured to output one or more phenotypic classifications corresponding to one or more phenotypic attributes of the corresponding user, and the natural language model being further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user; training a synthetic image generation model by one or more processors on multiple images corresponding to the one or more phenotypic classifications and one or more demographic classifications output by the natural language model; the synthetic image generation model being configured to output a digital twin image of the corresponding user based on multiple images corresponding to the one or more phenotypic classifications and one or more demographic classifications of the corresponding user as input; and training a product recommendation model by one or more processors on the digital twin image of the corresponding user output by the synthetic image generation model, the product recommendation model also being trained on the one or more phenotypic classifications and one or more demographic attributes of the corresponding user. The product recommendation model is trained on statistical classification and is configured to output product recommendations to the corresponding user based on digital twin images, one or more phenotypic classifications, and one or more demographic classifications as input. An image simulation model is trained by one or more processors on the digital twin images output by a synthetic image generation model, and this image simulation model is also trained on product recommendations output by the product recommendation model, which is also trained on one or more phenotypic classifications and one or more demographic classifications. The image simulation model is configured to generate simulated images based on the digital twin images, which have one or more graphical enhancements based on product recommendations. Each of the natural language model, synthetic image generation model, product recommendation model, and image simulation model is stored in one or more memories accessible by an application (app), which includes computational instructions stored in the one or more memories, which are communicatively coupled to the one or more processors.

[0016] When executed by one or more processors, this instruction can also enable one or more processors to implement a multi-AI model execution phase, which includes: (a) the application receiving user-specific natural language data, the natural language data defining user-specific phenotypic information and user-specific demographic data; (b) the application inputting the user-specific natural language data into a natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user; and (c) the application inputting one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into a synthetic image generation model.(d) The application inputs the user-specific digital twin image, one or more phenotypic classifications of the user, and one or more demographic classifications of the user into a product recommendation model, wherein the product recommendation model generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem identified in the user-specific digital twin image or one or more user-specific phenotypic classifications; (e) The application inputs the user-specific digital twin image, user-specific product recommendations, one or more phenotypic classifications of the user, and one or more demographic classifications of the user into an image simulation model, wherein the image simulation model generates a user-specific simulated image based on the user-specific digital twin image and the user-specific product recommendations, the user-specific simulated image being graphically depicted as a user-specific digital twin image graphically enhanced based on the user-specific product recommendations; (f) The application outputs natural language data describing the user-specific product recommendations to the user; and (g) The application displays the user-specific simulated image on a display screen.

[0017] Based on the foregoing and the disclosure herein, this disclosure includes improvements to computer functionality or other technologies, at least because this disclosure describes improvements to, for example, servers or other computing devices (e.g., user computer devices), wherein the intelligence or predictive capabilities of the server or computing device are enhanced by multiple AI models, which are trained on natural language processing (NLP) and dialogue with the user via a GPT-type interface, and are capable of generating and enhancing a digital twin image of the user, and / or associating and / or generating accurate predictions of real-world product recommendations based on user-specific natural language data. Multiple AI models executing on the server or computing device are capable of more accurately identifying one or more user problems based on natural language data and / or pixel data digital twin images, and can output user-specific product recommendations designed to solve the problem (e.g., a given skin problem). In other words, this disclosure describes functional improvements to the computer itself or "any other technology or technical field" because the server or user computing device is enhanced by multiple training images (e.g., tens of thousands of training images and associated pixel data as feature data) and / or natural language data, thereby enabling accurate prediction, detection, classification, or determination of user-specific problems based on pixel data of user-specific images with the same identified problem. This is an improvement over the prior art, at least because existing systems lack this predictive or classification functionality and, without providing user-specific images, cannot accurately analyze user-specific natural language data to output prediction results.

[0018] For similar reasons, this disclosure relates to improvements to other technologies or technical fields.At least because this disclosure describes or introduces improvements to computing devices in the fields of skin care and skin care products, wherein multiple AI models executed on imaging devices or computing devices improve the skin care field, chemical formulations and / or skin classification and their identification, and output predictive results by performing digital and / or artificial intelligence-based analysis on user-specific natural language processing (NLP) data to address problems that can be identified by NLP data as being related to at least a portion of a user's skin area.

[0019] Furthermore, this disclosure relates to improvements in other technical or technological fields, at least because this disclosure describes or introduces improvements to computing devices in the fields of skin care and / or skin care products, wherein trained AI models executed on imaging devices or computing devices improve underlying computing devices (e.g., servers and / or user computing devices), wherein such computing devices become more efficient through the configuration, tuning or adaptation of a given machine learning network architecture. For example, in some embodiments, computing resources can be reduced by abandoning the need to receive images from the user and reducing the required machine learning network architecture, thereby using fewer machine resources (e.g., processing cycles or memory storage). Additionally, this disclosure describes the generation of synthetic images and, at least in some aspects, reduces the depth, width, image size, or other image aspects of the synthetic images to meet the dimensionality requirements of machine learning. Such reduction frees up computational resources of the underlying computing system, thereby making it more efficient.

[0020] Furthermore, this disclosure relates to improvements in other technical or technological fields, at least because this disclosure describes or introduces improvements to computing devices in the field of security. In particular, when an AI model generates and / or outputs an image of a given user, it does not need to transmit or share any image of that user, i.e., it does not need to depict the user's personally identifiable information (PII). This specific implementation eliminates the need to transmit a user's private photographs over a computer network (where such images may be easily intercepted by third parties). Such features provide security improvements, i.e., where eliminating the transmission of PII (e.g., facial features) provides an improvement over previous systems because there is no need to transmit data over a network (e.g., the Internet), where data transmission in NLP is more secure and less data-intensive compared to photographic images of the user. Furthermore, in some aspects, digital twin images and / or analog-enhanced images may be preprocessed (e.g., cropped or otherwise modified) to define the extracted or depicted user skin regions. For example, the AI ​​model described herein can use a user's digital twin image and / or cropped or edited portions of a simulated augmented image. Therefore, the systems and methods described herein operate without requiring such basic information, providing improvements over previous systems, such as enhanced security. Furthermore, at least in some embodiments, the use of cropped images allows the underlying system to store and / or process images with smaller data sizes, resulting in improved overall performance of the underlying system.Because images with smaller data sizes require less storage memory and / or processing resources to store, process, and / or otherwise operate by the underlying computer system. Specification 6 / 33 pages 10 CN 122207052 A

[0021] Furthermore, this disclosure includes the application of certain claim elements using or by using a particular machine (e.g., a GPT interface) that can interact with or access a natural language model trained on NLP data to generate and enhance digital twin images using pixel data of a user-specific skin problem.

[0022] Furthermore, this disclosure includes specific features beyond those well-known, routine, and conventional activities in the art, or the addition of unconventional steps that limit the claims to a particular useful application, such as AI-based systems and methods for providing personalized skin product recommendations, as described herein.

[0023] The advantages will become more apparent to those skilled in the art from the following description of preferred embodiments shown and described by way of example. As will be appreciated, embodiments of the invention may have other and different embodiments, and their details can be modified in various aspects. Therefore, the accompanying drawings and descriptions should be considered substantially illustrative rather than limiting. Brief Description of the Drawings

[0024] The drawings described below depict various aspects of the systems and methods disclosed herein. It should be understood that each drawing depicts an embodiment of a specific aspect of the disclosed systems and methods, and each drawing is intended to be consistent with its possible embodiments. Furthermore, where possible, the following description refers to the reference numerals included in the following drawings, wherein features depicted in multiple drawings are designated with consistent reference numerals.

[0025] The arrangement currently discussed is shown in the drawings; however, it should be understood that this embodiment is not limited to the precise arrangement and tools shown, wherein: Figure 1 illustrates an example artificial intelligence (AI) based system according to various embodiments disclosed herein, configured to provide personalized skin product recommendations.

[0026] Figure 2A illustrates an example artificial intelligence (AI) based method for providing personalized skin product recommendations according to various embodiments disclosed herein.

[0027] Figure 2B also illustrates the example artificial intelligence (AI) based method of Figure 2A according to various embodiments disclosed herein.

[0028] Figure 3 illustrates example user interfaces according to various embodiments disclosed herein.The user interface illustrates a Generative Pre-trained Transformer (GPT) interface for capturing a user's natural language data.

[0029] Figure 4A illustrates a set of example digital twin images that can be used to train and / or implement artificial intelligence (AI) models according to various embodiments disclosed herein.

[0030] Figure 4B illustrates example digital twin images selected or generated for a given user according to various embodiments disclosed herein.

[0031] Figure 4C illustrates example images and their associated pixel data that can be used to train and / or implement artificial intelligence (AI) models according to various embodiments disclosed herein.

[0032] Figure 5A illustrates example images enhanced by image simulation models according to various embodiments disclosed herein.

[0033] Figure 5B illustrates another example image enhanced by image simulation models according to various embodiments disclosed herein.

[0034] Figure 6 illustrates an example user interface presented on a display screen of a user's computing device according to various embodiments disclosed herein.

[0035] The figures depict preferred embodiments for illustrative purposes only. Without departing from the principles described herein, as outlined on page 7 / 33 of the specification of the invention (CN 122207052 A), alternative embodiments of the systems and methods illustrated herein may be employed. Detailed Description

[0036] The disclosure herein provides a solution based on a multi-artificial intelligence (AI) model that enables a user to converse with a natural language model, thereby providing natural language data defining skin problems, conditions, or concerns, including but not limited to, for example, pigmentation, wrinkles and / or fine lines, acne, enlarged pores, sagging and / or loss of elasticity, uneven texture, thinning skin, dryness, oiliness, sensitivity, uneven skin tone, eczema, and / or dermatitis. In response, the multi-AI model-based solution can provide outputs that can be used to overcome user-specific skin problems. For example, such outputs may include generating or otherwise determining user-specific recommendations for skin products that have therapeutic and / or dermatological efficacy for predicted user-specific skin conditions and / or other skin conditions.

[0037] As an example, a user may have specific skin concerns. Users can use natural language to describe or otherwise provide their specific concerns and other information (e.g., demographic and / or phenotypic information) to AI-based systems and methods through dialogue. The AI-based systems and methods can then identify or capture natural language data from such dialogues. In one example, a specific skin concern might involve acne. Acne typically occurs when pores are clogged with oil and dead skin cells. Acne can cause skin problems or conditions such as whiteheads, blackheads, or papules. Such skin conditions can be treated with products containing ingredients such as retinoic acid, adapalene, and benzoyl peroxide. Through dialogue,Users can describe these features to AI-based systems and methods, which identify and recommend acne skin care products.

[0038] As another example, users can identify wrinkles, which may be caused by natural aging and / or environmental factors such as sun exposure, pollutants, smoking, etc. Wrinkles can refer to skin with a high degree of skin laxity. Wrinkles can be treated with products containing ingredients such as glycolic acid, retinol, vitamin C, and / or hyaluronic acid. Through dialogue, users can describe these features to AI-based systems and methods, which identify and recommend wrinkle-related skin care products with relevant active ingredients for treating wrinkles.

[0039] As another example, users can identify pigmentation spots (e.g., hemoglobin and / or melanin). In some cases, such spots can be effectively treated at low pH with a combination of hydroxycinnamic acid (HCA) and niacinamide, which can reduce melanin and hemoglobin in persistent spots or marks. Through dialogue, users can describe these characteristics to AI-based systems and methods, which identify and recommend pigmentation-therapeutic skincare products having relevant active ingredients for treating skin pigmentation or spots (e.g., hemoglobin and / or melanin).

[0040] It should be understood that additional and / or different skin concerns and / or relevant products having active ingredients for treating skin concerns can be identified and recommended separately.

[0041] Once a given skin concern is identified via natural language data, a synthetic image (e.g., a digital twin image) can be generated for the user, where the synthetic image includes the user's skin concern. The AI-based systems and methods can then determine which product can treat the skin concern, whereby the AI-based systems and methods can subsequently generate a graphically enhanced simulated image based on the digital twin image and its predicted appearance after applying the recommended product to the specific skin concern depicted therein. The user will then be able to determine whether to purchase the product for a real-world application to treat his or her skin.

[0042] The disclosure herein provides additional details describing AI-based systems and methods for identifying and / or classifying such skin conditions, problems, or concerns, thereby allowing for effective treatments, such as product and / or composition recommendations, selection, and use.

[0043] Figure 1 illustrates an example artificial intelligence (AI)-based system 100 according to various embodiments disclosed herein, configured to provide personalized skin product recommendations. In the example embodiment of Figure 1, the AI-based system 100 includes a server 102, which may include one or more computer servers. In various embodiments, server 102 includes multiple servers.The multiple servers may include multiple, redundant, or replicated servers as part of a server cluster. In another embodiment, server 102 may be implemented as a cloud-based server, such as a cloud-based computing platform. For example, server 102 may be any one or more cloud-based platforms, such as Microsoft Azure, Amazon AWS, etc. Server 102 may include one or more processors 104 (i.e., CPU and GPU) and one or more computer memories 106.

[0044] Memory 106 may include one or more forms of volatile and / or non-volatile, fixed and / or removable memory, such as read-only memory (ROM), electronically programmable read-only memory (EPROM), random access memory (RAM), erasable electronically programmable read-only memory (EEPROM) and / or other hard disk drives, flash memory, microSD cards, etc. Memory 106 may store an operating system (OS) (e.g., Microsoft Windows, Linux, UNIX, etc.) that enables the functions, applications, methods, or other software discussed herein. Memory 106 may also store AI model 108, which may include artificial intelligence-based models, such as machine learning models, trained on natural language processing (NLP) data, images, and information about products for treating skin conditions or problems as described herein. AI model 108 may include any of the natural language models, synthetic image generation models, product recommendation models, and / or image simulation models as described herein.

[0045] Additionally or alternatively, AI model 108 may also be stored in database 105, which may be accessible by server 102 or otherwise communicatively coupled to server 102. Furthermore, memory 106 may also store machine-readable instructions, including any of one or more applications, one or more software components, and / or one or more application programming interfaces (APIs), which may be implemented to facilitate or perform these features, functions, or other disclosures described herein, such as any methods, processes, elements, or limitations illustrated, depicted, or described in the various flowcharts, diagrams, charts, figures, and / or other disclosures herein. For example, at least some of the applications, software components, or APIs may be, may include, or otherwise constitute part of an imaging-based machine learning model or component (such as any one or more of AI model 108), wherein each may be configured to facilitate its various functionalities discussed herein. It should be understood that one or more other applications executed by processor 104 may be contemplated.

[0046] Processor 104 may be connected to memory 106 via a computer bus.The computer bus is responsible for transmitting electronic data, data packets, or other electronic signals to and from the processor 104 and memory 106 to implement or execute machine-readable instructions, methods, processes, elements, or limitations as illustrated, depicted, or described in the various flowcharts, diagrams, figures, and / or other disclosures herein.

[0047] The processor 104 may interface with the memory 106 via the computer bus to execute an operating system (OS). One or more processors 104 may also interact with the memory 106 via the computer bus to create, read, update, delete, or otherwise access or interact with data stored in or in the memory 106 and / or database 104 (e.g., a relational database such as Oracle, DB2, MySQL, or a NoSQL-based database such as MongoDB). Data stored in the memory 106 and / or database 105 may include all or part of any data or information described herein, including, for example, NLP data, training images, and / or other images and / or information of the user, including phenotypic data, demographic data, product recommendation data, or as otherwise described herein.

[0048] Server 102 may also include a communication component configured to communicate (e.g., send and receive) data via one or more external / network ports to one or more network or local terminals such as computer network 120 and / or terminal 109 (for presentation or visualization) as described herein. In some embodiments, server 102 may include client-server platform technologies such as ASP.NET, Java J2EE, Ruby on Rails, Node.js, web services, or online APIs, responding to receiving and responding to electronic requests. Server 102 may implement client-server platform technologies that can interact via a computer bus with memory 106 (including applications, components, APIs, data, etc. stored therein) and / or database 105 to implement or perform machine-readable instructions, methods, processes, elements, or limitations as illustrated, depicted, or described in the various flowcharts, diagrams, figures, and / or other disclosures herein.

[0049] In various embodiments, server 102 may include one or more transceivers (e.g., WWAN, WLAN, and / or WPAN transceivers) that operate according to IEEE standards, 3GPP standards, or other standards and are available for receiving and transmitting data via an external / network port connected to computer network 120, or interact with such one or more transceivers. In some embodiments, computer network 120 may include a private network or a local area network (LAN). Additionally or alternatively, computer network 120 may include a public network.Such as the Internet.

[0050] Server 102 may also include or implement an operator interface configured to present information to an administrator or operator and / or receive input from an administrator or operator. As shown in FIG1, the operator interface may provide a display screen (e.g., via terminal 109). Server 102 may also provide I / O components (e.g., ports, capacitive or resistive touch-sensitive input panels, buttons, keys, lights, LEDs) that may be directly accessed or attached to server 102 via server 102, or indirectly accessed or attached to terminal 109 via terminal 109. According to some embodiments, an administrator or operator may access server 102 via terminal 109 to view information, make changes, input training data or images, initiate training of AI model 108, and / or perform other functions.

[0051] As described herein, in some embodiments, server 102 may perform functionality as discussed herein as part of a “cloud” network, or may otherwise communicate with other hardware or software components within the cloud to send, retrieve, or otherwise analyze the data or information described herein.

[0052] Generally, computer programs or computer-based products, applications, or code (e.g., models such as AI models, or other computational instructions described herein) may be stored on a computer-usable storage medium or on a tangible, non-transitory computer-readable medium (e.g., standard random access memory (RAM), optical disc, universal serial bus (USB) drive, etc.) in which such computer-readable program code or computer instructions are embodied, wherein the computer-readable program code or computer instructions may be installed or otherwise adapted to be executed by processor 104 (e.g., in conjunction with a corresponding operating system in memory 106) to facilitate, implement, or perform machine-readable instructions, methods, processes, elements, or limitations as illustrated, depicted, or described in the various flowcharts, diagrams, figures, and / or other disclosures herein. In this regard, the program code may be implemented in any desired programming language and may be implemented as machine code, assembly code, bytecode, interpretable source code, etc. (e.g., via Golang, Python, C, C++, C#, Objective-C, Java, Scala, ActionScript, JavaScript, HTML, CSS, XML, etc.).

[0053] As shown in FIG1, server 102 is communicatively connected to one or more user computing devices 111c1-111c3 and / or 112c1-112c3 via computer network 120 and base stations 111b and 112b. In some embodiments, base stations 111b and 112b may include cellular base stations such as cellular towers.Thus, based on any one or more of various mobile phone standards (including NMT, GSM, CDMA, UMMTS, LTE, 5G, etc.), it communicates with one or more user computing devices 111c1-111c3 and 112c1-112c3 via wireless communication 121. Additionally or alternatively, base stations 111b and 112b may include routers, wireless switches, or other such wireless connection points that communicate with one or more user computing devices 111c1-111c3 and 112c1-112c3 via wireless communication 122 based on any one or more of various wireless standards, such wireless standards as non-limiting examples including IEEE 802.11a / b / c / g (WIFI), BLUETOOTH standards, etc.

[0054] Any one of the one or more user computing devices 111c1-111c3 and / or 112c1-112c3 may include mobile devices and / or client devices for accessing server 102 and / or communicating with the server. In various embodiments described on page 10 / 33 of CN 122207052 A, user computing devices 111c1-111c3 and / or 112c1-112c3 may include mobile phones (e.g., cellular phones), tablet devices, personal data assistants (PDAs), etc., including, as non-limiting examples, Apple iPhone or iPad devices or Google Android-based mobile phones or tablets. In various embodiments, one or more user computing devices 111c1-111c3 and / or 112c1-112c3 may implement or execute an operating system (OS) or mobile platform, such as Apple's iOS and / or Google's Android operating system. Any of one or more user computing devices 111c1-111c3 and / or 112c1-112c3 may include one or more processors and / or one or more memories for storing, implementing, or executing computing instructions or code (e.g., mobile applications or home or personal assistant applications), as described in the various embodiments herein. As shown in Figure 1, the AI ​​model 108a and / or application 109a, or at least a portion thereof, as described herein, may also be locally stored on the memory of a user computing device (e.g., user computing device 111c1). In some aspects, the AI ​​model 108a installed on the computing device may include the same AI model 108 installed on server 102. Additionally or alternatively, the AI ​​model 108a may include a portion of the AI ​​model 108 installed on server 102. It should be understood that, in some aspects,One or more AI models in the AI ​​models may be installed entirely on the user computing device, entirely on the server 102, or partially on the user computing device and partially on the server 102, wherein communication between AI model 108a and AI model 108 (e.g., via applications 109a and 109, respectively) is conducted via computer network 120. Generally, when any AI model (e.g., any of a natural language model, a synthetic image generation model, a product recommendation model, and / or an image simulation model) is mentioned herein, it may refer to one or both of AI model 108 and / or AI model 108a.

[0055] User computing devices 111c1-111c3 and / or 112c1-112c3 may include wireless transceivers to transmit wireless communications 121 and / or 122 to and from base stations 111b and / or 112b. In various implementations, natural language data from users 202a, 202b, and / or 202c may be transmitted via computer network 120 to server 102 for training and / or imaging analysis of models (e.g., AI model 108) as described herein. Such natural language data may be received from applications implementing a GPT interface, such as those described herein with reference to FIG3. As described herein, natural language data may be used to train or implement models, such as AI or machine learning models (e.g., AI model 108). Such natural language data may be transmitted to and / or stored on memory 106 and / or database 105 of server 102.

[0056] Furthermore, each of one or more user computer devices 111c1-111c3 and / or 112c1-112c3 may include a display screen for displaying graphics, images, text, classifications, natural language data, skin products, data, pixels, features, and / or other such visualizations or information as described herein. In various embodiments, graphics, images, text, classifications, natural language data, skin products, data, pixels, features, and / or other such visualizations or information may be received from server 102 for display on the display screen of any one or more of user computer devices 111c1-111c3 and / or 112c1-112c3. Additionally or alternatively, the user computer device (e.g., as described herein with reference to FIG6) may include, implement, access, present, or otherwise at least partially expose an interface or guided user interface (GUI) for displaying text and / or images on its display screen.

[0057] In some embodiments, computational instructions and / or applications executing at a server (e.g., server 102) and / or a mobile device (e.g., mobile device 111c1) may be communicatively connected to analyze the user's natural language data, thereby outputting natural language data describing user-specific product recommendations and / or displaying user-specific simulated images on the display screen.For example, as described herein. For example, one or more processors of server 102 (e.g., processor 104) may be communicatively coupled to a mobile device via a computer network (e.g., computer network 120). In such embodiments, the application may include a server application portion (e.g., application 109) configured to execute on one or more processors of the server (e.g., server 102) and a mobile application portion (e.g., application 109a) configured to execute on one or more processors of a mobile device (e.g., any one of one or more user computing devices 111c1-111c3 and / or 112c1-112c3). In such embodiments, the server application portion is configured to communicate with the mobile application portion. The server application portion or the mobile application portion may each be configured to implement or partially implement one or more of the following: (1) receiving user-specific natural language data of a user, the natural language data defining user-specific phenotypic information and user-specific demographic data of the user; (2) inputting the user-specific natural language data into a natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user; (3) inputting one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into a synthetic image generation model, wherein the synthetic image generation model generates a user-specific digital twin of the user. (4) Inputting a user-specific digital twin image of the user, one or more phenotypic classifications of the user, and one or more demographic classifications of the user into a product recommendation model, wherein the product recommendation model generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem identified in the user-specific digital twin image or one or more user-specific phenotypic classifications; (5) Inputting a user-specific digital twin image, user-specific product recommendations, one or more phenotypic classifications of the user, and one or more demographic classifications of the user into an image simulation model, wherein the image simulation model generates user-specific simulated images based on the user-specific digital twin image and the user-specific product recommendations.The user-specific simulated image graphically depicts a user-specific digital twin image graphically enhanced based on user-specific product recommendations; (6) outputs natural language data describing the user-specific product recommendations to the user; and / or (7) the user-specific simulated image is displayed on a screen by the application.

[0058] Figure 2A illustrates an example artificial intelligence (AI)-based method 250 for providing personalized skin product recommendations according to various embodiments disclosed herein. Figure 2A illustrates an implementation or execution of a multi-AI model training phase 251 of the AI-based method 250. The multi-AI model training phase 251 includes training multiple models, such as a natural language model, a synthetic image generation model, a product recommendation model, and an image simulation model. These models are trained on data (e.g., natural language data) and outputs, wherein, for at least some models, the output of one model is used as input for the training or implementation of another model. In this way, multiple AI models (e.g., AI model 108 and / or AI model 108a) may include ensemble models or downstream models, some of which receive inputs from other models to produce new or additional outputs. In various aspects, each of the AI ​​models (e.g., AI model 108 and / or AI model 108a) including natural language models, synthetic image generation models, product recommendation models, and image simulation models can be stored in one or more memories (e.g., memory 106, database 105, and / or the memory of a user computing device). Furthermore, the AI ​​models can be accessed by applications (apps) including computational instructions stored in one or more memories (e.g., application 109 and / or application 109a). The AI ​​models can be accessed and / or trained by one or more processors, including the processors of user computing devices (e.g., user computing device 111c1) and / or server 102.

[0059] At block 252, the AI-based method 250 includes training a natural language model by one or more processors using natural language data (e.g., text data, speech data, image data, and / or video data) and demographic data (e.g., ethnic groups, etc.) of the respective user. In various aspects, the natural language model includes a large language model (LLM). In some implementations, the natural language model is trained based on existing or general-purpose natural language models (LLMs) (such as OpenAI's GPT model (e.g., the GPT-4 model) or other LLM models (such as the Llama2 model)). In such implementations, the existing or general-purpose LLM is also trained on natural language data (e.g., audio, text, and / or image / video data) related to skin care information, including various skin problems and / or skin conditions. As a non-limiting example,This includes those described herein.

[0060] The natural language model is trained or otherwise configured to output one or more phenotypic classifications corresponding to one or more phenotypic traits of a given user. The natural language model may also be trained or otherwise configured to output one or more demographic classifications corresponding to one or more demographic attributes of a given user. For example, in some embodiments, phenotypic classifications and / or demographic classifications may include one or more of the following: (a) gender type classification; (b) age type classification; (c) ethnic type classification; or (d) skin concern type classification. For example, a phenotypic classification may include any one or more skin concerns or problems of a user or associated with them, which, as a non-limiting example, may be skin health, skin appearance, dry skin, skin shine, skin radiance, etc. In some aspects, a phenotypic classification may include ordinal values ​​ranked based on the degree of skin health, skin appearance, dry skin, skin shine, and / or skin radiance, etc. Additionally or alternatively, phenotypic classifications can be numbered values ​​(e.g., values ​​1-10) based on the degree of skin health, skin appearance, skin dryness, skin shine, and / or skin radiance, etc.

[0061] Furthermore, as described above, the natural language model is further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the respective user. As a non-limiting example, the user's demographic attributes may include any one or more of gender, age, ethnicity, geographic region, health, and / or other user information. For example, the user may provide natural language data indicating that the user is 35 years old, of Asian descent, and lives in a geographic region (e.g., a city in China).

[0062] At block 254, the AI-based method 250 includes training a synthetic image generation model by one or more processors on multiple images of the respective individual or user corresponding to one or more phenotypic classifications and one or more demographic classifications output by the natural language model. For example, in one aspect, the images may be labeled or annotated, for example using metadata that defines the phenotypic and / or demographic information of the respective individual. In various aspects, metadata can be at the pixel level, for example, defining or identifying the location of skin problems or conditions (e.g., hemoglobin and / or melanin) identified in a given image depicting a user's skin.

[0063] The synthetic image generation model can be trained or otherwise configured using images, pixel data, metadata, and / or label data to output a digital twin image (e.g., a synthetic individual) of a corresponding individual or user based on multiple images corresponding to one or more phenotypic classifications and one or more demographic classifications of the corresponding user as input. For example, in various specific implementations,The synthetic image generation model includes an AI model configured to generate photorealistic images, for example, where the digital twin image includes a photorealistic image. A user-specific digital twin image may include a photorealistic image of a digital twin similar to the user. In some example implementations, the digital twin image may depict a photorealistic image of an individual that does not actually exist but appears very realistic. Furthermore, the digital twin image may be generated with skin problems or conditions (e.g., pigmentation spots, wrinkles and / or fine lines, acne, pores, sagging and / or loss of elasticity, uneven texture, thinning skin, dryness, oiliness, sensitivity, uneven skin tone, eczema, dermatitis), on which the synthetic image generation model is trained.

[0064] The synthetic image generation model may include a pre-trained deep learning model, a machine learning model, a generative adversarial network, a conditional generative adversarial network, a convolutional neural network, and / or a visual transformer. The synthetic image generation model can be pre-trained using a large number of facial images covering real-life imaging conditions (e.g., consumer images showing skin conditions or problems) and laboratory imaging conditions (e.g., clinical images showing skin conditions or problems). The synthetic image generation model is configured to produce photorealistic facial images based on natural language data obtained from the user through interaction with a natural language model.

[0065] Furthermore, in some embodiments, AI model 108 and / or AI model 108a also include an imaging model trained on one or more corresponding images of the corresponding user and trained or otherwise configured to output one or more phenotypic classifications corresponding to one or more phenotypic classifications of the corresponding user. The image model may include a visual transformer and / or a convolutional neural network (CNN) configured to determine phenotypic classifications from images. In such embodiments, the outputs of the natural language model and the imaging model may be combined (e.g., averaged or weighted based on the classifications output by each model) to determine one or more phenotypic classifications of the corresponding user. In some specific implementations, phenotypic information can be obtained solely based on imaging analysis of the imaging model, without the need for natural language data from the user.

[0066] At block 256, the AI-based method 250 includes training a product recommendation model by one or more processors on a digital twin image of the corresponding user output by a synthetic image generation model. The product recommendation model may also be trained on one or more phenotypic classifications and one or more demographic classifications. The product recommendation model is trained or otherwise configured to output product recommendations for the corresponding user based on the digital twin image, one or more phenotypic classifications, and one or more demographic classifications as input. In various specific implementations, the product recommendation model may include a model that has been trained,This system can determine the efficacy of one or more skin care products corresponding to one or more phenotypes of a given user. The efficacy of a given product may be based on the active ingredients of a given skin care product, and data on the efficacy of a given product and its impact on skin conditions or problems can be used to train a product recommendation model. Product recommendations may be targeted at user-specific skin care products, and each skin care product can be mapped or otherwise associated with the treatment of known skin care conditions or problems (e.g., pigmentation, wrinkles and / or fine lines, acne, pores, sagging and / or loss of elasticity, uneven texture, thinning skin, dryness, oiliness, sensitivity, uneven skin tone, eczema, dermatitis). For example, in some implementations, the synthetic image generation model may include lookup tables, statistical models, machine learning models, deep learning models, recommendation engines developed or trained based on consumer information (e.g., demographics, skin concerns, skin goals) and clinical studies conducted to evaluate the efficacy of one or more skin care products.

[0067] At block 258, the AI-based method 250 includes training an image simulation model by one or more processors on a digital twin image output by a synthetic image generation model, and the image simulation model is also trained on product recommendations output by a product recommendation model. The image simulation model may also be trained on the product recommendation model, which may be trained on one or more phenotypic classifications and one or more demographic classifications. The image simulation model is trained or otherwise configured to generate simulated images based on the digital twin image, the simulated images having one or more graphic enhancements based on the product recommendation. In various specific embodiments, the graphic enhancements may include skin annotations or other graphic enhancements depicted by skin and / or pixel data from the digital twin image. The image simulation model may include an AI model trained to determine the influence of one or more product attributes (e.g., active ingredients, pigmentation, etc.) corresponding to the respective product in the product recommendation. For example, the image simulation model may include a machine learning model, a deep learning model, a generative adversarial network, a conditional generative adversarial network, a convolutional neural network, a visual transformer, and / or a statistical model. Image simulation models can be pre-trained based on clinical studies conducted to evaluate the efficacy of one or more skincare products on the skin of a corresponding individual, in order to generate simulated images based on digital twin images, which have one or more graphic enhancements based on product recommendations.

[0068] More generally, in various aspects, each of the AI ​​models (e.g., AI model 108 and / or AI model 108a) includes an artificial intelligence (AI)-based model trained using at least one AI algorithm. Training an AI model involves analyzing natural language data, phenotypic data, demographic data, images, or other training data as described herein to configure the weights of the respective AI model. For example,In various embodiments described herein, the generation of AI model 108 involves training the AI ​​model using natural language data, multiple training images of an individual, and / or data or classifications as described herein, wherein each piece of data includes lexical units, words, and / or image pixel data defining various features such as demographics, phenotypes, image features, or other features as described herein. In some embodiments, one or more processors of a server or cloud-based computing platform (e.g., server 102) may receive training data (e.g., natural language data, demographic data, phenotype data, training images, or other data as described herein) via a computer network (e.g., computer network 120). In such embodiments, the server and / or cloud-based computing platform may use such training data to train one or more AI models in the AI ​​model specification (page 14 / 33, 18 CN 122207052 A). For example, at least some of the AI ​​models may be trained using user demographic data (e.g., data indicating race, skin color, etc.) and environmental data (e.g., sunshine duration, geography, weather conditions, etc.) for the corresponding user. In this regard, a freckle classification or identifier can be generated by an AI model to provide the extent of a skin problem or condition.

[0069] In some specific embodiments, image modeling may include training a machine learning-based model (e.g., a synthetic image generation model and / or an image simulation model) on pixel data of images and / or digital twin images depicting the skin or skin regions of a corresponding individual. For example, in various specific embodiments, one or more features of the skin or skin regions may vary based on the demographics and / or ethnicities of one or more users of the corresponding individual and / or digital twin represented in the corresponding training images, such as being associated with or otherwise naturally occurring as different races, genomes, and / or geographic locations typically associated with such demographics and / or ethnicities. Synthetic image generation models and / or image simulation models may generate outputs (e.g., digital twin images and / or user-specific digital twin image graphics augmented images) based on the ethnicity and / or demographic values ​​of the corresponding individual. Additionally or alternatively, image modeling may include using a machine learning imaging model, as previously trained, to determine the image classification of a user's skin or skin regions based on pixel data (e.g., including their L, a, and b values ​​and / or RGB values) from one or more images of an individual. For example, the model's weights may be trained by analyzing various L ab values ​​of individual pixels in a given image. For instance, dark or low L values ​​(e.g., pixels with L values ​​less than 50) may indicate areas in the image containing hemoglobin and / or melanin. Similarly, slightly lighter L values ​​(e.g., pixels with L values ​​greater than 50) may indicate the absence of melanin or hemoglobin. Furthermore, high / low a values ​​may indicate skin regions containing more / less melanin and / or hemoglobin. In summary,When a pixel with a skin tone Lab value is located within a given image or otherwise surrounded by a set of pixels with melanin and / or hemoglobin hues, an image-based model (e.g., a synthetic image generation model and / or an image simulation model) can determine or generate an output recognized within the given image (e.g., a digital twin image and / or a user-specific digital twin image graphic augmentation image). In this way, a machine learning imaging model can be trained or used to determine, for example, an output associated with such an output as described herein, using pixel data from tens of thousands of training images (e.g., skin regions that describe the skin of the corresponding individual or digital twin).

[0070] In various embodiments, supervised or unsupervised machine learning procedures or algorithms can be used to train machine learning imaging models as described herein (e.g., any of the AI ​​models in AI model 108). The machine learning procedure or algorithm may employ a neural network, which may be a convolutional neural network, a visual transformer, a deep learning neural network, or a combined learning module or program that learns two or more features or feature datasets (e.g., natural language data and / or pixel data) in a particular region of interest. Machine learning programs or algorithms may also include natural language processing, semantic analysis, automated reasoning, regression analysis, support vector machine (SVM) analysis, decision tree analysis, random forest analysis, K-nearest neighbor analysis, Naive Bayes analysis, clustering, reinforcement learning, and / or other machine learning algorithms and / or techniques. In some embodiments, algorithms based on artificial intelligence and / or machine learning may be included as libraries or groups of libraries that execute on one or more servers 102. For example, libraries may include libraries based on TENSORFLOW, PYTORCH, and / or SCIKIT-LEARN Python libraries.

[0071] Machine learning may involve identifying and recognizing patterns in existing data (such as recognizing skin features, such as spots and / or color or discoloration-related features, in pixel data of an image and / or based on natural language data as described herein) to facilitate prediction or recognition of subsequent data (such as using models on new pixel data and / or natural language data to determine or generate digital twin images, user-specific product recommendations, and / or simulated images based on digital twin images with one or more graphic enhancements based on product recommendations).

[0072] Machine learning models (such as the AI ​​models described herein) can be created and trained based on example data (e.g., “training data,” such as natural language data and / or pixel data) or input data (which may be referred to as “features” and “labels”) to make effective and reliable predictions on new inputs (such as test-level or production-level data or inputs). In the supervised machine learning specification, page 15 / 33, 19 CN 122207052 A,Machine learning programs operating on servers, computing devices, or other processors may be set up with example inputs (e.g., “features”) and their associated or observed outputs (e.g., “labels”) to enable the machine learning program or algorithm to determine or discover rules, relationships, patterns, or other machine learning “models” that map such inputs (e.g., “features”) to outputs (e.g., “labels”), for example by determining weights or other metrics across various feature categories and / or assigning weights or other metrics to the model. Such rules, relationships, or other models may then be provided as subsequent inputs to enable the model executing on the server, computing device, or other processor to predict the expected output based on the discovered rules, relationships, or models.

[0073] In unsupervised machine learning, it may be required that the server, computing device, or other processor find its own structure in unlabeled example inputs, where, for example, multiple training iterations are performed by the server, computing device, or other processor to train multiple model generation until a satisfactory model is generated, for example, a model that provides sufficient predictive accuracy when given test-level or production-level data or inputs.

[0074] Supervised learning and / or unsupervised machine learning may also include retraining the model, relearning the model, or otherwise updating the model with new or different information, which may include information received, ingested, generated, or otherwise used over time. The disclosure herein may use one or both of such supervised or unsupervised machine learning techniques.

[0075] Figure 2B also illustrates an example of an artificial intelligence (AI)-based method 250 of Figure 2A according to various embodiments disclosed herein. Figure 2B illustrates an implementation or execution of a multi-AI model execution phase 260 of the AI-based method 250. The multi-AI model execution phase 260 includes accessing and executing multiple models, such as models trained during AI model training phase 251 (e.g., AI models 108 and 108a), to provide inputs and outputs for these models to generate user-specific product recommendations and to generate user-specific simulated images based on user-specific digital twin images and user-specific product recommendations. Multiple AI model execution phases 260 may be implemented by an application (e.g., application 109 and / or 109a) executing on one or more processors.

[0076] At block 262, the AI-based method 250 includes receiving user-specific natural language data of a user (e.g., user 202a) by the application. For example, natural language data may be received from an LLM interface 300 as described herein with respect to FIG3. The natural language data may define user-specific phenotypic information and user-specific demographic data of the user, as a non-limiting example,This includes any one or more of the user's gender, age, ethnicity, and / or skin concerns that identify skin problems or conditions. The natural language model may include or be part of a dialogue engine, for example, as part of an application (e.g., application 109 and / or application 109a), where the application accesses the natural language model to obtain natural language data by directly asking questions and / or inferring from user information such as age, gender, ethnicity, and skin concerns. For example, user 202a could be a 41-year-old female with hemoglobin-related skin concerns.

[0077] At box 264, the AI-based method 250 includes inputting user-specific natural language data into the natural language model by the application (e.g., application 109 and / or 109a). The natural language model can then generate one or more user-specific phenotypic classifications (e.g., skin health, skin appearance, dry skin, skin shine, skin radiance) and one or more user-specific demographic classifications (e.g., gender, age, ethnicity, geographic region, health, and / or other user information) based on the natural language data.

[0078] At block 266, the AI-based method 250 includes inputting one or more user-specific phenotypic classifications and one or more user-specific demographic classifications into a synthetic image generation model by an application (e.g., application 109 and / or 109a). The synthetic image generation model can then generate a user-specific digital twin image of the user based on the user's phenotypic classification and / or demographic classification. For example, the digital twin image may be an image designed to look like or resemble the user based on phenotypic classification and / or demographic classification (including age, ethnicity, skin condition, etc.). However, the digital twin image may look different from the user, for example, different from the digital photograph of the image. In this way, the digital twin image serves as a substitute or synthetic image for the user without requiring the user to transmit the actual image via a computer network (e.g., computer network 120).

[0079] In some specific implementations, a user-specific digital twin image may be provided to the user. Applications (e.g., applications 109 and / or 109a) may receive additional user-specific natural language data from the user in order to manipulate a user-specific digital twin image. For example, a user may wish to change certain aspects of the digital twin image to make it look more like the user. The application (e.g., applications 109 and / or 109a) that access the synthetic image generation model can then update the user-specific digital twin image based on the additional user-specific natural language data. In this way, the user can change the appearance and / or look of his or her digital twin image based on additional text, voice, video, or other natural language data and instructions (e.g.,Skin type, skin concerns, color, etc.).

[0080] At box 268, the AI-based method 250 includes inputting a user-specific digital twin image of the user, one or more phenotypic classifications of the user, and one or more demographic classifications of the user into a product recommendation model by an application (e.g., application 109 and / or 109a). The product recommendation model then generates user-specific product recommendations for the user. User-specific product recommendations may include recommendations for manufactured products (e.g., skin creams or lotions with ingredients for treating specific skin conditions or problems). User-specific product recommendations may be designed to address at least one problem identified in the user-specific digital twin image or one or more user-specific phenotypic classifications. The identified problem may include skin problems or conditions (e.g., melanin, hyperhemoglobin, and / or acne) as depicted in the pixel data of the user-specific digital twin image and / or as determined based on one or more user-specific phenotypic classifications as determined from the user's natural language data.

[0081] At block 270, the AI-based method 250 includes inputting a user-specific digital twin image, a user-specific product recommendation, one or more phenotypic classifications of the user, and one or more demographic classifications of the user into an image simulation model by an application (e.g., applications 109 and / or 109a). The image simulation model can then generate an enhanced image, such as a user-specific simulated image, based on the user-specific digital twin image and the user-specific product recommendation. The user-specific simulated image is generated or output to graphically depict the user-specific digital twin image graphically enhanced based on the user-specific product recommendation. Graphical enhancement of the user-specific simulated image may include updating or altering pixels of a first digital twin image of the user, wherein the user-specific simulated image newly depicts a reduction or elimination of the degree or appearance of a skin problem or condition, consistent with what is expected when the product is applied to the user's skin according to the recommendation.

[0082] At block 272, the AI-based method 250 includes outputting natural language data describing the user-specific product recommendation to the user by an application (e.g., applications 109 and / or 109a). For example, natural language data may include output from LLM interface 300, where the dialogue engine of the natural language model converses with the user regarding user-specific product recommendations, product usage, general advice about the causes of a user-specific skin condition or problem, and / or how to avoid that skin condition and / or problem in the future.

[0083] At block 274, AI-based method 250 includes displaying user-specific simulated images on a display screen by an application (e.g., applications 109 and / or 109a). The display screen may be the display screen of a user computer device (e.g., user computing device 111c1). In some aspects,User-specific simulated images may be displayed on the screen in real-time or near real-time during or after receiving the user's natural language data.

[0084] Furthermore, in some embodiments, user-specific product recommendations may be displayed on the screen (e.g., the screen of a user's computing device), accompanied by instructions to treat at least one feature identifiable in pixel data including a region of the user's skin using a manufactured product. The pixel data may be pixel data of a user-specific simulated image. For example, the instructions may provide information to the user (e.g., avoid direct sunlight exposure) to reduce or eliminate excessive melanin production in identifiable skin areas within the image.

[0085] In additional embodiments, the user-specific product recommendation includes a product recommendation for a manufactured product. The manufactured product may include a medicament, treatment, or other product for treating at least one spot feature identifiable in the pixel data. For example, the product may include a composition (such as a cream) having hydroxycinnamic acid (HCA) and nicotinamide, which can reduce melanin and hemoglobin in persistent spots or marks at low pH. For example, a product can then be recommended to treat the identified spot type, such as an anti-inflammatory product for hemoglobin-type spots. For melanin-type spots, a product for treating hyperpigmentation can be recommended. In some embodiments, the application delivers manufactured products to the user based on user-specific product recommendations.

[0086] Regarding manufactured product recommendations, in some embodiments, one or more processors (e.g., server 102 and / or user computing devices (such as user computing device 111c1)) can generate a modified image (e.g., a simulated or enhanced image) based on at least one image of the user, such as one initially received. In such embodiments, the modified image can depict what the user's skin or skin area is predicted to look like after treatment of at least one feature by the manufactured product. For example, the modified image can be modified by updating, smoothing, or changing the color of the pixels of the image to represent possible or predicted changes after treatment of at least one feature within the pixel data by the manufactured product. The modified image can then be presented on the display of the user computing device (e.g., user computing device 111c1).

[0087] Figure 3 illustrates an example user interface according to various embodiments disclosed herein, showing a Large Language Model (LLM) interface 300 for capturing a user's natural language data. In various aspects, the LLM interface 300 may be implemented, invoked, or executed by an application, such as application 109a executing on a user computing device (e.g., user computing device 111c1) and / or application 109 executing on a server 102. For example, in some specific embodiments,LLM interface 300 may include a graphical user interface (GUI) presented by application 109a, which may include a web browser-based application (e.g., implemented on a web browser such as Google Chrome). Additionally or alternatively, the LLM interface 300 presented on a user computing device (e.g., user computing device 111c1) may communicate with server 102, for example, via computer network 120. This communication allows application 109a to communicate with application 109 on server 102, and to receive and transmit data from and to that application, and to access data and / or AI model 108 that may be stored on memory 105 and / or database 105. In one example, LLM interface 300 communicates with a language-based AI model (e.g., a natural language model) stored on server 102. The language-based AI model may include instances of large language models (such as Llama2 or GPT-4).

[0088] Referring to FIG3, the LLM interface 300 includes a text-based interface, such as a dialogue engine, where a user (e.g., user 202a) can interact with the artificial intelligence (AI)-based system 100 of FIG1. ​​The LLM interface 300 can also be configured to provide personalized skin product recommendations. While FIG3 illustrates an example of capturing natural language data (e.g., text-based data) via an example text-based interface of the LLM interface 300, it should be understood that the LLM interface 300 may also capture and receive other natural language data, including, as a non-limiting example, voice data and / or video data.

[0089] As illustrated in the example of FIG3, the user interacts with the LLM interface 300 by providing text-based natural language data indicating skin type and skin condition or problem. The user also asks natural language questions about the ingredients of products used to treat skin condition or problem. The user's natural language data may be received by an application (e.g., application 109a and / or application 109) and then provided or transmitted to a GPT AI model (e.g., a GPT-4 AI model instance stored on server 102). The GPT AI model can respond with outputs providing relevant information and also seeks new natural language data from the user to obtain a complete dataset, which is then provided to the natural language model as described herein. The natural language model can use this natural language data (page 18 / 33, 22 CN 122207052 A) to determine or output user-specific phenotypic information and user-specific demographic data.Such as the user's gender, age, ethnicity, and / or skin concerns or problems.

[0090] Figure 4A illustrates a set of example digital twin images that can be used to train and / or otherwise implement artificial intelligence (AI) models, such as synthetic image generation models and / or image simulation models, according to various embodiments disclosed herein. Digital twin images 202a_dt1 (and, as a non-limiting example, 202b_dt1 and 202c_dt1 for the digital twin images depicted in Figure 4A) can be generated based on user-specific phenotypic classifications and user-specific demographic classifications received from or determined for the respective users (e.g., users 202a, 202b, and 202c) as described herein by a natural language model. For example, a synthetic image generation model can take into input user-specific phenotypic classifications and user-specific demographic classifications for a particular user to output a given digital twin image for that user. The user-specific phenotypic classifications and user-specific demographic classifications for a particular user may have been previously determined by a natural language model that inputs the user's natural language data and the user's specific demographic data. Each digital twin image in a digital twin image may include a photorealistic image with pixel data, for example, as if captured by a digital camera. In various ways, the synthetic image generation model is trained on multiple images of real-world individuals. Each real-world individual in the images will be associated or mapped (e.g., via supervised learning) to one or more phenotypic classifications and one or more demographic classifications, including those output by, for example, a natural language model. In this way, the synthetic image generation model will output digital twin images of the corresponding user (e.g., those in Figure 4A). For example, one individual may have brown skin, be of a certain age (e.g., 35 years old), and have a skin condition and / or skin concern type classification (e.g., pigmentation, wrinkles and / or fine lines, acne, pores, sagging and / or loss of elasticity, uneven texture, thinning skin, dryness, oiliness, sensitivity, uneven skin tone, eczema, dermatitis). Another individual may have white skin, be of a different age (e.g., 24 years old), and have a different skin condition and / or skin concern type classification (e.g., acne). In this way, digital twin images (e.g., those in FIG4A) are based on multiple images of training images of real-world individuals, which have been mapped to one or more phenotypic classifications and one or more demographic classifications of the corresponding user.

[0091] It should be understood that the digital twin images shown in FIG4A are non-limiting examples of example digital twin images, and the synthetic image generation model may also output additional and / or different digital twin images. In some specific implementations, the digital twin images (e.g., any one of digital twin images 202_dt1, 202b_dt1, and 202c_dt1) are unique for each user.This ensures that no user shares the same digital twin image with another user.

[0092] Figure 4B illustrates an example digital twin image 202a_dt selected or generated for a given user according to various embodiments disclosed herein. The digital twin image 202a_dt represents a digital twin or other synthetic image of user 202a of Figure 1. In at least one specific embodiment, the digital twin image 202a_dt is generated when, for example, natural language data of user 202a is received by application 109 and / or 109a, which defines user-specific phenotypic information of user 202a (e.g., gender is female, age is 41 years, etc.) and user-specific demographic information (e.g., the user is Caucasian). User-specific natural language data of user 202a can be input into a natural language model that generates one or more user-specific phenotypic classifications (e.g., gender type classification, age type classification, ethnic type classification, or skin distress type classification) and / or one or more user-specific demographic classifications (e.g., Caucasian).

[0093] For example, in the example of FIG4B, the digital twin image 202a_dt depicts the digital twin image of user 202a as a 41-year-old Caucasian woman. The digital twin image 202a_dt is generated based on information about user 202a provided via the LLM interface 300 as illustrated with respect to FIG3. Skin conditions or problems can be used to generate or update the digital twin image, wherein such skin conditions or problems are also based on the user's natural language data, such as information provided via the LLM interface 300. For example, as shown in the example of FIG4B, user 202a provides natural language data describing or defining a skin condition or problem associated with increased melanin (e.g., melanin-related spots 202ar3). User 202a may also provide natural language data describing the location of the melanin-related spots, for example, in the skin area 202ar on the user's left cheek. Synthetic image generation models can use such natural language data as input to generate user-specific digital twin images (e.g., digital twin image 202a_dt) with the same or similar skin conditions or problems (e.g., melanin-related spots, such as melanin-related spots 202ar3) in the same or similar skin regions (e.g., skin region 202ar).

[0094] FIG4C illustrates example image 202_dt1 and its associated pixel data that can be used to train and / or implement artificial intelligence (AI) models according to various embodiments disclosed herein. In the example of FIG4C, image 202_dt1 represents and is depicted as a scaled or cropped version of digital twin image 202a_dt. In various aspects, image 202_dt1 represents an image used to train a synthetic image generation model.The synthetic image generation model is configured to generate user-specific digital twin images (e.g., digital twin image 202_dt) that otherwise output to the user, wherein the user-specific digital twin image includes at least one problem identified in the image (e.g., skin problems such as acne, hemoglobin-related spots, and / or melanin-related spots). The specific problem identified for the user (e.g., a specific skin problem) can then be applied (e.g., overlaid or masked) to the digital twin image (e.g., digital twin image 202_dt) to generate an image depicting the skin problem in the user-specific digital twin image. In this way, the synthetic image generation model can be trained to associate or correlate the user's NPL data describing the skin problem (e.g., where the user describes melanin-related spots 202ar3) with pixel-related data of an image depicting that skin problem (e.g., image 202_dt1).

[0095] Furthermore, digital images (such as example images 202_dt1 and 202_dt1alt as described with respect to FIG. 5A) may be generated and stored at server 102, and may be analyzed by a synthetic image generation model and / or an image simulation model, and / or used to train the synthetic image generation model and / or the image simulation model, which may each be a machine learning model as described herein. The synthetic image generation model may be trained on images with skin problems (such as image 202_dt1) to generate digital twin images with such skin problems. However, the image simulation model may be trained on images (such as image 202_dt1) to determine how skin problems would be affected based on treatment with a given product (e.g., a skin cream), and to generate user-specific simulated images (e.g., enhanced digital twin images, such as 202adtalt) based on user-specific digital twin images and user-specific product recommendations. Each of these images (e.g., digital twin image 202a_dt, digital twin image region 202a_dt1, digital twin enhanced image region 202a_dt1alt, and digital twin enhanced image 202a_dtalt) may include pixel data that includes feature data within the respective image and corresponds to a skin region of the respective user.

[0096] With respect to the digital image as described herein, pixel data (e.g., pixel data 202ap of FIG. 4C) comprises individual points or squares of data within the image, where each point or square represents a single pixel within the image (e.g., each of pixel 202ap1, pixel 202ap2, and pixel 202ap3). Each pixel may be located at a specific location within the image. Furthermore, each pixel may have a specific color (or lack that specific color). Pixel color can be determined by the color format and associated channel data associated with a given pixel. For example,The popular color format is the 1976 CIELAB color format (also referred to herein as "CIE L-a-b" or simply "Lab" color format), which is configured to simulate human perception of color. That is, the Lab color format is designed such that the amount of numerical variation in the three values ​​representing the Lab color format (e.g., L, a, and b) roughly corresponds to the same amount of variation perceived by human vision. This color form is advantageous, for example, because the Lab color gamut (e.g., the complete subset of colors included as part of the color format) includes the color gamuts of red (R), green (G), and blue (B) (collectively referred to as RGB) and cyan (C), magenta (M), yellow (Y), and black (K) (collectively referred to as CMYK).

[0097] In the Lab color format, color is considered as a point in three-dimensional space, defined by a three-dimensional coordinate system (L, a, b), where each of the L, a, and b data can correspond to a separate color channel and can therefore be referred to as channel data. In this three-dimensional coordinate system, the L-axis describes the brightness (luminance) of the color, with values ​​ranging from 0 (black) to 100 (white). The a-axis describes the green or red ratio of the color, where a positive a value (+a) indicates a red hue, while a negative a value (-a) indicates a green hue. The b-axis describes the blue or yellow ratio of the color, where a positive b value (+b) indicates a yellow hue, while a negative b value (-b) indicates a blue hue. Typically, the values ​​corresponding to the a and b axes can be infinite, such that the a and b axes can include any suitable numerical values ​​to express the axis boundaries. However, the a-axis and b-axis can typically include a lower boundary and an upper boundary ranging from approximately 150 to -150. Therefore, in this way, each pixel color value can be represented as a triplet of L, a, and b values ​​to produce the final color for a given pixel.

[0098] As another example, popular color formats include the red-green-blue (RGB) format, which has red, green, and blue channels. That is, in the RGB format, pixel data is represented by three numeric RGB components (red, green, and blue), which can be referred to as channel data to manipulate the color of pixel regions within an image. In some specific implementations, the three RGB components can be represented as three 8-bit numbers for each pixel. Three 8-bit bytes (one byte for each of the RGB values) can be used to generate a 24-bit color. Each 8-bit RGB component can have 256 possible values, ranging from 0 to 255 (i.e., in a basic 2 binary system, an 8-bit byte can contain one of 256 numeric values ​​in the range of 0 to 255). This channel data (R, G, and B) can be assigned values ​​from 0 to 255 that can be used to set the pixel color. For example,Three values ​​such as (250, 165, 0) (meaning (red=250, green=165, blue=0)) can represent an orange pixel. As another example, (red=255, green=255, blue=0) means that each of the red and green is fully saturated (255 is the highest brightness that 8 bits can achieve), with no blue (zero), where the resulting color is yellow. As yet another example, the color black has RGB values ​​(red=0, green=0, blue=0) and white has RGB values ​​(red=255, green=255, blue=255). The nature of gray is having equal or similar RGB values, for example (red=220, green=220, blue=220) is light gray (approximately white), and (red=40, green=40, blue=40) is dark gray (approximately black).

[0099] In this way, the combination of three RGB values ​​produces the final color for a given pixel. Regarding a 24-bit RGB color image, 3 bytes are used to define a color, and there may be 256 shades of red, 256 shades of green, and 256 shades of blue. This provides a 256×256×256, or 16.7 million possible combinations or colors for a 24-bit RGB color image. Therefore, the RGB data value of a pixel indicates the degree of color or light composed of each of the red, green, and blue pixels. The three colors and their intensity levels are combined at that pixel in the image, that is, at that pixel location on the display screen, to illuminate the display screen with that color at that location. However, it should be understood that other bit sizes with fewer or more bits, such as 10 bits, can be used to produce fewer or more overall colors and ranges.

[0100] As a whole, individual pixels positioned together in a grid pattern (e.g., pixel data 202ap) form a digital image or a portion thereof. A single digital image may include thousands or millions of pixels. Images can be captured, generated, stored, and / or transmitted in a variety of formats such as JPEG, TIFF, PNG, and GIF. These formats use pixels to store or represent images.

[0101] Referring to FIG4C, example image 202_dt1 illustrates a skin region representing a digital twin image (e.g., digital twin image 202a_dt) of a user or individual (e.g., user 202a). More specifically, image 202_dt1 includes pixel data, including pixel data 202ap that defines a skin region representing the user's or individual's skin. Pixel data 202ap includes a plurality of pixels, including pixel 202ap1, pixel 202ap2, and pixel 202ap3. In example image 202a_dt1, each of pixel 202ap1, pixel 202ap2, and pixel 202ap3 represents a feature of the skin corresponding to an image classification or value of the skin region. Generally, in various embodiments,Features of a user's skin or other skin areas may include one or more of hemoglobin-related spots, melanin-related spots, and / or other skin conditions (e.g., acne). Each of these features, as described on pages 21 / 33 of the specification (CN 122207052 A), may be determined from or otherwise based on one or more pixels in a digital image (e.g., image 202_dt1). For example, with respect to image 202_dt1, each of pixels 202ap1 and 202ap2 may be a relatively light pixel (e.g., a pixel with a relatively high L value) and / or a relatively yellow pixel (e.g., a pixel with a relatively high or positive b value) within pixel data 202ap located in an area of ​​the user's skin, which may indicate the user's skin's normal or more common values. However, pixel 202ap3 may include darker pixels (e.g., with negative or lower relative L values) and / or redder or darker pixels (e.g., with positive or higher relative a values), which may indicate melanin (e.g., brown tones) or hemoglobin (e.g., red tones) related spots at that location in an image representing the user's skin, respectively. Such pixel features can be used to train AI models, such as synthetic image generation models to output similar images with similar features, and / or image simulation models to output enhanced images with similar but improved skin features, depicting skin after treatment with the recommended product. In addition to pixels 202ap1, 202ap2, and 202ap3, pixel data 202ap also includes a variety of other pixels, including the remainder of the user's skin, including various other skin regions and / or portions of skin that can be analyzed and / or used to train models (such as those described herein). For example, pixel data 202ap also includes pixels representing features of the spots, and in various respects, in addition to the color of the spots, groupings of such pixels at specific locations in the image (where such pixels have similar Lab and / or RGB values) provide training information for skin recognition, classification, and / or enhancement as described herein.

[0102] In some specific implementations, predicted or classification values ​​indicating spot types (e.g., hemoglobin and / or melanin spot types) may be generated or computed. For example, Figures 4B and 4C illustrate images 202a_dt and 202a_dt1 with classifications of melanin spot types (e.g., melanin-associated spots 202ar3), which may be based on pixel data, such as pixel 202ap3, at, for example, spot 202ar3. In some respects, spot types or other spot classifications or values ​​can be identified by spot identifiers (IDs), wherein the value or classification of the spot ID may include methods for identifying various degrees or intensities of pigmentation or hyperpigmentation of the skin (e.g., caused by hemoglobin, melanin, acne, etc.) as capable of being identified within pixel data based on pixel values ​​(e.g.,The range of RGB and / or Lab ID values ​​(e.g., ID 1-20) is used to identify the area. For example, as shown with respect to Figure 4C, pixel 202ap3 is associated with a spot ID value or classification "14" (indicated by the text presentation 202at1), indicating that the melanin-related spot 202ar3 has a relatively high value or classification in the spot ID range of 1-20. It should be understood that additional and / or different spot IDs (e.g., classification, value, and / or range) can be used to detect or classify additional and / or different types of spots or discoloration on a user's skin.

[0103] In some specific implementations, digital images (such as training images or images output by AI models or other digital images (e.g., any of digital twin image 202a_dt, digital twin image region 202a_dt1, digital twin enhanced image region 202a_dt1, and digital twin enhanced image 202a_dtalt)) may be or may include cropped images. Generally, a cropped image is an image with one or more pixels removed, deleted, or hidden from the original captured image. In some aspects, cropped images may include images depicting skin regions with one or more instances of skin problems, such as freckles. For example, referring to FIG4C, image 202_dt1 represents at least a portion of the original image (e.g., digital twin image 202a_dt). Cropped portion 202ac1 represents a first cropped portion of image 202_dt1 that removes portions of the skin region that may not include easily identifiable freckles and / or other skin problems (outside cropped portion 202ac1). As another example, cropped portion 202ac2 represents a second cropped portion of image 202_dt1 that removes portions of the image that may not include freckles and / or other skin problems as easily identifiable as those included in cropped portion 202ac2 (outside cropped portion 202ac2), and therefore may be less useful as training data and / or as the output of a given AI model (e.g., a synthetic image generation model and / or an image simulation model). In various implementations, analyzing cropped images and / or using cropped images for training improves the accuracy of AI models as described herein. (See specification 22 / 33, page 26, CN 122207052 A). It also improves the efficiency and performance of the underlying computer system, as such systems process, store, and / or transmit smaller-sized digital images. Furthermore, images can be stored in a cropped form or otherwise include extracted or depicted areas of the user's skin, but not depicting the user's personally identifiable information (PII). Such cropped images provide security improvements, i.e., where the removal of the PII provides improvements over previous systems because cropped or edited images, especially those that can be transmitted over a network (e.g., ...).Images accessed via the Internet are more secure without including the user's PII information. Importantly, the systems and methods described herein can operate without requiring such non-essential information, providing improvements over conventional systems, such as security and performance improvements. Furthermore, while Figure 4C may depict and describe cropped images, it should be understood that other image types, including but not limited to original, uncropped images (e.g., digital twin image 202a_dt) and / or cropped images of other types / sizes (e.g., cropped portion 202ac1 of image 202_dt1), may also be used or replaced.

[0104] It should be understood that the disclosure of image 202_dt1 of Figure 4C applies in kind or similarly to other digital images described herein, including, for example, digital twin images as described with respect to Figure 4A, wherein such images also include pixels that can be analyzed and / or used to train models as described herein.

[0105] Furthermore, digital images of skin regions (e.g., digital twin image region 202_dt1) can depict various skin features that can be used to train a given AI model and / or generate output from such AI models (such as synthetic image generation models and / or image simulation models as described herein). For example, as illustrated with reference to Figures 4B and / or 4C, these user skin regions include skin features (e.g., freckles) that can be identified in the user's skin region using pixel data from the corresponding image. These skin features include, for example, features indicating hemoglobin, melanin, and / or other skin conditions (e.g., acne), which may include discrete skin regions or features (e.g., freckles) distributed at one or more locations on the user's skin. Additionally, as described herein with reference to Figures 5A and 5B, examples of depicting skin regions with reduced or eliminated skin features (e.g., freckles) that can be identified in the user's skin region using pixel data from the corresponding image are also illustrated. These reduced or eliminated skin features include, for example, features of pixel data having reduced and / or eliminated hemoglobin, melanin, and / or other skin conditions (e.g., acne) (which can be reduced and / or eliminated by applying skin care products). These reduced and / or eliminated pixel regions may include discrete skin regions or features (e.g., spots) distributed at one or more locations on the user's skin, for example, as shown with respect to Figures 5A and / or 5B.

[0106] Pixel data from a given image can be used to train image-based AI models, including, for example, synthetic image generation models and / or image simulation models. For example, a synthetic image generation model can be trained using pixel data from multiple images of a user (e.g., different demographics and / or phenotypes) and pixel data from multiple images of skin conditions or problems (e.g., image twin image region 202_dt1 similar to that in Figure 4C).Such images contain pixel data representing different users with different demographics, phenotypes, and skin problems or conditions. A synthetic image generation model can be configured or trained to output digital twin images with skin conditions or problems of users exhibiting certain phenotypes and given demographics. As another example, an image simulation model can be trained on pixel data from multiple images of users (e.g., different demographics and / or phenotypes) and on pixel data from multiple images of skin conditions or problems (e.g., similar to image twin image region 202_dt1alt in Figure 5A), where the skin has been treated with specific skincare products. The pixels of the images can be characterized as reduced and / or alleviated skin condition problems as reduced and / or alleviated pixel features, for example, as shown for Figures 5A and 5B. In this manner, the output of the image simulation model may include a user-specific simulated image (e.g., the image twin image region 202_dt1alt of FIG. 5A and / or the image twin image 202_dtalt of FIG. 5B), which graphically depicts a user-specific digital twin image that is graphically enhanced (e.g., pixel-enhanced) based on a user-specific product recommendation.

[0107] FIG. 5A illustrates an example image enhanced by an image simulation model according to various embodiments disclosed herein (e.g., digital twin enhanced image region 202a_dt1alt). For example, as described herein (e.g., for FIG. 2A and FIG. 2B), the image simulation model may receive a user-specific digital twin image and a user-specific product recommendation as input. In the example of FIG. 5A, the user-specific digital twin image input to the image simulation model may include a digital twin image 202a_dt selected or generated for user 202a. User-specific product recommendations may include skin creams for treating skin conditions or problems of user 202a. For the examples of Figures 4B, 4C, 5A, and 5B, and relative to user 202a, the skin condition or problem involves increased melanin, for example, melanin-related spots 202ar3 as shown in Figure 4A. User 202a will describe the skin condition or problem during the provision of natural language data (e.g., via LLM interface 300). The natural language data may also include a description of the location of the melanin-related spots, for example, in the skin area 202ar on the user's left cheek as shown in Figures 4A, 4B, 5A, and 5B.

[0108] As shown with respect to Figure 5A, the digital twin enhanced image region 202a_dt1alt includes pixel data 202apalt, which corresponds to pixel data 202ap in Figure 4C. The pixel data 202apalt and the remainder of the digital twin enhanced image region 202a_dt1alt include individual points or squares of data.Each dot or square represents a single pixel within the image. As shown with respect to FIG5A, pixels associated with melanin (e.g., melanin-related spots 202ar3 as shown with respect to FIG4A) are enhanced or otherwise updated, displaying melanin-related skin conditions or problems (e.g., dark-toned pixels, such as pixel 202ap3) as reduced or eliminated, thereby demonstrating the effectiveness or other efficacy of a user-specific product recommendation (e.g., a skin cream product) for treating the skin condition or problem of user 202a. Furthermore, melanin-related skin conditions or problems are reduced or eliminated at the same location or skin area on the face of user 202a (e.g., in the skin area of ​​the user's left cheek).

[0109] Furthermore, as shown with respect to FIG5A, a graphical representation of the user (e.g., image 202_dt1alt) is annotated by one or more graphics (e.g., areas of pixel data 202apalt) or text presentation (e.g., text 202at2) corresponding to various features identifiable within the pixel data, which includes a portion of the user's skin area. For example, in some implementations, the region of pixel data 202apalt may be annotated or overlaid on top of the user's image (e.g., image 202_dt1alt) to highlight the region or feature identified by the image simulation model or another AI model 108 within the pixel data (e.g., feature data and / or raw pixel data). In the example of Figure 5A, the region of pixel data 202apalt indicates features as defined in the pixel data and defined by a spot ID, classification, or value (e.g., the value "3" as shown for text 202at2), thereby indicating a reduction or elimination of hemoglobin and / or melanin spots (e.g., a reduction and / or elimination of pixels at or near 202ap3 as described for Figure 4C). In other implementations, even if pixels at or near that skin region of pixel data 202apalt are associated with such spot IDs, classifications, and / or values, the spot IDs, classifications, and / or values ​​(e.g., the value "3" as shown for text 202at2) are not shown or displayed.

[0110] FIG5B illustrates another example image (e.g., digital twin augmented image 202a_dtalt) enhanced by an image simulation model according to various embodiments disclosed herein. FIG5B is an example of a graphically augmented simulated image according to the disclosure herein. For example, as described herein (e.g., with respect to FIG2A and FIG2B), the image simulation model may receive a user-specific digital twin image and a user-specific product recommendation as input. In the example of FIG5B, the user-specific digital twin image input to the image simulation model may include a digital twin image 202a_dt selected or generated for user 202a. In the examples of FIG5A and FIG5B,The digital twin augmented image region 202a_dt1alt in Figure 5A is represented and depicted as a scaled or cropped version of the digital twin image 202a_dtalt. Pixel data 202apalt corresponds to the skin region 202aralt in Figure 5B. In one embodiment, the digital twin augmented image 202a_dtalt may include an image output by an image simulation model, wherein, for example, the image simulation model is trained to output the digital twin augmented image 202a_dtalt with image augmentation. Additionally or alternatively, in different embodiments, the image simulation model may first output the image region 202a_dt1alt (as described with respect to Figure 5B). Then, the digital twin enhanced image region 202a_dt1alt can be overlaid or superimposed on the digital twin image 202a_dt at or around the skin region 202aralt to generate the digital twin enhanced image region 202a_dt1alt. In any specific implementation, and as shown with respect to FIG5B, pixels previously associated with melanin (e.g., melanin-related spots 202ar3 as shown with respect to FIG4A) are enhanced or otherwise updated, displaying melanin (e.g., dark-toned areas exemplifying skin conditions or problems in skin region 202ar) as reduced or eliminated, thereby demonstrating the effectiveness or other efficacy of a user-specific product recommendation (e.g., a skin cream product) for treating the skin condition or problem of user 202a. In this way, the digital twin enhanced image 202a_dt1alt includes a user-specific analog image that graphically depicts the user-specific digital twin image (i.e., digital twin image 202a_dt) as graphically enhanced based on a user-specific product recommendation.

[0111] FIG6 illustrates an example user interface 602 presented on a display screen 600 of a user computing device (e.g., user computing device 111c1) according to various embodiments disclosed herein. For example, as shown in the example of FIG6, the user interface 602 may be implemented or presented via an application (app) executing on the user computing device 111c1. For example, as shown in the example of FIG6, the user interface 602 may be implemented or presented via a native application executing on the user computing device 111c1. In the example of FIG6, the user computing device 111c1 is a user computer device as described with respect to FIG1, for example, where 111c1 is illustrated as an Apple iPhone implementing the Apple iOS operating system and having a display screen 600. The user computing device 111c1 may execute one or more native applications (apps) on its operating system, including, for example, imaging applications as described herein. Such native applications may be implemented or presented via a user computing device operating system (e.g.,The user interface 602 is implemented or encoded (e.g., as computational instructions) by a computational language (e.g., SWIFT) executed by the processor of the user computing device 111c1 (APPLE iOS).

[0112] Additionally or alternatively, the user interface 602 may be implemented or presented via a web interface, such as via a web browser application, such as Safari and / or Google Chrome applications, or other such web browsers.

[0113] As illustrated in the example of FIG6, the user interface 602 includes a graphical representation of the user's skin (e.g., image 202_dt1alt or a graphical representation of a portion thereof). Image 202_dt1alt may include an augmented digital twin image (or a graphical representation thereof) of the user (e.g., user 202a), which includes pixel data (e.g., pixel data 202apalt) of at least a portion of the skin region of the user's skin as described herein. In the example of Figure 6, the user's graphical representation (e.g., image 202_dt1alt) is annotated by one or more graphics (e.g., regions of pixel data 202apalt) or text presentations (e.g., text 202at2) corresponding to various features identifiable within the pixel data, which includes a portion of the user's skin region. For example, in some implementations, regions of pixel data 202apalt may be annotated or overlaid on top of the user's image (e.g., image 202_dt1alt) to highlight regions or features identified by the image simulation model or other AI model 108 within the pixel data (e.g., feature data and / or raw pixel data). In the example of Figure 6, regions of pixel data 202apalt indicate features as defined in the pixel data and defined by spot ID, classification, or value (e.g., the value "3" as shown for text 202at2), thereby indicating reduction or elimination of pigment spots (e.g., reduction and / or elimination of pixels at or near 202ap3 as described in Figure 4C). In various respects, pixels identified as having specific characteristics (e.g., any of pixels 202ap1-3) may be highlighted or otherwise annotated when presented on display 600. In other specific implementations, even if pixels at or near the skin region of pixel data 202apalt are associated with such spot IDs, classifications, and / or values, the spot IDs, classifications, and / or values ​​(e.g., the value "3" as shown for text 202at2) are not shown or displayed.

[0114] Text presentation (e.g., text 202at2) shows user-specific attributes or characteristics (e.g., the value "3") that may indicate that pixels near or at pixel 202ap3 (where the spot ID or value of the previously skin-colored area was 14 (see Figure 4C)) have been enhanced, thereby displaying skin problems or conditions (e.g., spots) as eliminated or reduced. For example,The value "3" can indicate a reduction (e.g., an improved result) in the value or category of the spot ID (e.g., within the range of 1-20) as depicted in image 202a_dt1alt, where pixel values ​​are updated to show improved skin. Such depictions are specific implementations illustrating the appearance expected of a user's skin after using the recommended product to improve its quality and / or appearance (e.g., normalizing spots or other skin discoloration). It should be understood that other text presentation types, categories, values, and / or ranges are considered herein, where text presentation types, categories, and values ​​may be presented as, for example, spot IDs such as melanin, hemoglobin, acne, and / or other skin conditions or problems. Additionally or alternatively, color values ​​may be used and / or overlaid on a graphical representation (e.g., image 202_dt1alt) shown on user interface 602 to indicate the degree or quality of a given spot ID, such as a high ID of 17 or a low ID of 2 (e.g., low RGB and / or Lab pixel values) or other cases. IDs may be provided as raw values, absolute scores, or percentage-based IDs. Additionally or alternatively, such IDs may be presented using textual or graphical indicators that indicate whether the ID represents a positive result (e.g., low discoloration indicating low sun exposure or skin irritation), a negative result (e.g., high discoloration indicating excessive sun exposure or skin irritation), or an acceptable result (average or acceptable value).

[0115] User interface 602 may also include or present a user-specific skin problem 610. In the embodiment of FIG. 6, the user-specific skin problem 610 includes a message 610m to the user, which is designed to instruct the user on a user-specific prediction, along with a brief description of any causes leading to the prediction. The output may include natural language data determined, for example, by AI models 108 and / or 108a.

[0116] The user interface 602 may also include or present user-specific product recommendations 612. For example, an imaging application may present at least one user-specific product recommendation on the display of a computing device based on user-specific predicted and / or simulated images. In various aspects, user-specific skin recommendations may include text recommendations, image-based recommendations, and / or a virtual representation of at least a portion of a user's skin area. For example, as shown in the example of FIG6, message 612m indicates to the user that a user-specific problem and / or condition is mild and further indicates to the user that the skin condition or problem is caused by pigmentation spots in the indicated area of ​​the user's skin. As shown in the example of FIG6, message 612m recommends that the user use a night cream to help reduce pigmentation spots. The night cream product may be a composition of hydroxycinnamic acid (HCA) and niacinamide at a low pH, as described herein. Product recommendations may be based on spot IDs (e.g., value 14) indicating that the user's image depicts a slightly discolored spot.The night cream product is designed to address discoloration detected or classified in the pixel data of image 202_dt1, or discoloration assumed in other ways, such as by a spot ID or classification output by an AI model based on natural language data and / or digital twin images. Product recommendations may be associated with identified features within the pixel data, and when features (e.g., pigmentation spots such as excessive melanin) are identified or classified, the user computing device 111c1 and / or server 102 may be instructed to output product recommendations.

[0117] The user interface 602 may also include or present a portion of a specific product recommendation 622 for the manufactured product 624r (e.g., the night cream as described above). Product recommendation 622 may correspond to a user-specific product recommendation 612, as described above. For example, in the example of FIG6, a user-specific product recommendation 612 may be displayed on the display screen 600 of the user computing device 111c1, accompanied by instructions (e.g., message 612m) to treat at least one feature (e.g., mild spot ID of 14 associated with a pigment spot (such as melanin) at or near pixel 202ap3) that can be identified in pixel data (e.g., pixel data 202ap), which includes pixel data of at least a portion of the skin area of ​​the user's skin.

[0118] As shown in FIG6, the user interface 602 recommends products (e.g., the manufactured product 624r (e.g., the night cream)) based on the user-specific product recommendation 612. The output or analysis of images (e.g., image 202_dt1) of AI models (e.g., AI models 108 and / or 108a) may be used to generate or identify recommendations for the corresponding products. Such recommendations may include products such as night creams, skin exfoliants, skin moisturizers, hydrating care products, information on avoiding excessive sun exposure, etc., to address user-specific problems detected within pixel data by AI models (e.g., AI model 108, specification pages 26 / 33, 30 CN 122207052 A and / or 108a).

[0119] In the example of FIG. 6, user interface 602 presents or provides recommended products (e.g., manufactured product 624r) determined by the AI ​​model and / or its related image analysis of image 202_dt1, its pixel data, and / or the user's natural language data (including phenotypic and / or demographic data). In the example of FIG. 6, this is indicated and annotated (624p) on user interface 602.

[0120] User interface 602 may also include selectable UI buttons 624s to allow a user (e.g., user 202a) to select to purchase or ship the corresponding product (e.g., manufactured product 624r). In some embodiments,The selection of selectable UI buttons 624s can enable one or more recommended products to be delivered to the user (e.g., the user of image 202a) and / or notify a third party of an individual's interest in a product. For example, user computing device 111c1 and / or server 102 can initiate the delivery of manufactured product 624r (e.g., night cream) to the user based on user-specific skin problem 610 and / or user-specific product recommendation 612. In such embodiments, the product can be packaged and delivered to the user.

[0121] In some specific embodiments, a graphical representation (e.g., image 202_dt1alt) with graphical annotations (e.g., a region of pixel data 202apalt), text annotations (e.g., text 202at2), user-specific skin problem 610, and user-specific product recommendation 612 can be transmitted to user computing device 111c1 via a computer network (e.g., from server 102 and / or one or more processors) for presentation on display screen 600. In other embodiments, no user-specific image is transmitted to the server, wherein user-specific skin issues 610, user-specific product recommendations 612 (and / or product-specific recommendations) may instead be generated locally by an AI model (e.g., AI model 108a) executed and / or implemented on the user's mobile device (e.g., user computing device 111c1) and presented on the mobile device's display 600 by the processor of the mobile device (e.g., user computing device 111c1).

[0122] In some embodiments, a graphical representation (e.g., image 202_dt1alt) with graphical annotations (e.g., region of pixel data 202ap), text annotations (e.g., text 202at2), user-specific skin issues 610, user-specific product recommendations 612 and / or product recommendations 622 may be presented in real-time or near real-time (e.g., locally presented on display 600) during or after receiving an image of a skin region having the user's skin. In embodiments where the image is analyzed by server 102, the image may be transmitted and analyzed by server 102 in real-time or near real-time.

[0123] In some embodiments, the user may provide additional natural language data, which may be transmitted to server 102 for updating, retraining, or reanalyzing by AI model 108. In other embodiments, the natural language data may be received locally on computing device 111c1 and analyzed on computing device 111c1 by AI model 108a. For example, as shown in the example of FIG6, the user may select a selectable button 612i for reanalyzing (e.g., locally at computing device 111c1 or remotely at server 102) new natural language data. The selectable button 612i may prompt the user interface 602 to initiate a chat, for example,Such as launching or displaying an LLM interface 300. Server 102 and / or user computing devices (such as user computing device 111c1) may receive natural language data about the user's phenotypic data and / or demographic data. Natural language data may be used to generate digital twin images, make product recommendations, and / or generate analog images of digital twin images after applying a recommended product. Analog images may include graphic enhancements, such as depicting and predicting how successfully a user will use a night cream to reduce melanin and / or hemoglobin, as detected by pixel data of the new image. Comments may include the user's need to correct features detected within the pixel data (e.g., any additional blemishes) by applying additional products (e.g., moisturizing oil, etc.).

[0124] Aspects of this Disclosure

[0125] The following aspects are provided as examples of what is disclosed herein and are not intended to limit the scope of the disclosure.

[0126] 1. An artificial intelligence (AI) based system configured to provide personalized skin product recommendations, the AI ​​based system comprising: one or more processors; one or more memories, the one or more memories being communicatively coupled to the one or more processors; an application (app) stored in the one or more memories and including computational instructions configured to execute on the one or more processors; a natural language model accessible by the application and trained using natural language data and demographic data of a corresponding user, the natural language model being configured to output one or more phenotypic classifications corresponding to one or more phenotypes of the corresponding user, and the natural language model being further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user; A synthetic image generation model, accessible by the application, is trained on a plurality of images corresponding to one or more phenotypic classifications and one or more demographic classifications output by the natural language model. The synthetic image generation model is configured to output a digital twin image of the corresponding user based on the plurality of images corresponding to the one or more phenotypic classifications and one or more demographic classifications as input. A product recommendation model, accessible by the application, is trained on the digital twin image of the corresponding user output by the synthetic image generation model. The product recommendation model is also trained on the one or more phenotypic classifications and one or more demographic classifications.The product recommendation model is configured to output product recommendations for the corresponding user based on the digital twin image, the one or more phenotypic classifications, and the one or more demographic classifications as input; an image simulation model, accessible by the application and trained on the digital twin image output by the synthetic image generation model, and also trained on the product recommendations output by the product recommendation model, the product recommendation model also being trained on the one or more phenotypic classifications and the one or more demographic classifications, the image simulation model being configured to generate simulated images based on the digital twin image, the simulated images having one or more graphic enhancements based on the product recommendations; and wherein the computational instructions of the application, when executed by the one or more processors, cause the one or more processors to: (a) receive user-specific natural language data from the user, The natural language data defines the user's user-specific phenotypic information and user-specific demographic data; (b) the user-specific natural language data is input into the natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user; (c) the one or more user-specific phenotypic classifications and the one or more user-specific demographic classifications are input into the synthetic image generation model, wherein the synthetic image generation model generates a user-specific digital twin image of the user; (d) The user-specific digital twin image of the user, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user are input into the product recommendation model, wherein the product recommendation model generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem identified in the user-specific digital twin image or the one or more user-specific phenotypic classifications; (e) the user-specific digital twin image, the user-specific product recommendations, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user are input into the image simulation model, wherein the image simulation model generates a user-specific simulated image based on the user-specific digital twin image and the user-specific product recommendations, the user-specific simulated image graphically depicting the user-specific digital twin image graphically enhanced based on the user-specific product recommendations; and (f) outputting natural language data describing the user-specific product recommendations to the user; and (g) displaying the user-specific simulated image on a display screen.

[0127] 2. The AI-based system according to aspect 1,The one or more phenotypic or demographic classifications include one or more of the following: (a) gender type classification; (b) age type classification; (c) ethnic type classification; or (d) skin distress type classification. Specification 28 / 33 pages 32 CN 122207052 A

[0128] 3. The AI-based system according to any one of aspects 1 or 2, wherein the natural language model includes a large language model.

[0129] 4. The AI-based system according to any one of aspects 1 to 3, the AI-based system further includes an imaging model trained on one or more corresponding images of the corresponding user, wherein the imaging model is further configured to output the one or more phenotypic classifications corresponding to one or more phenotypes of the corresponding user.

[0130] 5. The AI-based system according to aspect 4, wherein the output of the natural language model and the output of the imaging model are combined to determine the one or more phenotypes of the corresponding user.

[0131] 6. The AI-based system according to any one of aspects 1 to 5, wherein the synthetic image generation model comprises an AI model configured to generate photorealistic images, wherein the digital twin image comprises a photorealistic image, and wherein the user-specific digital twin image comprises a photorealistic image of the user.

[0132] 7. The AI-based system according to any one of aspects 1 to 6, wherein the product recommendation model comprises a model trained to determine the efficacy of one or more skin products corresponding to the one or more phenotypes of the corresponding user.

[0133] 8. The AI-based system according to any one of aspects 1 to 7, wherein the image simulation model comprises a model trained to determine the influence of one or more product attributes corresponding to the corresponding product of the product recommendation.

[0134] 9. The AI-based system according to any one of aspects 1 to 8, the AI-based system further comprising: providing the user with the user-specific digital twin image; receiving additional user-specific natural language data from the user to manipulate the user-specific digital twin image; and updating the user-specific digital twin image based on the additional user-specific natural language data.

[0135] 10. The AI-based system according to any one of aspects 1 to 9, wherein the user-specific simulated image is presented on the display screen in real time or near real time during or after receiving the user's natural language data.

[0136] 11. The AI-based system according to any one of aspects 1 to 10, wherein the user-specific product recommendation includes product recommendations for manufactured products.

[0137] 12. The AI-based system according to aspect 11,The system displays the user-specific product recommendation on the display screen, accompanied by instructions to treat at least one feature identifiable in pixel data including the user's skin region using the manufactured product.

[0138] 13. The AI-based system according to aspect 11, wherein the computational instructions further cause the one or more processors to initiate delivery of the manufactured product to the user based on the user-specific product recommendation.

[0139] 14. The AI-based system according to any one of aspects 1 to 13, wherein at least one of the one or more processors comprises a processor of a mobile device.

[0140] 15. The AI-based system according to any one of aspects 1 to 14, wherein the one or more processors comprises a server processor of a server, wherein the server is communicatively coupled to a computing device via a computer network, and wherein the application comprises a server application portion configured to execute on the one or more processors of the server and a computing device application portion configured to execute on the one or more processors of the computing device, the server application portion being configured to communicate with the computing device application portion, wherein the server application portion is configured to implement one or more of the instructions a to g according to any one of aspects 1 to x.

[0141] 16. An artificial intelligence (AI) based method for providing personalized skin product recommendations, wherein the AI-based method is described in the specification on pages 29 / 33 of CN 122207052 A. The method includes: implementing a multi-AI model training phase, the multi-AI model training phase comprising: training a natural language model by one or more processors using natural language data and demographic data of a corresponding user, the natural language model being configured to output one or more phenotypic classifications corresponding to one or more phenotypic traits of the corresponding user, and the natural language model being further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user; training a synthetic image generation model by the one or more processors on multiple images corresponding to the one or more phenotypic classifications and the one or more demographic classifications output by the natural language model, the synthetic image generation model being configured to output a digital twin image of the corresponding user based on the multiple images corresponding to the one or more phenotypic classifications and the one or more demographic classifications of the corresponding user as input; and training a product recommendation model by the one or more processors on the digital twin image of the corresponding user output by the synthetic image generation model, the product recommendation model also being trained on the one or more phenotypic classifications and the one or more demographic classifications.The product recommendation model is configured to output product recommendations to the corresponding user based on the digital twin image, the one or more phenotypic classifications, and the one or more demographic classifications as input; an image simulation model is trained by the one or more processors on the digital twin image output by the synthetic image generation model, and the image simulation model is also trained on the product recommendations output by the product recommendation model, the product recommendation model is also trained on the one or more phenotypic classifications and the one or more demographic classifications, the image simulation model is configured to generate a simulated image based on the digital twin image, the simulated image having one or more graphic enhancements based on the product recommendations, wherein each of the natural language model, the synthetic image generation model, the product recommendation model, and the image simulation model is stored in one or more memories accessible by an application (app), the application (app) including computational instructions stored in the one or more memories, the one or more memories being communicatively coupled to the one or more processors; and a multi-AI model execution phase is implemented, the multi-AI model execution phase including: (a) receiving user-specific natural language data of the user by the application, the natural language data defining user-specific phenotypic information and user-specific demographic data of the user. (b) The application inputs the user-specific natural language data into the natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user; (c) The application inputs the one or more user-specific phenotypic classifications and the one or more user-specific demographic classifications into the synthetic image generation model, wherein the synthetic image generation model generates a user-specific digital twin image of the user; (d) The application inputs the user-specific digital twin image of the user, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user into the product recommendation model, wherein the product recommendation model generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem identified in the user-specific digital twin image or the one or more user-specific phenotypic classifications; (e) The application inputs the user-specific digital twin image, the user-specific product recommendations, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user into the image simulation model, wherein the image simulation model generates user-specific simulated images based on the user-specific digital twin image and the user-specific product recommendations.The user-specific simulated image graphically depicts the user-specific digital twin image graphically enhanced based on the user-specific product recommendation; (f) the application outputs natural language data describing the user-specific product recommendation to the user; and (g) the application displays the user-specific simulated image on a display screen.

[0142] 17. The AI-based method according to aspect 16, wherein the one or more phenotypic or demographic classifications include one or more of the following: (a) gender type classification; (b) age type classification; (c) ethnic type classification; or (d) skin problem type classification.

[0143] 18. The AI-based method according to any one of aspects 16 to 17, wherein the natural language model includes a large language model.

[0144] 19. The AI-based method according to any one of aspects 16 to 18, the AI-based method further comprising an imaging model trained on one or more corresponding images of the corresponding user, wherein the imaging model is further configured to output the one or more phenotypic classifications corresponding to one or more phenotypic types of the corresponding user, wherein the output of the natural language model and the output of the imaging model are combined to determine the one or more phenotypic types of the corresponding user.

[0145] 20. A tangible non-transitory computer-readable medium storing instructions for providing personalized skin product recommendations, the instructions, when executed by one or more processors, causing the one or more processors to: implement a multi-AI model training phase, the multi-AI model training phase comprising: training a natural language model by the one or more processors using natural language data and demographic data of a corresponding user, the natural language model being configured to output one or more phenotypic classifications corresponding to one or more phenotypes of the corresponding user, and the natural language model being further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user; training a synthetic image generation model by the one or more processors on a plurality of images corresponding to the one or more phenotypic classifications and the one or more demographic classifications output by the natural language model, the synthetic image generation model being configured to output a digital twin image of the corresponding user based on the plurality of images corresponding to the one or more phenotypic classifications and the one or more demographic classifications of the corresponding user as input; and training a product recommendation model by the one or more processors on the digital twin image of the corresponding user output by the synthetic image generation model, the product recommendation model also being trained on the one or more phenotypic classifications and the one or more demographic classifications.The product recommendation model is configured to output product recommendations to the corresponding user based on the digital twin image, one or more phenotypic classifications, and one or more demographic classifications as input; an image simulation model is trained by the one or more processors on the digital twin image output by the synthetic image generation model, and the image simulation model is also trained on the product recommendations output by the product recommendation model, which is also trained on the one or more phenotypic classifications and one or more demographic classifications; the image simulation model is configured to generate simulated images based on the digital twin image, the simulated images having one or more graphic enhancements based on the product recommendations; wherein each of the natural language model, the synthetic image generation model, the product recommendation model, and the image simulation model is stored in one or more memories accessible by an application (app), the application (app) including computational instructions stored in the one or more memories, the one or more memories being communicatively coupled to the one or more processors. Or multiple processors; and a multi-AI model execution phase, the multi-AI model execution phase including: (a) receiving user-specific natural language data from the application, the natural language data defining user-specific phenotypic information and user-specific demographic data of the user; (b) inputting the user-specific natural language data into the natural language model by the application, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user; (c) inputting the one or more user-specific phenotypic classifications and the one or more user-specific demographic classifications into the synthetic image generation model by the application, wherein the synthetic image generation model generates a user-specific digital twin image of the user; (d) inputting the user-specific digital twin image, the one or more phenotypic classifications, and the one or more demographic classifications of the user into the product recommendation model by the application, wherein the product recommendation model is generated by the user. Page 35 CN 122207052 A User-specific product recommendations are designed to solve at least one problem in the identification of the user-specific digital twin image or the one or more user-specific phenotypic classifications; (e) The application inputs the user-specific digital twin image, the user-specific product recommendations, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user into the image simulation model, wherein the image simulation model generates user-specific simulated images based on the user-specific digital twin image and the user-specific product recommendations.The user-specific simulated image graphically depicts the user-specific digital twin image graphically enhanced based on the user-specific product recommendation; (f) the application outputs natural language data describing the user-specific product recommendation to the user; and (g) the application displays the user-specific simulated image on a display screen.

[0146] Additional Considerations

[0147] While this disclosure sets forth specific embodiments of several different implementations, it should be understood that the legal scope of this specification is defined by the content of the claims set forth at the end of this patent and their equivalents. The specific embodiments are to be understood as exemplary only and not as describing every possible implementation, as it would be impractical to describe every possible implementation. Numerous alternative implementations may be implemented using current technology or technology developed after the filing date of this patent, and these alternative implementations will still fall within the scope of the claims.

[0148] The following additional considerations apply to the foregoing discussion. Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Although the various operations of one or more methods are illustrated and described as separate operations, one or more of the operations may be performed simultaneously and need not be performed in the illustrated order. Structures and functionalities shown as separate components in the example configuration can be implemented as combined structures or components. Similarly, structures and functionalities shown as single components can be implemented as separate components. These types, modifications, additions, and improvements, as well as other variations, modifications, additions, and improvements, are all within the scope of this document.

[0149] Furthermore, some embodiments are described herein as including logic or multiple routines, subroutines, application programs, or instructions. These may constitute software (e.g., code embodied on a machine-readable medium or in transmitted signals) or hardware. In hardware, routines, etc., are tangible units capable of performing certain operations and can be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., standalone client or server computer systems) or one or more hardware modules (e.g., processors or processor groups) of a computer system may be configured by software (e.g., an application program or an application portion) as hardware modules for performing certain operations as described herein.

[0150] The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor implementation modules for performing one or more operations or functions. In some example embodiments, the modules mentioned herein may include processor implementation modules.

[0151] Similarly, the methods or routines described herein can be implemented at least in part by a processor. For example,At least some operations of the method can be performed by one or more processors or processor-implemented hardware modules. The execution of some operations can be distributed among one or more processors, which reside not only in a single machine but are deployed across multiple machines. In some example embodiments, one or more processors may be located in a single location, while in other embodiments, the processors may be distributed across multiple locations.

[0152] The execution of some operations can be distributed among one or more processors, which reside not only in a single machine but are deployed across multiple machines. In some example embodiments, one or more processors or processor-implemented modules may be located in a single geographic location (e.g., in a home environment, office environment, or server cluster). In other embodiments, one or more processors or processor-implemented modules may be distributed across multiple geographic locations. Specification 32 / 33 pages 36 CN 122207052 A

[0153] This specific embodiment is to be understood as exemplary only and not as describing every possible embodiment, as describing every possible embodiment would be impractical, even if possible. Numerous alternative embodiments can be implemented by those skilled in the art using current technology or technology developed after the date of this application.

[0154] Those skilled in the art will recognize that various modifications, alterations, and combinations can be made to the above embodiments without departing from the scope of the invention, and such modifications, alterations, and combinations should be considered to fall within the scope of the inventive concept.

[0155] The patent claims at the end of this patent application are not intended to be interpreted in accordance with 35 USC § 112(f) unless conventional component-plus-function language, such as “component for…” or “step for…”, as expressly referenced in the claims, is used. The systems and methods described herein relate to improvements in the functionality of computers and improvements in the operation of conventional computers.

[0156] The dimensions and values ​​disclosed herein should not be construed as strictly limited to the precise numerical values ​​referenced. Rather, unless otherwise specified, each such dimension is intended to represent the value and a range surrounding its functional equivalent. For example, a dimension disclosed as “40 mm” is intended to represent “about 40 mm”.

[0157] Unless expressly excluded or otherwise limited, every reference cited herein, including any cross-references or related patents or patent applications, and any patent application or patent claiming priority to or beneficial to this application with respect to them, is incorporated herein by reference in its entirety. Reference to any reference is not an endorsement of its existence as prior art to any disclosed or protected art herein, nor is it an endorsement of any such invention, either on its own or in combination with any one or more references. Furthermore,Where any meaning or definition of a term in this invention contradicts any meaning or definition of the same term in any document incorporated herein by reference, the meaning or definition given to that term in this invention shall prevail.

[0158] While specific embodiments of the invention have been illustrated and described, it will be apparent to those skilled in the art that various other changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, it is intended that all such changes and modifications falling within the scope of the invention be covered by the appended claims. Instruction manual, page 33 / 33; 37 CN 122207052 A; Figure 1; Instruction manual drawing 1 / 8 page; 38 CN 122207052 A; Figure 2A; Instruction manual drawing 2 / 8 page; 39 CN 122207052 A; Figure 2B; Instruction manual drawing 3 / 8 page; 40 CN 122207052 A; Figure 3; Instruction manual drawing 4 / 8 page; 41 CN 122207052 A; Figure 4A; Instruction manual drawing 5 / 8 page; 42 CN 122207052 A; Instruction manual drawing 6 / 8 page; 43 CN 122207052 A; Instruction manual drawing 7 / 8 page; 44 CN 122207052 A; Figure 6; Instruction manual drawing 8 / 8 page; 45 CN 122207052 A,

Claims

1. An artificial intelligence (AI)-based system configured to provide personalized skin product recommendations, the AI-based system comprising: One or more processors; One or more memories, the one or more memories being communicatively coupled to the one or more processors; An application (app) is stored in the one or more memories and includes computational instructions configured to execute on the one or more processors; A natural language model, accessible by the application and trained using natural language data and demographic data of a corresponding user, is configured to output one or more phenotypic classifications corresponding to one or more phenotypic traits of the corresponding user, and is further configured to output one or more demographic classifications corresponding to one or more demographic attributes of the corresponding user. A synthetic image generation model, accessible by the application and trained on multiple images corresponding to one or more phenotypic classifications and one or more demographic classifications output by the natural language model, is configured to output a digital twin image of the corresponding user based on the multiple images corresponding to the one or more phenotypic classifications and one or more demographic classifications of the corresponding user as input. A product recommendation model, accessible by the application and trained on the digital twin image of the corresponding user output by the synthetic image generation model, is also trained on one or more phenotypic classifications and one or more demographic classifications. The product recommendation model is configured to output product recommendations for the corresponding user based on the digital twin image, the one or more phenotypic classifications, and the one or more demographic classifications as input. An image simulation model, accessible by the application, is trained on the digital twin image output by the synthetic image generation model and on the product recommendation output by the product recommendation model, which is also trained on one or more phenotypic classifications and one or more demographic classifications. The image simulation model is configured to generate simulated images based on the digital twin image, the simulated images having one or more graphic enhancements based on the product recommendation. and The computational instructions of the application, when executed by the one or more processors, cause the one or more processors to: (a) Receive user-specific natural language data from a user, the natural language data defining the user's user-specific phenotypic information and user-specific demographic data. (b) Inputting the user-specific natural language data into the natural language model, wherein the natural language model generates one or more user-specific phenotypic classifications and one or more user-specific demographic classifications defining the user. (c) Inputting the one or more user-specific phenotypic classifications and the one or more user-specific demographic classifications into the synthetic image generation model, wherein the synthetic image generation model generates a user-specific digital twin image of the user. (d) Inputting the user-specific digital twin image of the user, the one or more phenotypic classifications of the user, and the one or more demographic classifications of the user into the product recommendation model, wherein the product recommendation model generates user-specific product recommendations for the user, the user-specific product recommendations being designed to solve at least one problem of identification in the user-specific digital twin image or the one or more user-specific phenotypic classifications. (e) Inputting the user-specific digital twin image, the user-specific product recommendation, the user's one or more phenotypic classifications, and the user's one or more demographic classifications into the image simulation model, wherein the image simulation model generates a user-specific simulated image based on the user-specific digital twin image and the user-specific product recommendation, the user-specific simulated image graphically depicting the user-specific digital twin image graphically enhanced based on the user-specific product recommendation, and (f) Output natural language data describing the user's specific product recommendations to the user; and (g) Display the user-specific simulated image on the display screen.

2. The AI-based system according to claim 1, wherein the one or more phenotypic or demographic classifications include one or more of the following: (a) sex type classification; (b) age type classification; (c) ethnic type classification; or (d) skin problem type classification.

3. The AI-based system according to any of the preceding claims, wherein the natural language model includes a large language model.

4. The AI-based system according to any of the preceding claims, the AI-based system further comprising an imaging model trained on one or more corresponding images of the corresponding user, wherein the imaging model is further configured to output the one or more phenotypic classifications corresponding to one or more phenotypic types of the corresponding user.

5. The AI-based system according to any of the preceding claims, wherein the output of the natural language model and the output of the imaging model are combined to determine the one or more phenotypes of the corresponding user.

6. The AI-based system according to any of the preceding claims, wherein the synthetic image generation model includes an AI model configured to generate photorealistic images, wherein the digital twin image includes photorealistic images, and wherein the user-specific digital twin image includes a photorealistic image of the user.

7. The AI-based system according to any of the preceding claims, wherein the product recommendation model comprises a model trained to determine the efficacy of one or more skin products corresponding to the one or more phenotypes of the corresponding user.

8. The AI-based system according to any of the preceding claims, wherein the image simulation model comprises a model trained to determine the influence of one or more product attributes corresponding to the product recommendation.

9. The AI-based system according to any of the preceding claims, wherein the AI-based system further comprises: Provide the user with a user-specific digital twin image; Receive additional user-specific natural language data from the user in order to manipulate the user-specific digital twin image; as well as The user-specific digital twin image is updated based on the additional user-specific natural language data.

10. The AI-based system according to any of the preceding claims, wherein the user-specific simulated image is presented on the display screen in real time or near real time during or after receiving the user's natural language data.

11. The AI-based system according to any of the preceding claims, wherein the user-specific product recommendation includes product recommendations for manufactured products.

12. The AI-based system of claim 11, wherein the user-specific product recommendation is displayed on the display screen, accompanied by instructions to treat at least one feature identifiable in pixel data including the user's skin region using the manufactured product.

13. The AI-based system according to claims 11 to 12, wherein the computation instructions further cause the one or more processors to: The manufacturer initiates the delivery of the manufactured products to the user based on the user-specific product recommendations.

14. The AI-based system according to any of the preceding claims, wherein at least one of the one or more processors comprises a processor of a mobile device.

15. The AI-based system according to any preceding claim, wherein the one or more processors include a server processor of a server, wherein the server is communicatively coupled to a computing device via a computer network, and wherein the application includes a server application portion configured to execute on the one or more processors of the server and a computing device application portion configured to execute on the one or more processors of the computing device, the server application portion being configured to communicate with the computing device application portion, wherein the server application portion is configured to implement one or more instructions of instructions a to g according to claim 1.

Images