Interface and analysis techniques for a personalized skincare regimen based on a skin diagnosis

Abstract

Interface and analysis techniques for a personalized skincare regimen based on a skin diagnosis. A computer system generates a computer-assisted personal care program based at least in part on a digital model of a user's face. In one embodiment, a computer system generates the model based on one or more images or scans of a user's face and generates a computer-assisted personal care program based on the model, environmental data, and user data. In one embodiment, a program generated on this basis includes multiple steps to be performed using a computer-controlled skincare device, as well as user prompts for performing these steps configured to be presented in a user interface. Figure for abstract: 1

Description

Title of the invention: Interface and analysis techniques for a personalized skin care regimen based on skin diagnosis

[0001] SUMMARY

[0002] This summary is provided to present a selection of concepts in a simplified form described in more detail below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0003] The present disclosure relates to the generation of personalized personal care regimens based on skin diagnosis. In described embodiments, a computer system generates computer-assisted personal care programs for smart personal care devices based on digital models of the user's skin.

[0004] In one aspect, a computer system performs operations including obtaining a digital model of the face of a living human subject; obtaining user data associated with the living human subject; and generating a computer-assisted skin care program based at least in part on the digital model of the face of the living human subject, the environmental data, and the user data. In one embodiment, the computer-assisted skin care program includes one or more operational parameters (e.g., brushing speed or intensity, cleaning duration) for a computer-driven skin care device. In another embodiment, the computer-assisted skin care program includes a product recommendation (e.g., a recommended device accessory, a skin cleanser, an eye cream, a sunscreen, a makeup product, etc.) presented by a user interface. In another embodiment, the computer-assisted skin care program includes a self-care recommendation (e.g., hydration recommendation, UV protection recommendation, stress reduction recommendation, sleep recommendation, etc.) presented by a user interface.

[0005] In one embodiment, the digital model is based on one or more digital scans of the face, and includes a plurality of facial features to be targeted in a skin care program, such as one or more blemishes, areas of hyperpigmentation, texture, or wrinkles, or a combination thereof.

[0006] In one embodiment, the computing system obtains environmental data (e.g., temperature data, humidity data, pollution data, or UV radiation data, or a combination thereof) associated with the environment of the living human subject, and the computer-assisted skin care program is further based on the environmental data. The environmental data is obtained from a remote computing system or one or more environmental sensors, or a combination thereof. In some embodiments, the environmental sensors are integrated into a skin care device or a smartphone or other computing device, in communication with a skin care device.

[0007] In some embodiments, a user interface includes one or more user interface elements for providing feedback on or modifying the computer-assisted skin care program, or other data upon which the program is based, such as user preferences (e.g., target age, device preferences). In such an embodiment, operations performed by the computer system further include obtaining user feedback (e.g., satisfaction rating or degree, adjusted age target or skin care goal, etc.) relating to the computer-assisted skin care program by the user interface, and / or modifying the computer-assisted skin care program.

[0008] Computer-implemented methods, computer-based data, computing devices, and computing systems are disclosed. Brief Description of the Drawings

[0009] The foregoing aspects and many of the advantages of the present invention derived therefrom will be better understood with reference to the following detailed description, taken in conjunction with the accompanying drawings, in which:

[0010] [Fig-1] [Fig.l] is a schematic diagram which provides a top-down overview system level that includes a skin care device according to aspects of the present disclosure;

[0011] [Fig.2] [Fig.2] is a block diagram illustrating an exemplary embodiment of a client computing device according to aspects of the present disclosure;

[0012] [Fig.3] [Fig.3] is a block diagram illustrating an exemplary embodiment of a skin care device according to aspects of the present disclosure;

[0013] [Fig.4] [Fig.4] is a flowchart illustrating an exemplary embodiment of a method for generating a skin care program assisted by computer, based at least in part on a digital model of the face of the living human subject according to aspects of the present disclosure;

[0014] [Fig.5] [Fig.5] is a screenshot diagram of a screen illustrating a skin care program selection screen, according to a described embodiment;

[0015] [Fig.6A] [Fig.6A] is a screenshot diagram illustrating a facial scan analysis screen, according to a described embodiment;

[0016] [Fig.6B] [Fig.6B] is another screenshot diagram illustrating a facial scan analysis screen, according to the same embodiment described as [Fig.6A];

[0017] [Fig.7] [Fig.7] is a screenshot diagram illustrating a questionnaire screen for obtaining user data, according to a described embodiment;

[0018] [Fig.8] [Fig.8] is a screenshot diagram illustrating a questionnaire screen for obtaining user data, according to the same embodiment described as [Fig.7];

[0019] [Fig.9] [Fig.9] is a screenshot diagram illustrating a questionnaire screen for obtaining user data, according to the same embodiment described as Figures 7 and 8;

[0020] [Fig. 10] [Fig. 10] is a screenshot diagram illustrating a results screen in which a skin care program has been generated based on a face scan and other inputs, according to a described embodiment;

[0021] [Fig. 11] [Fig. 11] is a screenshot diagram illustrating a skin care program tutorial screen, according to a described embodiment;

[0022] [Fig. 12] [Fig. 12] is a screenshot diagram illustrating a user preferences screen, according to a described embodiment;

[0023] [Fig. 13] [Fig. 13] is a screenshot diagram illustrating a skin care program editing screen, according to a described embodiment;

[0024] [Fig. 14] [Fig. 14] is a screenshot diagram illustrating a skin care program editing screen, according to the same embodiment described as [Fig. 13]; and

[0025] [Fig. 15] [Fig. 15] is a block diagram illustrating aspects of an exemplary computing device suitable for use with embodiments of the present disclosure. DETAILED DESCRIPTION

[0026] Although a wide variety of personal care products and personal care devices are available today, discovering how best to use these products or devices effectively often relies on a trial-and-error method. Although consumers may use search engines on Internet to find relevant information about the capabilities of these devices, they are often unable to configure or use these devices optimally based on this information. Furthermore, as these devices become more complex and include internet-connected and more programmable features, the act of configuring and using these devices also becomes more complex, especially for ordinary consumers. This leaves consumers unable to benefit from all the ways in which they can configure and use their devices.

[0027] Embodiments described herein provide technical solutions to one or more of the technical problems described above, or other technical problems.

[0028] In embodiments disclosed herein, a computer system generates a computer-assisted skin care program based at least in part on a digital model of a user's face. In one embodiment, a computer system obtains a digital scan of a user's face along with other inputs, such as environmental data associated with the user's environment, and user data, associated with the user. The system uses this data to generate a computer-assisted skin care program. In one embodiment, the computer system includes one or more computer servers that obtain the digital scan from a client device, such as a smartphone with an integrated digital camera, and generates a digital model of the user's face based on the digital scan.

[0029] In one embodiment, the computer-assisted skin care program includes user prompts, recommendations, or other output presented to a user via video, audio, or other output by a client computing device or by a human-machine interface integrated into a skin care device. In one embodiment, the generated program includes steps for using the skin care device tailored to the user's face, the user's environment, and other information about the user, such as the user's age and preferences, and skin care goals. The generated program may be a new program or a modified version of a previously existing program.For example, if a facial scan reveals a new defect that was not present at the time of development of an existing program, the generated program can be adapted to provide an additional step to treat the defect. As another example, an existing care program can be adjusted to account for new environmental data (e.g., reduced humidity, increased temperature, increased pollution, etc.). The generated program can be based on criteria such as characteristics. of a skincare device (e.g., available settings or accessories), previous results, user preferences or behaviors, time of day, current month / season of year, day of week, or similar.

[0030] A wide variety of operational parameters for skin care devices may be uniquely defined in assisted or programmed skin care programs. For example, adjustable parameters for electric skin brushes or massagers may include adjustable power settings, timers, speed settings (e.g., rotational or oscillation speed of a brush head), haptic feedback settings, or other parameters or combinations of parameters. These parameters may be automatically set and / or adjusted over time in a computer-assisted skin care program.

[0031] In one embodiment, the smart skincare device provides signals or prompts to users during a skincare program, such as audio signals, haptic signals, or visual signals, which may be used to guide a user through different phases of the skincare program, such as moving a skincare device to different skin areas, changing speeds, changing cleansing modes, and the like. In these embodiments, the smart skincare device may include speakers, displays, indicator lights, or haptic electronics, such as touch sensors or vibrating actuators.The assisted care program functionality of the smart skin care device can be activated in various ways, such as by pressing a physical button (e.g., an on / off switch or a dedicated button to activate a care program), by voice command or in any other way.

[0032] In one embodiment, a smart skincare device further includes a wireless communication circuit, such as a wireless transceiver for Wi-Fi communication. In this embodiment, the smart skincare device may access a voice interface service through a wireless communication circuit from a remote computing system. In another embodiment, the smart personal skincare device further includes a Bluetooth or Near Field Communication (NFC) circuit, either instead of or in addition to a Wi-Fi communication circuit. In this embodiment, the smart skincare device may communicate with an external client computing device, such as a smartphone or tablet computer, which may, in turn, access a voice interface service through its communication circuit from a remote computing system.In another embodiment, the smart skin care device includes circuitry for communicating with . a smart speaker, such as an Amazon Echo or Google Home device, which may include its own voice interface service or communicate with a remote computer system to access voice interface services.

[0033] In one embodiment, user data may be obtained from user profiles, which may be used to store system preferences, social network account or contact information, product / service provider preferences or ratings, location information, questionnaire responses, information from connected devices (e.g., smart personal care devices, such as hair dryers, electric skin brushes, or the like), or other information.

[0034] In one embodiment, a smart skincare device includes sensors to detect characteristics of the physical environment (e.g., humidity, temperature, ambient light levels, etc.) of the device during a skincare program. Alternatively, the system may obtain data from remote sensors that may not be included in the skincare device itself. The information generated by these sensors may be used by the skincare device (or computer systems that may be in communication with the device) so that it can guide the user through the program or make adjustments to achieve better results.

[0035] Information obtained from devices may be used to guide personalized product or care program selections, or for other purposes. This information may include device identification information, configuration information, usage information (e.g., number of times a device is used, duration of use, time of day, etc.), sensor information (e.g., measurements of environmental conditions, such as temperature, humidity, pollution levels, UV radiation levels, etc., or skin types, such as pH levels), location information, combinations of this information, or other information. Sensor information may be provided by Internet of Things (IoT) devices connected or already configured to receive sensors. These devices may include weather stations, mobile computing devices, and the like..

[0036] [Fig. 1] is a block diagram illustrating a system in which various aspects of the present disclosure may be implemented. As shown in [Fig. 1], the system 100 includes a skin care device 102 (in the illustrated example, a skin brush or an electric massager) with a wireless communication circuit. The skin care device 102 may send and / or receive information (e.g., usage data, device identification / configuration data, environmental data, or if military) to one and / or from a remote computing system 110, either directly or through one or more intermediary devices, such as a client computing device 104, as described below.

[0037] In the illustrated arrangement shown in [Fig. 1], a client computing device 104 connects to the remote computing system 110, which may generate personalized content for a user, such as computer-assisted skin care programs, product recommendations, or settings or parameters for operating a personal care device. In examples described herein, the personal care device is a skin care device 102, as described in more detail below. In one embodiment, a personalized care program for a skin cleansing device includes a defined pattern of skin area to be cleaned during the program, durations for each skin area, and brush head speeds or cleaning modes for each skin area.In one embodiment, customized programs or device settings may be downloaded to the client computing device 104 for later transmission to the skin care device 102. The skin care devices 102 may also include or communicate with environmental sensors and / or other computing devices. Illustrated components of the remote computing systems 110 and the skin care device 102 are described below.

[0038] The client computing device 104 may be used by a consumer, a skincare professional, or other entities to interact with other components of the system 100, such as remote computing systems 110 or the skincare device 102. In one embodiment, the client computing device 104 is a mobile computing device such as a smartphone or a tablet computer. However, any other suitable type of computing device capable of communicating over the network and having a user interface may be used, including but not limited to a desktop computer, a laptop computer, a smart speaker, or a smartwatch (or combinations of these devices).

[0039] Illustrated components and functionality of the remote computing system 110 will now be described. The remote computing system includes one or more computing servers that implement the illustrated components, for example, in a cloud computing device. As illustrated in [Fig.l], the computing system 110 includes a care program engine 112, a product data store 120, and a user profile data store 122. The care program engine 112 generates assisted care program information, which may then be transmitted, for example, to the client computing device 104. and / or to the skin care device 102. The assisted care program information may include, for example, care program instructions for programming or configuring the skin care device 102 in a particular manner, product recommendations, tutorials, or other information.

[0040] In one embodiment, the care program engine 112 generates assisted care program information based on receiving information from the product data store 120 and user information from the user profile data store 122, the skin care device, the client computing device 104, or a combination thereof, or any other source or combination of sources. The care program engine 112 may, for example, receive a request for an updated care program from the skin care device 102 or the client computing device 104, obtain information from the product data store 120 (e.g., available settings and configurations for the skin care device 102, available accessories, etc.), from the user profile data store 122 (e.g., user responses to questions, device usage data, preferred care programs, location, age, products used, etc., or from the client computing device 104 (e.g., information describing the user's current location, satisfaction with previous programs (indicated, for example, by means of a numerical rating system or a number of stars), current stress level and mood, etc.) and use this information to perform further processing. The care program engine 112 may use the obtained information, for example, to generate a new program or update a previously defined program.

[0041] The care program engine 112 may employ artificial intelligence or machine learning techniques (e.g., comparison to models, feature comparison and extraction, classification, artificial neural networks, deep learning architecture, genetic algorithms, or the like). In one embodiment, machine learning techniques are used to analyze data, such as a digital scan of a user's face, environmental data, and user data, and create or revise care programs, product recommendations, or other content based on that data. For example, to generate a personalized care program, the care program engine 112 may analyze digital scans to measure or map wrinkles, pigmentation, skin texture, etc., of the user's skin.In such a scenario, the skincare program engine 112 may use this information to recommend, generate, or modify a particular skincare program that adapts to the particular characteristics of the user's skin.

[0042] In one embodiment, location information obtained by a skin care device or client computing device may be used to search for and obtain other information from other devices or computing systems that may be relevant to a care program. For example, a client computing device 104 may provide location information to a remote computing system 110, which may, in turn, obtain current environmental data (e.g., weather information, pollution information, UV radiation information, etc.) for the respective location. The remote computing system 110 may then use the environmental data to generate or modify a care program.In one illustrated scenario, the remote computing system 110 uses the location information to determine whether the user is in a polluted, hot city, and generates or modifies a care program to account for that environment.

[0043] In one embodiment, information is stored, transmitted, and shared within the system 100 in a secure and reliable manner. For example, secure connections are established between client computing devices and skincare devices, between client computing devices and server systems, or any other set of communication devices. Users may be securely reminded of skincare program events, environmental alerts (e.g., temperature, humidity, pollution, or UV radiation warnings), product expirations, and the like via instant notifications to a client application, email, instant messaging, or any other communication channel.

[0044] [Fig. 2] is a block diagram illustrating an exemplary embodiment of a client computing device 104 according to various aspects of the present disclosure. [Fig. 2] represents a non-limiting example of features and configurations of client computing devices; many other features and configurations are possible within the scope of the present disclosure.

[0045] In the example shown in [Fig. 2], the client computing device 104 includes a camera 250 and a client application 260. The client application 260 includes a user interface 276, which may include interactive functionality, such as data collection or questionnaire items, tools for entering or editing user preferences, tools for editing skincare programs, tutorials, a virtual “try-on” functionality for virtually trying on different products or cosmetics, or other items. Visual elements of the user interface 276 are presented on a display 240, such as a touchscreen display. Personalized content, such as computer-assisted skincare programs, may be obtained by the client computing device 104 (e.g., from the system in remote computing 110) and presented by the user interface 276. Details of a user interface illustration are described below, with reference to FIGURES 5-14.

[0046] In one embodiment, the client application 260 also includes an image capture / scanning module 270, which is configured to capture and process digital images (e.g., color images, depth images, etc.) or scans upon which digital models described herein are based. In one embodiment, the digital images or scans are transmitted to the remote computing system 110 or any other external computing system in which the digital skin models are generated. In another embodiment, the digital models are generated in the client computing device 104, or at any other location.In one embodiment, the digital models include 3D topological and texture information that can be used to reproduce an accurate representation of the user's facial structure and overall appearance, as well as to perform skin diagnostics (e.g., to detect blemishes, areas of hyperpigmentation, visible pores, etc.). In one embodiment, the user interface 276 includes interface elements to assist in accurately capturing the digital images or scans upon which these digital skin models are based.

[0047] In one embodiment, a communication module 278 of the client application 260 is used to prepare information for transmission to, or to receive and interpret information from, other devices or systems, such as the remote computing system 110 or a skincare device 102. This information may include captured digital images, scans, or videos, skincare device settings, personalized skincare programs, user preferences, user identifiers, device identifiers, or the like.

[0048] Other features of client computing devices are not shown in [Fig. 2] for ease of illustration. A description illustrating computing devices is provided below, with reference to [Fig. 15].

[0049] [Fig. 3] is a block diagram illustrating components included in an exemplary embodiment of a skin care device, according to various aspects of the present disclosure. [Fig. 3] represents a non-limiting example of features and configurations of a skin care device; many other features and configurations are possible within the scope of the present disclosure.

[0050] In the example shown in [Fig. 3], the skin care device 102 includes a treatment application unit 302 configured to apply a treatment to a user, a power source 304, a human-machine interface device 306, a processor 310, a network interface 312, and a computing medium 314. A non-limiting example of a skin care device 102 is an electric facial cleansing brush, with a brush head that oscillates, rotates, or otherwise moves, so as to perform a cleansing action. Other non-limiting examples of skin care devices 102 include a massage device or a phototherapy device that can be used to expose the skin to light (e.g., at a particular wavelength and / or for a particular duration) to achieve therapeutic benefits.

[0051] In one embodiment, the treatment application unit 302 includes one or more devices that collectively apply a treatment to a user. For example, if the skin care device 102 is an electric skin brush, the treatment application unit 302 may include a drive motor, an armature coupled to the drive motor that accepts a removable brush head, and the brush head itself. As another example, if the skin care device 102 is a handheld phototherapy device, the treatment application unit 302 may include one or more light-emitting diodes (LEDs) or other light-emitting devices that emit light at wavelengths and intensity appropriate for phototherapy applications.

[0052] In one embodiment, the power source 304 is a rechargeable battery that powers the treatment application unit 302 for its operation. The power source 304 may also provide power to operate the other components of the skin care device 102. In other embodiments, instead of a battery, the skin care device 102 may be coupled to an external power source, such as an electrical outlet.

[0053] The human-machine interface (HMI) 306 may include any type of device capable of receiving user input or generating output for presentation to a user. In one embodiment, the HMI 306 includes a speaker 308 to enable the care device 102 to present audio content (e.g., care program signals in the form of recorded or synthesized speech, tones, etc.), and / or a microphone receives audio input, such as voice commands from the user. In addition to voice signals, the speaker 308 may present another output to accompany the care program, such as music, sounds of a relaxing nature, or the like. Other non-limiting examples of possible components of the HMI 306 include a pushbutton switch, a rocker switch, a capacitive switch, a rotary switch, a slide switch, and a touchscreen.

[0054] The processor 310 is configured to execute computer-executable instructions stored on the computing medium 314. In one embodiment, the processor 310 is configured to receive and transmit signals to and / or from other components of the skin care device 102 via a communication bus or other circuit. The network interface 312 is configured to transmit and receive signals to and from the client computing device 104 (or other computing devices) on behalf of the processor.The network interface 312 may implement any suitable communication technology, including, but not limited to, short-range wireless technologies, such as Bluetooth, infrared, near-field communication, and Wi-Fi; long-range wireless technologies, such as WiMAX, 2G, 3G, 4G, LTE, and 5G; and wired technologies, such as USB, FireWire, and Ethernet. The computing medium 314 is any type of computing medium on which computer-executable instructions may be stored, including, but not limited to, flash memory, ROM, EPROM, EEPROM, and an FPGA. The computing medium 314 and the processor 310 may be combined into a single device, such as an ASIC, or the computing medium 314 may include a cache memory, a register, or another component of the processor 310.

[0055] In the illustrated embodiment, the computing medium 314 stores computer-executable instructions that, in response to execution by one or more processors 310, cause the skin care device 102 to implement a care program assistance engine 316 and a treatment control engine 318. The treatment control engine 318 controls one or more aspects of the skin care device 102 in a care program, as described above. In one embodiment, the care program is generated and / or modified by the care program engine 112, as described above. In one embodiment, the treatment control engine 318 adjusts settings or configurations for the skin care device 102, which may be generated and / or modified by the care program engine 312, as described above.In one embodiment, the treatment control engine 318 controls basic functions, such as turning the treatment application unit on or off, controlling the speed of the brush head, or the like. In one embodiment, the treatment control engine 318 detects input from 1THM 306, and activates the treatment application unit 302 or modifies a function of the skin care device 102 in response to the input. The treatment control engine 318 may then detect subsequent input from the HMI 306 and deactivate the treatment application unit 302 or make other adjustments to the function of the skin care device 102 in response, or allow the application unit to . treatment 302 to operate for a predetermined period of time before automatically deactivating the treatment application unit 302.

[0056] In one embodiment, the care program assistance engine 316 controls the presentation of care program signals, such as by causing the speaker 308 to produce audio signals, so as to guide a user's actions during a care program. In one embodiment, the care program assistance engine 316 collects data, such as data describing care program execution, environmental data, or other data.In one embodiment, the data collected by the care program assistance engine 316 is transmitted via the network interface 312 to the remote computing system 110 (e.g., by the client computing device 104 or directly) for storage (e.g., in the product data store 120 or the profile data store 122) or for further processing (e.g., to update a computer-assisted care program, adjust settings of skin care devices, etc.). In one embodiment, the collected data is used with the facial scan data to make these updates or adjustments.

[0057] In one embodiment, the processor(s) 310 and the computing medium 314 provide circuitry that may be collectively referred to as a skin care unit of the skin care device 102. In one embodiment, the skin care unit is operatively coupled with the HMI 306 and the treatment application unit 302, and includes circuitry for controlling the treatment application unit 302. In one embodiment, the skin care unit includes circuitry for presenting voice signals, visual signals, or other signals by the HMI in a computer-assisted skin care program to guide the operator in using a skin care device. Alternatively, the skin care device 102 may include different circuitry, or the circuitry may be implemented in an entirely different manner.

[0058] The devices shown in FIGURES 1-3 or other devices used in described embodiments may communicate with each other via a network (not shown), which may include any suitable communication technology, including, but not limited to, wired technologies such as DSL, Ethernet, fiber optics, USB, and Firewire; wireless technologies such as WiFi, WiMAX, 3G, 4G, LTE, 5G, and Bluetooth; and the Internet. In general, communication between components of the systems in [Fig.l] or other computing devices may occur directly or through intermediary devices.

[0059] Many other arrangements disclosed and described with reference to FIGURES 1-3 are possible. For example, the functionality described as being implemented in multiple components may instead be combined into a single component, or functionality described as being implemented in a single component may be implemented in multiple illustrated components, or in other components not shown in FIGURES 1-3. As another example, devices in FIGURES 1-3, illustrated as including particular components may, instead, include more components, fewer components, or different components without departing from the scope of the described embodiments.

[0060] In components of the system shown in [Fig.l] or devices shown in FIGURES 2 and 3, or by components of these systems and devices operating in combination, several technical advantages are achieved. For example, the ability to automatically generate or modify care programs based on digital scans of a user's face in combination with additional data, such as user data and environmental data, overcomes the technical limitations of previous technologies that depended on the user's abilities to configure their own devices.As another example, the system 100 allows certain aspects of the process to be performed independently by skin care devices or client computing devices, while moving other processing is the responsibility of the remote computing system 110 (which may be a relatively reliable and powerful computing system), thereby improving performance and preserving battery life to support the functionality provided by skin care devices or client computing devices.

[0061] Generally, the word "engine," as used herein, refers to logic embedded in hardware or software instructions written in a programming language, such as C, C++, COBOL, JAVA™, PHP, Perl, HTML, CSS, JavaScript, VBScript, ASPX, Microsoft .NET™, and / or the like. An engine may be compiled into executable programs or written in interpreted programming languages. Software engines may be called by other engines or by themselves. Generally, the engines described herein relate to logic modules that may be merged with other engines or divided into sub-engines. Engines may be stored in any type of computer media or computer storage device, and be stored on and executed by one or more general-purpose computers, thereby creating a specialized computer configured to provide the engine or the functionality thereof.

[0062] As may be understood by those skilled in the art, a "data store" as described herein may be any suitable device configured to store data accessible by a computing device. An example of a data store is a fast and reliable database management system (DBMS) that runs on a or more computing devices, and accessible over a high-speed network. Another example of a data store is a key-value store. However, any other suitable storage technique and / or device capable of quickly and reliably providing the stored data, in response to queries, may be used, and the computing device may be accessed locally instead of over a network, or may be provided as a “cloud” service. A data store may also include data stored in an organized manner on a computing medium, as described below. Those skilled in the art will recognize that separate data stores described herein may be combined into a single data store, and / or a single data store described herein may be separated into multiple data stores, without departing from the scope of this disclosure.

[0063] [Fig. 4] is a flowchart illustrating an exemplary embodiment of a method for generating a computer-assisted skin care program based at least in part on a digital model of the face of the living human subject. As illustrated, the method 400 is implemented by a computer system. The method 400 may be implemented by a server computer system including features of the remote computer system 110, by a client computing device, by a skin care device 102, or by any other computer device or system.

[0064] From a starting block, the method 400 proceeds to block 402, where the computer system obtains a digital model (e.g., based on one or more digital images or scans) of the face of a living human subject. In one embodiment, the computer system includes a server computer that obtains one or more digital images or scans from a client device, such as a smartphone with an integrated digital camera. In such an embodiment, these images or scans are captured by the client device and uploaded to the server computer, which generates the digital model. In one embodiment, the digital model includes a highly accurate model of facial features, including the edges of the eyes and lips, the size and position of the iris, skin features including pimples, texture, and wrinkles, and the like.It is understood that the skin features described herein are exemplary only, and that other features or combinations thereof are also desirable and within the scope of this disclosure. In one embodiment, the source images are captured and the digital models are generated using Modiface software available from Modiface, Inc.

[0065] The method 400 proceeds to an optional step 404, in which the computer system obtains environmental data associated with the environment of the living human subject. In one embodiment, the computer system obtains environmental data from environmental sensors at the user's location (e.g., by sensors embedded in a client computing device or a skincare device used by the user). Alternatively, the computing system obtains environmental data from another device or computing system. In one embodiment, the computing system determines the user's approximate location (e.g., as reported by the user or a location-rendering computing device) and requests environmental data relating to that location from a computing system for providing weather or environmental data.

[0066] The method proceeds to step 406, in which the computing system obtains user data associated with the living subject. In one embodiment, user data may be obtained from user profiles stored in a user profile data store 122. In another embodiment, user data may be obtained directly from a client computing device 104, a skincare device 102, or from any other source. In one embodiment, the user data includes system or device preferences, device usage information, contact or social network account information, product / service provider preferences or ratings, location information, questionnaire responses, or a combination thereof, or other information.In one embodiment, the user data includes an age target, which a user may set to adjust a skin care program to a particular target age. For example, a 50-year-old user may select an age target of 45, which results in the generation of a moderate skin care program, to achieve a moderate difference in appearance. In another embodiment, the 50-year-old user selects an age target of 40, which results in the generation of a more intensive skin care program.

[0067] The method proceeds to step 408, in which the computer system generates a computer-assisted skin care program based at least in part on the digital model of the face of the living human subject, the environmental data, and the user data. In one embodiment, the skin care program engine 112 of the remote computer system 110 obtains these inputs and generates the computer-assisted skin care program. In another embodiment, the program is generated based on different combinations of inputs. For example, if environmental data is not obtained or is not available, the computer system generates a computer-assisted skin care program based on the digital model of the face and the user data.

[0068] In one embodiment, certain aspects of the skin care program computer-assisted skin care routines, such as user prompts, are configured to be presented in a user interface. In one embodiment, the computer-assisted skin care routine automatically includes activation of one or more voice signals or other prompts stored in the memory of the skin care device. Alternatively, the routine is presented by a display or other modality. The computer-assisted skin care routine may be controlled, for example, by the skin care routine assistance engine 316 of the skin care device 102.

[0069] In one embodiment, a user interface (UI) includes one or more user interface elements for providing or modifying user data, capturing facial images or scans, or otherwise obtaining or modifying data upon which computer-assisted skin care programs may be based, and / or for providing feedback on or modifying a previously generated computer-assisted skin care program. In such an embodiment, operations performed by the computer system further include obtaining user feedback (e.g., a satisfaction rating, an adjusted age target or skin care goal, etc.) relating to the computer-assisted skin care program by the UI and / or modifying the computer-assisted skin care program.

[0070] Aspects of an illustrative UI for collecting information and providing user feedback and control for generated computer-assisted skin care programs will now be described with reference to FIGURES 5-14. The UI shown with reference to FIGURES 5-14 is exemplary only, and it is understood that many other UI features, as well as other techniques for generating computer-assisted skin care programs, may be developed in accordance with the present disclosure.

[0071] [Fig.5] shows a screenshot of a selection screen illustration of skin care programs 500, according to a described embodiment. The screen 500 includes options for selecting a predefined skin care program or, if the user so desires, for generating a personalized skin care program. The screen 500 includes an illustration of UI elements in the form of a button 502 for initiating a process for capturing one or more facial images or scans, a button 504 for initiating a process for obtaining environmental data, and a button 506 for obtaining user data, in the form of a questionnaire or a "skin quiz." In one embodiment, upon activation of the button 502, a user is directed to take a self-portrait or series of such images, which is analyzed by facial scanning software to generate a digital model of the user's face that includes skin characteristics. In a embodiment, the features include eye and lip edges, skin features including blemishes, areas of hyperpigmentation, texture, or wrinkles, or other features or combinations thereof, these. The home screen includes an additional button 508 that can be used to add a predefined skin care program.

[0072] FIGURES 6A and 6B depict screenshots of an illustration of face scan analysis screens that may be presented to a user after generating a face scan, according to a described embodiment. As mentioned above, the model of the user's face may include detected facial features, such as lip and eye edges, as well as skin features, such as wrinkles, texture, and blemishes. In [Fig. 6A], the screen 600 includes a self-portrait image of the user with graphical overlays depicting areas of interest or concern on a user's face, including detected areas of wrinkles, sagging, and visible pores. These overlays are developed based on the results of the face scan.The example shown on the display 600 also includes skin ratings, including an overall condition rating for the user's face, as well as sub-ratings relating to visible pores, wrinkles, sagging, and pigmentation, and labeled graphical overlays corresponding to detected problem areas. The display includes a scrollable tab 610 that allows a user to scroll the display to view personal skin care recommendations. In [Fig. 6B], the display 602 shows the tab 610 in the scrolled position, revealing additional feedback relating to a skin problem. In addition, the region in which that problem is most prominent is displayed in the graphical overlay 612, which is generated based on the facial scan.

[0073] FIGURES 7-9 depict screenshots of an illustration of questionnaire screens 700, 800, and 900, according to a described embodiment. The illustrated screens include UI elements that are used to collect user data on topics such as skin type, skin care goals, and facial areas, which may be of particular concern to the user. Other UI elements may be used to collect information on similar or different topics.

[0074] [Fig. 10] depicts a screenshot of an illustration of a results screen in which a skin care program has been generated based on a facial scan and other inputs, according to a described embodiment. On the results screen 1000, the user is informed by a UI element 1010 that a program has been generated based on the facial scan, environmental data, and user data. For example, the program may include steps to reducing visible pores identified in the facial scan, and using a cleanser appropriate for hot and humid weather identified in the environmental data, and a longer cleansing time for areas of attention identified by the user. The user has the option, via button 1020, to add this new program to their skin care device. In one embodiment, activation of button 1020 causes the program data and instructions to execute the new program to be transmitted by the client computing device 104 to the skin care device 102 via Bluetooth. In another embodiment, the program data and instructions corresponding to the generated skin care program may be transmitted, stored, or implemented in some other manner.

[0075] [Fig. 11] shows a screenshot of an illustration of a tutorial screen skin care program 1100, according to a described embodiment. On the tutorial screen 1100, the UI provides graphic animations that are overlaid on a selfie image and that show, for example, usage pathways for cleansing steps, such as the forehead cleansing step illustrated on the screen 1100. In one embodiment, proper placement of the graphic animation relative to the selfie image is guided by the previously generated template based on the facial scan.

[0076] [Fig. 12] shows a screenshot of an illustration of a screen of user preference 1200, according to a described embodiment. The screen 1200 includes an example of how the UI may be designed to not only set user preferences, such as reminders to follow a schedule, but also to provide opportunities to update previously generated schedules with new facial scans or other data. The screen 1200 includes a reminder for a “selfie check every 7 days.” This may be used to remind a user to record a new facial scan every 7 days (or in any other desired interval) and then update a previously generated schedule, if applicable. In addition to the features shown in [Fig. 12], other user preferences (e.g., device settings, sound or voice settings, haptic feedback settings, visual UI settings, etc.) can also be defined using similar or different UI elements.

[0077] In one embodiment, the computer-assisted care program includes a plurality of steps, and the step of generating (e.g., creating or updating) the computer-assisted care program includes selecting from available options for one or more steps based on a combination of data including a digital scan of the user's face and other user data. liser (e.g., user preferences, device usage patterns, questionnaire / survey responses, etc.), as well as environmental data. In one embodiment, these selections are further based on selection criteria, such as skincare device characteristics (e.g., available brush heads, available brush head speeds), previous results, time of day, day of week, day of year, or other selection criteria, or a combination thereof.

[0078] [Fig. 13] depicts a screenshot of a skincare program editing screen illustration 1300, according to a described embodiment. Screen 1300 includes further examples of how the UI may be designed to update previously generated programs. Screen 1300 includes UI elements showing current steps in a program (neck and chin cleansing steps), and a button element 1310 configured to allow a user to add a step to the program, as well as a button element 1320 configured to allow a user to view and / or edit a serum or cream for use in the program. In one embodiment, activating button 1310 brings up another program editing screen 1400, as shown in [Fig. 14].In the illustration of screen 1400, the UI includes sliders 1410, 1420, which allow the user to select parameters (e.g., intensity and duration) for adding a step to a program, as well as a button element 1430, which allows a user to select a desired position (e.g., chin, cheek, etc.) for adding the step.

[0079] In one embodiment, the parameters to be added to the steps depend on previously collected data. In one scenario illustration, a user has previously indicated that they have sensitive skin, and the user's facial scan indicates a problem area that may benefit from a longer treatment time. In the context of [Fig. 14], this scenario involves a default value for the new step set at a minimum intensity level ("1"), and a maximum value ("3") of slider 1420 is reduced to avoid intensity levels that are not appropriate for sensitive skin. Furthermore, slider 1410 is set to a default duration (e.g., 20 s.) that is appropriate for a problem area identified in the facial scan.

[0080] [Fig. 15] is a block diagram illustrating aspects of an exemplary computing device 1500 suitable for use with embodiments of the present disclosure. While [Fig. 15] is described with reference to a computing device implemented as a device on a network, the description below is applicable to servers, personal computers, mobile phones, smartphones, tablets, embedded computing devices, and other devices positives, which may be used to implement portions of embodiments of the present disclosure. In addition, those skilled in the art and others will recognize that the computing device 1500 may be any of any number of devices currently available or yet to be developed.

[0081] In its most basic configuration, the computing device 1500 includes at least a processor 1502 and system memory 1504 connected by a communication bus 1506. Depending on the exact configuration and type of device, the system memory 1504 may be volatile or non-volatile memory, such as read-only memory (“ROM”), random access memory (“RAM”), EEPROM, flash memory, or similar memory technology. Those skilled in the art and others will recognize that the system memory 1504 generally stores data and / or program modules that are immediately accessible to and / or currently being operated on the processor 1502. In this regard, the processor 1502 may serve as the computing center of the computing device 1500 by supporting the execution of instructions.

[0082] As further illustrated in [Fig. 15], the computing device 1500 may include a network interface 1510 comprising one or more components for communicating with other devices on a network. Embodiments of the present disclosure may access basic services that utilize the network interface 1510 to perform communications using common network protocols. The network interface 1510 may also include a wireless network interface configured to communicate via one or more wireless communication protocols, such as WiFi, 2G, 3G, LTE, 5G, WiMAX, Bluetooth, and the like.

[0083] In the exemplary embodiment depicted in [Fig. 15], the computing device 1500 also includes a storage medium 1508. However, services may be accessed using a computing device that does not include a means for recording data to a local storage medium. Therefore, the storage medium 1508 illustrated in [Fig. 15] is represented by a dashed line to indicate that the storage medium 1508 is optional. In any event, the storage medium 1508 may be volatile or non-volatile, removable or non-removable, implemented using any technology capable of storing information, such as, but not limited to, a hard disk drive, a solid state drive, a CD ROM, a DVD, or other disk storage, magnetic cassettes, magnetic tape, a magnetic disk storage unit, and / or the like.

[0084] As used herein, the term "computer medium" includes volatile and non-volatile media and removable and non-removable media implemented in any method or technology capable of storing information, such as computer-readable instructions, data structures, program modules, or other data. To this In this regard, the system memory 1504 and the storage medium 1508 depicted in [Fig. 15] are simple examples of computing media. In one embodiment, the computing media are used to store data for use by programs.

[0085] Suitable commercially available implementations of computing devices are known, including a processor 1502, system memory 1504, a communications bus 1506, a storage medium 1508, and a network interface 1510. For ease of illustration, and since it is not important to understand the claimed subject matter, [Fig. 15] does not show some of the typical components of many computing devices. In this regard, computing device 1500 may include input devices, such as a keyboard, keypad, mouse, microphone, touch input device, touchscreen, tablet, and / or the like. These input devices may be coupled to computing device 1500 by wired or wireless connections including RF, infrared, serial, parallel, Bluetooth, USB, or other suitable connection protocols using physical or wireless connections.Similarly, computing device 1500 may also include output devices, such as a display, speakers, a printer, etc. Since these devices are well known in the art, they are not illustrated or described in further detail herein. Extensions and other solutions

[0086] Many other solutions for the systems and devices described herein are possible. For example, individual modules or subsystems may be separated into additional modules or subsystems, or combined into fewer modules or subsystems. As another example, modules or subsystems may be omitted or supplemented by other modules or subsystems. As another example, functions indicated as being performed by a particular device, module, or subsystem may instead be performed by one or more other devices, modules, or subsystems. Although some examples of the present disclosure include descriptions of devices comprising specific hardware components in specific arrangements, the techniques and tools described herein may be modified to accommodate different hardware components, combinations, or arrangements.Further, although some examples in this disclosure include descriptions of specific use scenarios, the techniques and tools described herein may be modified to accommodate different use scenarios. Functionality described as being implemented in software may instead be implemented in hardware, or vice versa.

[0087] Many other solutions for the techniques described herein are possible. For example, processing steps in the different techniques may be separated in additional steps or combined into fewer steps. As another example, processing steps in the different techniques may be omitted or supplemented by other techniques or processing steps. As another example, processing steps described as occurring in a particular order may instead occur in a different order. As another example, processing steps described as being performed in a series of steps may instead be handled in a parallel manner, by multiple software modules or processes simultaneously handling one or more of the illustrated processing steps.

[0088] Many other solutions for the user interfaces described herein are possible. In practice, the user interfaces described herein may be implemented as separate user interfaces or as different states of the same user interface, and the different states may be presented in response to different events, e.g., user input events. The user interfaces may be customized for different devices, input and output capabilities, and the like. For example, the user interfaces may be presented differently depending on the display size, display orientation, whether the device is a mobile device, etc. The information and elements of the user interfaces shown in the user interfaces may be modified, supplemented, or replaced by other elements in different possible implementations.For example, different combinations of graphical user interface elements including text boxes, sliders, drop-down menus, radio buttons, soft buttons, etc., or any other user interface elements, including hardware elements, such as buttons, switches, scroll wheels, microphones, cameras, etc., may be used to accept user input in different forms. As another example, the user interface elements used in a particular implementation or configuration may depend on the particular input and / or output capabilities of a device (e.g., touchscreen). Information and user interface elements may be presented in different temporal, logical, and spatial arrangements in different possible implementations.For example, information or user interface elements described as being presented simultaneously on a single page or screen may also be presented at different times, on different pages or screens, etc. As another example, certain information or user interface elements may be presented under certain conditions based on previous input, user preferences, or the like.

[0089] Although embodiments have been illustrated and described, it will be appreciated that various modifications may be made herein without departing from the spirit and scope of the invention.

Claims

Claims

1. A non-transitory computer medium having instructions stored thereon, configured to, upon execution of the instructions by one or more computing devices of a computer system, cause the computer system to perform operations comprising: obtaining a digital model of the face of a living human subject; obtaining the digital model of the face includes obtaining a plurality of skin characteristics selected from one or more blemishes, areas of hyperpigmentation, texture, or wrinkles, or a combination thereof, obtaining user data associated with the living human subject; and generating a computer-assisted personal care program based at least in part on the digital model of the face of the living human subject and the user data;obtaining user feedback relating to the computer-assisted personal care program using a user interface; and modifying the computer-assisted personal care program based at least in part on the user feedback.;

2. The computer medium of claim 1, wherein the computer-assisted personal care program comprises one or more operational parameters for a computer-controlled personal care device.

3. The computer medium of claim 1, the operations further comprising: obtaining environmental data associated with the environment of the living human subject, wherein the computer-assisted personal care program is further based on the environmental data wherein obtaining the environmental data includes obtaining temperature data, humidity data, pollution data, or UV radiation data, or a combination thereof.

4. A computer system comprising a processor and a non-transitory computer medium on which instructions are stored, configured to, once the instructions have been executed by one or more computing device(s) of a computer system, causing the computer system to perform operations including: obtaining a digital model of the face of a living human subject; obtaining user data associated with the living human subject; obtaining environmental data associated with the environment of the living human subject; generating a computer-assisted skin care program based at least in part on the digital facial model, user data, and environmental data of the living human subject, obtaining user feedback relating to the computer-assisted personal care program by means of a user interface; and modification of the computer-assisted personal care program based at least in part on user feedback.

Images