Systems and methods for hair coverage analysis

Abstract

The present invention discloses a hair analysis system and method, the hair analysis system and method comprising: (a) a step of capturing an image of at least the top of a user's head at an image capture unit and sending the image from the image capture unit to a hair analysis unit; (b) a step of analyzing the user's hair coverage condition and / or scalp coverage condition at the hair analysis unit based on the image from the image capture unit by using a deep neural network, which predicts the user's hair coverage condition and / or scalp coverage condition relative to a gender group and is trained on class labels obtained through crowdsourcing, and providing the analysis result to a display unit; and (c) a step of displaying the analysis result to the user at a display unit. The present invention provides the system and method with improved sensitivity.

Description

Technical Field

[0001] The present application as a whole relates to a hair coverage analysis system and method for capturing an image of at least the top of a user's head, analyzing the user's hair coverage status and / or scalp coverage status by using a deep neural network trained on class labels collected through crowdsourcing; and predicting the user's hair coverage status and / or scalp coverage status relative to a gender group, and providing the analysis results to the user. The present invention provides a system and method with improved sensitivity. Background Art

[0002] Globally, premature hair loss and thinning hair are one of the biggest unmet consumer needs affecting more than half of the population. Most people who worry about their hair quantity status do not take immediate action because they do not understand the true condition of their hair. Although there are available diagnostic technologies, these technologies have been supervised by clinics and doctors' offices to provide more simulated analysis. If individuals have access rights to assess their conditions early on, they may be able to choose to do better in maintaining their current hair quantity. The latest advances in digital imaging technology (especially consumer images or selfies) have improved the ability to utilize image analysis technology, and have therefore improved the accessibility and speed of "on-hand" consumer diagnosis. However, various consumer characteristics and "selfie" conditions have caused it to be difficult to accurately assess hair conditions and recommend treatment plans without the need for more artificial consultation. In addition, these methods, systems and assessments rely on predetermined information about hair physical properties and appearance, and therefore cannot be summarized as true hair conditions. Therefore, there remains a need to provide an improved method of conveniently determining the amount of hair a person currently has, which method can then be used to help provide customized anti-hair loss product or regimen recommendations.

[0003] Assessment of hair condition is of interest in order to understand, for example, the extent of hair coverage and / or scalp coverage. Such assessment is also of interest in order to demonstrate the efficacy of a therapy for preventing and / or treating hair loss.

[0004] Thus, the present invention has satisfied the need for: systems and methods for evaluating consumer hair loss with increased sensitivity to assess true hair condition and providing such evaluation results; customized product recommendations based on the evaluation results; and customized hairstyle recommendations based on the evaluation results. Summary of the invention

[0005] A hair analysis system, comprising:

[0006] a) an image capture unit for capturing an image of at least the top of the user's head and sending the image to the hair analysis unit;

[0007] b) a hair analysis unit: analyzing the hair coverage condition and / or scalp coverage condition of the user based on the image by using a deep neural network, the deep neural network predicting the hair coverage condition and / or scalp coverage condition of the user relative to a gender group;

[0008] and providing an analysis result to a display unit, wherein the analysis result is at least one of the following:

[0009] The hair coverage and / or scalp coverage analyzed;

[0010] hair prediction based on the analyzed hair coverage condition and / or scalp coverage condition;

[0011] hair product recommendations based on the analyzed hair coverage condition and / or scalp coverage condition;

[0012] hair product usage recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; and

[0013] Hair style recommendations based on the analyzed hair coverage condition and / or scalp coverage condition.

[0014] c) A display unit for displaying the analysis results to the user.

[0015] A hair analysis method, comprising:

[0016] a) the step of capturing an image of at least the top of the user's head at an image capture unit and sending the image from the image capture unit to a hair analysis unit;

[0017] b) a step of analyzing the hair coverage condition and / or scalp coverage condition of the user based on the image from the image capture unit by using a deep neural network at the hair analysis unit, the deep neural network predicting the hair coverage condition and / or scalp coverage condition of the user relative to a gender group; and providing the analysis result to a display unit, wherein the analysis result is at least one of the following:

[0018] The hair coverage and / or scalp coverage analyzed;

[0019] hair prediction based on the analyzed hair coverage condition and / or scalp coverage condition;

[0020] hair product recommendations based on the analyzed hair coverage condition and / or scalp coverage condition;

[0021] hair product usage recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; and

[0022] Hair style recommendations based on the analyzed hair coverage condition and / or scalp coverage condition;

[0023] c) The step of displaying the analysis result to the user at a display unit.

[0024] A system and method for analyzing a user's hair coverage condition and / or scalp coverage condition improves the sensitivity of assessing the true hair condition and provides such analysis results. By using a deep neural network (DNN) in the method and system, a hair analysis of the user's appearance in an image is provided to the user, in which the hair coverage condition and / or scalp coverage condition on the top of the user's head is shown. The DNN-based system uses very little image preprocessing, which reduces the reliance on predetermined information about the image and facilitates generalization, thereby estimating the consumer's hair condition and / or scalp condition with improved sensitivity to assess the true hair condition and / or scalp condition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] It should be understood that both the foregoing general description and the following detailed description describe various non-limiting examples and are intended to provide an overview or framework for understanding the nature and character of the subject matter protected by the claims. The accompanying drawings are included to provide a further understanding of the various non-limiting examples and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate various non-limiting examples described herein and, together with the description, serve to explain the principles and operation of the subject matter protected by the claims.

[0026] Figure 1 As a non-limiting example, a system for capturing user images, analyzing the images, and providing customized product recommendations in accordance with the present invention described herein is depicted.

[0027] Figure 2 , by way of non-limiting example, depicts a flow chart for providing customized product recommendations in accordance with the present invention as described herein. DETAILED DESCRIPTION

[0028] A "deep neural network" is a feed-forward artificial neural network with multiple layers of neurons or units that build a hierarchy of learned features or concepts representing the input. Examples of these DNNs may be convolutional neural networks (CNNs) or deep capsule networks (DCNs).

[0029] When referring to various components of the systems herein, "coupled" means that the components are in electrical, electronic, and / or mechanical communication with each other.

[0030] "Data augmentation" refers to changing the data associated with training images or other images to create additional samples of images.

[0031] "Feature vector" means a series of features containing information describing one or more characteristics of an object in a digital image. Each feature in a feature vector is usually represented by one or more numbers, but any suitable indicator (letter, symbol, color, etc.) can be used as desired.

[0032] An "image capture device" refers to a device capable of capturing an image of a user, such as a digital camera; it may also be a "video capture device", such as a digital camera capable of capturing a video of a user; it may also be a 3-D image capture device.

[0033] "Macro features" are relatively large physical features that exist on or at the top of a person's head or near a person's face. Macro features include, but are not limited to, the top of the head, face shape, ears, eyes, mouth, nose, hair, and eyebrows.

[0034] "Micro features" are features such as scalp area, amount of hair, amount of hair thickness, amount of hair coverage, amount of scalp coverage, hair distribution, etc. Micro features do not include macro features.

[0035] A “model” in this context refers to a mathematical formula, algorithm, or computer software used to predict, describe, or simulate a set of situations, systems, or naturally occurring phenomena.

[0036] "Selfie" means a digital image, digital video, still image, or 3-D scan of a person taken by the person, by another person, or by an automated image capture system (e.g., a photo booth or security camera). In addition, a selfie may include a digital image, digital video, still image, or 3-D scan of the top of a person's head,

[0037] The term "user" herein refers to a person who uses at least the features provided herein, including, for example, a device user, a product user, a system user, and the like.

[0038] Build the model

[0039] The inputs for training the diagnostic model may be images, videos, 3D scans, etc., of the tops of heads of at least hundreds of people (covering variations in gender, race, hair color, location, background, etc.). These inputs may be rated by a panel of expert raters or a group of novice raters (also known as crowdsourcing) for hair coverage conditions. One approach may be to arrange the inputs into groups based on hair coverage patterns (e.g., low, medium, high, etc.). Another approach may be to use a paired comparison approach in which a pair of inputs is displayed to the rater and the rater is asked to select an input with a higher (or lower) amount of hair coverage. The collection of these paired selections from a single or multiple raters may be aggregated into a hair coverage score or grade for the individual whose input is rated. The hair coverage measure or score or grade may be further divided into several hair coverage levels (e.g., 0 to 3) or used as a continuous hair coverage value (e.g., 0 to 1). The inputs may then be used together with their corresponding hair coverage group labels or levels or scores to train a deep neural network model to predict a hair coverage label, level, or score given the input. Different hair coverage prediction models may be trained for individuals with different attributes, such as gender, race, etc. Preliminary steps in the diagnostic process may include asking the individual to specify these attributes, such as gender, race, etc., or using different deep neural network models to predict these attributes from the input in order to direct the input to corresponding attribute-specific hair coverage models.

[0040] Another preliminary step in diagnosis can include identifying input quality issues (e.g., input is too blurry, too dark, too bright, does not clearly include the top of the head, etc.) and providing corrective feedback to generate improved input. Input quality identification can also be predicted via a deep neural network model that is trained on inputs with corresponding input quality issue labels.

[0041] Neural Networks

[0042] In the present invention, for a neural network, an input is received (e.g., an image) and an output is produced (e.g., a prediction about the image, such as classifying its content). This consists of several (hence "depth") "hidden" (or intermediate) layers that successively transform the input data (e.g., pixel values) to produce output values ​​(e.g., the probability of an image classification). The weights or parameters of the hidden layers are "learned" (e.g., by gradient descent via backpropagation) by being shown pairs of network input-output (hence the need for labeled data - e.g., images with class labels). The idea of ​​using depth (multiple hidden layers) is to create a hierarchy of learned features / layers that will build on each other to produce a complex understanding of the input (e.g., from raw pixels in an image -> identifying lines / edges / colors -> object parts (circles / squares) -> small objects (wheels / windows) -> larger objects / scenes (cars)).

[0043] Convolutional Neural Networks (CNN)

[0044] In the present invention, for a convolutional neural network (CNN), the hidden layers use a specific operation called "convolution" to process only the data of the "receptive field" (for example, a convolution kernel may "look" at a 3×3 pixel window in the input image at a time, and apply transformations locally and repeat the same process across the entire image grid). Typically, a CNN may include several consecutive convolution, pooling, activation layers (for feature extraction), leading to a fully connected layer that produces the desired output (for prediction).

[0045] Capsule Network (CapsNet)

[0046] In the present invention, CNN may not directly use the relative relationship of learned features (e.g., eyes are above nose and mouth) for classification. Capsule Network (CapsNet) attempts to explicitly understand the pose (translation and rotation) of the object parts that make up a complex object. Therefore, CapsNet can potentially use much fewer labeled examples to achieve the same classification performance of CNN.

[0047] Image capture unit

[0048] The image capturing unit is used to capture an image of the user and send the image to the hair analyzing unit.

[0049] The image of the user herein is a 3-D video showing the top of the user's head, or the entire head, or an image of the user's hair and face. In the image, the ratio of the face size to the image size may be about 20% to 70%, so that the image shows more than 70% of the hair outline, or more than 80% of the hair outline, or more than 90% of the hair outline, or more than 95% of the hair outline. The image herein may be any image such as a selfie and a video. The image may also undergo quality inspection or preprocessing to ensure that there is a full hair view with acceptable quality. The image may have automatic guidance to capture the best head selfie. A non-limiting example would be that such guidance may be an automatic number, such as a measured distance from the camera or a certain angle or via an audio command. Another non-limiting example would be guidance on adjusting lighting conditions. In addition, for the hair coverage on the scalp, the image may also observe different areas on the scalp, for example, which area on the scalp may have less or more hair coverage, and access the measurements of a certain area, which may lead to product recommendations.

[0050] The image capturing unit may be connected to the hair analyzing unit via a wired connection or a wireless connection.

[0051] Q&A User Interface Unit

[0052] The unit optionally included in the systems and / or methods of the present invention is to provide questions to the user at the user interface; receive answers from the user; and send the answers to the hair analysis unit.

[0053] The unit may provide the consumer with a list of questions at a user interface, wherein each question has a defined set of answers; the answer selected by the consumer at the user interface is sent to the hair analysis unit.

[0054] The questions herein include, for example, those relating to the user's hair coverage condition and / or scalp coverage condition, those relating to the user's hair-related habits; those relating to the user's product preferences, those relating to the user's hairstyle preferences, those relating to the user's geographic information, those relating to the user's gender, those relating to the user's age; and those relating to the user's lifestyle.

[0055] The answer may be used to provide a hair analysis result at the hair coverage and / or scalp coverage analysis unit.The answer may be sent to the hair analysis unit in any form, for example, it may be sent as is, or may be sent as a score calculated from the answer.

[0056] The Q&A interface unit may be connected to the hair coverage and / or scalp coverage analysis unit via a wired connection or a wireless connection. The Q&A interface unit may be connected to the image capture unit via a wired connection or a wireless connection, or may be independent of the image capture unit, or may be physically located with the image capture unit, for example, within the same mobile computing device.

[0057] Hair analysis unit

[0058] The hair analysis unit analyzes the user's hair coverage condition and / or scalp coverage condition based on an image of at least the top of the user's head by using a deep neural network; and provides an analysis result to a display unit, wherein the analysis result is at least one of the following: the analyzed hair coverage condition and / or scalp coverage condition; hair prediction based on the analyzed hair coverage condition and / or scalp coverage condition; hair product recommendation based on the analyzed hair coverage condition and / or scalp coverage condition; hair product usage recommendation based on the analyzed hair coverage condition and / or scalp coverage condition; and hairstyle recommendation based on the analyzed hair coverage condition and / or scalp coverage condition.

[0059] The hair analysis unit may further pre-process the image, wherein the pre-processing comprises determining an anchor feature on the image and altering the image such that the anchor feature is at a predetermined position.

[0060] The hair condition analysis can be performed in the hair analysis unit by the following steps, which include:

[0061] Pretreatment;

[0062] applying a deep neural network (DNN) to extract macro- and micro-features including both facial features and hair features; and

[0063] The analyzed hair coverage condition and / or scalp coverage condition is provided.

[0064] The hair condition to be analyzed herein is at least one of the following conditions: hair coverage and / or scalp coverage, scalp area, hair volume, hair thickness, hair distribution, and combinations thereof.

[0065] To analyze these hair coverage conditions and / or scalp coverage conditions, the present invention may provide increased sensitivity by incorporating the capture of an image of at least the top of the user's head.

[0066] Hair predictions, hair product recommendations, hair product usage recommendations, and hairstyle recommendations are all based on such analyzed hair coverage conditions and / or scalp coverage conditions.

[0067] The hair analyzing unit may be connected to the display unit via a wired or wireless connection.

[0068] Display unit

[0069] The display unit will display the analysis results to the user, wherein the analysis results are at least one of the following: the analyzed hair coverage condition and / or scalp coverage condition, which may include hair coverage amount and / or scalp coverage amount, scalp area, hair amount, hair thickness amount, hair distribution and combinations thereof; hair predictions based on the analyzed hair coverage condition and / or scalp coverage condition; hair product recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; hair product usage recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; and hairstyle recommendations based on the analyzed hair coverage condition and / or scalp coverage condition.

[0070] A display showing hair product recommendations and / or hair product usage recommendations also shows an option for the user to purchase the product.

[0071] The analysis results can be represented by, for example, numerical data such as absolute values ​​with or without indication, relative values, indices and / or colors. Alternatively or simultaneously, the analyzed hair coverage condition and / or scalp coverage condition can be represented by, for example, cartoons and / or with indications and / or highlights on the image to show areas to be improved.

[0072] The display unit may be physically located together with the image capture unit and / or the Q&A user interface unit, for example, within the same mobile computing device. Alternatively, the display unit may be located separately from any one of the image capture unit and the Q&A user interface unit.

[0073] Example

[0074] The systems and methods herein can use a trained deep neural network such as a CNN or a DCN to analyze the user's hair condition by analyzing the captured user image. The CNN includes a multi-layered set of neurons that use the same filter for each pixel in the layer. Using the same filter for each pixel in various combinations of partial and fully connected layers reduces the memory and processing requirements of the system.

[0075] In some cases, the system may include a pre-processing stage followed by a CNN or DCN training and image analysis stage. During pre-processing, one or more hair features common to most users, such as scalp area, hair volume, hair thickness, hair coverage, scalp coverage, hair distribution ("anchor features"), may be detected in the received image. The system may detect one or more anchor features using known edge detection techniques, shape detection techniques, etc. Based on the location of the one or more anchor features, the image may be scaled and rotated so that the image is substantially horizontal, and the one or more anchor features are arranged at predetermined locations in the final image. In this way, the training images may be consistently aligned, thereby providing more consistent training and analysis. The image may then be cropped to a predetermined area of ​​pixels as input for further processing.

[0076] Preprocessing may also include image normalization. For example, global contrast normalization may be used to normalize training images (and / or user images) to account for variability that may be introduced by real selfie capture conditions.

[0077] In some cases, data augmentation can be performed to create additional samples from the input image. Additional samples are used to train the CNN or DCN to allow for changes in the input image. This helps to improve the accuracy of the model. In other words, the CNN or DCN is able to extract information and relationships of important features required for appropriate analysis, despite differences in, for example, the way people take photos, the conditions for taking photos, and the hardware used for taking photos. Additional samples generated by data augmentation can also force the CNN or DCN to learn to rely on multiple features for hair condition analysis, rather than one specific feature, and can prevent overtraining of the CNN or DCN. Some non-limiting examples of data augmentation include randomly enlarging or reducing images, randomly rotating images in a clockwise or counterclockwise direction, randomly cropping images, and / or randomly changing the saturation and / or exposure of images. In some cases, image data can be enhanced by subjecting the input image to random vertical discarding, in which random columns of pixels are removed from the image.

[0078] The CNN or DCN herein may be trained using deep learning techniques that allow the CNN or DCN to learn which parts of an image contribute to skin, facial features, hair characteristics, etc., much in the same way that a mammalian visual cortex learns to recognize important features in an image. In some cases, CNN training may involve the use of mini-batch stochastic gradient descent (SGD) with Nesterov momentum (and / or other algorithms). An example using stochastic gradient descent is disclosed in US 8,582,807.

[0079] DCN is composed of many capsules. A capsule is a small group of neurons that learn to detect a specific object (e.g., a rectangle) in a given area of ​​an image, and it outputs a vector (e.g., an 8-dimensional vector) whose length represents the estimated probability of the existence of the object, and whose orientation (e.g., in 8D space) encodes the pose parameters of the object (e.g., precise position, rotation, etc.). Very similar to conventional neural networks, DCN is organized into multiple layers. The capsules in the lowest layer are called main capsules: each of them receives a small area of ​​the image as input (called its receptive field), and it attempts to detect the presence and pose of a specific pattern (e.g., a rectangle). Capsules in higher layers (called routing capsules) detect larger and more complex objects, such as ships. The main capsule layer can be implemented using some regular convolutional layers. For example, two convolutional layers can be used, which output 256 6×6 feature maps containing scalars. These feature maps can be reshaped to obtain 32 6×6 feature maps containing 8-dimensional vectors. Finally, a squeezing function can be applied to ensure that these vectors have a length between 0 and 1 (to represent probability).

[0080] The capsules in the next layer may also try to detect objects and their poses using an algorithm called routing by agreement. The routing by agreement algorithm may involve several iterations of agreement detection + routing updates (this may happen for each prediction, not just once, and not just at training time).

[0081] Source: https: / / www.oreilly.com / ideas / introducing-capsule-networks

[0082] In some cases, the DNN may be trained by providing an untrained DNN with a large number of captured images from which it learns. In some cases, the DNN may learn to identify portions of an image that contribute to a particular hair coverage condition and / or scalp coverage condition through a process known as supervised learning. "Supervised learning" generally means training the DNN by analyzing images in which the hair coverage and / or scalp coverage of the person in the image is predetermined. Depending on the desired accuracy, the number of training images may vary from a small number of images to a large number of images (e.g., hundreds or even thousands), with images continuously input (i.e., providing continuous training).

[0083] The systems and methods herein utilize a trained DNN that is capable of accurately analyzing the hair condition of a user of a wide variety of hair types and styles. To provide the analyzed hair condition, an image of the user is forward propagated through the trained DNN. The DNN analyzes the image and identifies portions of the image that contribute to the hair condition, such as hair coverage and / or scalp coverage. The DNN then uses the identified portions to analyze the hair condition of the user.

[0084] In some cases, the DNN analysis of the analyzed hair coverage condition and / or scalp condition and / or target condition, optionally in combination with habit and practice input provided by the user, can be used to help provide hair predictions, hair care product recommendations, hair product usage recommendations and / or hairstyle recommendations.

[0085] Figure 1 A system 10 for capturing user images, analyzing the images, and providing customized product recommendations is depicted. The system 10 may include a network 1, which may be embodied as a wide area network (such as a mobile phone network, a public switched telephone network, a satellite network, the Internet, etc.), a local area network (such as Wireless Fidelity, Wi-Max, ZigBee, etc.), or a wireless network. TM 、Bluetooth TM Coupled to the network 1 are a mobile computing device 2, a remote computing device 4, an information kiosk computing device 5, and a training computing device 6.

[0086] The mobile computing device 2 may be a mobile phone, a tablet computer, a laptop computer, a personal digital assistant, and / or other computing devices configured to capture, store, and / or transmit images such as digital photos. Therefore, the mobile computing device 2 may include an image capture device 3, such as a digital camera, and / or may be configured to receive images from other devices. The mobile computing device 2 may include a memory component 7A that stores image capture logic 8A and interface logic 8B. The memory component 7A may include random access memory (such as SRAM, DRAM, etc.), read-only memory (ROM), registers, and / or other forms of computing storage hardware. As described herein, the image capture logic 8A and the interface logic 8B may include software components, hardware circuits, firmware, and / or other computing infrastructure. As described in more detail below, the image capture logic 8A may facilitate capturing, storing, preprocessing, analyzing, transmitting, and / or performing other functions on the user's digital images. The interface logic 8B may be configured to provide one or more user interfaces to the user, which may include questions, options, etc. The mobile computing device 2 may also be configured to communicate with other computing devices via the network 1.

[0087] The remote computing device 4 may also be coupled to the network 1 and may be configured as a server (or multiple servers), a personal computer, a mobile computer, and / or other computing devices configured to create and train a convolutional neural network capable of analyzing the user's hair condition by identifying portions of the captured image that contribute to a particular hair condition. The remote computing device 4 may include a memory component 7B that stores training logic 8C and analysis logic 8D. The training logic 8C may facilitate the creation and / or training of a DNN, which may facilitate the creation and / or operation of a DNN. For example, the DNN may be stored as logic 8C, 8D in the memory component 7B of the remote computing device 4. The analysis logic 8D may cause the remote computing device 4 to receive data from the mobile computing device 2 (or other computing device) and process the received data for providing analyzed hair coverage conditions and / or scalp coverage conditions, product recommendations, hairstyle recommendations, etc.

[0088] The system 10 may also include a kiosk computing device 5, such as Figure 1 As shown. Kiosk computing device 5 may operate similarly to mobile computing device 2, but may also be able to dispense one or more products and / or receive payment in the form of cash or electronic transactions. In some cases, kiosk computing device 5 may also be configured to facilitate training of a DNN, as described in more detail below with respect to training computing device 6.

[0089] A training computing device 6 may be coupled to the network 1 to facilitate training of the DNN. For example, a trainer may provide one or more digital images of a face or skin or hair to the DNN via the training computing device 6. The trainer may also provide information and other instructions to inform the DNN which assessments are correct and which assessments are incorrect. Based on input from the trainer, the DNN may automatically adapt, as described in more detail below.

[0090] It should be understood that although the kiosk computing device 5 is depicted as a vending machine type device, this is a non-limiting example. Additional non-limiting examples may utilize mobile devices that also provide payment and / or production distribution. Similarly, the kiosk computing device 5, the mobile computing device 2, and / or the training computing device 6 can be used to train the DNN. Therefore, the hardware and software depicted for the mobile computing device 2 and the remote computing device 4 may be included in the kiosk computing device 5, the training computing device 6, and / or other devices. Similarly, the hardware and software may be included in one or more of the mobile computing device 2, the remote computing device 4, the kiosk computing device 5, and the training computing device 6.

[0091] It should also be understood that although Figure 1 The remote computing device 4 is depicted as performing deep neural network processing, but this is only an example. The deep neural network processing may be performed by any suitable computing device as desired.

[0092] Figure 2 A flow chart for providing customized product recommendations is depicted and described herein. In box 11, a user image may be captured. In box 12, a question may be provided to the user. In box 13, an answer to the question may be received from the user. In box 14, the analyzed hair coverage condition and / or scalp coverage condition may be provided to the user. In box 15, a customized product recommendation may be provided to the user.

[0093] Additional embodiments / combinations

[0094] A. A hair analysis system, comprising:

[0095] a. an image capture unit for capturing an image of at least the top of the user's head and sending the image to the hair analysis unit;

[0096] b. a hair analysis unit: analyzing the user's hair coverage condition and / or scalp coverage condition based on the image by using a deep neural network, the deep neural network predicting the user's hair coverage condition and / or scalp coverage condition relative to a gender group;

[0097] and providing an analysis result to a display unit, wherein the analysis result is at least one of the following:

[0098] The hair coverage and / or scalp coverage analyzed;

[0099] hair prediction based on the analyzed hair coverage condition and / or scalp coverage condition;

[0100] hair product recommendations based on the analyzed hair coverage condition and / or scalp coverage condition;

[0101] hair product usage recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; and

[0102] Hair style recommendations based on the analyzed hair coverage condition and / or scalp coverage condition;

[0103] c. A display unit that displays the analysis results to the user.

[0104] B. A system according to paragraph A, wherein the deep neural network is trained on class labels obtained through crowdsourcing.

[0105] C. A system according to paragraphs AB, wherein the system also includes a Q&A user interface unit to provide questions to the user at the user interface; receive answers from the user; and send the answers to the analysis unit.

[0106] D. A system as described in paragraphs AC, wherein the answer is used to provide an analysis result.

[0107] E. A system according to paragraphs AD, wherein the system uses a convolutional neural network.

[0108] F. A system according to paragraphs AE, wherein the system uses a deep capsule network.

[0109] G. A system as described in paragraphs AF, wherein the display displaying hair product recommendations and / or hair product usage recommendations also displays an option for the user to purchase the product.

[0110] H. A system according to paragraphs AG, wherein the hair coverage condition and / or scalp coverage condition to be analyzed is at least one of the following: hair coverage amount and / or scalp coverage amount, scalp area, hair amount, hair thickness amount, hair distribution and combinations thereof.

[0111] I. A method of hair analysis according to paragraphs AH, comprising:

[0112] a) the step of capturing an image of at least the top of the user's head at an image capture unit and sending the image from the image capture unit to a hair analysis unit;

[0113] b) analyzing the hair coverage condition and / or scalp coverage condition of the user based on the image from the image capture unit by using a deep neural network at the hair analysis unit, the deep neural network: 1) trained on the class labels obtained by crowdsourcing; and 2) predicting the hair coverage condition and / or scalp coverage condition of the user relative to the gender group; and providing the analysis result to the display unit, wherein the analysis result is at least one of the following:

[0114] The hair coverage and / or scalp coverage analyzed;

[0115] hair prediction based on the analyzed hair coverage condition and / or scalp coverage condition;

[0116] hair product recommendations based on the analyzed hair coverage condition and / or scalp coverage condition;

[0117] hair product usage recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; and

[0118] Hair style recommendations based on the analyzed hair coverage condition and / or scalp coverage condition;

[0119] c) The step of displaying the analysis result to the user at a display unit.

[0120] J. According to the method described in paragraph AI, a deep neural network is trained on class labels obtained through crowdsourcing.

[0121] K. A method according to paragraphs AJ, wherein the method also includes the steps of providing questions to the user in the Q&A user interface unit; receiving answers from the user; and sending the answers to the analysis unit.

[0122] L. A method according to paragraphs AK, wherein the answer is used to provide an analysis result.

[0123] M. The method of paragraph AL, wherein the display unit that displays hair product recommendations and / or hair product usage recommendations also displays an option for the user to purchase the product.

[0124] N. A method according to paragraph AM, wherein the hair coverage condition and / or scalp coverage condition to be analyzed is at least one of the following: hair coverage amount and / or scalp coverage amount, scalp area, hair amount, hair thickness amount, hair distribution and combinations thereof.

[0125] The dimensions and values ​​disclosed herein are not to be understood as being strictly limited to the exact numerical values ​​cited. Instead, unless otherwise indicated, each such dimension is intended to represent the stated value and a functionally equivalent range around that value. For example, a dimension disclosed as "40 mm" is intended to mean "about 40 mm".

[0126] Unless expressly excluded or otherwise limited, each document cited herein, including any cross-referenced or related patent or patent application and any patent application or patent to which this application claims priority or the benefit of, is hereby incorporated by reference in its entirety. The citation of any document is not an admission that it is prior art to any of the present invention disclosed or claimed herein, or an admission that it, by itself or in combination with any one or more of the references, proposes, suggests, or discloses any such invention. In addition, to the extent that any meaning or definition of a term in this invention conflicts with any meaning or definition of the same term in a document incorporated by reference, the meaning or definition assigned to that term in this invention shall govern.

[0127] Although specific embodiments of the present invention have been illustrated and described, it will be apparent to those skilled in the art that many other changes and modifications may be made without departing from the spirit and scope of the present invention. Therefore, it is intended that all such changes and modifications within the scope of the present invention be covered in the appended claims.

Claims

1. A hair analysis system, comprising: a) an image capture unit for capturing an image of at least the top of the user's head and sending the image to the hair analysis unit; b) a hair analysis unit: analyzing the hair coverage condition and / or scalp coverage condition of the user based on the image by using a deep neural network, wherein the deep neural network predicts the hair coverage condition and / or scalp coverage condition of the user relative to a gender group, wherein the deep neural network is a deep capsule network; and providing an analysis result to a display unit, wherein the analysis result is at least one of the following: The hair coverage and / or scalp coverage analyzed; hair prediction based on the analyzed hair coverage condition and / or scalp coverage condition; hair product recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; hair product use recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; as well as Hair style recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; c) a display unit for displaying the analysis results to the user, wherein the deep neural network is trained on class labels obtained through crowdsourcing; wherein the hair analysis system also includes data augmentation capabilities to create additional samples from the input image, and the deep capsule network is trained using the additional samples to allow for changes in the input image.

2. The system according to claim 1, wherein the system further comprises a Q&A user interface unit to provide questions to the user at the user interface; receive answers from the user; and send the answers to the analysis unit. The system of claim 2 , wherein the answer is used to provide the analysis result.

4. The system of claim 1, wherein the system uses a convolutional neural network.

5. The system of claim 1, wherein the display unit that displays the hair product recommendation and / or hair product usage recommendation also displays an option for the user to purchase the product.

6. A system according to any one of claims 1-5, wherein the hair coverage condition and / or scalp coverage condition to be analyzed is at least one of the following: hair coverage amount and / or scalp coverage amount, scalp area, hair amount, hair thickness amount, hair distribution and their combinations.

7. A hair analysis method, comprising: a) the step of capturing an image of at least the top of the user's head at an image capture unit and sending said image from said image capture unit to a hair analysis unit; b) a step of analyzing, at a hair analysis unit, a hair coverage condition and / or a scalp coverage condition of the user based on the image from the image capture unit by using a deep neural network, the deep neural network predicting the hair coverage condition and / or the scalp coverage condition of the user relative to a gender group, and providing an analysis result to a display unit, wherein the deep neural network is a deep capsule network; The analysis result is at least one of the following: The hair coverage and / or scalp coverage analyzed; hair prediction based on the analyzed hair coverage condition and / or scalp coverage condition; hair product recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; hair product use recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; as well as Hair style recommendations based on the analyzed hair coverage condition and / or scalp coverage condition; c) a step of displaying the analysis result to the user at a display unit, wherein the deep neural network is trained on class labels obtained through crowdsourcing; wherein the hair analysis unit also includes a data augmentation function to create additional samples from the input image, and the deep capsule network is trained using the additional samples to allow for changes in the input image.

8. The method according to claim 7, wherein the method further comprises the following steps at a Q&A user interface unit: providing questions to the user; receiving answers from the user; and sending the answers to the analysis unit.

9. The method of claim 8, wherein the answer is used to provide the analysis result.

10. The method of claim 7, wherein the display unit displaying the hair product recommendation and / or hair product usage recommendation also displays an option for the user to purchase the product.

11. A method according to any one of claims 7-10, wherein the hair coverage condition and / or scalp coverage condition to be analyzed is at least one of the following: hair coverage amount and / or scalp coverage amount, scalp area, hair amount, hair thickness amount, hair distribution and their combinations.

Images