SYSTEM INCLUDING DEVICES AND ALGORITHMS FOR ASSESSING THE HEALTH OF THE SCALP AND HAIR

Abstract

SYSTEM INCLUDING DEVICES AND ALGORITHMS FOR ASSESSING SCALP AND HAIR HEALTH A device for detecting a scalp condition, wherein the scalp condition comprises at least a presence or absence of dandruff, including at least one camera configured to view a user's scalp, a diffusion cone disposed around the at least one camera, a lighting module comprising at least one light source, wherein the at least one light source is configured to emit light under a plurality of lighting conditions, and a processor configured to receive image data from the at least one camera captured during at least one lighting condition of the plurality of lighting conditions, and transmit the image data to a smart device for detecting the scalp condition. Figure for abstract: 1

Description

Title of the invention: SYSTEM INCLUDING DEVICES AND ALGORITHMS FOR ASSESSING THE HEALTH OF THE SCALP AND HAIR ABSTRACT

[0001] This summary is provided to present a selection of concepts in simplified form which are described more fully below in the detailed description. This summary is not intended to identify key features of the claimed subject matter or to be used as an aid in determining the scope of the claimed subject matter.

[0002] In one aspect, disclosed herein is a device for detecting a scalp condition, a hair condition, or both, wherein the scalp condition includes at least a presence or absence of dandruff, and the hair condition includes a presence or absence of the hair condition, including at least one camera configured to view a user's scalp, a diffusion cone disposed around the at least one camera, an illumination module including at least one light source, wherein the at least one light source is configured to emit light under a plurality of illumination conditions, and a processor configured to receive image data from the at least one camera captured during at least one illumination condition of the plurality of illumination conditions, and transmit the image data to a smart device for detecting the scalp condition, the hair condition, or both.

[0003] In some embodiments, the image data comprises a set of images at a location on the user's scalp under each of the plurality of lighting conditions. In some embodiments, the plurality of lighting conditions includes a white light condition, a cross-polarized light condition, an ultraviolet (UV) light condition, or a combination thereof. In some embodiments, in the white light condition, the white light is emitted in at least one of a plurality of brightness modes.

[0004] In some embodiments, the at least one camera is a three-lens camera, and wherein each lens of the three-lens camera has a different focal length.

[0005] In another aspect, disclosed herein is a system for detecting a scalp condition, a hair condition, or both, where the scalp condition includes at least a presence or absence of dandruff, and the hair condition includes a presence or absence of the hair condition, the system including the device as described herein and a smart device, where the smart device includes a processor of a smart device, and wherein the smart device processor is configured to receive the image data from the device, input an image of the image data under one of the plurality of lighting conditions, determine one or more areas of one or more features detected at a pixel level of the image, and display the one or more areas overlaid on the image data.

[0006] In some embodiments, the smart device processor is further configured to segment, at a pixel level, the image data, locate the segmented films, classify the segmented films based on a film count, film size, and film shape based on film statistical data, output a film severity score, and display contour segments of the films overlaid on the image data.

[0007] In some embodiments, the smart device processor is further configured to segment hair strands in the image data based on color, determine a position and direction of the hair strands, measure individual diameters of one or more of the hair strands, determine an average hair diameter, output a hair score based on hair condition, where the hair score includes an average hair strand diameter, a percentage of white or gray hair, a root color detection, or a combination thereof. In some embodiments, the smart device processor is further configured to estimate a number of hair roots in the image data and output the hair score, where the hair score further includes a probability of hair loss.

[0008] In some embodiments, a 3-layer convolutional neural network (CNN) having a same structure with different weight sets is used to predict dandruff score severity and scalp redness level.

[0009] In some embodiments, the smart device processor is further configured to display hair strands and roots of the image data in a plurality of colors on a black background, and where the plurality of colors correspond to individual diameters of one or more of the hair strands. In some embodiments, the smart device processor is further configured to indicate on the displayed image data when a hair strand is less than a diameter threshold.

[0010] In some embodiments, the smart device processor is further configured to detect one or more colors on a pixel level in the image data indicative of porphyrins; determine a number, size, form, or a combination thereof, of porphyrins, output a porphyrin score and display outlines of the detected porphyrins overlaid on the image data.

[0011] In yet another aspect, disclosed herein is a method of detecting a scalp condition, wherein the scalp condition includes at least a presence or absence of dandruff, the method comprising placing the device described herein on the scalp of the user, emitting light under a plurality of lighting conditions, capturing image data of the scalp of the user under each lighting condition of the plurality of lighting conditions, transmitting the image data to a smart device processor, analyzing each image of the image data under each lighting condition of the plurality of lighting conditions, detecting one or more areas of one or more detected features at a pixel level of the image, and displaying the detected areas overlaid on the image data.

[0012] In some embodiments, the method further comprises segmenting, at a pixel level, films in the image data, classifying the segmented films according to film number, film size, and film shape based on statistical data about the films, outputting a film severity score, and displaying contour segments of the films overlaid on the image data.

[0013] In some embodiments, the method further includes segmenting, at a pixel level, hair strands in the image data based on color, determining a position and direction of the hair strands, measuring individual diameters of one or more of the hair strands, determining an average hair diameter, and outputting a hair score, wherein the hair score includes an average hair strand diameter, a percentage of white or gray hair, a root color detection, or a combination thereof.

[0014] In some embodiments, the method further comprises estimating a number of hair roots in the image data and outputting a hair score, wherein the hair score comprises a probability of hair loss.

[0015] In some embodiments, the method further includes storing the image data, the one or more areas, the one or more scalp conditions, or a combination thereof as historical data, and tracking the historical data over time to determine a change in the one or more scalp conditions.

[0016] In some embodiments, the method further comprises detecting one or more colors on a pixel level in the image data indicative of porphyrins, determining a number, size, shape, or combination thereof, of the porphyrins, outputting a porphyrin score, and displaying outlines of the detected porphyrins overlaid on the image data.

[0017] In some embodiments, the method further comprises recommending one or more treatments for the one or more scalp conditions. Description of the drawings

[0018] The foregoing aspects and numerous related advantages of this invention will be more readily appreciated as they are better understood with reference to the following detailed description, when taken in conjunction with the accompanying drawings, in which:

[0019] [Fig.lA] [Fig.lA] is a perspective side view of an exemplary scalp condition detection device, in accordance with the present technology;

[0020] [Fig.IB] [Fig.IB] is a front perspective view of an exemplary scalp condition detection device, in accordance with the present technology;

[0021] [Fig.lC] [Fig.lC] is a front perspective view of another example of a scalp condition detection device, in accordance with the present technology;

[0022] [Fig. 1D] [Fig. 1D] is an example of an optical sensor for detecting a scalp condition, in accordance with the present technology;

[0023] [Fig.2] [Fig.2] is an example illustration of a plurality of brightness modes, in accordance with the present technology;

[0024] [Fig.3A]-[Fig.3C] Figures 3A to 3C are examples of image data of a scalp under different lighting conditions of a plurality of lighting conditions, in accordance with the present technology.

[0025] [Fig.4A] [Fig.4A] is a block diagram of an exemplary scalp condition detection system, in accordance with the present technology;

[0026] [Fig.4B] [Fig.4B] is another block diagram of an exemplary scalp condition detection system system, in accordance with the present technology;

[0027] [Fig.4C] [Fig.4C] is an example of a scalp condition detection system, in accordance with the present technology;

[0028] [Fig.5] [Fig.5] is an illustration of an exemplary device in use, in accordance with the present technology;

[0029] [Fig.6A]-[Fig.6J] Figures 6A to 6J are examples of visualizations, in accordance with this technology;

[0030] [Fig.7] [Fig.7] is an example of a method for detecting a scalp condition, in accordance with the present technology;

[0031] [Fig.8] [Fig.8] is another example of a method of detecting a scalp condition, in accordance with the present technology;

[0032] [Fig.9] [Fig.9] is yet another example of a method of detecting a scalp condition, in accordance with the present technology; and

[0033] [Fig.l0A]-[Fig.l0G] Figures 10A to 10G show an example of model training and validation for scoring, dandruff segmentation and hair segmentation. Detailed description Dandruff is a common condition that causes the skin on the scalp to flake. Symptoms include scaling and sometimes mild itching. This can lead to social or self-esteem problems. Although dandruff can be detected by visual inspection, it can be difficult to determine a dandruff classification, whether a dandruff treatment works, and what may be causing dandruff. For example, dandruff can be caused by the fungus Malassezia, a natural organism found on the skin and scalp. Dandruff can also be caused by oily hair, psoriasis, or eczema. Treatments vary in each case.

[0035] Porphyrins are intermediate metabolites in the biosynthesis of vital molecules, including thiamin, cobalamin, and chlorophyll. Bacterial porphyrins are known to be pro-inflammatory, with elevated levels linked to inflammatory skin diseases. Porphyrins on the scalp can cause inflammation, redness, sores, and acne. It can also be difficult to detect porphyrins on the scalp and therefore treat them.

[0036] Additionally, an individual may wish to know about and / or treat other scalp conditions, such as hair health, presence of oiliness, hair strand diameter, hair diversity / thickness, gray hair, and the like.

[0037] A scalp and / or hair reader, or a device with machine learning algorithms for quantitatively analyzing the health of consumers' scalp and hair based on images, is described herein. The device is used to capture images of the scalp with multiple illumination modes. The detection results on the health of the scalp and hair are analyzed and delivered to the consumers. The scores on the scalp dandruff level, porphyrin level, hair diameter and hair root number based on the images captured by the device are predicted by algorithms. In some embodiments, the analysis is based on an overall scalp health index and / or an overall hair loss risk index, which are correlations based on the output of the algorithms and / or responses to a user questionnaire. Areas in need of care on the observed scalp and the diversity of fiber diameters are also detected and visualized by algorithms. Recommended products and treatments may be provided based on the analysis results.

[0038] In some embodiments, a set of algorithms is used that take the raw scalp images as input and predict scores corresponding to the scalp dandruff level, porphyrin level, redness level, average hair diameter, and number of roots, respectively. In addition to the score predictions, visualizations of segmented dandruff, porphyrins, hair fibers, and hair roots are masked by algorithms. The algorithms for score prediction and visualization of all the aforementioned scalp and hair parameters are built based on deep learning models and conventional image processing methods. In one example, the deep learning models include convolutional neural network (CNN)-based deep learning models.Statistical data, such as the number, size distribution of dandruff / porphyrins, and hair fiber diameters, are analyzed by the algorithms. To ensure the accuracy of the algorithms, a large dataset of scalp images was collected. The algorithms were trained and tested based on the specially collected dataset for high accuracy.

[0039] [Fig. 1 A] is a perspective side view of an exemplary device 100 of detecting a scalp condition, in accordance with the present technology. In some embodiments, the device 100 includes a diffusion cone 105, a body 110, one or more actuators 115A, 115B, 115C, a processor 120, a power indicator 125, and a battery 130.

[0040] In some embodiments, the diffusion cone 105 is a protrusion located on a front side of the body 110. The diffusion cone may be configured to be widest toward the body 110 and taper as it extends from the body 110. In some embodiments, a length of the diffusion cone is determined based on a distance needed for an optical sensor (as shown in [Fig. 1B]) to capture image data of a scalp when one end of the diffusion cone 105 contacts the scalp (as shown in [Fig. 5]). In some embodiments, the diffusion cone 105 is configured to block light from a light source (as shown in Figures 1B and 1D) so that an optical sensor can capture image data of the scalp under a particular light condition, as described in detail herein.

[0041] While the body 110 is illustrated as a cylinder in [Fig. 1A], it should be understood that the body 110 may be of any form factor. In some embodiments, the body 110 is rounded, angular, organically shaped, or the like. In some embodiments, the body 110 is composed of plastic, metal, ceramic, or a combination thereof. In some embodiments, the body 110 is configured to be held in a hand of an operator or user. In some embodiments, a hair clip (not shown in [Fig. 1A]) may be used to hold hair in place and allow the device 100 to better visualize the scalp and / or hair.

[0042] The one or more actuators 115A, 115B, 115C may be configured to perform a variety of functions, including, but not limited to, turning on / off the device 100, emitting or stopping the emission of light from a light source (as shown in Figures 1B-1D), changing a lighting condition or light mode of the light, as explained herein, capturing scalp image data, and / or pairing the device 100 with another device (such as the smart device 1000 in [Fig. 4C]) via a wireless connection, such as Wi-Fi, Bluetooth LTETM, Zigbee, or the like. In some embodiments, actuators 115A, 115B, 115C are buttons, but in other embodiments, actuators 115A, 115B, 115C may be switches, capacitive touch-type buttons, levers, or the like.

[0043] Those skilled in the art should understand that the processor 120 is internal to the body 110 but is illustrated in [Fig. 1A] for clarity. In some embodiments, the processor 120 is configured to capture and store image data from one or more optical sensors (as shown in Figures 1B-1D), control one or more light sources, and / or transmit image data to another device (such as the smart device 1000 in [Fig. 4C]). In some embodiments, the processor is configured to communicatively couple with a smart device over a wireless or wired connection.

[0044] In some embodiments, the device 100 includes a power indicator 125. In some embodiments, the power indicator 125 indicates a power level of the device. In some embodiments, the power indicator 125 is configured to emit a variety of colors to convey a status of the device 100, such as green for charged, red for low power, yellow for charging, or the like. In some embodiments embodiment, the power indicator 125 may indicate a status of the device that is unrelated to the power of the device 100, such as by flashing to indicate that the device 100 is paired with an external device, by means of a flash to indicate that image data has been captured, a color associated with a particular light condition, or the like.

[0045] The battery 130 may be internal or external to the device 100. In some embodiments, access to the battery 130 may be by removing one end of the device 100. In some embodiments, the battery 130 is a single-use battery, but in other embodiments, the battery 130 is a rechargeable battery, or a capacitor. In some embodiments, the battery 130 may be charged via a wired or wireless connection.

[0046] In operation, the device 100 is configured to be held by a user or operator, such as a salon technician or pharmacist. In some embodiments, the operator places the diffusion cone 105 on a client's scalp, as shown in [Fig. 5]. In some embodiments, the operator clears the client's hair, for example by manually moving it, brushing it, or securing it with a clip or other fastener. In some embodiments, the diffusion cone 105 separates one or more optical sensors, light sources, or the like from the scalp, so as to capture image data and / or emit light under a plurality of light conditions (as depicted in Figures 1B-1D). The operator may actuate the one or more actuators 115A, 115B, 115C to begin and / or terminate image data capture.After capturing the image data under one or more light conditions, the processor 120 may transmit the image data to an external device for processing and / or detecting a scalp condition. As used herein, a scalp condition includes at least one of a presence or absence of dandruff, a presence or absence of porphyrin, a presence or absence of grease, a presence or absence of gray hair, hair thickness, a likelihood of hair loss, hair diversity, and / or a presence or absence of redness. In some embodiments, the diagnosis of the hair and / or scalp condition is determined with a single “spot,” i.e., when a plurality of images are taken from a single location on the scalp. In other embodiments, the diagnosis of the hair and / or scalp condition is determined with multiple “spots.”In such embodiments, a user or operator may move the device 100 over multiple locations on the scalp. Image data from each of these locations may be analyzed separately, or together, to determine a condition of the hair and / or scalp. In some embodiments, the device . 100 may prompt the user to move the device 100 to multiple locations, for example with a vibration, alert, tone, or other instruction.

[0047] [Fig. 1B] is a front perspective view of an exemplary scalp condition detection device 100, in accordance with the present technology. In some embodiments, the device 100 includes a diffusion cone 105, a body 110, a light source 135, and an optical device 140. In [Fig. 1B], one end of the diffusion cone 105 is shown. In some embodiments, on the body 110, but within the diffusion cone 105, are the light source 135 and the optical device 140.

[0048] In some embodiments, the light source 135 is a light emitting diode (LED). In some embodiments, the light source 135 is configured to emit light under a plurality of light conditions (as shown in Figures 3A-3C). In some embodiments, the plurality of light conditions includes white light (i.e., light emitted at a wavelength of 400-700 nm), cross-polarized light (i.e., where two polarizers oriented perpendicular to each other are used on incident and reflected lights), and / or ultraviolet (UV) light (i.e., light emitted at a wavelength of 100-400 nm). In some embodiments, the light source 135 is also configured to emit white light at a plurality of brightness modes (as shown and described in [Fig. 2]).

[0049] In some embodiments, the optical device 135 is a camera. In some embodiments, the optical device 135 is configured to capture image data of a scalp. In some embodiments, the optical system 135 is a single-lens camera. In some embodiments, the optical device 135 is communicatively coupled to a processor (such as the processor 120). In some embodiments, the optical device 135 transmits the captured image data to the processor.

[0050] In operation, a user or operator places the diffusion cone 105 on the scalp. The light source 135 emits light at a first light condition of the plurality of light conditions. The optical sensor 140 may then capture image data of the scalp in the first light condition. In some embodiments, the image data is one or more snapshots, but in other embodiments, the image data may be a video. The light source 135 may then emit a second light condition of the plurality of light conditions. The optical sensor 140 may then capture image data of the scalp in the first light condition. This repeats until the optical sensor 140 captures image data under all of the plurality of light conditions or under a predetermined subset of the plurality of light conditions. A processor (such as the processor 120) may then transmit the image data captured by the optical sensor 140 to an external device (such as the smart device 1000 in [Fig.4C]).

[0051] [Fig. 1C] is a front perspective view of another example of a device 100 for detecting a scalp condition, in accordance with the present technology. In some embodiments, the device 100 includes a plurality of light sources 135A, 135B, 135C.

[0052] In some embodiments, each light source of the plurality of light sources 135A, 135B, 135C is configured to emit a different light condition of the plurality of light conditions. For example, if there are three light sources of the plurality of light sources 135A, 135B, 135C, a first light source 135A may emit white light, a second light source 135B may emit cross-polarized light, and a third light source 135C may emit UV light.

[0053] [Fig. 1D] is an example of an optical sensor 140 for detecting a scalp condition, in accordance with the present technology. In some embodiments, the optical sensor 140 is a camera. In some embodiments, the optical sensor 140 is a three-lens camera. In some embodiments, the optical sensor has a first lens 145A, a second lens 145B, and a third lens 145C. In some embodiments, each lens 145A, 145B, 145C of the three-lens camera has a different focal length.

[0054] In operation, each lens in the optical sensor 140 is configured to capture image data at a predetermined focal length, such that multiple focal length images of the scalp can be captured. In some embodiments, instead of a three-lens camera, the optical sensor 140 is a plurality of optical sensors, each optical sensor configured to capture image data at a plurality of focal lengths.

[0055] [Fig. 2] is an example illustration of a plurality of brightness modes, in accordance with the present technology. In some embodiments, the one or more light sources (as shown in Figures 1B and 1C) are configured to emit light in a plurality of brightness (or "light") modes. While five modes are shown in [Fig. 2], it is to be understood that any number of light modes may be emitted by the one or more light sources. In some embodiments, as the number of light modes increases, the brightness level of the light decreases (i.e., Mode 5 is the darkest and Mode 0 is the brightest). In some embodiments, a processor (such as processor 120) controls the brightness mode based on light detected by an optical sensor (such as optical sensor 140). In some embodiments, one or more actuators (such as one or more actuators 115A, 115B, 115C) may be actuated to control the brightness mode.

[0056] Figures 3A-3C are examples of image data of a scalp under different light conditions of a plurality of light conditions, in accordance with the present technology. [Fig. 3A] shows image data of a scalp under white light, [Fig. 3B] shows an image of the scalp under cross-polarized light, and [Fig. 3C] shows an image of a scalp under UV light. In each light condition, different scalp conditions may be detectable, such as dandruff, redness, irritation, acne, inflammation, and / or porphyrins.

[0057] For example, under white light, as shown in [Fig. 3A], hair segmentation algorithms can be applied to detect hair characteristics such as hair color (such as black hair and white hair), hair roots, hair root color, and the like. Under cross-polarized light, as shown in [Fig. 3B], acne, inflammatory dermatoses, redness, dandruff, and bruises may be detectable. Under UV light, as shown in [Fig. 3C], porphyrins may be detectable, as shown by the lighter spots (or "detected areas").

[0058] [Fig. 4A] is a block diagram of an exemplary scalp condition detection system 4000, in accordance with the present technology. In some embodiments, the system 4000 includes a device 100 (as shown in Figures 1A-1D), a smart device (or "scalp reader system") 1000, and an application 1040. In some embodiments, the application 1040 is downloaded to the smart device 1000, but in other embodiments, it may be separate from the smart device, such as on a browser of another external device.

[0059] As shown in Figures 1A-1D, the device 100 may include a camera module (or optical device) 140, a lighting adjustment module (or light source) 135, and a transmission module (or processor) 120. As described herein, in operation, the device 100 captures image data with the camera module 140 under a plurality of light conditions emitted by the lighting adjustment module 135 and transmits this image data to the smart device 1000 with the transmission module 120.

[0060] In some embodiments, the scalp reader system 1000 includes an image capture module 10, a feature scoring module 20, and a result analysis module 30.

[0061] In some embodiments, the image capture module 10 is configured to receive a set of scalp images (or "image data") under three or more lighting conditions (as shown in Figures 3A-3C).

[0062] In some embodiments, the feature scoring module 20 receives an image in a specific lighting condition as both input and outputs a corresponding score that reflects one or more scalp conditions (such as dandruff or redness level for an image in a cross-polarized lighting condition). In some embodiments, the feature scoring module 20 applies a deep learning algorithm to the input to determine the score. For example, in some embodiments, the feature scoring module 20 applies deep learning models (such as a 3-layer convolutional neural network (CNN)) having a same structure but different weight sets to predict the dandruff level and redness level score, respectively.

[0063] In some embodiments, the result analysis module 30 is configured to generate, based on the above score, a scalp and hair health analysis report to provide scores on a scalp condition, such as dandruff, redness, hair fiber diameter, hair root count, hair loss risk level, porphyrin level, etc. In some embodiments, the result analysis module 30 applies a deep learning algorithm to the input to determine the score. The smart device 1000 may include any number of additional modules, as shown in [Fig.4B].

[0064] The application 1040 may be located on the smart device 1000. In some embodiments, the application 1040 includes a live streaming module 1041, a transmission module 1042, and one or more display modules 1043.

[0065] In some embodiments, the live streaming module 1041 is configured to display or show the image data received by the device 100. For example, if the image data is a video or a live stream, the live streaming module 1041 may display the image data in real time as an operator moves the device from an individual's scalp. In some embodiments, the image data is one or more photos. In such embodiments, the live streaming module 1041 may be omitted, or may be configured to display the image data as photos.

[0066] The transmission module 1042 is configured to transmit information to an operator of the device and / or to the one or more algorithms of the smart device 1000 for processing.

[0067] In some embodiments, the display module 1043 is configured to display one or more overlays on the image data associated with a detected scalp condition, as explained in detail herein.

[0068] In one example, the system of [Fig.4A] is configured to detect dandruff and / or redness of a scalp. In such embodiments, the image capture module 10 obtains a set of images of the scalp at a location in three lighting modes. The feature scoring module 20, inputs one image of the image data in a specific lighting mode at a time and outputs the corresponding score that reflects the scalp conditions (for dandruff level or redness). Deep learning models (such as 3-layer CNN) having the same structure, but different sets of weights are used to predict the score of dandruff level and redness level, respectively.

[0069] [Fig.4B] is another block diagram of an exemplary scalp condition detection system 4000, in accordance with the present technology. In some embodiments, the system 4000 also includes one or more scoring modules, one or more segmentation modules, one or more post-processing modules, a statistics module, a segment tracking module, a score calculator, and a visualization module. In some embodiments, each of these modules is located on a smart device (such as the smart device 1000). In some embodiments, one or more of these modules may be omitted.

[0070] In block 405, raw images are obtained. Generally, the system 4000 is configured to receive image data from a device (such as the device 100). This image data may be referred to as "raw images." Raw images may also be images taken by the device that have not been filtered or modified.

[0071] At block 410A, a scoring module applies an algorithm to the raw images based on the lighting conditions under which each image was captured, one or more colors in the image, or both. For example, if the raw image was taken in white light, the scoring module may generate a redness score, such as a percentage of redness present in the image, a shade of redness, or the like. As another example, if the raw image was taken in UV light, the scoring module may detect porphyrins based on one or more colors in the raw image.

[0072] At block 415A, scores are generated by the scoring module. In some embodiments, a single score is generated, such as a redness score. In some embodiments, multiple scores are generated at the same time, such as a dandruff score and a redness score.

[0073] Returning to block 405, in some embodiments, the raw images are also input into one or more segmentation modules.

[0074] At block 410B, the one or more segmentation modules receive the raw images as input and output individual segments of hair and / or scalp features. In some embodiments, one or more areas of one or more features detected at a pixel level of the image are determined by the segmentation module.

[0075] At block 415B, individual segments of the detected features are output. In some embodiments, the features include dandruff, hair strands (or "fibers"), hair roots, or the like. In some embodiments, blocks 410A and 410B occur either simultaneously or sequentially. In some embodiments, only block 410A or 410B is performed.

[0076] At block 420, one or more post-processing modules receive the scores and / or individual feature segments from blocks 415A and / or 415B, respectively. In some embodiments, the post-processing modules include a statistics module 425A, a segment tracking module 425B, a score calculator 425C, and / or a visualization module 425D.

[0077] At blocks 430A and 430B, the output of one or more of the post-processing modules is used to generate an analysis report (430A) and / or visualize one or more areas or regions in need of care (430B).

[0078] In block 435, products and / or treatments are recommended. In some embodiments, the products and / or treatments are recommended by the application 1040.

[0079] In one example, the system 400 is configured to determine one or more scalp conditions, such as dandruff, hair quality, and / or hair roots. In such examples, the image capture module (such as the image capture module 10) is used to obtain a set of images of the scalp at a location under a plurality of lighting conditions. In some embodiments, the plurality of lighting conditions includes white light, UV light, and / or cross-polarized light. Then, the feature segmentation module 410B receives an input of one frame of the image data under a specific light pattern at a time and outputs detected feature areas at a pixel level with a probability (between 0 and 1) in terms of scalp or hair conditions. In some embodiments, learning models Deep segmentation models (such as U-net) with the same structure and different weight sets are used to segment dandruff and hair fiber / root, respectively. For the dandruff segmentation model, the outputs are dandruff class and scalp class by pixels. For the hair segmentation model, the outputs are 5 classes of black fiber, black fiber root, white fiber, white fiber root, scalp by pixels.

[0080] A segment tracking module (425B) is implemented specifically for hair fiber detection. As part of the segment tracking module, a fiber tracking function is configured to locate the position and direction of the fibers at a pixel level based on the segmented hair fiber result.

[0081] In some embodiments, a statistical analysis module (425A) is implemented. For dandruff, the statistical analysis module locates the segmented dandruff class at a pixel level and calculates the number, size, and shape of the dandruff. For dandruff, dandruff types are classified based on the statistical data of the dandruff segments. For hair fibers, a diameter sampling function was performed to collect individual fiber diameters by sampling. For hair roots, a density estimation function may count the number of hair roots.

[0082] In some embodiments, a score calculation module (425C) may be implemented. In some embodiments, the score calculation module determines the mean, median quarters of the dandruff and / or hair fiber size. The diameter may be calculated by a function. For hair fibers, the hair fiber diameter score is calculated based on the mean fiber diameter. Similarly, a hair loss risk score is calculated by a hair diversity function and, optionally, information collected by a questionnaire. The hair diversity function may classify the sampled fibers based on diameter into 3 classes ranging from fine to thick. A hair diversity score is calculated based on the comparison result of the mean diameter and fiber count for the 3 fiber classes.

[0083] In some embodiments, a visualization module (425D) is implemented. For dandruff, the contours of the segmented dandruff on the raw images are masked by visualization functions (as shown in Figures 6A-6F). In some embodiments, for hair, the hair fibers and roots are masked in different colors with a black background of functionalized scalp. In some embodiments, for fine fibers whose diameter is less than a threshold value, a pointer will be shown on the raw image by a visualization function.

[0084] In another example, the system 4000 may detect a presence / absence and / or severity score of porphyrins. In such embodiments, the image capture module obtains a set of images of the scalp at a location in three illumination modes. The feature detection module receives one image at a time in a UV light mode. The feature detection module detects areas of a specific color that indicates porphyrins in the UV images. In some embodiments, a traditional image processing method based on a python-opencv package may be used to filter pixels whose color falls within the range of determined RGB values. Then, the feature scoring module may determine the number, size, and shape of detected porphyrins calculated by a function. A porphyrin score may be calculated based on the area fraction of porphyrins in the raw image.The visualization module can display the contours of detected porphyrins on raw images masked by the visualization functions.

[0085] [Fig.4C] is an example of the system 4000 for detecting a scalp condition, in accordance with the present technology. As explained herein, in some embodiments, the system 4000 includes a device 100 and a smart device 1000. The smart device 1000 may host an application 1040 described herein. In some embodiments, the device 100 and the smart device 1000 are communicatively coupled, for example by a wireless connection. Examples of wireless connections include Wi-Fi, Zigbee, and Bluetooth LTE.

[0086] [Fig.5] is an illustration of an example of the device 100 in use, in accordance with the present technology. An operator, such as a hairdresser, pharmacist, dermatologist, or the like, can visualize an individual's scalp by placing the diffuser cone on the scalp. In some embodiments, an operator can use the device and application to provide a more professional and personalized scalp and hair care service in hair salons. Highly accurate and quantitative results describing scalp and hair health based on multiple parameters can be delivered to help establish a scientific treatment product / service based on the diagnosis in a hair salon.

[0087] Figures 6A-6J are examples of visualizations, in accordance with the present technology. In some embodiments, after processing the raw image data and detecting the one or more scalp conditions, the smart device 100, via an application, displays the image data including one or more overlays.

[0088] Figures 6A-6F are visual representations of the visualizations used to show concepts with clarity, while Figures 6G-6J are photographs of examples of similar visualizations. Accordingly, in Figures 6A to 6F, the smart device 1000, the image data 1002, the hair strands 1007A, 1007B, 1007C... and a score 1003 are shown.

[0089] For example, Figures 6A and 6G both show a scalp image without overlay. In such examples, no scalp conditions were detected. As shown in 6A, the image data 1002 represents multiple hair strands 1007A, 1007B, 1007C... and no detection areas, since no scalp conditions were detected. A score 1003 is output depending on whether any scalp conditions (and at what severity) were detected.

[0090] In [Fig.6B], porphyrins 1008A, 1008B... are shown superimposed on the image data 1002. Accordingly, the score 1003 would reflect the presence of porphyrins and / or an intensity of porphyrins based on the size, shape and / or number of porphyrins.

[0091] In [Fig.6C], films 1009A, 1009B... are shown superimposed on the image data 1002. Accordingly, the score 1003 would reflect the presence of films, a classification of films, or both, based on the size, shape, and / or number of films.

[0092] In [Fig.6D], hair strands are shown with different varieties of dashes to represent different thicknesses. [Fig.6D] is a visual representation of the photograph of [Fig.6J]. In some embodiments, hair strand thickness is indicated by different colors. In some embodiments, hair strand thickness may be classified in other ways, for example with labels as shown in [Fig.6J]. Hair strand thickness, as explained herein, is determined based on hair strand diameter. The score 1003 of [Fig.6D] would be based on a number of hair strands in each of the diameter categories. In some embodiments, a low score 1003 is indicative of potential hair loss.

[0093] In some embodiments, fine hair is indicated by one or more indicators, as shown graphically in [Fig. 6E] and photographically in [Fig. 6I]. In some embodiments, the one or more indicators 101 1A, 101 1B, 101 1C are pointers or arrows.

[0094] In [Fig. 6H], the roots of the hair are identified. In some embodiments, the overlay visualizes the strands of hair and / or the roots of the hair.

[0095] In [Fig.6F], multiple overlays are shown on an image 1002. In some embodiments, several of the visualizations shown in Figures 6A-6J may be generated and overlaid on a single image. For example, [Fig.6F] shows multiple skin conditions, including porphyrins 1008A, 1008B..., dandruff 1009A, 1009B... and fine hair indicators 101 IA, 101 IB, 101 IC detected. In such cases, all of these features may be shown on a single image 1002. Although the score 1003 may be a general hair or scalp health score, in some embodiments, an independent score 1003 is generated for each condition (i.e., a dandruff score, a porphyrin score, a hair diversity score, and the like).

[0096] [Fig. 7] is an example method 700 of detecting a scalp condition, in accordance with the present technology. In some embodiments, method 700 may be performed with a device (such as device 100) having an optical sensor (such as optical sensor 140) and a light source (such as light source 135). The device may be communicatively coupled to a smart device (such as smart device 1000) having an application (such as application 4000). The device, smart device, and application may be referred to herein as a system (such as system 4000). In some embodiments, the system includes one or more scoring modules, one or more segmentation modules, one or more post-processing modules, a statistics module, a segment tracking module, a score calculator, and a visualization module (as shown in Figures 4A-4C).In some embodiments, each of these modules is located on a smart device. In some embodiments, one or more of these modules may be omitted.

[0097] In block 705, the device is placed on a user's scalp. In some embodiments, this may be done by a hairdresser, pharmacist, dermatologist, or the like.

[0098] At block 710, light is emitted from the light source in a plurality of lighting modes (or conditions). In some embodiments, the plurality of lighting conditions includes white light, UV light, and / or cross-polarized light. In some embodiments, the light is emitted at one of a plurality of brightness modes (as shown in Figure 3).

[0099] At block 715, image data of the user's scalp is captured with the optical device. The image data (or "raw images") may be video or static images. In some embodiments, the image data is a live stream of the scalp in real time. In some embodiments, the image data is captured during each lighting condition (or a subset of lighting conditions) of the plurality of lighting conditions. For example, an image of the image data may be captured in white light, cross-polarized light, and UV light.

[0100] At block 720, the image data is transmitted to the smart device. In some embodiments, this is facilitated by a processor (such as processor 120) over a wired or wireless connection (such as Wi-Fi, Bluetooth LTE, or Zigbee).

[0101] At block 725, each image of the image data is analyzed with one or more algorithms as described herein. For example, in some embodiments, each image of the image data is analyzed with the one or more scoring modules, the one or more segmentation modules, the one or more post-processing modules, the statistics module, the segment tracking module, the scoring calculator, and / or the visualization module.

[0102] At block 730, areas of detected features are detected at a pixel level (as with the segmentation module described herein). In some embodiments, the detected features include dandruff, redness, porphyrins, hair thickness, hair diversity, and the like.

[0103] At block 735, the detected areas (or features) are visually shown overlaid on the image data. Examples of this visual overlay are shown and described in Figures 6A-6J. In some embodiments, the visual overlay is accompanied by a score (such as score 1003) and / or a report. In some embodiments, the score and / or report provide a diagnosis of the one or more scalp conditions. In some embodiments, the report may also provide a treatment recommendation.

[0104] [Fig. 8] is another example of a method 800 for detecting a scalp condition, in accordance with the present technology. In some embodiments, the method 800 may be performed with a device (such as device 100) having an optical sensor (such as optical sensor 140) and a light source (such as light source 135). The device may be communicatively coupled to a smart device (such as smart device 1000) having an application (such as application 4000). The device, smart device, and application may be referred to herein as a system (such as system 4000).In some embodiments, the system includes one or more scoring modules, one or more segmentation modules, one or more post-processing modules, a statistics module, a segment tracking module, a score calculator, and a visualization module (as shown in Figures 4A-4C). In some embodiments, each of these modules is located on a smart device. In some embodiments, one or more of these modules may be omitted. In some embodiments, method 800 is a subset of method 700.

[0105] At block 805, the films are segmented at a pixel level, as described in [Fig.4B]. In some embodiments, a 3-layer CNN algorithm is used to segment the films.

[0106] In block 810, the segmented films are classified. The classification may include film size, film shape, a number of films, and / or film type.

[0107] At block 815, a score is output regarding the severity / presence of dandruff. In some embodiments, the score is based on the classification of block 810. In some embodiments, the visual overlay is accompanied by a report. In some embodiments, the score and / or the report provide a diagnosis of the one or more scalp conditions. In some embodiments, the report may also provide a treatment recommendation.

[0108] At block 820, the identified (or segmented) films are displayed superimposed on the image data, as shown in [Fig.6B].

[0109] [Fig. 9] is yet another example of a method 900 for detecting a scalp condition, in accordance with the present technology. In some embodiments, the method 900 may be performed with a device (such as device 100) having an optical sensor (such as optical sensor 140) and a light source (such as light source 135). The device may be communicatively coupled to a smart device (such as smart device 1000) having an application (such as application 4000). The device, smart device, and application may be referred to herein as a system (such as system 4000).In some embodiments, the system includes one or more scoring modules, one or more segmentation modules, one or more post-processing modules, a statistics module, a segment tracking module, a score calculator, and a visualization module (as shown in Figures 4A-4C). In some embodiments, each of these modules is located on a smart device. In some embodiments, one or more of these modules may be omitted. In some embodiments, method 900 is a subset of method 700.

[0110] At block 905, the hair strands are segmented at a pixel level, as described in [Fig.4B]. In some embodiments, a U-net algorithm is used to segment the hair strands.

[0111] In block 910, the segmented hair strands are measured to determine a diameter of the hair strands. In some embodiments, a direction of the hair strands and / or a color of the hair strands may also be determined.

[0112] In block 915, an average hair diameter is determined based on the measurements in block 910.

[0113] At block 920, a number of hair roots is estimated. This can be done by detecting hair roots as shown in [Fig.6H].

[0114] At block 925, a score is output for the hair. In some embodiments, the score is based on the measurements and metrics of blocks 910, 915, and 920. In some embodiments, the hair score includes hair thickness, likelihood of hair loss, number (or percentage of) gray hairs, hair diversity, and the like. In some embodiments, a visual overlay may also be displayed, as shown in Figures 6G and 6J. In some embodiments, the visual overlay is accompanied by a report. In some embodiments, the score and / or report provide a diagnosis of the one or more scalp conditions. In some embodiments, the report may also provide a treatment recommendation.

[0115] It should be understood that all of the methods 700, 800, and 900 are to be construed as merely representative. In some embodiments, the process blocks of all of the methods 700, 800, and 900 may be performed concurrently, sequentially, in a different order, or even omitted, without departing from the scope of the present disclosure. Examples

[0116] Figures 10A-10E show an example of model training and validation for film scoring and segmentation. The algorithm development goes through several steps, including data collection, manual labeling, algorithm training, algorithm validation, and so on.

[0117] [Fig. 10A] is an example of a method of the process of generating a film score.

[0118] In block 305, a raw image under a cross-polarized (Xpol) lighting condition is obtained.

[0119] At block 310, a single image of the raw images is read (such as by system 400).

[0120] At block 315, a function (defined as img_matrice_norm_320) normalizes the image and determines the presence or absence of films by segmentation (as shown in [Fig.lOD]).

[0121] At block 320, the segmented image is modified with a function (defined as model_score_pellicules) that compares the detected films to the model described herein.

[0122] At block 325, a predicted film score is generated with about 0~5 as the base.

[0123] At block 330, this score is converted.

[0124] At block 335, a predetermined film score is generated with approximately 0-100 as a basis.

[0125] The model was trained on 4000 cross-polarized images for different individuals covering multiple types of scalp and hair problems to build a large dataset for algorithm development. Each image was labeled with an average Eurosyn dandruff score by experts, using the raw cross-polarized images. This data was then fed to a model having a 3-layer CNN network structure.

[0126] [Fig. 10B] is a plot of the mean error (mean standard deviation for experts). The full dataset of raw images was collected from over 200 volunteers covering multiple scalp and hair tones / types of Asian, Caucasian, and African American individuals. The full dataset consists of over 10K images. For the full dataset, over 10 experts were engaged to rate the degree of dandruff on a single image. The labels are determined as the average of the experts' scores for model training, which serve as the ground truth for the dataset. The average manual error (mean standard deviation) of the experts' scores is 0.49 (for a scale of 0 to 5) regarding the level of dandruff, which serves as a benchmark for algorithm evaluation.For the film rating prediction module, a 3-layer CNN model was trained based on a training dataset consisting of 4000 images in cross-polarized light mode. During the training process, the weights, hyperparameters, and structure of the neural network were tuned to obtain the best candidate. As shown in [Fig.lOB], the average error was 0.49 for the Eurosyn score.

[0127] [Fig.10] is a plot of the mean absolute error (MAE) of the model on the test data. A test dataset with a size of 10% of the training dataset (400 images for the dandruff scoring model) was constructed to evaluate the trained model. For model evaluation, the predicted dandruff score was compared with the ground truth (average of the experts' scores) and the absolute error was recorded for each image. The mean absolute error of the trained model on the test dataset is 0.150, which is much lower than the manual error of the experts' scoring as discussed earlier, thus showing the progress of the trained algorithm.

[0128] [Fig. 10D] is a representative image of a model training for film segmentation for visualization. Similarly, the film segmentation model was trained similarly to the film scoring model. The structure of the segmentation model was chosen as a U-net model. The training dataset consists of hundreds of images with manually masked film contours (of different types). The test dataset also follows 10% of the training dataset.

[0129] [Fig.lOE] shows a receiver operating characteristic (ROC) plot. For a segmentation task, the Fl score or ROC_AUC which can be considered as a combination of precision and recall was used to evaluate the trained model. For film segmentation, the average ROC_AUC reaches 0.959 for the test dataset, which is considered a high-accuracy model.

[0130] Figures 10F and 10G show an example of model training and validation for hair root scoring and segmentation. In [Fig. 10F], a representative method of hair root scoring and segmentation for visualization is shown. The model was trained on 172 masked images of white light illumination conditions (124 from a hair reader and 48 from a scalp reader (such as device 100)). Each image was manually labeled. In this case, a U-net structure model was used.

[0131] The model was validated for root detection (test on 20 images - 10 normal and 10 dark). An Fl score of 0.886 was determined. Table 1 shows the results of this validation.

[0132] [Table 1] Table 1 10 normal scalp test images 10 dark scalp test images 20 test images (10 normal + 10 dark) Fl score 0.94 0.82 0.88 Precision 0.91 0.82 0.87 Recall 0.98 0.84 0.91 [Fig.10G] is a graph showing a representative test of the predicted and actual test results. Accuracy is defined as shown in Equation 1.

[0133]

[0134]

[0135]

[0136]

[0137]

[0138]

[0139]

[0140]

[0141]

[0142]

[0143] Accuracy = Equation 1 Where TP is the true positive and FP is the false positive. The recall is defined as shown in Equation 2. TP Reminder — pp + p^ Equation 2 Where TP is the true positive and FN is the false negative. The Fl score is defined by equation 3. „ n ; details-reminder H — y 1 pre-session + reminder Equation 3

[0144] The model was also validated for hair diameter. 85% agreed with the QNR response by HD (N = 66) in Quali@JP.

[0145] Although illustrative embodiments have been illustrated and described, it will be appreciated that various changes may be made therein without departing from the spirit and scope of the invention.

[0146] The present application may refer to quantities and numbers. Unless specifically indicated, said quantities and numbers should not be considered restrictive, but representative of the possible quantities or numbers associated with the present application. Also, in this regard, the present application may use the term "plurality" to refer to a quantity or number. In this regard, the term "plurality" is intended to be any number greater than one, e.g., two, three, four, five, etc. The terms "about," "approximately," "near," etc. mean within plus or minus 5% of the stated value. For the purposes of the present disclosure, the expression "at least one of A, B, and C," for example, means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C), including all other possible permutations when more than three elements are listed.

[0147] Embodiments disclosed herein may employ circuitry to implement technologies and methodologies described herein, operatively connect two or more components, generate information, determine operating conditions, control an apparatus, device, or method, and / or the like. Circuitry of any type may be used. In one embodiment, the circuitry includes, among other things, one or more computing devices such as a processor (e.g., a microprocessor), a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like, or any combination thereof, and may include separate digital or analog circuit elements or electronics, or combinations thereof.

[0148] One embodiment includes one or more data stores that, for example, store instructions or data. Non-limiting examples of one or more data stores include: volatile memory (e.g., random access memory (RAM), dynamic random access memory (DRAM), or the like), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), or the like), persistent memory, or the like. Other non-limiting examples of one or more data stores include erasable programmable read-only memory (EPROM), flash memory, or the like. The one or more data stores may be connected, for example, to one or more computing devices by one or more instruction, data, or power buses.

[0149] In one embodiment, the circuitry includes a computer-readable media player or a memory slot configured to accept a signal-carrying medium (e.g., computer-readable memory medium, computer-readable recording medium, or the like). In one embodiment, a program for causing a system to perform any of the disclosed methods may be stored, for example, on a computer-readable recording medium (CRMM), a signal-carrying medium, or the like.Non-limiting examples of signal-carrying media include a recordable-type medium such as any form of flash memory, magnetic tape, floppy disk, hard disk, compact disc (CD), digital video disc (DVD), Blu-ray disc, digital tape, computer memory, or the like, as well as a transmission-type medium such as a digital and / or analog communication medium (e.g., fiber optic cable, waveguide, wired communication link, wireless communication link (e.g., transmitter, receiver, transceiver, transmit logic, receive logic, etc.).Other non-limiting examples of signal-carrying media include, but are not limited to, DVD-ROM, DVD-RAM, DVD+RW, DVD-RW, DVD-R, DVD+R, CD-ROM, Super Audio CD, CD-R, CD+R, CD+RW, CD-RW, video compact discs, super video discs, flash memory, magnetic tape, magneto-optical disc, MINIDISC, non-volatile memory card, EEPROM, optical disc, optical storage, RAM, ROM, system memory, web server, or the like.

[0150] The detailed description presented above in connection with the accompanying drawings, where like numerals refer to like elements, is intended to be a description of various embodiments of the present disclosure and is not intended to represent the only embodiments. Each embodiment described in the present disclosure is offered solely by way of example or illustration and should not be construed as being preferred or advantageous over other embodiments. The illustrative examples provided herein are not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Also, any steps described herein may be interchangeable with other steps, or combinations of steps, to achieve the same or a substantially similar result.In general, the embodiments disclosed herein are not limiting, and the inventors contemplate that other embodiments within the scope of the present disclosure may include structures and features from more than one specific embodiment shown in the Figures and described in the patent specification.

[0151] In the foregoing description, specific details are presented to provide a thorough understanding of exemplary embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the embodiments described herein may be practiced without incorporating all of the specific details. In some instances, well-known process steps have not been described in detail so as not to unnecessarily obscure various aspects of the present disclosure. Furthermore, it should be appreciated that embodiments of the present disclosure may employ any combination of features described herein.

[0152] The present application may include references to directions, such as "vertical", "horizontal", "front", "back", "left", "right", "above" and "below", etc. These references, and other similar references in the present application, are intended to assist in describing and understanding the particular embodiment (such as when the embodiment is positioned for use) and are not intended to limit the present disclosure to these directions or locations.

[0153] The present application may also refer to quantities and numbers. Unless specifically indicated, these quantities and numbers should not be considered as restrictive, but as examples of the possible quantities or numbers associated with the present application. Also, in this regard, the present application may use the term "plurality" to refer to a quantity or number. In this regard, the term "plurality" is intended to be any number greater than one, e.g., two, three, four, five, etc. The term "about," "approximately," etc. means within plus or minus 5% of the stated value. The term "based on" means "based at least partially on."

[0154] The principles, representative embodiments, and modes of operation of the present disclosure have been described in the foregoing description. However, aspects of the present disclosure, which are intended to be protected, should not be construed as being limited to the particular embodiments disclosed. Furthermore, the embodiments described herein should be considered illustrative rather than restrictive. It should be appreciated that variations and changes may be made by others, and equivalents employed, without departing from the spirit of the present disclosure. Accordingly, it is expressly intended that all such variations, changes, and equivalents fall within the spirit and scope of the present disclosure, as claimed.

Claims

Claims

1. A device for detecting a scalp condition, a hair condition, or both, wherein the scalp condition includes at least a presence or absence of dandruff, and the hair condition includes a presence or absence of the hair condition, comprising: at least one camera configured to view a user's scalp; a diffusion cone disposed around the at least one camera; a lighting module comprising at least one light source, wherein the at least one light source is configured to emit light under a plurality of lighting conditions; and a processor configured to: - receive image data from the at least one camera captured during at least one lighting condition of the plurality of lighting conditions; and - transmit the image data to a smart device for detecting the scalp condition, the hair condition, or both.

2. The device of claim 1, wherein the image data comprises a set of images at a location on the user's scalp under each of the plurality of lighting conditions.

3. The device of claim 1, wherein the plurality of illumination conditions includes a white light condition, a cross-polarized light condition, an ultraviolet (UV) light condition, or a combination thereof.

4. The device of claim 1, wherein the at least one camera is a three-lens camera, and wherein each lens of the three-lens camera has a different focal length.

5. A system for detecting a scalp condition, a hair condition, or both, wherein the scalp condition comprises at least a presence or absence of dandruff, and the hair condition comprises a presence or absence of the hair condition, the system comprising: the device of claim 1; and a smart device, wherein the smart device comprises a smart device processor, and wherein the smart device processor is configured to: - receive the image data from the device; - input an image of the image data under one of the plurality of lighting conditions; - determine one or more areas of one or more features detected at a pixel level of the image; and - display the one or more areas overlaid on the image data.

6. The system of claim 5, wherein the smart device processor is further configured to: - segment, at a pixel level, the image data; - locate the segmented films; - classify the segmented films based on a film count, a film size, and a film shape based on the film statistical data; - output a score for a film severity; and - display outline segments of the films superimposed on the image data.

7. The system of claim 5, wherein the smart device processor is further configured to: - segment hair strands in the image data based on color; - determine a position and direction of the hair strands; - measure individual diameters of one or more of the hair strands; - determine an average hair diameter; and - output a hair score based on hair condition, wherein the hair score comprises an average hair strand diameter, a percentage of white or gray hair, a root color detection, or a combination thereof.

8. The system of claim 5, wherein a 3-layer convolutional neural network (CNN) having a same structure with different weight sets is used to predict the severity of dandruff score and a level of scalp redness.

9. The system of claim 5, wherein the smart device processor is further configured to: - detect one or more colors on a pixel level in the image data indicative of porphyrins; - determine a number, size, shape, or combination thereof, of the porphyrins; - output a porphyrin score; and - display outlines of the detected porphyrins superimposed on the image data.

Images